I haven't had to replace my O2 sensor yet, but I remember on another car I had in the past, the O2 sensor went, and I wasn't planning on keeping the car, so I didn't change it. After a few months, the car ran like crap, and my cat. converter rotted out. What I didn't know at the time was that with a bad O2 sensor, the computer gets readings it doesn't like, and compensates. This can mean adding fuel when it's not needed, or cutting back on fuel when fuel is needed, etc... There's actually nothing wrong with the way the engine is running, but the computer changes things based on the readings from the faulty O2 sensor, and the engine starts running poorly. And in my case, the resulting change in emissions rotted my converter. I can't say for sure that this is your problem, but if your computer is getting a bad signal from you O2 sensor, it might be the problem.
Now a more obvious thing to consider is the plugs/wires. I'm sure you've done that already, but really bad plugs/wires will also trigger the check engine light, and cause stuttering.
Oh...and maybe I'm just old, but don't just make it "louder". Loud exhaust is retarded. A nice deep throaty exhaust that you only hear under hard acceleration...that's nice. Those ricer-burners you hear from 5 miles away while they're only doing 5 MPH...that's just stupid. | 
