As a guesstimate, the quality of your sound is somewhere about like this:
10% driver quality
10% crossover quality
30% amp quality
50% cabinet/enclosure design
so if you have a super high end amp, super high end driver, and super high end crossover, but your enclosure is poorly designed you're worse off than if you have low end amp/crossover/driver and a well-made enclosure. Most car amps include a crossover in them, so you usually don't have much choice there.
A carpeted box with two speaker cut-outs and vent hole won't cut it unless you designed the shape and volume of the box correctly and made the vent hole the correct area with a tube of the correct size. Most people have to use insulation to give the enclosure extra volume, shape it to avoid standing waves, and make sure the port is properly tuned.