regarding your speaker adapters did you use silicon between the adapter ring and the door panel
as you need to make an air tight seal between the two and the same goes for the speaker to adapter,
between these pieces you could you the white foam type material that you find around a tv / pc monitor
to protect the screen this will help to create a seal, a speaker needs to create pressure in an enclosure
i.e. inside door... if this pressure escapes on the front side of the panel this maybe a reason for poor sound quality,
you could use dynamat behind the speaker on the door skin as this will stop the panel from resonating ,
as a cheap option you could use some self adhesive flashing on a roll as its dense and will stop the door skin
from vibrating.