Well, as I mentioned b4, it is not all the time and it seems to get better the longer I drive. I now have about 500 km and the gear shift felt much worse in the first km's... I believe the "problem" will be gone in a few 100 more km's.
I could live with an auto on my IS300 and I probably could live with an auto on a IS350, but to be honest, the IS250 isn't a rocket and it somehow feels more sporty with a manual box. OK, the 250 is quite even compared to the 300 in terms of power, but the acceleration definitely feels different.