The LS400 is probably the best in terms of over engineering. The new LSs have a fit and finish that far exceeds the originals - body panel gaps and consistency for example were hugely improved upon with the LS430 and again with the LS460 due to new technology introduced on the LS production lines in the factory.
Whilst the IS may be built in the same factory the LS, I believe, uses thicker gauge steel for the body panels, you have the better panel alignment, the paining process involves two stages of rubbing down by humans, the leather is buffed for many hours more than on the IS, 100% of all the speakers and amps are tested with a full burn-in at the ML factory, the headlamp glass is specially designed to look like crystal, and hundreds of other examples.
There IS is a great car for the class it is in. It is refined and comfortable but not to the level of the GS, which itself isn't at the same level as an LS. There is a reason why the LS costs around three times that of an IS, and only a very small fraction of that cost is due to the vehicle physically being larger and therefore needing more basic materials.
The IS has always had quite a lot of road noise, I'd suspect an LS driver would notice that more than most. Handling is much better in the IS compared to the LS, especially the series I and series III IS, but an LS driver isn't typically looking for that type of engaging drive.