It's the age old argument, done to death on many websites/forums. Yes Lexus cars share much from the Toyota line-up. What makes a Lexus a Lexus is the additional sound deadening, higher quality materials in the cabin and yes, the badge.
As Toyota and Lexus are essentially the same company one should naturally expect a degree of cross-over but one can certainly tell the two apart.
Lexus don't have to defend their price premium, they offer, one buys, or one doesn't. The choice is one's own.