Usually when watching the Phantom Menace I wouldn't think much of this situation, other than that it's another step for Palpatine to rise in power. But after reading the Darth Plagueis novel, this short scene got a lot more depth:

In the years and months prior to this scene, things had gradually been looking worse and more complicated than it had in a long time for the senate and most of all for Valorum.

 During these trying times he did not have many "real" friends to go to for support, except for one; Palpatine. Palpatine was the one person Valorum trusted and went to for support and advise in both political matters and some more personal matters. At a party for the elite on coruscant, Valorum even told Palpatine that he wasn't sure who else, amongst all the people at the party, he could count on.

Then move to the day of the scene in the senate:

So when the queen of Naboo votes for no confidence, visibly advised by Palpatine, the one person who had Valorums back, Valorum has a realization: He has no one left. His one trusted friend, is now advising against him. This realization is what truly breaks him leaving, the usually talkative and confident man, speechless, with nothing to do but sit down.

That's my take on this scene. Hopefully this can make the scene more powerful for you too. I hope you found this as interesting as me, or slightly intriguing at the least:)