You do need to book in advance for most games, at least at West Ham, where tickets range from between around £70 to £50, depending on where you sit and the status of the opposition. It's generally accepted even amongst rival supporters that there's a really good atmosphere at Upton Park compared with the other London PL sides. West Ham would be a really good team to see ... so long as you don't expect them to win.
You can get all the ticket information you need here:
http://www.whufc.com/page/TicketNews/0,,12562,00.html