I agree, most of the hardware works great (out of the box) with linux.

An exception is the dial-up modem, to make a long story short:

Use an external SERIAL modem. better yet, get an ADSL connection or you can use internet sharing through your windows box.

If you insist on using one of the "cheap" modems available, your best bet is to buy a connexant modem. Don't even think about creative, it's really good but completely unsupported under linux.

Personal note: All VGA cards work nicely, but if you're interested in games (3d accelerated), then get a nVIDIA card (GeForce, TNT or whatever). they have the best linux drivers. Again if you don't need acceleration, then any card will do.