Hop on one of the river boats for a trip up and down the river. If you get a one day pass (150 baht when I was there 12 months ago) you can get on and off as many times you like. One of the landings (#8) is located near the Grand Palace and reclining Buddha, and accross the river is Wat Arun (Temple of the Dawn). Have a walk around Wat Saket (Golden Mount) - from the top you get a reasonable view of Bangkok.
Visit the Vimanmek Teak Palace - built for one of the Kings in late 1800's early 1900's. There's a few other sights around there as well (one I liked was part of the Kings vehicle fleet - 6 ~ 8 VW Kombi's from the late 50's to early 70's all in different colours and immaculate condition).
For shopping there's MBK - 6 floors of anything imagineable and reasonable priced. Next to Lumpini Park there is (or was - there was talk of it closing but I don't know if it has) Suan Lum night bazaar. One of the better places to buy reasonably priced knock-off's and other stuff.
Of course Patpong Road is worth a look - there's a night market there as well, but you pay higher prices.
As noted above - grab a guide book and get out and explore!