If you're into serious low-cost transportation, consider a Tomos moped. I have one, rode 7500 km since I got it last spring, averaging 107 mpg. Prices are $1000 - 1700, depending on model and dealer. Look at
Tomos. Tomos has been making mopeds since 1954, and their engines are quite good. Their whole moped is good -- durable and simple for maintenance and repair.
If you want a good hobby, add an engine to a stout bicycle, as suggested by other posts. Golden Eagle makes a good kit, sold via internet, with several engine options. But these add-ons are not really serious transportation -- some of the name-brand engines are quite good, but adding an engine to a bike puts a lot of extra stress on spokes, bearings, tires, and brakes. You're adding 12-20 lbs, usually hung out over the rear wheel, and you will be consistently running at higher speeds.
For a cool-looking ready-made motorized bicycle that is strong enough for transportation, check out a Whizzer Bike. A guy in town has one, and it is something people will stop and look at. They are sold over the internet. I think the price is a little high compared to a Tomos moped, but it comes down to personal taste. The original US-made Whizzers are real collector's items when restored.