Setup an Apple ID for your child - With no Credit Card
Most of the information in this article is now out of date and not accurate. I've left it here for legacy sake.
If your child has an iDevice (iPhone, iPad, or (more likely) an iPod Touch) you can create an Apple ID for them to use in the App Store without any credit card attached. That way they can download free apps they find without bugging you to do it (or you giving them your password), and there's no risk of racking up a huge credit card bill. If they use their Apple ID when they download an app, they can also download any updates themselves.
You can add credit to these accounts using iTunes gift cards you purchase from many retail outlets around town. So they can buy that copy of Angry Birds they've been bugging you about with that $5 iTunes gift card they got from Grandma for their birthday.
Age-based content restrictions can be put in place for Apps, Music, Movies, etc so they can't download age-inappropriate material. Restrictions can also be put in place to prevent deleting apps, accessing certain apps (like Safari and YouTube), changing mail account details and adding friends in the Game Center.
You can read how to create a "credit card free" Apple ID here. Make sure you follow Step 4:
4. Important: Before proceeding to the next step, you must download and install the free application by tappingFree followed by tapping Install.
If you already have an Apple ID and want to remove the credit card go to this site and follow the directions under "How do I delete my credit card number?"
You can also learn more about Restrictions in iOS devices here.