Here's our family's favorite kid-friendly videos. Scan through to find videos suitable for your own child. Favorite them with same YouTube account used by your Apple TV, and these videos will be easily accessable via your Apple TV's YouTube app.
They Might be Giants - Here Comes Science. This album (including videos) is available from iTunes. If you enjoy, you'd probably want to support TMBG by purchasing it. But I also suggest you Favorite these videos with your child's YouTube account too... both have their own advantages: An iTunes purchase is the only way to enjoy these videos on your iPad in an airplane! (But YouTube lets you Favorite all types of videos together.)
And some non-Here-Comes-Science videos from They Might Be Giants, mostly from "Here Come the 123s".