Apple TV is our favorite way to watch YouTube videos on our son. Many otherwise excellent YouTube videos have pre-roll ads. Some of those ads (playing before children's content) are not suitable for children. Lack of advertising is a great reason to consume YouTube videos via Apple TV.
The embedded video below is a "Playlist". A Playlist is a collection of videos created by a YouTube user. Apple TV directly supports only a single Playlist... the "Favorites" of the YouTube user signed in to Apple TV. The Playlist below is my son's, created with his very own YouTube account. We can access it directly from our Apple TV (which we leave signed-in to his YouTube account at all times). By looking at this webpage with an iPod, iPad or iPhone, you can AirPlay this Playlist to your Apple TV.

AirPlay is the only way to play a Playlist all the way from beginning to end on Apple TV. Even on our own Apple TV, looking directly at our son's Favorites, videos can only be played back one-by-one.

So if you are viewing this page with an iOS device, and you're sitting in your living room next to your HDTV and Apple TV, then you are exactly where you want to be to start experimenting with AirPlay, and the playing back of an embedded Playlist (above).
If you are watching this playlist with an iPad, be aware the forward/back buttons are only available full-screen! That means (with an iPad) you must start playing back the video, expand it to full screen, and only then will you see video chapter buttons.
Also, as your iOS device reaches the end of one video chapter, Apple TV briefly flickers back to a menu display, then returns back play the next video on the Playlist. Anyone familiar with Apple products will find this jarring and un-Apple like. Hopefully it will be addressed in future iOS releases, but it is normal behavior for today.
Of course, my son's Playlist isn't going to meet the exact needs of your child. You really do want to create your own, and I'd suggest creating an independent YouTube account for them. YouTube doesn't allow super-young people to have their own Google Accounts, so this is essentially another account for YOU to presumeabley hand-off when they're old enough.
You might also want to install a seperate browser on your computer specific to your child's YouTube account. This way when you're surfing with your own browser and see a YouTube video you'd like to add to their favorites, you simply copy/paste the URL into the child's web browser and [Add to Favorites]. That new favorite'd video will now appear on your Apple TV's favorite list.
The process of finding and adding quality videos to a YouTube account is painful on Apple TV, compared to how fast one can find and critique a video on a computer. If you'd like to crib YouTube videos from my son's favorites, I suggest you move to a computer and sign in to their YouTube account before continuing this process...