My Kid and TV

My little toddler loves to watch TV. Sometimes I fear that I allow him to watch too much. Even though he only watches the kids' channels, and I usually watch the shows with him, I still don't feel comfortable when he focuses on the TV, instead of playing with his toys. I do turn the TV off, when I think he has watched quite enough for a stretch of time. There are some shows that I just cannot stand - not for any particular reason, but just personal taste. My toddler is so worried that I'll turn off the TV, instead of switching channels, that he'll tell me, "Mommy, it'll (the show I don't like) be over soon!"