The privilege level page for protected questions gives a simple explanation. Anybody with 3,500 rep or more can protect questions with just a single action.
A protected question prevents answers by anonymous and very new users.
Questions should be protected when they are garnering lots of views and newbies are adding "me too!", "thanks!" and possibly even spam non-answers.
The result is both simple and subtle. Only users with 10+ rep may add new answers. This means that before a newbie can answer one of these questions, they must answer some other question and get at least one upvote. One upvote is all it takes. This is still a very low bar for entry, but what it does is stop drive-by users who have landed on the site with no clue what is going on jumping in before they have looked around at at least a few pages and started to get an idea about how the site works.
It's quite true that over the past few weeks I have protected maybe a couple dozen questions. I realized at some point that this tool was an obvious little aide in dealing with increased site traffic that we were not taking advantage of. There has been a clear trend in a few troublesome questions attracting low quality answers. When people's first contributions to the site are on these popular questions, they almost always go awry. Whenever these pop up on the home page, I've been protecting them so they don't fall victim over and over again.
I haven't been using any hard numbers to draw the line, but the indicators to consider are:
Number of answers: Questions with 5+ answers already might be headed off course and are candidates for protection. If there are already several really good quality answers and new answers are showing up that are lower quality than what is already there, it might be time to protect the question. Of course there is always room for a better or more throughout answer, but people just shoveling their opinion into the bucket isn't what we're after.
Number of deleted answers: If, for whatever unknown reason, a particular question has already attracted a number of failed attempts where very low quality answers have been deleted, it is worth considering whether the question should be reserved for people who have successfully answered at least one other question first.
Number of views: Questions with very high numbers of views are probably getting hit using links from outside the SE network. These often attract answers that don't contribute anything new.
Number of votes: High vote count on the question is often an indicator more of the topic being popular than the question being high quality. These questions tend to draw attention and make people feel like throwing in their 2c. New site users often browse for the highest voted questions. Often these are the ones that are actually hardest to give high quality answers too. Restricting these to users that have participated in other questions where it is easier to follow and learn site guidelines is a good idea.
I would encourage all our regular users to keep an eye out for problematic questions. Lots of you have enough rep to protect them yourselves, and if not you can always flag with a note for a moderator to do so. It's really a simple and subtle tool, but can help keep the mess from getting out of hand.