Can we reverse the trend on low quality posts?

Question

In the beginning, there was Christianity.SE

Congratulations community, we are now just over half-way through the minimum stay in public beta! We have lots of participation and a full seven weeks of asking and answering. Honestly we have a lot of good content and some wonderful users.

However, many of us who have been here from the beginning have become increasingly concerned at the prevalence of low quality questions. At first we stood by - hoping that we could get the low quality basic questions out of the way and grow into expert, high quality questions. Unfortunately this hoped for direction is not materializing and we have realized that Christianity.SE is not improving. Instead, it seems that as we gain new users, the number of low quality questions seems to increase while the number of high quality questions diminish.

Over the past several weeks, several people have raised this issue in chat. Other concerns about low quality can even be seen much earlier in our history in meta posts like How can we get better expert-level questions?. This problem has become even more tangible since Biblical Hermeneutics went into private beta. Those of us who have participated in that beta so far have been impressed by the high quality of that site.

With all of this in mind, several of us have tried to nail down what our problem is and we have a possible solution to propose. After several long discussions, we’ve concluded the problem is two-fold: unfocused questions and unsupported answers.

Unfocused Questions

The primary problem that we’ve identified that makes a question low quality is a lack of focus. Questions like Is it a sin to get a tattoo? invite low quality responses due to their unfocused nature. Pure opinions and speculation would be completely valid for this type of question. However, opinions and speculation make for very low quality answers. Unfocused questions like these have drawn a large number of low quality answers and encouraged a general trend in opinion-based answers.

The fact that unfocused questions lead to poor quality answers can be clearly seen if we compare the previous example question (about tattoos) to a high quality, focused question such as What is the Mormon doctrine regarding becoming a god?. Clearly this second question invites high quality answers that are focused and well supported. Anyone that posts an answer to the second question will post high quality, supported answer based on doctrine*. Low quality posts (speculation, opinion, or unverifiable doctrine etc.) quite simply do not answer the question and can easily be identified, downvoted, flagged or removed as appropriate.

* _{Thus we see the other advantage of focused questions: they make it harder for trolls to get a word in edge-wise :)}

Unsupported Answers

While unfocused questions are clearly the primary problem (as they lead to unsupported, low quality answers), we have also seen a lot of low quality answers in general. This answer in response to an excellent question regarding Catholic prayers is currently the highest voted answer. However, the answer itself is completely unsupported and purely opinion.*

*_{I [Richard] suggested this example but am not encouraging you to downvote it, it is after all my answer!}

If we compare that low quality, unsupported answer with this answer regarding the translation of Elohim, we can see that what makes it good is its thoroughly support using biblical references and external sources. Obviously, these are the types of answers that make StackExchange a better place.

What could happen?

Our goal when we committed to Christianity.SE was to see this site succeed. (I’m sure that is true of almost everyone reading this long post.) We wanted Christianity.SE to be a place where people could come and ask questions or find answers to the theological and doctrinal issues that they struggle with.

We believe that the quality of the questions and answers around here needs to be raised. We believe this is essential in order for Christianity.SE to become a haven on the internet for intelligent questions and for doctrinal experts. We believe that without raising the bar on the questions and answers, this site may never graduate from public beta and may eventually end up failing as a StackExchange site.

Therefore, we believe we need to refine our quality guidelines in order to raise the bar on both questions and answers.

So what now?

Lets hear some voices! Do you agree that something needs to be done? Or do you think things are fine the way they are? This is an opportunity to either agree or disagree with our conclusion. This is not our site; it does not belong to the moderators or even just those with the highest reputation. It is a community effort and the whole community needs to pitch in to help choose a direction.

We will deal with how to fix this in later meta posts. At this point we are just looking to establish a baseline that there is a problem. Please use the answer section here to voice your support or disagreement and explain why.

Furthermore, you can help out with this effort by getting involved.

Answer 1

I've been harping about this for a while, but what really turns me off to the majority of questions here is how utterly untestable they are: "what does X in the Bible mean?", "what's the Biblical basis for X?", "what do Christians think about X?"

These are soft questions: if you spent at least a modicum of effort to quote any arbitrary amount of Biblical passages, you get a ton of upvotes. But they don't actually teach people who don't already agree with the answer's interpretation anything new. If I'm not also familiar with the Biblical interpretation an answer uses, how do I know it's correct? I can't test it: the interpretation is not cited, or worse: completely synthesized by the answerer.

The value in Stack Exchange is providing not only expert answers, but verifiable answers. You can see this in action on a site like Stack Overflow. How do I know an answer is correct? I can test it myself. Obviously, testing an answer in the same vein as Stack Overflow doesn't work for cerebral sites like Christianity.SE, but we should be relying on the next best thing: reliable secondary sources. I know an answer is correct because I can read the secondary source myself and confirm it.

Ah, you may be thinking: "well why should I trust Augustine or Luther over a random Christianity.SE user? It's all just opinions, man!" That's the problem: questions that just ask for interpretations are, for the purposes of verifiability, opinions. For someone who doesn't already have a background in the question, random Christianity.SE user's interpretation is no better or worse than the interpretation of whale like Augustine or Luther.

So the solution isn't to allow people to provide personal synthesis: instead, we need to stop asking open-ended interpretation questions. Questions must provide parameters for testability. If you'll excuse my reliance on questions I've answered (I do so because they are ones I believe to be testable), I'm talking about questions like these:

• Why doesn't the Catholic Bible include all books from Septuagint?
• List of papal teachings considered infallible
• Why does Vatican require "Yahweh" not to be used?

Not questions like these:

• If Adam and Eve were perfect, why did they choose to sin?
• Can I believe in evolution and still be a Christian?
• Can there be free will in heaven?

Yes, these are popular, "fun" questions that let people exercise their intellectual curiosity and taste for discourse, but they don't provide answers. I'd much rather see a less-popular, dry Q&A site that actually teaches people things than a wildly-popular, fun discussion board that's only a step above Reddit or Yahoo! Answers. The hope of a fun discussion site where I can riff and wax intellectual about a topic is not why I invest my time in Stack Exchange.

Answer 2

Yes there is a problem, and that problem is exacerbated by the existence of BH.SE as a separate site. The problem is confounded by the fact that some beliefs and practices - doctrines if you like - have a wealth of reference material while others have relatively little.

The older, more traditional denominations place a great emphasis on their long histories and written doctrines, and much more has been written about them than about things like the New Church Movement. I think the point I'm trying to make is this:

We are not Wikipedia.

Wikipedia famously has a criterion for inclusion which is "verifiability, not truth". This works well to meet that project's aims of consolidating information that's already published; but that's not SE's purpose. We're here to help people, to answer questions, to give the right answer - and that "right answer" doesn't necessarily have a shed load of reference material to back it up, especially where newer developments are concerned.

So I agree that there's a problem but I wouldn't want us to end up insisting that every statement in every answer needs to cite a reliable source as a solution.

Answer 3

Yes, we can!

^{(reverse the trend on low-quality posts.)}

Seriously, we have a lot of good questions, too. And we do tend to get excellent answers on good questions. It's just that the good questions get too little attention because of the bad(ly-focused) ones. I agree we have a definite problem.

I personally have asked many questions that are vague, unfocused or in general don't encourage good answers.*

• Is smoking sinful?
• Is "Bible In a Minute" accurate?
• What are the benefits of a red letter edition Bible?
• What does the Bible say about ecology?
• Does it matter which day I celebrate Sabbath / Lord's day on?
• Should a Christian read books known to be heretical / blasphemous / against Christianity?

Similarly, I've posted bad answers.* But I think we should focus on the questions, as not all questions are even possible to answer well.

Me, I'll need to give up or greatly revise many of my own posts if we're to change guidelines. But I think it's worth it. The SE system has been proven excellent in crowdsourcing factual answers to questions. My dream from the beginning has been to leverage that ability, and create a high-quality Christianity site. One where knowledge really trumps ignorance. One where trolls and ungrounded opinions have little room, whereas elsewhere on the internet they roam free.

So I say, let's do this! We already have much high-quality content and much activity. If we can come up with revised guidelines that encourage good content and filter out bad, we have a good chance of making the site a great success!

* _{I've posted excellent questions and answers, too, but they're too numerous to list... ;)}

Answer 4

Yes, there is a problem and we can turn it around. (This is my second answer here, this one takes a different angle as a counter point to @Waggers answer).

One of my early concerns on the site was this line in the stock FAQ:

Your questions should be reasonably scoped. If you can imagine an entire book that answers your question, you’re asking too much.

Given the long history of Christianity and it's propensity for disagreements and elaborate explanations of the minutest details, none of us could imagine a point doctrine that did not have at least one book devoted to the topic. I even went so far as to suggest removing that line.

In retrospect I think we should have paid more attention the warning because it turns out one of our largest problems is with popular questions that are unanswerable and it turns out generally have whole shelves of books on the topic.

Waggers suggested "newer developments". This is partially my doctrinal bias (Reformed Protestant) but I don't think there are any such things. Ecclesiastes tells us there is nothing new under the sun. Several posts here have shown how old heresies just resurface in different forms. New developments are typically NOT doctrinal, they are technological or archiological. These developments are typically well documented. In our modern day and age few beliefs slip through the cracks without somebody else documenting them.

Note that I am not suggesting that every answer must have a bibliography, but I think it is reasonable to require that answers do at least represent doctrine as believed by somebody other than self-proclaimed-opinion. With the diversity around it should never be hard to find some group that holds any given doctrine. Not identifying specific doctrines only leads to communication nightmares where we are using different terms to fight over the same ground.

I would not request every answer have written references to printed doctrines of the New Church Movement, but an answer that makes a doctrinal statement should identify itself as "The position of the NCM on this issue is X." If you can source it with references to statements of faith or other documents great. If you can show "and this is how they defend that doctrine using scripture" then more bonus points. But just answering "X" without any form of support is not a useful or constructive answer.

Answer 5

Yes, I agree this is a significant problem and we need to reform our guidelines in order to foster an environment for experts!

I have a number of questions that are unreasonably broad and difficult to answer without creating a voting contest. They encourage opinion-only answers. I think in order to provide a reasonable environment for high quality answers, these should be edited to ask for the doctrines of particular tradition or a comparison between a specific set.

• Under what conditions can a "good" or "charitable" act actually be sin?
• When did Christianity originate?
• Can I go to church on the internet?
• Can Christians be demon possessed?
• Is the Israel of the NT the modern country?
• Can man judge whether God is fair?
• What happens to people who have never heard about Jesus?

I also have a number of answers that are just me documenting my own beliefs. I think it is unreasonable to consider these high quality and they should be prompted for references to who believes this and in some cases, why.

• Did Satan really not understand God's salvation plan?
• Why did people live so long before the Flood?
• What is the interpretation of the bow in the cloud after the Flood?
• Are questions that begin "Is it ok to ___" of primary theological importance?