Recap of events

There was some discussion here and in chat with concerns of the quality of new questions and the quality of questions visible on the front-page. The consensus (or more honest: people I agree with so who I will selectively quote) is that if a users sees better questions on the front-page he/she is more inclined to ask better questions. To quote: "Make the front page represent the sorts of questions we want". Putting bad questions on-hold does not help with accomplishing this goal, even if moderators use their power to quickly close bad questions they are still very visible.

Implications

So closing questions does not give us a good front-page, does not inspire new users, and frankly pisses of existing users (who are irritated by all the bad/closed questions while trying to find something interesting to answer/comment/moderate).

Question

My question to meta is: how can we make bad questions less visible on the front page.

My thoughts on a good solution

While I don't have a direct answer to this question I think a good solution should not censor/remove bad questions. Visibility is the keyword here.

I definitely think improving the front-page question view is a good idea. I don't think we should do it by "closing less," (closing less is essentially another way to say, "allow softer questions," which I would much prefer we discuss -- if we're going to discuss it -- as a separate topic where we actually use that terminology; it is a viable option, but it also has serious risks).

Rather, I think we should do it by promoting pre-existing good questions to the front. There's only one effective way to do this, and fortunately that way goes hand-in-hand with what I perceive to be the fundamental key problem on this site (as discussed in the "be nice" threads): lack of engagement by existing active users. As I write this, we have 1279 (that's one thousand two hundred and seventy-nine!) unanswered (zero answer) questions.

A good portion of those questions are bad, many are duplicates, some are unclear. But there are also some really good questions in there, with difficult and/or interesting answers. There are also a lot of questions that could be good with a little love and editing. Answering or editing those questions not only increases user engagement, it improves the quality and utility of the site overall and (particularly relevant to this discussion) it bumps them to the front page.

Consequently, my call to action here would be for everybody who has participated in this thread (or the other one) to go out and start working through that backlog of unanswered questions. If you can't vote, flag. If you can't edit, propose a suggested edit instead. And every single one of you can answer. This is supposed to be your community. Take charge of it.

  • 2
    Also, refer to the 2013 Winter Refactor post for some guidelines for dealing with the unanswered queue. – Josh Aug 15 '14 at 15:19
  • For comparison: Stack Overflow has a 0.2% rate of unanswered-unclosed questions (39,388 out of 21,265,972 questions). Superuser has a 0.4% rate (2,997 out of 607,266). Our rate is currently 1.1% (654 out of 62,056). (I'm not sure why you're showing 1,279 questions, Josh; I only get 654 in the Data Explorer. – Trevor Powell Aug 16 '14 at 1:08
  • That query is not what the "Unanswered" tab shows, that's why :) (specifically the "no answers" sub-tab). – Josh Aug 16 '14 at 6:16
  • Hey, cool, I'd been struggling to find a way to get to just the unanswered questions. Still would be interested to know how the 'no answers' sub-tab chooses which questions to show, because it clearly is showing a lot more questions than the data explorer claims exist, which either means that my query terms are wrong, or the data explorer's data is suspect. (The former is more likely, but I can't see a problem in my query..) – Trevor Powell Aug 16 '14 at 8:53
  • @TrevorPowell the Unanswered tab counts questions with 0 answers with score 1 or higher, which is not the same as having 0 answers. – congusbongus Aug 19 '14 at 7:27
  • 1
    @congusbongus Thanks so much for posting that clarification! It totally explains the values I was getting from Data Explorer. – Trevor Powell Aug 19 '14 at 8:38

This isn't a real answer, just some stats that are too lengthy to fit into a comment:

Looking over the current front page, the vote score for a question actually looks like a pretty good heuristic for predicting the question's quality. That is, questions voted +2 or higher look pretty legitimate. And the ones voted -1 or lower look less ideal, in terms of being the type of questions we want to encourage on this site. This suggests that the voting members of the community are still successfully identifying good questions -- the problem we're perceiving is just the ratio of bad questions to good ones.

At the time of writing, out of 48 questions on our front page, 11 of them have a vote score of +2 or higher. 6 have a vote score of +4 or higher. 10 have a vote score of -1 or lower.

Comparing to other SE sites: On Stack Overflow, 9 of the 48 front page questions have a vote score of +2. 2 have a vote score of +4 or higher And 2 have a vote score of -1 or lower. (Note that Stack Overflow gets a much higher volume of questions; the vast majority of their questions have zero votes in total)

On Superuser, 12 out of their 48 front page questions have a vote score of +2 or higher, and 10 have a vote score of +4 or higher. 4 have a vote score of -1 or lower.

Per-site front page stats ("voted up" == +2 or higher, "voted down" == -1 or lower)

  • Stack Overflow: 20% voted up, 4% voted down
  • Superuser: 20% voted up, 8% voted down
  • GameDev: 20% voted up, 20% voted down

It's interesting (and perhaps surprising) that the percentage of up-voted questions on the front page turns out to be almost identical between the three sites, even with Stack Overflow's substantially higher churn rate. The real difference between the questions shown on the front-page of the different sites seems to be in how many of the non-great questions are bad enough to attract down-votes.

I wonder whether any other SE sites have down-vote rates similar to ours? We might be able to learn from them, if they've dealt with this situation in the past.

Edit:

For the sake of interest, here's a graph showing the percentage of GameDev.SE questions asked in different months that were downvoted to -1 or below:

GameDev's percent of questions downvoted per month

Interestingly, this shows a big peak of downvoted questions a little over a year ago, after which it fell right off, only to have a few brief peaks later on, including a big one that appears to be going on right now, which lends some support to @Anko's suspicions. But even these big peaks are topping out only at about 15% of questions being downvoted.

But here's another interesting graph, this one showing the percentage of questions asked in GameDev.SE each month which would eventually wind up with a score of at least 4:

Percentage of questions which score >= 4 points

Note that this isn't a cumulative graph; pretty regularly, month-on-month, we've had a lower and lower percentage of questions reaching a +4 score ever since the site opened. In July of 2011, nearly 50% of all questions asked were reaching a score of at least +4. In July of this year, only 4% of questions reached +4. (To be fair, though, we got almost twice as many questions in July of this year as we did in July of 2011)

The query I used is saved publicly here. It prompts you to specify a minimum threshold, and graphs the percentage of questions which scored beneath the specified value.

Please feel free to mock my rudimentary SQL skills, and let me know about the terrible blunders I've undoubtedly made which completely invalidate these graphs.

  • Nice insights. Actual numbers to work with! Maybe someone with a bit more rights can try to deduce these statistics over a longer period of time, to see when (if) things started changing and to reduce the interference of this being a bad (or good) day. – Roy T. Aug 15 '14 at 9:34
  • @Anko Absolutely. Definitely worth running the numbers again in a few days to see whether they come out the same way, or whether it was a once-off thing. – Trevor Powell Aug 15 '14 at 11:57
  • 1
    Can probably figure it out from the data explorer - I'm not the best with this kind of stuff but I might give it a go. Maybe someone with better SQL skills and better understanding of SE data could do it faster though. – SpartanDonut Aug 15 '14 at 13:44
  • I probably lied. I don't know if we can get "front page" stats from Data Explorer but we can get a general idea of close rates over time. – SpartanDonut Aug 15 '14 at 14:38
  • 1
    This is great data! But the last graph is terrifying! – Roy T. Aug 16 '14 at 8:16
  • 1
    Well, maybe (that the last graph is terrifying). In theory we should be accumulating questions that cover many topic, so it's only natural to expect the number of new good questions to diminish. Any time askers cover ground in questions asked previously, they should be marked as duplicates and never reach that +4 number. – MrCranky Aug 17 '14 at 11:28
  • @MrCranky I should have been clearer about how I generated those graphs -- closed questions are completely ignored in both of the graphs above. (But there are other reasons to not panic over the shape of the graph) – Trevor Powell Aug 17 '14 at 12:26

What I notice about the problem is that Gamedev seems to have a lack of high-rep questioners. I have a shakey hypothesis that the more knowledge a person has, the fewer questions they ask. Also that the more familiar with SO rules a person is, the fewer questions. Obviously, this isn't a problem unique to gamedev, it's just more pronounced here.

Here's an anecdotal comparison (big rep difference, not a big membership duration difference):

https://gamedev.stackexchange.com/users/1430/josh-petrie?tab=questions https://stackoverflow.com/users/22656/jon-skeet?tab=questions

One thing to note in those lists is that they both have self-answered questions. That's better than OK, it's really good. (Also note the extreme Answer-to-Question ratio.)


We could ask more good questions. We have talked about doing that before. Good questions do not rely on the asker not knowing the answer.


So the hard part about this solution is that it takes a LOT of individual work. I haven't followed my own suggestion from that meta post above, because Laziness. (As far as I've noticed, Josh is the only one who has. I'll take credit even if he would have done it anyway.) It takes far more time to construct a high-quality question AND an answer than it does to fire off close votes, comment on passable questions, and upvote the rare good question. But it would certainly help fix the lack of good questions.

I'm sure everyone has information they want to share. That's why we frequent the site in the first place. I just think we ought not wait for someone to ask the perfect question. Perhaps we could treated this site more like a blog with a wacky, 2-part format. You have a schizophrenic conversation with yourself. One side asks a question, and the other answers it.

  • 1
    I think your theory is pretty sound. The only question I have ever "asked" on the site was to self-answer it, since it had been coming up a lot. I expect that's the only reason I'd ever ask a question here, as the overwhelming majority of the "questions" I have at this point in my career are not appropriate here (either better fit for another SE, or just discussion oriented). – Josh Aug 17 '14 at 4:29
  • 3
    We could also definitely use some "canonical questions" for some things (dealing with resolution independence in object positioning, orienting objects to face targets, et cetera); we have existing questions that we use as duplicate targets for these common topics, but mainly by virtue of age, not by virtue of the question being a good, generic, well-searchable question. – Josh Aug 17 '14 at 4:33
  • I'm pretty much in the same boat as Josh; the only question I've asked on this site was an experiment (testing whether the community would accept a "what library should I use" question if it was re-framed as "how do I decide what library to use?") I wonder how many of our higher-rep users are also in that "answering, but never asking" boat. – Trevor Powell Aug 17 '14 at 9:47
  • Here's a graph of the average number of questions asked per user, grouped by user reputation. Note that we don't have enough users > 7K reputation to draw valid conclusions about the top end of this graph, but I'm not seeing any obvious trends between questions-asked and rep, in the <7K bands where we do have copious data. – Trevor Powell Aug 17 '14 at 14:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .