13
$\begingroup$

See the relevant thread at Meta.SE. As of Sep 8, answers on SO are sorted differently than before: previously, accepted answers were always shown first, but now, answers are strictly sorted by votes. SE are collecting feedback until Sep 19 as to whether other sites would like to also implement this.

How do we feel about this change? Do we want this on CV?

$\endgroup$
2
16
$\begingroup$

Where the most upvoted and accepted answers differ, I agree that the most upvoted is usually better. So I'm for the change but don't feel very strongly about it. It seems likely to affect a relatively small proportion of answers (anyone know what % we're talking about?) and the difference in quality in these cases is often fairly obvious.

EDIT: Based on @Scortchi's SEDE query above, this change presently affects 2899 questions, or about 1.5% of all non-deleted questions the site has ever received.

$\endgroup$
2
  • 9
    $\begingroup$ (+1) When there is a really big difference, the explanations seem to range from timing (the OP accepted an early answer and didn't change even though a much better answer came in later) to perversity (the OP accepted what they wanted to hear, despite being told that their question was misguided or that there is a much better idea). Again, quantifying those various reasons would be interesting but difficult. Optimists would emphasise timing, pessimists perversity. $\endgroup$
    – Nick Cox
    Sep 15 at 9:15
  • 1
    $\begingroup$ @NickCox Indeed. Your comment reminded me of this entertaining example, which happens to hold the CV record for biggest score difference between most upvoted and accepted: stats.stackexchange.com/questions/20836/… $\endgroup$
    – mkt
    Sep 21 at 12:44
6
$\begingroup$

I agree that this is a worthwhile change, but I think a slight augmentation would be ideal. Currently, you can sort answers several ways (by votes, by default, but also by recent activity or age). It would be helpful to have a sort option that returns the old style ordering (accepted on top). There can be cases with lots of answers and it might be helpful to see the accepted one quickly without having to scroll though everything.

$\endgroup$
6
$\begingroup$

This isn't a proposal for action but rather an extended comment that flags up some things we may want to keep an eye on if this change goes ahead. I suggest distinguishing static considerations, whereby orderings of historic posts already in our repository may change, from dynamic effects this change might produce on voting on new questions. Much of the discussion, and indeed cited data, concerns the former but the latter effect is less clearly evidenced. It's well known that answers listed at the top of a page tend to receive more votes, because not everyone scrolls down to read the lower ones. This is one of the problems with treating the voting system as if it were a rating system (as @SextusEmpiricus points out).

Sometimes an excellent answer is posted weeks or months after a question was first answered, and such answers inevitably struggle to catch up with ones posted when the question had its higher initial traffic. Occasionally these answers represent changes in the state of the art, e.g. when a new paper has been published (or particularly on StackOverflow, when there are a changes to a programming language or library, as anyone who regularly refers to the ggplot2 resources there will tell you while tearing any remaining hair out). Such answers often really deserve to be the top-listed, and a voting system will only rarely achieve this. There are ways to try to influence the votes, like high-rep users promoting the answer by posting a bounty for it and deliberately leaving it in the bounty queue to encourage more eyeballs/voters, but when a question was initially popular and the original answer garnered dozens or even hundreds of votes, it's very hard for a new answer to overcome that kind of lead even if it technically supersedes prior ones.

These dynamic considerations suggest that voting is not only dubious as a rating system, but also as a curation system. There is an advantage to having curated content. Future readers, whom this site is intended to benefit, are helped when deprecated answers no longer appear at the top of the page, regardless of vote count. Question pages "owned" by active users can be curated this way under the "green-ticked shows first" system, if superior answers come along later. This is not flawless since many of our most-viewed questions were asked by drive-by users or long-time users who have become inactive, and as @NickCox points out, a proportion of green-ticked selections seem perverse. (I think, perhaps more charitably than Nick, that some, but not all, of the unusual-looking decisions by OPs come when a question addresses their individual practical needs or clears up a point of confusion that was very specific to them, whereas a more general or comprehensive answer may be more valuable to everybody else.) I remain unconvinced that changing to a "best-voted shows first" system will solve the curation problem on old or abandoned questions due to the way voter traffic tails off so rapidly, and only gets a temporary bump when a new answer is posted. If we were concerned about curation of such questions — and I think arguably we ought to be, since many of the more important or persistent questions in statistics, the ones which readers will keep flocking to for years to come, have already been asked on this site — I think we might need more radical changes.

Perhaps SE devs could establish a "black tick" for "moderated choice" of the answer most likely to be useful for future readers, and any black-ticked answers appear first, regardless of votes or which answer was green-ticked. This would necessitate a dedicated review queue accessed by high-rep users for questions which have been flagged as having a deprecated, incorrect, or clearly technically inferior answer currently appearing in prime position. I can see obvious risks of misuse or contention in the case of subjective questions, but on a largely technical site where some answers are either objectively incorrect or outdated, and with a voting system that doesn't always correctly rate answers, some kind of solution along these lines seems inevitable eventually. This is just a piece of brainstorming rather than a solid suggestion but I can't see how our direction of travel can avoid such discussions in future, as our task becomes increasingly to curate the body of material the site has already produced. Voting in its current form doesn't seem to be a sufficiently powerful curation tool; again, it wouldn't surprise me if eventually higher voting power for high-rep users is introduced as an attempt to address the curation issue, or editing of existing, highly placed answers by inactive users becomes normalised. At some point, something will clearly have to be done to keep top-positioned answers to highly viewed question correct and updated.

If we do opt for "best-voted shows first" we need to pay attention to the dynamics of voting on new questions, and any weakening of the link between answer quality, green-ticked answer and best-voted answer. Timing affects voting not just when updated answers appear months later, but even within the first few hours or days of a question being asked, when voting activity is at its peak. Until the OP green-ticks an answer, we do operate under a "best-voted shows first" regime, and it's very clear we have a Matthew Effect ("the rich get richer") as a consequence. Answers even a few minutes earlier than another have a few more votes, get a higher position on the page, and as a result tend to attract even more votes, since not all readers/voters scroll down far enough to see the newer answers. I think most long-time users of this site will have witnessed cases where the OP influences voting by selecting what appears to them to be the best answer — at the point they make this choice, they have access to answers early voters could not see, and they are more likely than later voters to have read the full set of answers, so despite occasional perverse choices there are reasons to think their choice is better informed than the recorded vote distribution is. It often happens on questions with multiple answers that the OP green-ticks an answer which has climbed its way up to midway through the answer list on vote terms (and sometimes stalled there for a while), but after some time in prime position on the page, the green-ticked answer accumulates votes more rapidly to catch up or surpass the previously best-voted answer. If it had been left halfway down the page, I am sure it would have attracted a bit more attention and a few more votes by dint of having the green tick, but I very much doubt the effect would be so dramatic as when it gets a positional promotion too. This is why I'm not convinced data about agreement between green-ticked and best-voted answers on our existing question are a good guide to the effect of such a change in future, and some caution and a beady eye would be needed.

$\endgroup$
6
  • 4
    $\begingroup$ "the green-ticked answer accumulates votes more rapidly to catch up or surpass the previously best-voted answer" This is a very interesting mechanism that should be easy to test. In this question on meta I assembled a bunch of graphs displaying how the voting system works. I will see if I can get some time to make insightful graphs about the 'battle between answers' and the effect of being on the top of the page. $\endgroup$ Sep 20 at 15:39
  • 1
    $\begingroup$ My comment cited optimistic and pessimistic guesses at why voting doesn't match a perception of quality without saying how often I lean towards either. There is no evidence there for how charitable or uncharitable I am compared with anybody else. I'd say anyone who answers just one question on CV is thereby more charitable than the large tribe of those who lurk but don't even try, despite having appropriate expertise and experience. (Not offended or irritated at all ; I take your comments as typically good-natured.) $\endgroup$
    – Nick Cox
    Sep 21 at 9:29
  • 1
    $\begingroup$ @NickCox Duly edited to better represent what I intended to convey! (I do think there's some truth in your inner pessimist's observation that some people are just waiting for an answer that agrees with their preconceptions, and mkt's example is one of the more famous cases. But often the perverse choices strike me as occurring when some minor, obscure or idiosyncratic point was the trigger that allowed the OP's penny to drop. I suppose I might restrict myself to saying that most of the time I feel more charitable about perverse green-tickers' intentions than Nick Cox's inner pessimist!) $\endgroup$
    – Silverfish
    Sep 21 at 19:28
  • 2
    $\begingroup$ "Answers even a few minutes earlier than another have a few more votes, get a higher position on the page, and as a result tend to attract even more votes, since not all readers/voters scroll down far enough to see the newer answers." Slightly off-topic, but I think this is a good argument for randomising answer order for perhaps 1 day or 1 week after the question is asked. $\endgroup$
    – mkt
    Sep 22 at 8:19
  • 1
    $\begingroup$ @mkt Indeed, is an answer that has been on the site for 10 hours and accrued 15 upvotes "really" better than one that has been on the site for 4 hours and accrued 12 upvotes? Rather than a purely random order, you might take the current vote totals and fuzz them a bit by adding some random noise, perhaps granting "younger" answers more standard deviation to their noise to reflect greater uncertainty about their ultimate vote count. Ultimately, we want answer order to reflect usefulness for future readers. But ordering interacts with vote count which interacts with ordering... $\endgroup$
    – Silverfish
    Sep 22 at 12:32
  • 1
    $\begingroup$ @mkt Answers are originally written for the benefit of the OP, who likely reads all answers & ordering matters little to them. The initial viewership wave largely is site users who saw the question on a feed (or sister site users via Hot Network Questions) who also form the main votership. Perhaps presenting a "correct" ordering to "browsing & just curious" viewers/voters is less important than getting ordering "right" for future readers who actually have that problem and arrive via search engine. If randomising during the main voting wave improves ordering, it may be a worthwhile tradeoff $\endgroup$
    – Silverfish
    Sep 22 at 12:40
4
$\begingroup$

The 'accepted answer' is a way for the questioner to

  • grant a specific answer a bonus
  • notify that their problem/question has a sufficient answer (it provides a way to close the question; at least from the point of view of the questioner)

There seems no good reason to me why the questioner should be granted powers to influence the order of answers except for one potential problem...

  • Some questions have a long history and because of this there develops a large discrepancy between votes. For instance, at Cross Validated, a 'hot network question' can gather a large amount of votes in just a day. For that situation, a late answer will never catch up with the votes but might be a better answer (it breaks the voting system, which in my opinion doesn't work correctly because it is not a rating system).

    In such a case it is nice when the questioner can bump up an answer beyond discrepancies in votes. The accepted answer can be a better measure than the votes, or at least it is good to have it in addition to the votes.

So in principle I do not see why the 'accepted answer' should be on top. But in practice there is a good reason to have it on top. I guess that on CV it would be good to stick with the old way and choose for the practical reason to have it on top. There is not much harm in having the accepted answer on top. (One case would be when the accepted answer would be bad, but that problem should be dealt with differently.)

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .