19
$\begingroup$

(UPDATE 28-8-2014 : Following suggestions, this post now appears also in the CV blog, http://stats.blogoverflow.com/2014/08/answering-narrowly-or-more-broadly-some-evidence-from-green-checks-and-upvotes/)

Although not that controversial, there is always the issue: "Should we just answer the question, or opt for a more comprehensive treatment of the issue to which the question is associated?" There has been some discussion about this here and on other SE meta-sites also, and various opinions have been registered – although, as I said, it is not an issue that creates hot debates (and this is perhaps why most of the opinions were aired in comments or in the context of an answer to a meta-post with a different focus).

My intention here is to provide a tiny bit of statistical evidence that, "If we opt for a more comprehensive treatment of the issue to which the question is associated, we are also being more helpful to the OP and to the OP's specific question – according to the OP."

Naturally I expect that the true statisticians in this site will be able to tear my argument apart (always in good spirit and with good intentions of course) – but this is one of my secret motives for this post: to see applied statistics at work, from the point of view of judging the degree to which postulated conclusions are indeed supported by the statistical evidence, even tentatively.

What I will do is to examine the relation between upvotes per answer and accepted/non-accepted answers, or "green/ungreen" answers. At least in theory, the number of upvotes, at least conditional on the popularity of the issue answered, is a rough indication of the general quality of the answer, while the green mark indicates strictly the opinion of the OP, who, having a specific question in mind looks for an answer to his specific question, and may find a more general treatment to be an obstacle rather than helpful.

So one could argue that there may be a conflict here: the more focused an answer is, the more helpful will be to the OP (and so the more the chances that it will get the green check), but the less interesting will be for the broader community (and so it will have less chances of being highly upvoted).

So here is what I've done: I examined myself as an answerer here on CV. I have $379$ answers currently. My answers usually get few upvotes: only $14$% of them have got $5$ or more, and there are only $7$ answers that got $10$ upvotes or more. Partly, one could attribute this to the fact that it appears that I do tend to answer questions that do not interest the majority of users here – but most probably it is also an indication that my answers are just "OK".

Now, according to this query, I have the top accepted-answers-ratio ($48$%) among users that have $\pm 25$% answers than me $[285 , 475]$. This could be considered as evidence that I am good at getting the green mark. Following the previous conjecture, this could also contribute to the low-upvotes finding: I am answering narrowly, so I may be getting the green mark from the OP, but my answers do not excite the wider community.

Given these preliminary findings, let's look at the counts, as well as the joint-marginal and conditional empirical relative frequencies here (rounded). I have separated the "upvote" space in three bins. The "one or less" bin seems a natural choice. I have used "5 or more", just getting the idea from our network profiles, where shown are only answers with five or more upvotes. But it proved an interesting binning.

COUNTS

\begin{array}{| r | r | r | r|} \hline \text{Upvotes} / \;\;\text{Accepted} \rightarrow & \text {Green} & \text{Ungreen} &\Sigma \\ \hline \hline \leq 1 & 43 & 66 & 109 \\ \hline 2,3,4 & 106 & 110 & 216 \\ \hline \geq 5 & 33 & 21 & 54 \\ \hline \Sigma & 182 & 197 & 379 \\ \hline \end{array}

Joint & Marginals

\begin{array}{| r | r | r | r|} \hline \text{Upvotes} / \;\;\text{Accepted} \rightarrow & \text {Green} & \text{Ungreen} &\text{Marg} \\ \hline \hline \leq 1 & .114 & .174 & .29 \\ \hline 2,3,4 & .28 & .29 & .57 \\ \hline \geq 5 & .087 & .055 & .14 \\ \hline \text{Marg} & .48 & .52 & 1.00 \\ \hline \end{array}

Conditional: Upvotes given Color

\begin{array}{| r | r | r |} \hline \text{Upvotes} / \;\;\text{Accepted} \rightarrow & \text {Green} & \text{Ungreen} \\ \hline \hline \leq 1 & .24 & .33 \\ \hline 2,3,4 & .58 & .56 \\ \hline \geq 5 & .18 & .11 \\ \hline \Sigma & 1.00 & 1.00 \\ \hline \end{array}

Conditional: Color given upvotes

\begin{array}{| r | r | r | r|} \hline \text{Upvotes} / \;\;\text{Accepted} \rightarrow & \text {Green} & \text{Ungreen} &\Sigma \\ \hline \hline \leq 1 & .39 & .61 &1.00 \\ \hline 2,3,4 & .49 & .51 & 1.00 \\ \hline \geq 5 & .61 & .39 & 1.00 \\ \hline \end{array}

The above seem to indicate that color and upvotes are uninformative for each other regarding the middle of the distributions involved, but they are informative for the two tails. Especially the 2nd conditional table, where an apparently clear positive association emerges: knowing that an answer has got $\geq 5$ upvotes visibly increases the chances that it is also a green one, compared to the marginal "probability" of being green.

To me, this is evidence that answering in a more broad fashion is more beneficial both for the users of CV in general, but also for the OP. And I believe this is strengthened by the fact that I tend not to answer narrowly, but broadly, which means that we do not have to condition also on my "answering approach". So "when my broad answers are liked by the community, they are more helpful to the OP also" (this last statement is just an example of the attempt to turn science into rhetoric).

But this is just one user, and not "representative" of the community, by his own words. One can understand that performing such kind of analysis on data related to other users has the danger of being considered indiscreet, and rightfully so. I am leaving it at that, and hope for some statistical discussion on the issue. Thanks for reading thus far.

$\endgroup$
  • 6
    $\begingroup$ +1 Thanks for bringing some data. $\endgroup$ – Glen_b Aug 24 '14 at 3:15
  • 6
    $\begingroup$ This would have been a good post for the now dormant blog. $\endgroup$ – Andy W Aug 25 '14 at 12:09
  • $\begingroup$ @AndyW Migrate then? $\endgroup$ – Alecos Papadopoulos Aug 25 '14 at 12:11
  • 1
    $\begingroup$ No the blog is not a migration option. The general point of the blog was for generic discussion that did not really fit the Q/A mold, bit it fell in disuse. It still exists though, and if others wanted to contribute we could open it back up. If your interested, the blog will more likely get more general exposure than a post on meta, although is not as likely to generate as much back and forth with members of this site. $\endgroup$ – Andy W Aug 25 '14 at 12:49
  • $\begingroup$ @AndyW I am ok with copy-paste my post to the blog and close it here, what is the procedure for the former? $\endgroup$ – Alecos Papadopoulos Aug 25 '14 at 13:31
  • $\begingroup$ @AndyW, +1 (or 2), the blog is an excellent idea. $\endgroup$ – gung - Reinstate Monica Aug 25 '14 at 18:28
  • $\begingroup$ @gung Can you tell me please how can I post something on the blog? $\endgroup$ – Alecos Papadopoulos Aug 25 '14 at 18:38
  • $\begingroup$ I don't actually know, I've never done it. @AndyW is the expert there. $\endgroup$ – gung - Reinstate Monica Aug 25 '14 at 18:40
  • $\begingroup$ @gung Thanks, I will wait for him then, because: CONTD $\endgroup$ – Alecos Papadopoulos Aug 25 '14 at 18:48
  • $\begingroup$ CONTD @AndyW Woes of inactivity: In the blog's "About" the instruction is to go the the chat room "Skewed Distribution" in order to get an "author account" for the blog. But the specific chat room is frozen. $\endgroup$ – Alecos Papadopoulos Aug 25 '14 at 18:49
  • $\begingroup$ I'm working on it @AlecosPapadopoulos. I don't have the admin rights to add someone to the blog, but I know chl does. I've posted it in the chat room. Once you get access it is just a wordpress blog that you can copy-paste your post into. $\endgroup$ – Andy W Aug 25 '14 at 18:49
  • $\begingroup$ @Alecos I have unfrozen that room and checked that it has "public" status, so anyone may enter and participate. $\endgroup$ – whuber Aug 26 '14 at 14:18
  • $\begingroup$ Thanks @Andy. Sorry about the delay but I didn't get any notification from Ten Fold yesterday. I will grant writing permissions for Alecos on the blog ASAP. $\endgroup$ – chl Aug 26 '14 at 17:00
  • $\begingroup$ @chl Following your info on chat yesterday, should I also open a "bug" request here? $\endgroup$ – Alecos Papadopoulos Aug 27 '14 at 19:06
  • $\begingroup$ Following my reply on Skewed Distribution, it seems that you need to register yourself on the blog using your OpenID. A moderator will then change your rights to allow post editing. $\endgroup$ – chl Aug 28 '14 at 11:36

You must log in to answer this question.

Browse other questions tagged .