43
$\begingroup$

Is there a rich get richer phenomenon on CV and can we do something about it?

I have a feeling that we, as a community (including myself), have a bias in up-voting, so that high-reputation users are more prone to get up-voted then low-reputation users. It would be interesting to see that empirically using our data, but I don't have any good idea for testing it right now (maybe you have?). Nonetheless, it seems for me that the bias exists. For example, if there are two identical answers, then the one by high-reputation user tends to get more up-votes. The same with questions (recall e.g. short questions by high-reputation users, or long and complicated ones -- those are the kinds that are often ignored if posted by newcomers).

Can we do something to diversify our voting and encourage the new users?

$\endgroup$
9
  • 10
    $\begingroup$ Excellent topic. I think the bias does exist, I can feel it too. For me it's not so much the number of reputation points that a user has though, but my own respect for them. It's well correlated, but far from perfect: there are some high-rep users whom I do not particularly respect/trust and I am not more likely to view/upvote their posts (rather less likely). Similarly there are some not-so-high-rep users whom I respect/trust very much and I will be similarly biased towards them (more likely to view their posts, more likely to upvote an answer without full scrutiny etc.) $\endgroup$
    – amoeba
    Commented Feb 3, 2017 at 12:27
  • 6
    $\begingroup$ It's an interesting question, but I don't know how you could possibly quantify the effect. You would need to measure the inherent merit of an answer to work out if it was overvoted or undervoted. It's inevitable that highly active members get noticed (usually very positively, occasionally not quite so much!). If you like people's work, you are more likely to read more of it and upvote it. What's pathological about that? I have no conscious bias against the "poor"; everyone with high reputation once had a very low reputation and many people will surge up through the ranks in the near future. $\endgroup$
    – Nick Cox
    Commented Feb 3, 2017 at 12:50
  • 1
    $\begingroup$ @NickCox it's not pathological and I'm not suggesting anything that we should not up-vote high-reputation users or sth. I'm saying that maybe we should pay more attention on newcomers, since maybe there is a disproportion in the voting trends. $\endgroup$
    – Tim
    Commented Feb 3, 2017 at 13:01
  • 12
    $\begingroup$ I am all for being generous. But speaking for myself: (a) I answer some stuff and (b) I read much more, the latter for both personal profit and sheer pleasure. But I have enormous bias to reading answers by people I respect. It's just like reading novels; once you've identified what appeals, you read more of the same. But naturally you're also open to learning about new authors. I don't think I see a point to changing my habits by deliberately reading more answers by low-rep users just because they might deserve more reward. If they're any good, someone will notice. $\endgroup$
    – Nick Cox
    Commented Feb 3, 2017 at 13:11
  • 5
    $\begingroup$ I think it exists. I personally get more credit than I deserve. But it also shouldn't be surprising that vote counts aren't independent of contributor; there are individual differences in statistical knowledge & the ability to explain topics clearly. To test / estimate the effect, you need to control for views & answer quality distinct from vote count. $\endgroup$
    – gung - Reinstate Monica
    Commented Feb 3, 2017 at 13:43
  • 2
    $\begingroup$ Many will recall that Scott Fitzgerald supposedly remarked to Ernest Hemingway, ‘You know, the rich are different from you and me.’ He replied, ‘Yes. They've got more money.’ (I Googled this for the text; there are many small variants.) In the same way, it is an easy short-hand to wave arms about the rich as if they were a qualitatively identifiable subgroup, but quantitative thinking is what we supposedly do best. $\endgroup$
    – Nick Cox
    Commented Feb 4, 2017 at 15:52
  • 7
    $\begingroup$ There is substantial evidence against this bias (which doesn't mean it doesn't exist). If it existed and were as strong as you suggest, then (after controlling for as many variables as possible, such as accepted percentage of answers and length of time posts have existed) upvotes per post would tend to be higher among those with high reputations. As a matter of fact, the regression is U-shaped: there is a tendency for votes per post to decline for total post counts between 10 and 30, level off to 100, and only then increase. High-rep users get the votes because they tend to be good! $\endgroup$
    – whuber Mod
    Commented Feb 5, 2017 at 18:13
  • 2
    $\begingroup$ @whuber but how do you assess quality? Isn't it possible that sometimes users, who don't understand very well the answer, may just upvote it if they see it has already many upvotes? Mea culpa, mea maxima culpa. I've done a few times, and to some of your answers, glen_b and gung. =D This was some time ago. And once in a while, I have to fight that urge. =D $\endgroup$
    – An old man in the sea.
    Commented Feb 12, 2017 at 18:50
  • 1
    $\begingroup$ @Anoldmaninthesea Assessing quality is controversial. Because ultimately this is determined by people who are part of a community (even at the highest levels of research), the use of a voting mechanism can scarcely be impugned. However, because this site allows (and practically encourages) members to vote after little or no deliberation, and all votes have equal weight, whether from newcomers or established contributors, the accuracy and even the meaning of vote totals have been challenged. $\endgroup$
    – whuber Mod
    Commented Feb 12, 2017 at 18:57

6 Answers 6

Reset to default
32
$\begingroup$

Let's look at the data. First up is a plot of acceptance rates versus the number of answers, with a weighted GAM smooth superimposed (courtesy ggplot2). It shows $11,609$ users.

Figure 1

The strongest signal is a strong "learning effect" for those with $25$ or fewer answers. (There are many possible explanations: it's not necessarily due to learning to write high-quality answers, but could be a matter of self selection.)

(The data are from a query at https://data.stackexchange.com/stats/query/edit/623920#resultSets.)

Next is a Pareto curve showing Total reputation versus rank for the top 2000 users.

Figure 2

The points show a strikingly linear relationship for ranks 51 - 2000 (the blue line is a linear fit to those points), but is sublinear for ranks 1-50. The top five reputations are only one-third to one-tenth what would be predicted by the extrapolated line.

(The data are from https://data.stackexchange.com/stats/query/578093/elite-users-those-who-hold-x-of-all-reputation#resultSets.)

Finally, look at reputation per post. I have limited this to people with at least 10 posts: 1725 users in all.

Figure 3

A correct interpretation would require incorporating the amount of time each post has been viewable on the site: posts, especially upvoted ones, tend to continue accumulating votes (growing by 10-20% per year).

(These data were obtained from http://data.stackexchange.com/stats/query/106236/users-with-highest-reputation-per-post.)

I will let you ponder these plots (and hope you might make some for yourself), so I will refrain from much commentary. Let me just summarize.

Any adequate analysis would have to take time into account. However, these raw data can be explained well by supposing that people who are good at solving statistical problems and communicating those solutions well have tended to be active on this site and that their high reputations partially reflect those skills.

Because plenty of the most active users (say, those with more than 100 answers) have relatively low acceptance rates and reputation per post, there is plenty of evidence that a "rich get richer" phenomenon, if it exists at all, explains only a small part of the reputation and is not universal.

$\endgroup$
1
  • 4
    $\begingroup$ To compliment this between person analysis I previously had conducted analysis of within person analysis for SO users. There I showed that people with high reputation tended to have higher scores for their earlier answers. You could actual do the analysis trimming off older votes by using the data dump, archive.org/details/stackexchange. Takes alittle more work than the query, but is not as annoying as SO due to the size @DJohnson. $\endgroup$
    – Andy W
    Commented Feb 6, 2017 at 17:58
11
$\begingroup$

The question is asking about rich-get-richer at the level of users (high rep users getting undeserved attention as they get higher rep). I'll address that first, but I'll then discuss a slightly different issue.

  1. The effect may exist, but if it did, I'd presume that my upvotes per post would be higher now than it was say three or so years ago.

    I don't think it has gone up at all in that time. (It did go up for a while when I first started answering questions regularly, but the average quality of my answers was improving; once that more-or-less plateaued, I don't think anything changed much since).

    If anything, even without a rich-get-richer phenomenon, that average should have gone up simply because a larger proportion of my answers are old - and they still accumulate some votes over time, so unless my new answers are getting less attention than they used to, upvotes per answer should tend to increase a little anyway, and I think they've actually gone down a bit).

    I guess the decrease may partly be having spent a bit on bounties, but that's not all that much -- it might explain the small decrease, and maybe even just a tad more than that -- which even if that adjustment resulted in an underlying small increase, it would be so small as to be easily consistent with the increasing age of my average post.

    So for me at least I don't know that there's any indication of rich-get-richer, but if there is one in my case it must be really tiny.

    Maybe others have seen more of an effect.

  2. However, amoeba (in comments below) raises the problem at the level of questions and there I think there is perhaps more of an issue. Some of my answers get more attention than they seem to deserve while other (clearly better in several senses) answers languish; and numerous other people have recognized a problem at that level. So while we might be ending up about right on average (if the informal analysis under point 1. of my average points per answer as my reputation rose is not atypical), that doesn't mean the best posts are always well recognized.

    For that I can suggest a few things:

    • where you can, look at all the answers, not just the top few

    • try varying the sort order for answers using the choices at the top of the set of answers (e.g. to look at "active"):

    Image showing the answer-list sort order links

    • keep your eye open for low-voted answers that are gems and upvote them

    • give bounties to especially noteworthy answers

$\endgroup$
3
  • 4
    $\begingroup$ Actually I did experience a strong effect but only in one specific case: with my PCA-for-grandmother answer. When I wrote it, there were around 15 answers above and the top one had 200+ upvotes. My answer was all the way down and was barely getting any upvotes. However, as it started rising, the upvote rate kept increasing. The higher it rose, the faster it was being upvoted. At certain moment it got accepted by the OP which made it appear on top, and boy did the upvote rate increase. But the answer stayed (almost) the same! So for answers in that thread -- yes, definitely, rich do get richer. $\endgroup$
    – amoeba
    Commented Feb 3, 2017 at 21:57
  • 3
    $\begingroup$ @amoeba that kind of sounds like things working more or less as they should. Rich-get-richer would be an issue where your initially low-ranked post stayed low forever (in spite of being really good) because the top posts always got all the attention; that story sounds like the moral is "the-low-ranked-but-worthy-can-get-richer-too, even if it may take a bit longer than we'd like". Nowadays its getting plenty of attention, which might seem less ideal, but your own experience would suggest that a sufficiently better answer, were one to arise, might eventually scale the heights as well. .. ctd $\endgroup$
    – Glen_b
    Commented Feb 4, 2017 at 0:37
  • 2
    $\begingroup$ ctd... however on the level of individual posts I accept that it's sometimes an issue. On the level of people, on average, it's less clear that there's as much of an issue - while some of my posts get more attention relative to their intrinsic value and some better posts languish, across many posts it's probably more or less even. I'll modify my answer to be clear that my discussion is averaging over many answers so there may still be an issue to worry about on the level of individual posts. $\endgroup$
    – Glen_b
    Commented Feb 4, 2017 at 0:40
10
$\begingroup$

One example where rich definitely do get richer causally (i.e. after controlling for quality), is questions, in particular newly posted questions.

  1. The more upvotes a newly posted question gets, the more likely it is to be noticed by somebody scrolling down the front page, and hence to be opened, and hence to be upvoted.

  2. If a newly posted question gets sufficient number of upvotes, it gets into the week/month hot questions list and is more likely to be noticed/opened/upvoted after it disappears from the front page.

  3. If a newly posted question gets huge enough number of upvotes (and possibly a couple of answers too), it can get into the Hot Network Questions list which usually makes the number of views/upvotes explode.

$\endgroup$
1
  • 4
    $\begingroup$ Wow, a downvote? Care to explain? $\endgroup$
    – amoeba
    Commented Feb 9, 2017 at 18:46
2
$\begingroup$

Supposedly rich get richer in economy (and life). Those who claim this to be a fact, usually, state that the reason is because the income from labor (wages) grows slower than return on capital. So capitalists accumulate wealth faster than workers.

If you'd like to claim that it's happening on CV, then you'd have to map CV's domain to capital and labor. You say that upvotes are capital (wealth), and labor is answering questions. If that's the case then by simply owning (managing) capital you must be getting upvotes. I don't think that's the case here. You can't simply do nothing and get upvotes on a new question. You have to answer it at the very least.

Your hypothesis is that people with many upvotes get new upvotes faster. However, that's more like getting higher wage based on length of service and prior achievements in real life.

So, I'm not saying that rich are not getting richer here at CV. Yet, I'm saying that if that's the case it's a completely different phenomenon from the economy. The mechanism would be very different.

UPDATE To answer @amoeba comment. I thought about this too. This scenario is not what OP was talking about. He's not talking about getting upvotes on the questions that you answered long ago, I think. Of course if you have a lot of reputation it means that you answered a lot of questions, and those questions got a lot upvotes. People who come here to ask the same question, will be likely to upvote you. So, it's a different kind of phenomenon still

UPDATE 2: Let's look at a few top performers in Year 2017. I pulled this data CV, ranked down by number of reputation points this year:

156,068  4,446  whuber       0.028487582
 10,009  1,775  mathew gunn  0.177340394
 33,083  1,256  amoeba       0.037965118
  2,129    690  studentt     0.32409582
    907    644  morganball   0.710033076

The guy with least amount of total reputation has the highest return on the reputation, i.e., ratio of reputation points in 2017 vs. total is 0.71. Otherwise, there doesn't seem to be clear pattern here.

Update 3: Here's the chart with OP's reputation growth. If OP's claim were true, then he should have seen the exponential growth in reputation: while his reputation grows, his upvote collection should accelerate. However, this chart shows linear growth in reputation. My chart looks very similar too.

enter image description here

$\endgroup$
2
  • 4
    $\begingroup$ I do nothing since New Year (no questions, no answers), and I earned around 1.5k rep since then (some of that already spent on awarding bounties). $\endgroup$
    – amoeba
    Commented Feb 9, 2017 at 15:58
  • 1
    $\begingroup$ This is an interesting perspective. I can't tell if it's quite the same phenomenon as what the OP wants to understand or not, but it is thought-provoking. $\endgroup$
    – gung - Reinstate Monica
    Commented Feb 9, 2017 at 19:43
1
$\begingroup$

The OP raises an interesting question. However, I feel "Can we do something to diversify our voting and encourage the new users?" may be an example of an XY Problem.

Personally, I'm not sure the "rich getting richer" phenomenon (if it exists) on SE is an issue. I've come to Stack Exchange off and on for several years now with statistics questions and found many helpful experts here. To be honest, I don't think having lots of points is terribly meaningful - more helpful is clicking on a respondent's profile to learn their background and expertise. Surely there are qualified statisticians/data analysts who simply don't have the time to write responses in their field of expertise.

$\endgroup$
3
  • 1
    $\begingroup$ This seems more like a comment than an answer. I have never seen anyone argue that non-participation on SE is equivalent to zero reputation in any worthwhile sense. Sure, Sir David Cox and Brad Efron don't hang around here that we can see, and fine. My own wild guess is that lack of inclination is more crucial than lack of time. Everyone has to have some rules for what they don't do. For one interesting story on why someone dropped out, see kbroman.wordpress.com/2013/05/13/… $\endgroup$
    – Nick Cox
    Commented Apr 6, 2017 at 15:16
  • $\begingroup$ @NickCox It does take time to formulate a thorough answer to a question, even when the inclination is there. Some of the excellent responses I've read here must have taken several hours at a minimum to write. Thanks for the link. Karl Broman's answer to drop off of SE seems a bit extreme - I think the answer is just don't get hung up on rep pts. It can be addictive, almost like a drug, tracking your rep pts. I'm not sure that's a good thing. $\endgroup$
    – RobertF
    Commented Apr 6, 2017 at 15:40
  • $\begingroup$ Indeed: I would never deny that serious participation here takes a chunk of personal time. But if reputation is really not important, then not even talking or thinking about it is implied, just as I don't know or care about <I won't say what; someone will be irritated>. $\endgroup$
    – Nick Cox
    Commented Apr 6, 2017 at 15:46
0
$\begingroup$

My own experience is that I've posted a lot of answer recently and get very little rep for them. This is fine if the answers aren't top quality, but I have no indication either way (no up-votes, no down-votes, no comments).

My observation is that the site isn't really doing anything. For example, see all the questions with bounties raised due to lack of attention. So, when a small number of super-users do post answers, they are either the only answers on the site, or people just read the rep and up-vote.

$\endgroup$
2
  • 1
    $\begingroup$ This is likely to get downvoted as a rant. However - you seem to have high rep on Workplace. Can you compare your experience there with your experience here? This would be interesting. $\endgroup$
    – amoeba
    Commented Feb 13, 2017 at 10:58
  • 2
    $\begingroup$ The first paragraph is personal. I doubt that you want, or that anyone wants to give, a discussion of how far your reputation is what you deserve. The second paragraph is puzzling. We have had some excellent answers and I don't see how this one pushes the discussion forward, unless you flesh out the comments. $\endgroup$
    – Nick Cox
    Commented Feb 13, 2017 at 11:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .