Do answers with equations and formulas in them get more votes?

Question

I have this hypothesis. Ceteris paribus the answers that have formulas (equations) in them get more votes than plain English answers. Do you agree?

Answer 1

In short: I think that the bottom-line is that in terms of correlations, there is not a clear effect. It differs per user, and if we would scale the number of equations by the size of the post then actually the post scores become lower for more equations.

To see if there are causal effects one might still do some alternative experiments, but the experimental correlations below suggest that the patterns are small, with lot's of noise and confounding variables.

---

It is easy to gather the data:

-- Enter Query Title
-- Enter Query Description

DECLARE @UserId int = ##UserId##

SELECT
   Id as [Post Link],
   Score,
   Len(Body) As CharacterNum,
   round((Len(Body)-len(replace(Body, '$$', '')))/4,1) As center_FormulaNum,
   round((len(replace(Body, '$$', ''))-len(replace(Body, '$', '')))/2,1) As total_FormulaNum
FROM
   posts
WHERE 
   OwnerUserId = @UserId
   and posttypeid = 2 
   group by Id, Score,  Len(Body),
   round((Len(Body)-len(replace(Body, '$$', '')))/4,1),
   round((len(replace(Body, '$$', ''))-len(replace(Body, '$', '')))/2,1)
ORDER BY Score desc;

But it is difficult to make something out of it.

If the number of equations in a post is high (counted by $$), then there is somewhat an increase of the scores of the posts. See the density function of the scores below. With four different selections, 0 equations, 1 equations, 2 equations, >2 equations. (Figure 1)

But more equations also correlate with more text, and if we scale the number of equations per post by the number of characters per post, then there is much less difference or sometimes we see lower scores for posts with (relatively) more equations. (Figure 2)

These correlations are a lot different from user to user. Maybe this may be a lead to think about causal relationships (although I suspect it is widely varying).

Fig1 : score density functions categorized by number of equations per post

Fig2 : score density functions categorized by number of equations per post divided by characters per post

------------------------------------------------------------------------------------------------------------

Because the above picture are small and noisy, I have created another, different view.

This time no split into 4 categories (making the numbers per category small, and instead a split into either functions with and without equations made by double $$). Also, the densities are not each by themselves scaled to 1 but instead are together scaled to 1. In this way also the relative difference can be seen how often a post has equations and how often not.

Behold, the Whuber-Amoeba-Polarization-Effect:

It is interesting to see that the post with high scores are fifty-fifty with and without equations. But at lower scores Whuber makes relatively more posts with equations, and Amoeba makes relatively more posts without equations. Of course it is just guessing what could be the cause of this effect, but the correlation is certainly interesting.

------------------------------------------------------------------------------------------------------------

Another plot inspired by EngrStudent's comment:

I think the difference between with/without after a score of 20 for Amoeba is interesting.

The above histogram/distribution for Whuber and Amoeba is now plotted for the nine members, and as a cumulative score on a log scale on the y-axis and an inverted square scale on the x-axis.

The differences above 20 were not so well visible on the previous plot and it looked much the same. This new plot suggest some type of exponential distribution or some hyper-exponential distribution.

The differences between 'with' and 'without' equations seem to be in

1. the rate parameter(s) of the components (exponential terms) in these distributions
2. the relative ratio that a member writes answers with and without equations

While using these correlations still leaves us guessing for interpretations in terms of causal effects, I believe that it is very likely that the majority of the differences between answers with and without equations is due to the correlation with confounding variables and that it is more like the topic of the answer/question is influencing the scores, instead of the 'fact that there is an equation in the post'. I believe that the widely varying distributions and answering styles of different users supports this, and we may rightfully wonder about the common adagio that 'adding an equation to a book/presentation is going to reduce the attention'.

small note: given the variations in the slopes of these cumulative distributions, I guess that the age of the posts may be important, and improvements of these plots can be made when we also take the age of the post into account (maybe I get back to that some time).