15
$\begingroup$

As I think about how to craft better questions, I wonder if the length of a question influences the probability of receiving an answer? Or increases the time to receive answer?

I know as a community we are simply not just interested in providing answers. However, I have often noticed long questions that provide a lot of information about the problem seem to receive less attention than short questions where the problem is less clear. Further, I often see those short question get a lot of comments, which help the questioner edit/develop a better question. As a result of this iterative process, it seems an answer is provided. Perhaps this is simply some sort of "confirmation bias" on my part, however.

| |
$\endgroup$
16
$\begingroup$

IMO this is because long questions take longer to read, which exhausts the attention before the main matter is stated.
A good lengthy SE question is also possible, but it should follow

question ~> details ~> more details

scheme rather than

details ~> previous results ~> more details ~> version of software ~> self-assessment of desperation ~> ... ~> question*.

(*) Nothing personal, general inspiration.

| |
$\endgroup$
  • 3
    $\begingroup$ +1, it is a good model for answers to try to follow as well. Be concise up front but elaborate as needed afterwords. $\endgroup$ – Andy W Sep 18 '12 at 12:58
  • 2
    $\begingroup$ In the American newspaper business, the first scheme is called the inverted pyramid. $\endgroup$ – gung - Reinstate Monica Sep 18 '12 at 13:09
12
$\begingroup$

I have been going on a bit of a data explorer binge recently, so here are a few stats to bear on the discussion. Here is query I used to pull the data for these tables and graphs. Of most note, I proxy "length of question" via the number of characters (in html) per question (i.e. the QLength variable in the query).

Now this is certainly an imperfect proxy, and I encourage others to do some text munging and improve upon it. But to proceed with imperfection ...

The distribution of question length comparing questions with an accepted answer vs no accepted answer is amazingly similar. Below is a table of some summary statistics, and following that a chart of boxplots for accepted answers and not accepted answers (with question length on a log scale). Labels in the box plots are post ids.

enter image description here

enter image description here

If there is a length of question too long to answer, we have not yet reached it! (That question is a good example of how character length of the html fails as well, it is only long because of a long code snippet, not because the actual question is long, a better procedure would not count such things in question length)

One might also consider not only if the post has an accepted answer, but the number of actual answers the post has received. Again (surprisingly to me) there appears to be little to no relationship between the question length and the number of answers. Below the graph is plotted on what SPSS calls "Safe Log" scale, where values of 0 are assigned to zero on the log scale. The points are also jittered because of extreme striation in the lower count values (over 90% of questions have answer counts 3 or fewer).

It is a bit strange to plot answers on a log scale, but it ends up producing a usefully weird separation in the point cloud. Points near the bottom are posts with no answers, and above have one or more answers.

enter image description here

I encourage everyone to download the data themselves and improve upon my analysis! An easy improvement may be just eliminating code snippets from the question length and seeing if the same results still hold. One might also include a host of other characteristics of the questions in analysis as well.

| |
$\endgroup$
  • 1
    $\begingroup$ +1, nicely done! This is the most useful answer here. $\endgroup$ – gung - Reinstate Monica Sep 30 '12 at 19:58
  • 4
    $\begingroup$ (+1) When you make almost any reasonable regression model of whether a question has any answer at all and include the question's score as a variable, then question length becomes highly significant and has a negative coefficient. There are signs of a score-length interaction. In brief, when you focus on questions having a similar score, there usually is a significant negative association between question length and whether it received an answer. This could be the starting point for a good EDA, which would want to explore how linear these associations are, etc. $\endgroup$ – whuber Sep 30 '12 at 22:05
  • $\begingroup$ I wonder if this changes over time. Is there a 2017-era change to this? $\endgroup$ – EngrStudent Mar 8 '17 at 20:11
5
$\begingroup$

There must be an optimal length of a question, assuming there is a smooth relation between the length of the question and the likelihood of the answer. Clearly, questions of zero length cannot receive valid answers, as there is not enough information to formulate a response. Likewise, infinitely long questions cannot receive an answer, either, because one's lifetime span would run out while reading the question, and one won't be able to apprehend all of the information contained in the question. Let us now apply Rolle's theorem: there must exist a point where the derivative of the likelihood of the answer is zero. The derivative is not identically zero on the positive semi-line, however, since we observe some questions that do get answered, so the probability is strictly positive for some nonzero lengths. By smoothness assumption, one of the critical points with the derivative = 0 must correspond to a global maximum of the probability of receiving a response wrt to a question of a certain length. QED.

(This is an old joke, initially formulated by a physicist Paul Dirac with regards to an optimal distance to admire a female face: from zero distance, you see something like a wall of flesh, and cannot really see anything; at infinity, that's just a small dot, and you can't see anything either, so there must exist an optimal distance. See http://www.dirac.ch/PaulDirac.html)

| |
$\endgroup$
  • $\begingroup$ See my response, too. There clearly must be an optimal length, but that optimal length may depend on other factors. Here's an example of a very short question I once saw on SAS-L: "I have some data. Can I do a t-test?" (Short, but hard to give non-flippant answers) $\endgroup$ – Peter Flom Sep 29 '12 at 20:02
  • $\begingroup$ I would just answer "Yes". Stupid question => stupid answer. Actually, a shorter answer is "No", it is one letter shorter. $\endgroup$ – StasK Sep 30 '12 at 3:09
  • $\begingroup$ Nice joke, but the math is flawed. Your argument would apply to, say, the function $\exp(x)(1+\sin(x)^2)$ but this has no global maximum. $\endgroup$ – whuber Sep 30 '12 at 21:36
2
$\begingroup$

I don't think question length alone is the key. If I was trying to model the likelihood of response, I would include measures of question length, question complexity and question novelty, together with their interactions (at least 2 way), along with question answerability.

I don't know of objective ways to measure complexity or novelty of questions.

But some complex questions demand a long question. Also, if the question involves code, it's nice to see the code. Very short questions cannot possibly include context, but context is often key.

And, while I am probably more toward the "keep" than "close" end of the continuum on question novelty, it is tiresome (and unproductive) to answer a question that has been asked already in the same way.

| |
$\endgroup$
1
$\begingroup$

As a member who tries to answer as many good questions as I can I find that I sometimes give up on very long questions. I would say keep the question to the point and concise. Further clarification can be brought up later in discussion if you find it necessary. Sometimes long questions with figures and/or R code may be necessary. If a long question is essential I would follow mbq's guidelines in his answer.

| |
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .