10
$\begingroup$

The question R/mgcv: Why do te() and ti() tensor products produce different surfaces? has had a very interesting history. It was asked by a new user who may not be quite so aware of where we draw the (controversial!) on-topic/off-topic line on questions with substantial coding content. The question itself is about generalized additive models, but included substantial R code using the mgcv package.

What I find particularly interesting about this question is that it was answered in some detail (not a lengthy answer but one in which significant thought had clearly been invested) by a highly experienced user, who felt the question touched on underlying statistical questions and was on-topic.

Five other users, also very experienced, voted to close as off-topic, presumably unhappy with how much the question turned on R code.

The answerer argued that the question should be reopened, arguing the level of implementation details in mgcv was unsurprising since he was "unaware of any other off-the-shelf software for GAMs that allows this ANOVA-like decomposition of bivariate smooths".

The question ultimately gathered four reopen votes before its time in the review queue was ended by a "leave-closed" vote from a moderator. Of course this doesn't preclude the question being reopened (a "leave-closed" vote from a moderator on a reopen review doesn't have the same finality as a "close" vote on a close review, since any earlier "reopen" votes are not wiped out) but falling off the review queue does make accruing sufficient reopen votes less likely.

I'm not here to protest the decision (I was one of the reopen-voters, but don't know nearly enough about GAMs to argue the technical points in anything resembling an authoritative way) but the fact that many experienced, high-reputation users were so split on this suggests this is worthy of further discussion. It reminds me to some extent of questions on lme4 or nlme which we're generally fairly tolerant of because there tends to be an underlying model specification issue, or questions on coding in JAGS/Stan - the sheer fact you are coding in JAGS or Stan indicates there must be something statistical lurking in the background! But even these are not cartes blanches and such questions can and do get closed, if the "underlying issue" is ultimately deemed to be non-statistical in nature.

Does anybody have any thoughts on which side of our dividing line this question should fall, or whether we should take special care with our treatment of mgcv questions in general?

Edit: the question was reopened as a result of the attention brought to it by this Meta question, but the issue of how to review questions with substantial code content is still relevant and it would be good to see what we can learn from the episode. Clearly a difficulty here is that not all reviewers on the Review Queue felt comfortable with the rather "niche" statistical material involved — it's not a topic many of us know much about, and even fewer could answer, which can make it hard to judge the borderline between a "primarily code" and "primarily statistical" question. Since there aren't that many active reviewers, this is bound to be a recurring problem here.

$\endgroup$
10
  • 1
    $\begingroup$ I voted to re-open and it got re-opened. Does anybody know how one can access the history of this Q to see who voted to close it and who voted to open? Usually this information is accessible in the revision history but in this case the Q has never been edited and so there is no revision history link. $\endgroup$
    – amoeba
    Sep 17, 2016 at 13:45
  • $\begingroup$ Ah, here it is: stats.stackexchange.com/posts/234809/revisions. $\endgroup$
    – amoeba
    Sep 17, 2016 at 13:46
  • $\begingroup$ @amoeba FWIW this was the Reopen review, I don't know how one can access the reviews themselves though (I only can see that through my review history because I took part in it). I don't know if there were any "leave open" votes during the close review, for example. $\endgroup$
    – Silverfish
    Sep 17, 2016 at 13:48
  • 1
    $\begingroup$ Thanks. Your link contains one piece of information that my link does not: it shows who voted to Leave Closed. I notice that Peter has lately been getting quite a bit of attention on Meta for his tendency to close things :-) $\endgroup$
    – amoeba
    Sep 17, 2016 at 13:52
  • 2
    $\begingroup$ @amoeba When a question is voted-closed by five experienced users, and then enters the reopen queue without being edited, I suppose it's probably not that surprising it picks up at least one "leave-closed" in the reopen queue! I do think a (surprisingly controversial) question like this might benefit from a bit more discussion and a bit less voting, but the nature of our review queue mechanism (even the interface itself) tends to favour voting over discussing. $\endgroup$
    – Silverfish
    Sep 17, 2016 at 14:02
  • $\begingroup$ I tried to find relevant Meta discussions in the last one-two years, and found this one: meta.stats.stackexchange.com/questions/2775. Any other relevant discussions? $\endgroup$
    – amoeba
    Sep 17, 2016 at 22:53
  • $\begingroup$ @amoeba There's meta.stats.stackexchange.com/q/3116/22228 but the discussion about topicality there was mostly in the comments. $\endgroup$
    – Silverfish
    Sep 17, 2016 at 23:54
  • $\begingroup$ @Silverfish, I'm not sure who can see this page, but the close vote history can be seen here. There was 1 leave open vote. $\endgroup$
    – gung - Reinstate Monica
    Sep 18, 2016 at 0:32
  • 1
    $\begingroup$ Speaking for myself, I did not vote on closure (for or against) b/c even after reading the Q, I just couldn't tell which side of the line it fell on. (Among other things, I don't have sufficient expertise on the topic.) Later, based on reading the answer & the comment arguing for reopening, I voted to reopen. It isn't clear to me if it was especially ambiguous, or if the topic was sufficiently esoteric that it was hard for people to adjudicate. I think discussing it on meta.CV is a fine idea, but I'm not sure if it will lead to better outcomes in future cases for that reason. $\endgroup$
    – gung - Reinstate Monica
    Sep 18, 2016 at 1:26
  • $\begingroup$ @gung I believe I also skipped the original review for a similar reason, but found Gavin's arguments convincing for reopening on closer inspection. I'm also unsure what about this question led to the difficulty for the community in reaching a decision; both things you suggested are plausible! I'm hoping there are some learning points here, but "hard cases make bad law" in terms of setting a precedent. It might be interesting to know how people judged a case which lay outside their domain of expertise - all reviewers need to do this sometimes $\endgroup$
    – Silverfish
    Sep 18, 2016 at 1:38

1 Answer 1

Reset to default
7
$\begingroup$

I know nothing about GAMs, but to me this Q looks clearly on-topic; I am surprised to see that there is so much controversy about it. The question did not simply ask about the meaning of te() and ti(); OP wrote that they are aware of the difference but expect them to coincide in this particular case for this particular reason; however they observe slight discrepancy. To me, this practically screams "on-topic": the models must be different but presumably in a tricky way, and it must require quite a bit of statistical knowledge to tell them apart. This is not at all a programming question.

Moreover, this must have become even more clear after @Gavin posted his insightful answer which is, frankly, quite obviously statistical and not programming-ish.

We have lots of open, popular, well-upvoted and highly insightful threads starting with questions about why some two functions in some statistical environment(s) produce different results. Like e.g. why an anova call produces different p-values from an lm call in R, or why lme4 in R does something different from PROC MIXED in SAS, or why eig and svd yield different signs in Matlab, or whatnot. I strongly believe that all of that should be on topic here.

Gavin's answer was posted on the 13th. It was accepted on the 13th too, indicating that this indeed resolved OP's issue. The post then got closed on the 14th meaning that some of the people voting to close must have already been able to see Gavin's answer. But okay -- after the closure on the 14th, Gavin posted a comment explaining why this should be on-topic and started the reopen voting. And then, after four reopen votes the thread was kicked out of the queue by Peter's leave-closed vote. Update: @gung made some fair comments about how the review queue works and I am now "striking" part of my original answer that was about specifics of how this particular Q got closed. This is not the most important here anyway.

$\endgroup$
5
  • 3
    $\begingroup$ FWIW, it is (was) not as clear to me as it seems to have been for you. Also, to the extent that votes were cast after Gavin's answer was posted, note that you cannot see the answers from the review queue, which is where I suspect the votes were cast. $\endgroup$
    – gung - Reinstate Monica
    Sep 18, 2016 at 1:34
  • 2
    $\begingroup$ Let me also mount a small defense of Peter. When voting in the review queue, you cannot see who has previously voted or how many reopen / leave closed votes have been cast so far. (This is for the best, IMO.) The thread was ambiguous (IMHO), had been closed by 5 high-rep users, & was in the reopen queue w/o having been edited or improved in any way. That could leave the impression of the losers of the previous close queue process re-litigating the same issue. Since I voted to reopen, I clearly disagree w/ Peter's decision here, but then, I disagree w/ everyone from time to time. (cont) $\endgroup$
    – gung - Reinstate Monica
    Sep 18, 2016 at 1:41
  • 3
    $\begingroup$ (I am rather disagreeable.) As you mentioned in your comment above, Peter has been coming under fire of late. I think that reasonable people will disagree occasionally, mistakes will be made every now & then, even by people working in good faith, & I agree w/ the bulk of Peter's decisions. Peter does a certain amount of necessary work in keeping the review queues clear. The fact is, there are a lot of reviews that need to be done & there really aren't enough experienced, dedicated reviewers. If it weren't for Peter, our review queues would be steadily growing. I hope he continues. $\endgroup$
    – gung - Reinstate Monica
    Sep 18, 2016 at 1:50
  • 1
    $\begingroup$ @gung Thanks, +1 to all your comments. I did not quite realize that when reviewing, one does not see existing answers or already cast votes. In any case, I did not want to attack Peter and I am also happy with his moderation most of the time and hope that he keeps up the good job he is doing. $\endgroup$
    – amoeba
    Sep 18, 2016 at 10:25
  • 1
    $\begingroup$ @gung I think it's a good point that the review screen doesn't show the answer, which clearly casts the question in a more statistical light. (The poor arrangement of the review screen has been a bugbear of mine for some time...) I think your comments under this answer but also under the main Q are actually insightful enough to be worth converting into an answer if you have time. $\endgroup$
    – Silverfish
    Sep 19, 2016 at 19:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .