6
$\begingroup$

This locked question "exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed."

Now, the question has +55,000 views, so it is understandable why it's not deleted. However, the answers are misleading, and thus dangerous for the audience.

Both answers stress that the R function lm does accept weights. However, they fail to mention that what lm understands by weights is a very precise, narrow definition of weights. As the help page states:

... the values in weights being inversely proportional to the variances.

This is quite different from the two more common type of weights, namely frequency weights and sampling weights. This blog entry makes this difference too.

Thus, as it stands, the locked post is potentially misleading. Sure, users should always read functions' help file, but still, I think this should be clarified in the locked question. I cannot add an edit or comment, but maybe someone else can.

UPDATE:

What would I change to the post? I would add a comment to the question, or to the first answer, with something like this:

"Notice that, whereas lm (and related commands like plm) do accept weights, the definition of this weights in the documentation is very precise: weights must be "inversely proportional to the variances". These weights are not necessarily equivalent to that of frequency weights or sampling weights. For the latter, R has a dedicated package called survey.

| |
$\endgroup$
  • 6
    $\begingroup$ Can you suggest an edit to the accepted answer that would sufficiently clarify this point? By "suggest" I mean suggest right here in your Q? If your suggestion gets support, one of the mods could unlock the thread, let you edit the answer, and lock it again. $\endgroup$ – amoeba Jul 17 '18 at 16:59
  • 1
    $\begingroup$ What is a common usage is often user dependent. I would have said frequency weights were very rare and survey weights quite a specialised subject but your experience is clearly different. $\endgroup$ – mdewey Jul 17 '18 at 21:08
  • 1
    $\begingroup$ Weights inversely proportional to variances is not "quite different" from frequency weights. Under iid sampling, frequency weights are "inversely proportional to variance"; you might regard frequency weights as a special case of that more general precision weighting. $\endgroup$ – Glen_b Jul 18 '18 at 0:51
  • $\begingroup$ @Glen_b Interesting. Do you have a reference where I can read more about it? Under iid sampling, then, pretty much every type of weight is, in practice, the same. Frequency and sampling weights, in practice, might be so too. $\endgroup$ – luchonacho Jul 18 '18 at 10:24
  • $\begingroup$ @amoeba Done. Glen_b or others would like to see if the comment is accurate enough. $\endgroup$ – luchonacho Jul 18 '18 at 10:36
  • $\begingroup$ The quote from the documentation stops at the semi-colon. The rest of that sentence says "or equivalently, when the elements of weights are positive integers w_i, that each response y_i is the mean of w_i unit-weight observations (including the case that there are w_i observations equal to y_i and the data have been summarized). However, in the latter case, notice that within-group variation is not used. Therefore, the sigma estimate and residual degrees of freedom may be suboptimal; in the case of replication weights, even wrong." $\endgroup$ – mdewey Jul 18 '18 at 10:57
  • $\begingroup$ @mdewey I'm not sure I understand the objective of the comment. $\endgroup$ – luchonacho Jul 18 '18 at 12:19
  • $\begingroup$ I was suggesting that the documentation and functionality were not quite so limited as you outline. $\endgroup$ – mdewey Jul 19 '18 at 9:55
  • $\begingroup$ I asked a very similar question which might be relevant to this thread. $\endgroup$ – Clarinetist Jul 30 '18 at 13:32

You must log in to answer this question.

Browse other questions tagged .