# ab-test is not a technical term

Should the tag test be removed from the list or made into a synonym? It is jargon for the BI community and we should encourage use of proper technical terms to preserve the consistency of the field of statistics. Should we redirect to "hypothesis-testing" or some other existing tag? Is there a term in the community of statisticians for a multi-grouped randomized experiment where the hypothesis is that one is the best in a well defined metric?

• I'm curious why you say A/B test is not a technical term. Arguably, jargon is synonymous with "technical term", and you give it this much credit within the BI (bioinformatics?) community. Furthermore, the answers seem to suggest some disagreement on its technical meaningfulness. Would you care to explain your initial opinion, even if it's changed? I note that you've seen it misused, but doesn't that imply a technically correct usage? – Nick Stauner Jul 19 '14 at 21:37
• I meant that I don't think it is a technical term in the academic fields of statistics and probability but jargon from another field in which it is clearly a technical term. However, nobody seems to know what the term is in academia so I guess there is not one and we inherit "A/B testing" by default. Comparing two samples is a super common thing to do so I had assumed a term had been coined hundreds of years ago and wanted to try to proliferate appropriate semantics. The down votes seem to imply my assumption and/or goal was ill advised. – Keith Jul 19 '14 at 21:52
• FYI, BI is Business Intelligence – Keith Jul 19 '14 at 21:54
• Downvotes on meta-posts can mean disagreement with the proposal in addition to the usual meaning; I'd guess it's only disagreement in this case. – Nick Stauner Jul 19 '14 at 21:55
• OK good to know. I am tempted to open a new question asking specifically for the term used by statisticians but fear the downvotes. :) – Keith Jul 19 '14 at 22:03
• I don't think you'd get any downvotes on the main site. Seems like a perfectly good question to me, and our community is generally pretty friendly about voting. However, I think @gung's link to the Wikipedia page gives a pretty good answer (binary hypothesis testing). Still, might be interesting to see if the community agrees with that though. In fact, asking a sort of leading question like, "Is binary hypothesis testing a better statistical term for what business intelligence often refers to as A/B testing?" seems like an even better question to me – Nick Stauner Jul 19 '14 at 22:07
• The policy for voting differs between the main site & meta. Downvotes here do not mean that the question appears "egregiously sloppy, [with] no-effort-expended" etc. Instead, meta posts that appear to make a concrete proposal (even if marked w/ [discussion]) are upvoted if people agree w/ the implied proposal & downvoted if people disagree. Note that you do not gain or lose reputation on meta.CV from votes. I am one of the downvoters here, as I disagree w/ the suggestion, however, I see no reason your proposed CV (main) Q should be downvoted. – gung - Reinstate Monica Jul 19 '14 at 22:29
• Hi @Keith, consider accepting one of the answers if you think the issue raised by your question has been resolved/settled. – amoeba Dec 10 '15 at 21:26

Organizing tags is a delicate balancing act between being too comprehensive and too specific. It is no surprise, then, that discussions about tags occur frequently. I am grateful that our community is constantly thinking about them and proposing improvements.

Tags are primarily intended to help people find information. Secondarily, they organize threads into related groups that are used to recognize people who are active in various specialties.

The proposal to synonymize with runs counter to both these objectives. It loses the semantic differentiation implied by "A/B testing." It would seem prudent not to make one a synonym of the other.

Supporting this conclusion are 49 threads currently endowed with the tag. This number, which has developed organically by original posters and community members applying the tag as they see fit, is evidence that is perceived to be useful.

• Good answer. I understand that there is need for such a topic. The question was more semantically motivated. If there is no technical term referring to this method of experimental analysis then clearly it should be kept. I have updated the question accordingly if you care to comment further. – Keith Jul 18 '14 at 14:07
• I agree strongly with the general and specific arguments here. Moreover, the repertoire of allowed tags should be not be based on arguments that one term is better or more widely used than another; it's sufficient that a term is widely used by a substantial fraction of statistical people. Otherwise, a large number of tags could be objected to on some such grounds. – Nick Cox Jul 18 '14 at 16:44

I don't really agree here. A/B testing is a specific kind of experimental design that is important and common in a specific setting (most typically website development, I gather). It could be a duplicate for , but I don't think it's a duplicate of [hypothesis-testing]. Moreover, I think it is specific enough, and used often enough, to merit its own tag; we do have tags for other specific designs (e.g., ).

People working in particular areas tend to embrace slightly different terminology from people working in other areas, and both may use the terminology differently to mainstream statisticians.

So for example you might see 'force of decrement' used by an actuary (moreso in the past than now) where a statistician might say 'hazard', for example, and such differences persist in every corner of statistics.

These differences come in from psychology, biology, physics, astronomy, econometrics, marketing, and so on (and on).

I don't think it's really up to us to dictate what's 'proper' terminology -- if enough people are using it to talk to each other, even if it's in some small niche application area, it should as far as possible be embraced.

The main caveat would be in the case where there's a very strong chance that a terminology will be very directly misleading, in which case at the very least we need to be very careful to clarify what's going on and in some cases (as few as possible) perhaps call for going with the more conventional terminology.

• Awesome thanks. I am still curious if there is a more formal term than A/B test use by statisticians. It is especially awkward when people use it to refer to experiments with three variants. – Keith Jul 18 '14 at 23:50