EDIT BELOW (August 25th, 2017)
I concur with what @amoeba proposed, except point 4.
- statistical-learning should be merged into machine-learning, see also the discussion in the comments above where everybody agrees with that.
@amoeba changed point 4, so now we are in agreement
- statistical-learning should become a synonym of machine-learning, see also the discussion in the comments above where everybody agrees with that.
@gung had already said in the comments
[statistical-learning] a synonym of
[machine-learning] seems like a good idea. I think in principle SL could be a valid tag, but it's very unlikely to work out--it'll just create more fragmentation instead. – gung Jan 5 at 13:22
For now I'm against the merging and in favor of proposing as synonym. But I can't still pinpoint if SL and ML warrant two tags. As @gung commented in this answer perhaps this warrants a separate question as well.
Below, I collected some evidence SL might not be simply ML.
Alright, found a somewhat (blurry) contrast between statistical-learning and machine-learning. While I was searching for it in Elements, it was actually in An Introduction.
Right at Chapter 1 - Introduction (emphasis mine):
Statistical learning refers to a set of tools for modeling and
understanding complex datasets. It is a recently developed area in
statistics and blends with parallel developments in computer science
and, in particular, machine learning. The field encompasses many
methods such as the lasso and sparse regression, classification and
regression trees, and boosting and support vector machines.
In "A Brief History of Statistical Learning" (sorry, long quote)
By the end of the 1970s, many more techniques for learning from data
were available. However, they were almost exclusively linear methods,
be-cause fitting non-linear relationships was computationally
infeasible at the time. By the 1980s, computing technology had finally
improved sufficiently that non-linear methods were no longer
computationally prohibitive. In mid 1980s Breiman, Friedman, Olshen
and Stone introduced classification and regression trees, and were
among the first to demonstrate the power of a detailed practical
implementation of a method, including cross-validation for model
selection. Hastie and Tibshirani coined the term generalized additive
models in 1986 for a class of non-linear extensions to generalized
linear models, and also provided a practical software implementation.
Since that time, inspired by the advent of machine learning and other
disciplines, statistical learning has emerged as a new subfield in
statistics, focused on supervised and unsupervised modeling and
prediction. In recent years, progress in statistical learning has been
marked by the increasing availability of powerful and relatively
user-friendly software, such as the popular and freely available R
system. This has the potential to continue the transformation of the
field from a set of techniques used and developed by statisticians and
computer scientists to an essential toolkit for a much broader
In Chapter 2 - Statistical Learning there's also some definition of the term.
Following the next chapters, you also have staples in Machine Learning: Linear Regression, Classification (LR, LDA, QDA, KNN), Resampling, Linear model selection and Regularization (subset selection, shrinkage, PCR, PLS), Non-linear regression (regression splines, GAM), Trees (CART, Random Forest), Support Vector Machines (SVC, SVR), Unsupervised Learning.
Sadly, nowhere SL and ML are directly compared one against the other.
I'd like to foment some discussion on the term, because it's not sure in what does it deviate from machine learning, if in anything at all.
*Now I'm under the impression it's a synonym (i.e. the ML framework under the statistics culture and jargon), but why not use the more vendible term then? Though in the scientific literature SL is a really popular term.
Perphaps the difference is simply cultural, like many discussions in the main site pointed. Consider Stanford, where two courses are taught: Stats 315a/315b - Statistical Learning and CS 229 - Machine Learning. Apart from being named different and being in different concentrations areas, they also attract different students.
Tibshirani even shares his views in his page comparing both courses and then both terms:
Machine learning research focusses more on low noise situations, eg
engineering applications like robotics and physical sciences
Statistical learning focusses more on high noise, observational data
like medicine and genomics, and problems where interpretation of the
fitted model is important
But more and more overlap in application areas!
I've come to the conclusion Statistical Learning is the application of Learning algorithms to classical statistical problems (I think the small phrase at the Machine Learning article in Wiki and the ISLR book description corroborate this notion). The distinction to Machine Learning is better shown with examples:
Machine Learning is concerned with optimizing generalized predictive power. So the focus is mostly on loss functions. Eg.: Studies trying to predict if a person has Alzheimer from neuroimaging, thus producing biomarkers of Alzheimer, but not focused on the biological meaning on the features, just on performance.
Statistical Learning on the other hand wants to make inference over this scenario. Eg.: "How do learning algorithms trained to predict the biological aging from neuroimages of healthy people perform on the presence of Alzheimer? Why?"
Another possible scenario for Statistical Learning is predicting states, such as task paradigms, from neuroimaging using linear models with shrinkage, such as SVMs, producing interpretable weight maps. Yet another scenario is in the introduction of a new imaging technique, where Statistical Learning can help the scientific community to uncover if said technique improves the diagnosis of a disorder.
*I'm mostly talking about neuroimaging because that's my area of expertise.
Said all that, I'm of the opinion the tag would be mostly useless here on CV, and wouldn't be used for it's true meaning.
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning (Vol. 6). New York: Springer.