What is our stance on questions “Where can I find an implementation of statistical algorithm X from scratch so that I can learn it”? Are they on-topic or off-topic?


Can anyone give me a simple SVM implementation in python?

I am learning SVM classification. But I'm unable to understand how to implement SVM classification. It would be really helpful if anyone could give an very easy example of an SVM implementation in python.

Implementation of CNN:

I am new to the field of vision. To get a good understanding of the concepts, I wanted to look at source code of some CNN.

Can anybody suggest some place where I can find implementation of simple CNN like LeNet 5(preferred as I'm following it's tutorial) from starch in Matlab or python not using any library like tensorflow

  • 1
    $\begingroup$ These questions seem in some ways like those asking for book recommendations. So far, so good. But I would rather people ask about algorithms than about code in some specific language, just as asking for code in some specific language is generally off-topic here. A difficulty with these questions is that often expectations are unclear or impossible to satisfy, just as "explain it to a seven year old" questions can't be usually answered without assuming high school mathematics. The answer is often likely to be, trivially, just look at the implementations in software X. $\endgroup$ – Nick Cox Feb 17 '17 at 19:04
  • 1
    $\begingroup$ What is scratch any way? What are the black boxes taken as standard? Someone might well regard FFT or SVD as basic while someone else might want to avoid matrix algebra. $\endgroup$ – Nick Cox Feb 17 '17 at 19:05
  • 1
    $\begingroup$ @NickCox Looking at the implementation in software X can be much more challenging than looking at a "educational" implementation. E.g., to understand a recurrent neural network, looking at gist.github.com/karpathy/d4dee566867f8291f086 is much easier than looking at TensorFlow. I agree that scratch is subjective but that can be made more explicit, e.g. implementation of RNN without any neural network libraries or autodifff. $\endgroup$ – Franck Dernoncourt Feb 17 '17 at 19:07
  • $\begingroup$ I don't understand the distinction you're making. Software implementations clearly differ in lots of ways, but I don't follow what is "educational" and what is not. I don't know TensorFlow and I can't judge the code example you cite, so unfortunately the examples don't help. What I had in mind was not just that all R code (e.g.) is public but also that large chunks of (e.g.) MATLAB or Stata are visible even though there is a proprietary core to each. $\endgroup$ – Nick Cox Feb 17 '17 at 19:32
  • 2
    $\begingroup$ @NickCox For example, in some cases, implementations of algorithms in a large library may be difficult to read as they might incorporate many optimizations, and is spread over many different classes. By educational I meant some implementations that we think match of the level and time of the readers. Pointing to R/Matlab code can definitely be good in many cases. $\endgroup$ – Franck Dernoncourt Feb 17 '17 at 20:26
  • $\begingroup$ @NickCox, Matlab's built-in t-test (ttest.m) is 165 lines. This is a lot, particularly since the t-test algorithm is very simple. Find the critical t-value from alpha and DF, then compare it with the mean/se(mean). The "production" version has to handle a lot of edge cases (all nan inputs, multiple alpha values, old argument format, etc) that obscure the simplicity of this algorithm. A "teaching" version wouldn't do any of that and might focus more on the mechanics of the test while avoiding some vectorized bsxfun() monstrosity that runs a bit faster. $\endgroup$ – Matt Krause Feb 25 '17 at 1:26
  • $\begingroup$ @Matt Krause Fair enough. I write programs too that are public, so am familiar with a spectrum of code styles. But all this exposes, I think, an ambiguity in the original (kind of) question. I am reading the question as Where do I see an implementation exposing the basic principles? whereas you and Franck seem to be thinking more about Where do I see an implementation that focuses on simple, well behaved inputs without extra bells and whistles? There need not be a contradiction there, but original posters need to be clear what they want precisely (what else is new?). $\endgroup$ – Nick Cox Feb 26 '17 at 10:49
  • $\begingroup$ @NickCox, Coincidentally, someone sent me this github repo, which is almost exactly what I had in mind github.com/eriklindernoren/ML-From-Scratch It's clearly not something you would use in production, but it's meant to show how the algorithms work while letting you run them. $\endgroup$ – Matt Krause Feb 27 '17 at 17:24
  • $\begingroup$ Does seems bang on. Once more I note that reading "ML" as maximum likelihood, as I do, shows that you are on the statistical side of a (virtual) wall. (Here ML means machine learning.) $\endgroup$ – Nick Cox Feb 27 '17 at 18:04

An implementation of a statistical algorithm is a way to describe the statistical algorithm, and is therefore on-topic on this Stack Exchange website. It is distinct from a pure programming question as the goal is to learn the algorithm.

  • 4
    $\begingroup$ The distinction seems along the right lines in principle (+1). Where to draw the line in practice is much trickier. I wonder how many people want to try to write really general pseudocode for more than about 10 lines. But that could be enough. I've seen lots of posts with the flavour "I use here language X but other people should be able to translate" and I've used that line myself (but I think only when giving code chunks as incidental to more general answers; not as an attempt at a full answer). $\endgroup$ – Nick Cox Feb 17 '17 at 19:37
  • 4
    $\begingroup$ I agree with Nick's assessment of the difficulty of drawing the line in practice. I'm not sure of what alternative answer to even suggest. $\endgroup$ – Glen_b Feb 18 '17 at 0:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .