10
$\begingroup$

I frequently post mock ups of my data to help get better answers to my questions. But it occurred to me that I would probably be better off posting my real data. Is there anything wrong with asking questions in which I upload my real data?

| |
$\endgroup$
16
$\begingroup$

Your real data are often exactly what are needed to help us understand your question. Indeed, it is often very frustrating for people who offer comments or answer questions to be told that someone's data are really quite different from that first stated or implied, so that a thread has to be reoriented with time-consuming secondary questions, revised answers, and the like. (Alternatively, people may decide not to give (more) time to a poorly asked question.)

Posting of data need not imply that the dataset is included within your question. If it is small, say a few hundred values, that can be fine. But often a link to a publicly accessible datafile is sufficient. It's best not to assume that everyone uses the same software as you, so avoid proprietary or idiosyncratic data formats. Comma- or tab-separated text files may be considered universally accessible. (Nor are software-specific questions always on-topic here, so the defence of being an X user is not much of a defence on Cross Validated for posting datasets only X can read. Your question should always have a statistical core, and therefore is presumed to interest people who use quite different software.)

There are some areas in which caution is recommended.

  1. If the data you are working with are not strictly yours, but unpublished data from other scientists or scholars, you should seek their permission before publication.

  2. Publication of data should not breach the anonymity, confidentiality or privacy that subjects, institutions or clients may require, deserve or expect. In general, this is between you and those persons or institutions and it is your responsibility to check on any restraints or constraints on publication.

  3. It is possible that the real data are just too large or too complicated in ways that do not bear on your question.

  4. (Nod to @Glen_b) Remember that the web location should remain stable and accessible. The point of postings here is not just to provide immediate gratification to the original poster, but to build up an archive of questions interesting and useful to others. A web link that is no longer valid weakens your question as something helpful to as many as possible for as long as possible.

A rule of thumb I suggest is that helps to see real data of the number and kind that drives your question. Even if you cannot post the full real data, telling us about measurement and sampling protocols, actual sample sizes, number and kinds of variables, and the like often helps greatly.

UPDATE On how you do it:

Copy and paste of data as text is fine. If you copy data as you would copy code, alignment of rows and columns is easier to preserve.

The overarching principle is blindingly obvious, but I will spell it out, as it is sometimes ignored: The key point is not just that others can see your data, but that they can read them easily into their own favourite software to carry out their own analyses.

Sometimes people save a screenshot of a data listing as an image, but a dataset presented in that form is not easy to copy as data.

| |
$\endgroup$
  • 6
    $\begingroup$ One difficulty with linked files is link-rot (old links die leaving a question without context) - questions should, as far as possible, stand alone. If it is possible to illustrate all (or essentially all) issues with a subset that is small enough to be included in the question, that will help, since that way if a link to the fuller data dies, at least only part of the context is lost. Some excellent points in this answer. $\endgroup$ – Glen_b Apr 19 '14 at 17:31
  • 1
    $\begingroup$ How should I include data? I was actually expecting there to be a button to upload .csv files but I cannot find this. The only ways I can think of is to paste data as text or use one of R's download from Dropbox functions $\endgroup$ – luciano Apr 20 '14 at 7:39
  • 2
    $\begingroup$ @Glen_b Would it maybe be a good feature request then to make it possible to upload data sets (maybe upto a certain file size) on the SE server? $\endgroup$ – rbm Apr 20 '14 at 7:56
  • 1
    $\begingroup$ @rbm Yes, I think it would be very useful to be able to upload data sets that are too large to include in-post, but aren't necessarily big. Even if it was just ascii formats it would be a good start. $\endgroup$ – Glen_b Apr 20 '14 at 8:21
  • $\begingroup$ @rbm Please see meta.stats.stackexchange.com/questions/1035/…. $\endgroup$ – whuber Apr 21 '14 at 16:40
  • 1
    $\begingroup$ It is worth noting that you can overcome 1 and 2 by scrambling it. If you take data from, say, a bunch of astronomical objects, but take away coordinates, modify the dates, and add a bit of noise/offset, they are essentially useless for science, but the analysis can be the same. $\endgroup$ – Davidmh May 3 '14 at 7:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .