Anybody have any experience with software (preferably free, preferably open source) that will take an image of data plotted on cartesian coordinates (a standard, everyday plot) and extract the coordinates of the points plotted on the graph?
Essentially, this is a data-mining problem and a reverse data-visualization problem.
There are many different options, but all basically use the same workflow:
The program returns each point as an x-y matrix.
Often it helps selecting points if the image is zoomed, either by uploading a zoomed version of the image or using the zooming feature available in some of the programs.
There are many programs, and they vary in extra features, usability, licensing, and cost. I have listed them below.
All of the ones I have used work fine. Except in contexts where measurement error is very small, error from graph scraping is insignificant (e.g. error from digitization << size of error bars or uncertainty in the estimate). If have not tested the accuracy of any of these programs, but it would be interesting to compare among users, among programs, and against the results of reproduced statistical analyses.
TL;DR: WebPlotDigitizer is available as a web application as well as a chrome plugin
Other answerers assume that you deal with raster image of a graph. But nowadays the good practice is to publish graphs in vector form. In this case you can achieve much higher exactness of the recovered data and even estimate the recovery error if you work with the code of the vector graph directly, without converting it to raster image.
Since the papers are published online as PDF files, I assume that you have a PDF file which contains vector plot with data you wish to recover from it (get in numerical form) and estimate introduced recovery error.
First of all, PDF is a vector format which is basically textual (can be read by a text editor). The problem is that it can (and almost always) contain compressed data streams which require to be uncompressed in order to read them by a text editor. These compressed data streams usually contain the information we need.
There are several ways to uncompress data streams in order to convert PDF file to a textual document with readable PDF code. Probably the simplest way is to use free QPDF utility with --stream-data=uncompress option:
qpdf infile.pdf --stream-data=uncompress -- outfile.pdf
Some other ways are described here and here.
The generated outfile.pdf can be opened by a text editor. Now you need PDF Reference Manual 1.7 to understand what you see. Do not panic at this moment! You need to know only few operators described in the "TABLE 4.9 Path construction operators" on pages 226 - 227. The most important operators are (the first column contains coordinate specification for an operator, the second contains the operator and the third is operator name):
x y m moveto
x y l lineto
x y width height re rectangle
h closepath
In most cases it is sufficient to know these four operators for recovering the data.
Now you need to import the outfile.pdf file as text into some program where you can manipulate the data. I'll show how to do it with Mathematica.
Importing the file:
pdfCode = Import["outfile.pdf", "Text"];
Now I assume the simplest case: the graph contains a line which consists of many two-point segments. In this case each segment of the line is encoded like this:
268.79999 408.92975 m
272.39999 408.92975 l
Extracting all such segments from the PDF code:
lines = StringCases[pdfCode,
StartOfLine ~~ x1 : NumberString ~~ " " ~~ y1 : NumberString ~~ " m\n" ~~
x2 : NumberString ~~ " " ~~ y2 : NumberString ~~ " l\n"
:> ToExpression@{{x1, y1}, {x2, y2}}];
Visualizing them:
Graphics[{Line[lines]}]
You get something like this (the paper I am working with contains four graphs):
Each two adjacent segments share one point. So in this case you can turn the sequences of adjacent segments into paths:
paths = Split[lines, #1[[2]] == #2[[1]] &];
Now you can visualize all the paths separately:
Graphics[{Line /@ paths}]
From this figure you can select (by double-clicking) the path you are looking for, copy graphics selection and paste as new Graphics. For converting it backward to list of points you take the element {1, 1, 1}. Now we have the points not in the coordinate system of the graph but in the coordinate system of the PDF file. We need to establish relationship between them.
From the above plot you select ticks by hand (holding Shift for multiple selection), then copy them and paste as new Graphics. Here is how you can extract coordinates of horizontal ticks:
Now check the differences between ticks:
Differences[reHorTicks]
From these differences you can see how precise is positioning of the ticks in the PDF file. It gives an estimate of error introduced by converting original datapoints into vector graph included in the PDF file. If there are appreciable errors in ticks positioning you can reduce the error by fitting the coordinates of ticks to a linear model. This linear function now can be used to get original coordinates of points of the path (that is in the coordinate system of the plot).
I haven't used it, but UWA CogSci lab recommend DataThief (shareware).
You can also try im2graph (http://www.im2graph.co.il) to convert graphs to data. Works in Linux and Windows.
'g3data' is a software which can be used to serve your purpose. It's a free software and I have used it. You can download it from here: http://www.frantz.fi/software/g3data.php
I had to do this so many times in my career I eventually put together a javascript program which is available here:
http://kdusling.github.io/projects/DataGrab/index.html
Sorry, but you will still need to click on every single point. Though you can use the arrow keys which does save some wrist strain.
STIPlotDigitizer has been newly released.
http://stiwww.com/product/software-techniques-plot-digitizer
For R users, the package grImport (on CRAN) can import vector graphics and convert them into objects that R can interpret. It assumes that one can convert PDF (or other vector format of interest) to PostScript format. This can be done for example with Inkscape: import (File > Import) your PDF page with your figure into Inkspace and File > Save As > Save as type: > PostScript *.ps. Once you have your *.ps file fallow the grImport vignette Importing Vector Graphics, more relevant being section '4.1. Scraping data from images'.
You will need ghostscript on your Operating System - try to download it from here.
Note, if you run somehow into ghostscript error 'status 127' when you call grImport::PostScriptTrace, then fallow the recommendation from here, which says to manually set the path to ghostscript on your machine.
Here is some sample R code to import PostScript file into R:
install.packages("grImport")
require(grImport)
# if you get the ghostscript error 'status 127' then set the path to ghostscript, e.g.:
Sys.setenv(R_GSCMD = normalizePath("C:/Program Files/gs/gs9.22/bin/gswin64c.exe"))
PostScriptTrace(file = "graph.ps", outfilename = "graph.ps.xml")
my_fig <- readPicture(rgmlFile = "graph.ps.xml")
grid.picture(my_fig)
Note, if your graph is on a page in a multi page PDF file, then you can split the multi-page document with PDFTK builder. Import your one page PDF file in Ikscape and delete any extra elements (extra text, extra graph elements). This wil ease your work in R when trying to catch the coordinates of the graph elements you are interested in.
