4

I have a signal coming from a measurement device (e.g., Volt vs., time) and perform a fit to the signal. I would like to do a goodness of fit test like the chi-squared test. However, I am not sure whether it is correct to apply it in my case.

In most online-tutorials (e.g. here http://www.stat.yale.edu/Courses/1997-98/101/chigf.htm), the first step of the chi-squared test is to construct the relative residuals as follows: $\sum (O_i - E_i)^2/E_i$ where $O_i$ would be the observed value and $E_i$ the expected value. This is most often done with data which are merely counts, e.g., counts in bins.

However, on the other hand I know that often in science the relative residuals are done in the following way: $\sum (O_i - E_i)^2/\sigma_i^2$ where $\sigma_i$ denotes the error of data point $i$.

I have two questions:

  1. Is the first version only true when the noise on the data is Poisson, and thus $\sigma_i^2 = E_i$.
  2. Since the examples I found are always for counts in some categories, is the chi-squared test even the right one to measure the goodness of a fit for, e.g., a voltage vs. time signal (or more mathematically: I don't draw multiple random variables according to some distribution and categorize them into bins)?

Edit: Some more information about the fit and my data. For every timestep dt I have exactly one voltage value and I fit (using scipy curve_fit) a hyperbolic curve of the form $f(x) = \frac{mx}{k+x}$ to it. I know the error of the data points. Here is a plot of the points used for the fit (red) and the fitted curve (blue). The way I would think to apply the test is calculating $\frac{1}{n_{\textrm{points}}-2}\sum (y_i - f(y_i))^2/\sigma_i^2$ (in the first denominator -2 as we have two parameters in the fitting function). And to my knowledge this should be around 1. As for the precise test, I have to say that I don't know which chi square distribution to use (the number of degrees of freedom) to make the hypothesis test. enter image description here

numberCruncher
  • 141
  • 5
  • 1
    +1 The test does apply when numbers are put into bins, but there are several subtle pitfalls for the unwary (which seems to include almost everybody). See my account of it buried in an answer at https://stats.stackexchange.com/a/17148/919. – whuber May 26 '17 at 15:06
  • does this mean the test does not apply in the case of a voltage vs time signal or is this equivalent? – numberCruncher May 26 '17 at 15:08
  • 2
    It can be made to apply. However, how you intend to apply it is unclear. Your question actually says very little about your data, your model, your fitting procedure, or how you propose to test the fit. – whuber May 26 '17 at 15:16
  • I would be interested in what you mean with "it can be made to apply" – numberCruncher May 28 '17 at 19:05
  • Please read the link I supplied. – whuber May 28 '17 at 19:34

1 Answers1

1

Is the first version only true when the noise on the data is Poisson, and thus $σ^2_i=E_i$.

Not quite; for example it works for the multinomial (see Pearson 1900 ); $E_i$ is no longer the standard deviation but the dependence between cells in the multinomial exactly compensates for it; see also the test of independence.

Since the examples I found are always for counts in some categories, is the chi-squared test even the right one to measure the goodness of a fit for, e.g., a voltage vs. time signal

Under some very particular assumptions perhaps, including conditionally independent Gaussian response and known $\sigma_i$. I often see it applied where it clearly doesn't apply (e.g. where there's substantial observation error in the $x$'s and its a situation where an errors-in-variables model would apply, or where the supposedly-known $\sigma$ values are clearly inconsistent with the spread of the data around the fit).

To my recollection it doesn't actually apply to nonlinear models when estimating parameters (except approximately/asymptotically).

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • With conditionally independent Gaussian response you mean that the the single data points are independent from each other and the noise is Gaussian? As it is a digital measurement which claims to be at equidistant t-values I guess I can assume the x-error to be zero. But more importantly: I am not sure whether the noise is actually Gaussian, neither is my measurement of the sigma necessarily right (although both assumptions might hold in first order). What would be a good method to measure the goodness of my fit if I don't know the y-error? – numberCruncher May 26 '17 at 19:11
  • This might be a new question, perhaps. What's this measure of goodness of fit to be used to do? – Glen_b May 27 '17 at 02:24
  • I will post a new question about what measure to use – numberCruncher May 28 '17 at 19:04