Should I use a paired sample t-test to compare two methods of measuring absorption lines?

Question

Background

I'm doing a research project in astronomy and measuring equivalent widths of absorption lines using Gaussian fits from a star's spectrum in order to determine the star's chemical abundances. Each absorption line corresponds to a specific electronic transition in an atom of a specific element. One type of atom, say iron, produce many absorption lines at different wavelengths that we observe. In addition, different atoms produce different absorption lines. The equivalent width of a Gaussian fit allows one to quantify the strength of an absorption line observed in a spectrum (i.e., how strong a luminous intensity the line appears to have).

My data

I have measured equivalent widths for 54 absorption lines with Method A and B, as shown in Tables 1 and 2 respectively. Each "Equivalent width" value in tables is an average of two or three measurements for each line.

This is how the measurements are done (the full method is described here) but are summarized here https://gitlab.com/evgenyneu/2018_summer_project_logbook/raw/master/a2019/a01/a23_measuring_lines/three_measurements.gif. I fit the Gaussian profile to the observed data. For the first measurement, I make a fit with a Gaussian curve with a wide base. For the second I fit a narrower Gaussian to the peak region. If the observed line is good, I make an optional third measurement by making a what looks like a moderate fit, close to the full width at half-maximum (actually half-minimum as absorption curves are upside down) which is between the narrow and wide measurements.

The "uncertainty" column is the half of the range of the measurements of equivalent widths. For example, if we have two measurements, 10 mA and 16 mA, then the uncertainty is 3 mA ( (16-10)/2 ).

Table 1: Equivalent widths of absorption lines and their uncertainties measured with Method A. The units are angstrom (A) and milliangstrom (mA).

+----------------------------------------+------------------+
| Wavelength (A) | Equivalent width (mA) | Uncertainty (mA) |
+----------------+-----------------------+------------------+
| 4730.03        | 37                    | 8                |
| 4731.45        | 79                    | 9                |
| 4788.76        | 52                    | 5                |
| 4890.75        | 153                   | 20               |
| 4891.49        | 171                   | 19               |
| 5814.81        | 8                     | 2                |
...
... Skipped 47 rows
...
| 6703.57        | 22                    | 1                |
+----------------+-----------------------+------------------+        

Table 2: Equivalent widths of absorption lines and their uncertainties measured with Method B.

+----------------------------------------+------------------+
| Wavelength (A) | Equivalent width (mA) | Uncertainty (mA) |
+----------------+-----------------------+------------------+
| 4730.03        | 26                    | 4                |
| 4731.45        | 52                    | 7                |
| 4788.76        | 39                    | 3                |
| 4890.75        | 116                   | 18               |
| 4891.49        | 139                   | 18               |
| 5814.81        | 6                     | 1                |
...
... Skipped 47 rows
...
| 6703.57        | 16                    | 2                |
+----------------+-----------------------+------------------+ 

Question

Should I compare methods A and B using $t$-testing or should I do something else?

Alternative methods

If paired sample t-test is not suitable, what alternative statistical tests should I use (Wilcoxon signed-rank test, chi square test of independence etc.)?

Sample size

54 measurements.

Independence

Equivalent widths measurements of 54 different lines are not be independent. For example, higher abundance of iron may result in wider equivalent width values of all its absorption lines. In my data it means that, for example, if measurement of a 4730.03 A line is wider, it is likely that measurement of 4731.45 A will be wider as well, if the two lines are produced by the same element.

Answer 1

I would recommend using the multivariate Hotelling's paired $T^2$-statistic since you have three measurements at each wavelength for each method. Essentially, what you are doing with this method is testing, simultaneously (and thereby controlling the error rate) if the measurements in total (i.e. 54 wavelengths) from Method A are equivalent to Method B. This is the multivariate equivalent to the paired-sample T-test.

There is a good description of this method listed here at Penn State's Multivariate Statistics Course. If you are using R, you'll want to check out the Hotelling package. If you are using SAS for your analysis, you can carry out this test using proc glm along with the MANOVA statement. Please pay careful attention to the assumptions listed before using this method. You must meet the assumptions for this test to be valid.

An excellent textbook reference is the classic multivariate statistics introductory text by Johnson and Wichern, Applied Multivariate Statistical Analysis 6th ed. (page 273-279).

Answer 2

The text (after editing due to comments exchanged) shows that a Gaussian is not a good fit to the spectral lines. This link shows the origin of the different widths of the spectral lines depending on how the lines are fit.

Half ranges are not a common method of making observations concerning error of location. For two observations this reduces to $\sqrt{2} $ standard deviations. However, for three observations it would have a different, more-variable relationship to standard deviation. Regardless, the half-range is a small number biased estimate, and would be perhaps better expressed as its square to eliminate that bias, i.e., analogous to variance being unbiased. The reasoning behind this is perhaps not obvious, so see Why are we using a biased and misleading standard deviation formula for $\sigma$ of a normal distribution? by way of explanation. In this case, a better measurement is the area under the curve, which (square measurement) should be proportional to luminous flux despite variable width of the absorption lines from whatever cause.

The sloppiness of the fitting with Gaussian curves is that the spectral lines are not Gaussian curves. That is, the first step in such problems is the numerical identification of the distribution type appropriate to the problem. That is the first thing I would test in the original data. As it turns out, this obviates $t$-testing by eliminating the modelling uncertainty.

A short-cut that obviates the need for testing fit quality is a search on prior work. That search yielded, among other things, Modeling Stellar Absorption Lines: The FeI 6546.25 $\overset{\small{_{\text{o}}}}{\text{A}}$ Line, which suggests better results from Chi-squared fitting of Voigt profile combination of the Gaussian and Lorentz distributions (Bowers & Deeming 1984), and is often used to model spectral absorption line features. If you can implement this, you may also be able to reduce the uncertainty resulting from Gaussian only modelling. That is, it would appear that there are three different criteria for Gaussian fitting because a Gaussian is not the best possible shape to be fitting to the data. I would encourage you to undertake further searches and consultations with potential coauthors on this approach, precisely because I am not an astrophysicist myself, although I probably should disclose that I have coauthored papers with two outstanding ones. Finally, it was not inappropriate to ask your question on CrossValidated, and although there is a StackExchange companion site for physics, there is no AstroPhysics site per se, and many of the astro questions posed do appear on this site.