Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [method-comparison]

Questions about comparisons of different statistical methods

141 questions
14
votes
1 answer

Predictive performance depends more on expertise of data analyst than on method?

I've come across a rumour that some study showed that the performance of predictive models depends more on the expertise of the data analyst with the chosen method than on the choice of the method. In other words, the claim is that it is more…
cbeleites unhappy with SX
  • 34,156
  • 3
  • 67
  • 133
13
votes
1 answer

How to compare the strength of two Pearson correlations?

I've been asked by a reviewer if the Pearson correlations (r-values) presented in a table can be compared to each other so one can claim that one is "stronger" than another (other than just eyeballing the actual r-values). How would you go about…
pecorino
  • 303
  • 1
  • 3
  • 9
13
votes
1 answer

How to compare forecasting methods?

I have several intermittent data. Based on those data, I would like to compare several forecasting methods (Exponential Smoothing, Moving Average, Croston, and Syntetos-Boylan), and decide whether Croston or Syntetos Boylan is better than SES or MA…
Fikri
  • 131
  • 5
12
votes
5 answers

What ways are there to show two analytical methods are equivalent?

I have two different analytical methods that can measure the concentration of a particular molecule in a matrix (for instance measure the amount of salt in water) The two methods are different, and each has it's own error. What ways exist to show…
PaulHurleyuk
  • 1,549
  • 3
  • 16
  • 18
10
votes
3 answers

2-Gaussian mixture model inference with MCMC and PyMC

The problem I want fit the model parameters of a simple 2-Gaussian mixture population. Given all the hype around Bayesian methods I want to understand if for this problem Bayesian inference is a better tool that traditional fitting methods. So far…
user2304916
  • 443
  • 3
  • 8
9
votes
2 answers

Propensity score matching vs non-parametric regression

I am trying to understand the benefit of propensity matching over non-parametric regression for causal inference from non-experimental data. As background: the way I understand it, parametric regressions are generally a poor choice for causal…
Shade
  • 93
  • 3
9
votes
5 answers

How to assess repeatability of multivariate and method-specific outcomes?

Method "A" describes biological samples using multivariate "fingerprints" that consist of about 30 different variables. Different variables show different typical distribution and many of them closely correlate one with another. From prior…
David D
  • 717
  • 5
  • 13
8
votes
3 answers

Clarification on how to perform nested cross-validation

I'm currently trying to get lifetables for our population based on death and population counts. My first idea was to follow this paper methodology but after some discussions we are planning to use different models, such as GLMMs and GAMs and explore…
psoares
  • 606
  • 3
  • 15
7
votes
0 answers

mixed effect model for method comparison of time series of paired measurements

A coworker and I are trying to analyze agreement between two measurement methods. I apologize in advance for needing some extra explanation due to the fact I'm an engineer whose statistics background is mostly geared toward the relationship between…
Ben Voigt
  • 170
  • 8
7
votes
1 answer

Is it wrong to compare multiple models on the same test set and choose the best model?

Suppose we split a dataset into 3 parts (train, validation, and test). I know that it's important to make sure the test set doesn't influence our decisions during model selection or hyperparameter tuning or else we may end up overfitting the test…
Abdulwahab Almestekawy
  • 71
  • 2
7
votes
2 answers

What is better in Monte Carlo integration: product of means or mean of products?

Let $X$ and $Y$ be two independent continuous random variables with pdfs $f_X$ and $f_Y$, respectively. Let $\varphi_1$ and $\varphi_2$ be two continuous functions from ${\mathbb R}$ to ${\mathbb R}$. I want to calculate…
Monaco
  • 73
  • 3
7
votes
1 answer

Is stratified meta-analysis more or less objective than meta-regression?

Reviewer asked me why I use meta-regression as a way how to deal with heterogeneity among effect sizes instead of conducting stratified meta-analysis. I tried to google "stratified meta-analysis" and probably the most useful explanation…
Ladislav Naďo
  • 2,202
  • 4
  • 21
  • 45
6
votes
1 answer

What is an "Unpaired Bland-Altman plot"?

I found this quote: Unpaired Bland-Altman plots were used to check systematic or bias errors in the SPECT myocardial uptake coefficients with respect to the reference standard PET data. in the following paper: ALHASSEN, Fares, et al.…
Alessandro Jacopson
  • 699
  • 9
  • 25
6
votes
2 answers

Taxonomy/overview of machine learning techniques

My question: I'm looking for a taxonomy/bestiary/overview of machine learning techniques. I would like to learn 1) how these methods relate to each other, and 2) the relative costs and benefits (and perhaps typical applications) of the different…
mkt
  • 11,770
  • 9
  • 51
  • 125
6
votes
1 answer

Most well-known set-similarity measures?

I know of the Jaccard index and the Sørensen-Dice coefficient for computing set similarity, but have been unable to find any other algorithms related to set similarity. This site contains quite a few resources for vector similarity, but that's not…
blacksite
  • 614
  • 1
  • 10
  • 22
1
2 3
…
9 10