0

I evaluated a logistic regression using mnrfit function in Matlab. However, I am not sure if I did this correctly, because my graph does not looks like standard logistic regression. The image below represent my logistic regression, there are 11 logistic regression curves, which represent the same variable with different parameters. My questions are:

1) why my 11 curves are not crossing in 'response' equal to 0.5, but around 0.25?

2) why my curves are not symmetrical to 'response' 0.5, why all of them are below 'response' 0.5, any why it is not possible to get 'response' equal to 1?

3) why some of my curves (the gray ones) have the opposite trend than the other ones (some of them with the increase of the 'analyzed variable' value have descending trend? Is it possible?

enter image description here

Haitao Du
  • 32,885
  • 17
  • 118
  • 213
user122618
  • 1
  • 1
  • I'm not familiar with MATLAB but [mnrfit documentation](https://nl.mathworks.com/help/stats/mnrfit.html) suggests that it is for fitting multinomial logistic regression, not a 'standard logistic regression' as you name it. Perhaps a confusion there? – Ozan Öğreden Jul 08 '16 at 08:32
  • Thank you for your answer, by standard logistic regression I meant only the shape of the curve which you can find on images with logistic regression in the internet, like e.g here https://upload.wikimedia.org/wikipedia/commons/thumb/8/88/Logistic-curve.svg/2000px-Logistic-curve.svg.png – user122618 Jul 08 '16 at 09:08
  • 1
    Your model, what you mean with 'representing the same variable with different parameters' (How did you get more than one estimate for the same model?), what those grey dots represent are not clear to me. I may be off the mark: Considering the outlıne of the dots, I guess the inflection point is not at .25. Did you try plotting a wider range for X axis, perhaps [-1, 3] here? (Check the difference between the plot in the question and the accepted answer [here](https://stats.stackexchange.com/questions/31597/graphing-a-probability-curve-for-a-logit-model-with-multiple-predictors).) – Ozan Öğreden Jul 08 '16 at 10:23
  • The 'different parameter' refers to my input data, not to some parameter in logistic regression. To make it more easier we can say that on my figure are 11 logistic regression curves for 11 different data sets, and for all these data sets I have one individual estimate. The grey dots represent logistic regression curve. The x axis on the original figure are[0.5, 2,5] - I just cutted the figure, because above 0.5 there is nothing, just empty space. – user122618 Jul 08 '16 at 12:24
  • 1
    If there are 11 different predictors and if you are fitting 11 models to get parameter estimates, you must have a very specific reason to do so otherwise don't do it. If you have any reason, why would the difference be surprising? Different predictors may have different relationships to the outcome. If there are 11 different data sets, i.e. both different predictor and a different outcome, why would you plot them on top of each other? And again, why the difference would be surprising? I'll post an answer to describe what I think is going on with the plot. – Ozan Öğreden Jul 08 '16 at 13:02
  • 1
    My logistic regression curves, are related each other, therefore I plotted them on one figure. However, it is difficult to explain this on my data, so I will use an example from here https://en.wikipedia.org/wiki/Logistic_regression (scroll to: Example: Probability of passing an exam versus hours of study). Let say that I add one additional factor (a cup of coffee) which may influence on the results of the exam. So I would like to check if the probability in passing of the exam will increase if I will add all students: 1 cup of coffee, 2 cups of coffee, and so on. Is it now more clear? – user122618 Jul 08 '16 at 13:33
  • The differences are not surprising for me. Surprising is that my curves are only under 'response' 0.5, and that they are not cutting each other in 'response' 0.5. Shouldn't they be symmetrical with respect to 'response' 0.5, and cut in 'response' 0.5? – user122618 Jul 08 '16 at 13:37
  • Now I understand your point. I still hold that your plot need not cover all of the 0-1 range. But I would expect them to intersect at 0.5 too. Perhaps you may post some of your MATLAB code and a bit of example data to make this reproducible? – Ozan Öğreden Jul 08 '16 at 14:06
  • @OğuzhanÖğreden: The curves need only intersect at a response of 0.5 if the estimate of the intercept parameter is the same for each regression. I see nothing in the q. or comments to suggest why it would be. – Scortchi - Reinstate Monica Jul 08 '16 at 14:18
  • I could, but I do not see here any option to add some attachment (it is my first question on this forum). – user122618 Jul 08 '16 at 14:18
  • @user122618: See the formatting options along the buttons at the top of the question box - & you can also link to off-site data-sets. But most importantly explain what you're doing & why you're doing it in English (& perhaps Maths) if you want a useful answer: "there are 11 logistic regression curves, which represent the same variable with different parameters" gives us nothing to go on. – Scortchi - Reinstate Monica Jul 08 '16 at 14:28
  • @Scortchi, you are right. I got astray while considering some other possibilities. – Ozan Öğreden Jul 08 '16 at 14:32
  • I think I cannot add the code here without the user account, therefore, the code and data are here https://boxup.uni-potsdam.de/public.php?service=files&t=fa8145d0463659c961ab939345548423 password: math – user122618 Jul 08 '16 at 14:46

1 Answers1

3

So here is what I think is going on with the plot may be something similar to this. Let's assume that this is how the predictor and outcome is related to each other:Relationship

If we happen to have a certain portion of this data and model that portion, our predictor and expected probabilities will look like this:

A portion of the data

Since I'm still not able to understand your thinking about 11 different models part, I'll have to skip that.

Edit after the comment: x-axis labels give the impression that I've just plotted selectively. However I've selected a portion of the data and fit a new model. y-axis of the second plot is the predicted probabilities of this new model and x-axis is the predictor. and the A similar case could have occured in practice due to sampling. Below is the R code, based on the code here:

set.seed(666)
# Let's say this is the way nature works:
x1 = rnorm(500)          
z1 = 1 + 5*x1        
pr1 = 1/(1+exp(-z))  
y1 = rbinom(500,1,pr)
# Fit a model: 
df = data.frame(y=y1,x1=x1)
model1 <- glm( y~x1,data=df,family="binomial")
plot(x1,model1$fitted.values,xlim=c(-2,2),xlab = NULL,ylab = NULL)

# Now let's assume that our observation was limited to this:
x2 <- x1[order(pr)][100:250]
y2 <- y[order(pr)][100:250]

# With the data we 'collected', let's fit a new model:
model2 <- glm(y2 ~ x2, family='binomial')
plot(x2, model2$fitted.values,xlim=c(-2,2),xlab=NULL,ylab=NULL)
Community
  • 1
Ozan Öğreden
  • 186
  • 6
  • I did not select a portion of my data. The curves are generated for the whole analyzed data set. – user122618 Jul 08 '16 at 13:55
  • 1
    (+1) It might help to simply consider the logistic regression equation $\operatorname{E} Y =\frac{1}{1+\mathrm{e}^{-(\beta_0+\beta_1 x)}}$ - how do plots look for different values of $ \beta_0$ & $\beta_1$ (& of course over different ranges of $x$)? Whether or how or why estimates of $\beta_0$ & $\beta_1$ might differ between the 11 regressions - or even whether each estimates a common parameter - is I think unanswerable given the current state of the question. – Scortchi - Reinstate Monica Jul 08 '16 at 14:31
  • 1
    @user122618: You might not have selected a portion of your data but Nature will have. If the chance of banging your head in any given week is a logistic function of your height, & the tallest man in your sample turns out to be 6'7" with a estimated chance of "only" 0.4, why would you be surprised? – Scortchi - Reinstate Monica Jul 08 '16 at 15:00
  • So you mean, that my logistic regression could be fine? I had a doubt, because everywhere in the internet the graphs with logistic regression are always passing through 'response' 0.5 and are symmetrical to this value, but mine not. However, your explanation fit the data which I have and are logic for me. So in general you mean, that with my data it is not possible to have high or even medium probability in predicting the dependence of analyzed phenomenon? – user122618 Jul 08 '16 at 15:13
  • The curve you've fitted will be perfectly symmetrical with an inflection point at a predicted response of 0.5. The observed values of your predictor may be strewn along that curve any-old-how. When people want to illustrate the shape of the logistic function they of course pick a suitable range on the x-axis over which to do so. – Scortchi - Reinstate Monica Jul 08 '16 at 15:28
  • Yes I know, but this figure is only a part of bigger image, therefore the range of my axis might look strange. The whole figure is here: https://boxup.uni-potsdam.de/public.php?service=files&t=fa8145d0463659c961ab939345548423 password: math – user122618 Jul 08 '16 at 15:38