1

This is related to a data classification problem having a Boolean output variable.

Summary: Once I perform the ML task using Logistic regression, I get the required coefficients. I use the multivariate model $\theta_0 + \theta_1X_1 + .. = Y$. Having obtained the $\theta$'s, am looking for some straightforward formulas that I can use to get the following:

  1. For the coefficients:
    • Standard Error: I think it is $(X^TX)^{-1} * (X^TY)$. Please correct me if wrong
    • Wald Stats: Is it square of $\text{Coefficient} / \text{StdError}$?
    • 95 %CI: Is it $(\text{lower, upper}) = (\text{coefficient}_i - 1.96 SE_i,~ \text{coefficient}_i + 1.96 SE_i)$?
    • p-value and significance
  2. For the whole model
    • Null model -2 Log likelihood
    • Full model -2 Log Likelihood
    • Chi-squared
    • DF
    • Significance level
  3. ROC Curve
    • Area under the curve (AUC)

I am new to ML and trying out C# based simulation, so any vector-based formulas would be highly appreciated.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
sppc42
  • 163
  • 1
  • 1
  • 5
  • $(X'X)^{-1} X'Y$ is the least squares solution for the coefficient vector and is unrelated to your question. The SEs for logistic regression coefficients are not expressible in closed form but you can get approximate standard errors using the fisher information matrix. I think these standard errors are what's outputted with standard statistical software. You can calculate the AUC using its close relationship to the Mann-Whitney U - see [the wikipedia page](http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Area-under-curve_.28AUC.29_statistic_for_ROC_curves) for more info. – Macro Aug 08 '13 at 14:41
  • If you're literally just looking for formulas, they should be readily available in the literature. Are you just asking us to look this stuff up for you? Because if that's your question, please look these formulas up yourself. Otherwise, please clarify what you're asking for. It looks like you may actually have multiple questions here (or you just want us to give you a list of formulas). – David Marx Aug 08 '13 at 17:50

0 Answers0