Questions tagged [weights]

"Weights" may refer to: (1) observation weights that come from sample surveys -- consider tagging "survey-sampling"; (2) Monte Carlo sample weights that arise when sampling from intractable distributions -- consider tagging "weighted-sampling"; (3) variable weights in statistical or machine learning models such as regression, factor analysis, or learning networks -- consider tagging with that specific model. Other odd uses of weights go here.

Weights generally indicate that a certain piece of information has a greater or a lesser importance than others. This may arise in almost any data analysis situation. As the concept of weights in statistics is quite ambiguous, questions tagged simply as weights may not attract the appropriate group of experts to answer them. If you came to this page while writing, editing or tagging a question, please consider tagging your question in a more specific way.

"Weights" may refer to:

  1. observation weights that come from sample surveys -- please tag "survey-sampling".
  2. Monte Carlo sample weights that arise when sampling from intractable distributions -- please tag "weighted-sampling".
  3. variable weights in statistical or machine learning models such as regression, factor analysis, or learning networks -- please tag with that specific model.

Other odd uses of weights go here.

A useful resource describing the various type of case/observation/row weights is http://www.ats.ucla.edu/stat/stata/faq/weights.htm.

Other related concepts and tags:

340 questions
19
votes
4 answers

Dealing with ties, weights and voting in kNN

I am programming a kNN algorithm and would like to know the following: Tie-breaks: What happens if there is no clear winner in the majority voting? E.g. all k nearest neighbors are from different classes, or for k=4 there are 2 neighbors from class…
Fletcher Duran
  • 191
  • 1
  • 1
  • 3
18
votes
2 answers

How do sample weights work in classification models?

What does it mean to provide weights to each sample in a classification algorithm? How does a classification algorithm (eg. Logistic regression, SVM) use weights to give more emphasis to certain examples? I would love going into the details to…
18
votes
0 answers

When should I use the Normal distribution or the Uniform distribution when using Xavier initialization?

Xavier initialization seems to be used quite widely now to initialize connection weights in neural networks, especially deep ones (see What are good initial weights in a neural network?). The original paper by Xavier Glorot and Yoshua Bengio…
15
votes
2 answers

What is the benefit of the truncated normal distribution in initializing weights in a neural network?

When initializing connection weights in a feedforward neural network, it is important to initialize them randomly to avoid any symmetries that the learning algorithm would not be able to break. The recommendation I have seen in various places (eg.…
14
votes
2 answers

Why do we use Gaussian distributions in Variational Autoencoder?

I still don't understand why we force the distribution of the hidden representation of a Variational Autoencoder (VAE) to follow a multivariate normal distribution. Why this specific distribution and not another one ? This is maybe linked with…
11
votes
2 answers

Can Weights and Offset lead to similar results in poisson regression?

In "A Practioner's guide to Generalized linear models" in paragraph 1.83 it is stated that: "In the particular case of a Poisson multiplicative GLM it can be shown that modelling claim counts with an offset term equal to the log of the exposure…
11
votes
4 answers

Neural network - meaning of weights

I am using feed-forward NN. I understand the concept, but my question is about weights. How can you interpret them, i.e. what do they represent or how can they be undestrood (besied just function coefficients)? I have found something called "space…
Martin Perry
  • 205
  • 1
  • 2
  • 10
10
votes
1 answer

Ordinary kriging example step by step?

I have followed tutorials online for spatial kriging with both geoR and gstat (and also automap). I can perform spatial kriging and I understand the main concepts behind it. I know how to build a semivariogram, how to fit a model to it and how to…
Pigna
  • 163
  • 1
  • 8
9
votes
1 answer

Algorithms for weighted maximum likelihood parameter estimation

What are the computational or algorithmic considerations for weighted maximum likelihood parameter estimation? That is, I want to get $$ \theta^* = \arg\max\limits_\theta \sum_i w_i \log(\mathcal{L}(\theta|x_i)) $$ assuming we have a weight $w_i$…
9
votes
4 answers

How to "highlight" an input feature of an artificial neural network?

I'm trying to solve a binary classification problem by using an artificial neural network implemented in Torch. My neural network has 82 input features (=neurons). After implementing a plain version that gives to the all the 82 input neurons the…
DavideChicco.it
  • 682
  • 1
  • 10
  • 24
8
votes
2 answers

What's the recommended weight initialization strategy when using the ELU activation function?

For deep neural networks using ReLU neurons, the recommended connection weight initialization strategy is to pick a random uniform number between -r and +r with: $r = \sqrt{\dfrac{12}{\text{fan-in} + \text{fan-out}}}$ Where fan-in and fan-out are…
MiniQuark
  • 1,930
  • 3
  • 16
  • 29
7
votes
2 answers

What is the distribution of the (arbitrarily) weighted Maximum Likelihood Estimator?

Suppose you observe vector $X_i$ of independent variables, and $y_i$ dependent variables, with likelihood $l\left(\theta;X_i,y_i\right)$. Assume the $y_i$ are independent. Furthermore assume you are given positive weights, $w_i$ which are arbitrary,…
7
votes
1 answer

Use BIC or AIC as approximation for Bayesian Model Averaging

I want to compare "real" Bayesian Model Averaging (BMA) performed with the EM algorithm and information-criterion based BMA. Which one, BIC or AIC, is a "closer" approximation to the "real" BMA? BIC as the name is indicating? I would like to compare…
user3165675
  • 139
  • 7
6
votes
0 answers

Derivation of a doubly robust estimator with clever covariate and inverse probability weighting

With notation: outcome $Y$, (binary) treatment $A$, and covariates $L$. In Hernan and Robins (2020) causal inference textbook: To obtain a doubly robust estimate of the average causal effect, first estimate the IP (inverse probability) weight W =…
6
votes
1 answer

How to understand the vertical bar (pipe) in R formulas

I came upon this because I wanted to emulate Welch's t-test using gls. I found the answer here: https://stats.stackexchange.com/a/144480/141304 and it says to add weights with gls(y ~ group, data = dat, weights=varIdent(form = ~ 1 | group)) y and…
abalter
  • 770
  • 6
  • 18
1
2 3
22 23