0

Question

I'm aware that generating features from existing data can be a valid method for adding new features for a regression/ML algorithm*, but can you add observations generated from existing data?

*related SO question: Do combinations of existing features make new features?

Example

Language: R

Given a data frame df of three dependent variables (dv1, dv2, dv3) and one response variable (rv)

dv1 <- c("gr1", "gr2", "gr3", "gr3", "gr3", "gr3", "gr1", "gr2", "gr2", "gr1", "gr3", "gr2")
dv2 <- c("grA", "grA", "grB", "grB", "grB", "grA", "grB", "grA", "grB", "grA", "grB", "grB")
dv3 <- c(1,1,1,1,2,2,1,1,1,1,2,1)
rv <- c(1,2,3,3,2,1,1,2,3,3,2,1)

df <- data.frame(dv1, dv2, dv3, rv)

> head(df)
  dv1 dv2 dv3 rv
1 gr1 grA   1  1
2 gr2 grA   1  2
3 gr3 grB   1  3
4 gr3 grB   1  3
5 gr3 grB   2  2
6 gr3 grA   2  1

Does it make statistical sense to engineer observations by grouping the variables, finding the 'total' rv value for that group...

library(dplyr)
df_t <- df %>%
  group_by(dv1, dv2) %>%
  summarise(dv3 = sum(dv3),
            rv = sum(rv))

> head(df_t)
Source: local data frame [6 x 4]
Groups: dv1

  dv1 dv2 dv3 rv
1 gr1 grA   2  4
2 gr1 grB   1  1
3 gr2 grA   2  4
4 gr2 grB   2  4
5 gr3 grA   2  1
6 gr3 grB   6 10

... and then combining it with the original data...

df2 <- rbind(df, df_t)

> df2
   dv1 dv2 dv3 rv
1  gr1 grA   1  1
2  gr2 grA   1  2
3  gr3 grB   1  3
4  gr3 grB   1  3
....
13 gr1 grA   2  4
14 gr1 grB   1  1
15 gr2 grA   2  4
16 gr2 grB   2  4
17 gr3 grA   2  1
18 gr3 grB   6 10

... and then using that data to train a regression model?

Community
  • 1
tospig
  • 165
  • 1
  • 7
  • 1
    It sounds like you are rather interested in mixed models e.g. http://stats.stackexchange.com/questions/134515/why-are-the-beta-values-provided-in-lmer-different-than-simple-group-means-of/134528#134528 or http://stats.stackexchange.com/questions/118473/measurements-from-two-raters-should-i-use-multilevel-random-effects-model/122024#122024 - aggregating values by groups is generally a bad idea and it is better to use model that includes both individual and group effects. – Tim Apr 24 '15 at 07:42

1 Answers1

-1

regression is finding the function which can map your attributes to the target values

f(x,y,z)=target

consider a simple function:

0*x+(y+1)=target

so: . .

f(a,1)=2

f(a,2)=3

f(a,3)=4

f(a,4)=5

. .

Follow your "add" assumption , there will be

f(a,10)=14 , and this is wrong.Seems that you can't simply "add".

Steven Du
  • 141
  • 1
  • 5
  • (-1) this does not answer the question... – Tim Jan 04 '16 at 15:29
  • @Tim I think it is in fact an attempt to answer the question, but the explanation for why you shouldn't do it is unclear. Du Sijun, please consider trying to expand on your explanation of why it's wrong to do so. – Glen_b Jan 07 '16 at 01:53