0

Background: Every time a customer buy something from us we will ask if the customer wants to become a member. I'm trying to build a Logistic Regression model to forecast how likely a customer will become a member based on some attributes and I'm facing a problem about how to deal with missing data properly in this case.

Question: I found there is a significant correlation between "Time Waiting in Line" and "Become a Member". However, "Time Waiting in Line" only make sense when a customer walks in our store but doesn't make sense when a customer buys things online or using phone calls. Therefore, I don't think it's OK to fill in the missing "Time Waiting in Line" using median or average... What should I do here? enter image description here

Frank Lin
  • 13
  • 4
  • 2
    If you fill in "Time Waiting in Line" for persons who didn't wait because they ordered by phone with *anything* then you end up with a rubbish data and rubbish results. See https://stats.stackexchange.com/questions/231817/how-is-missing-data-calculated-on-likert-scale/231848#231848 for a similar example. – Tim Jun 16 '17 at 07:47
  • 1
    See https://stats.stackexchange.com/q/6563/17230. (You can fill in "Time Waiting in Line" with an arbitrary value provided that you decouple it from valid values using a dummy variable.) – Scortchi - Reinstate Monica Jun 16 '17 at 09:03
  • @Scortchi Thank you for the link! It looks like a promising way to go and I'm trying to understand it better. Do you by any chance know any other reference related to this type of solution? (Like a book or online resource?) – Frank Lin Jun 16 '17 at 18:25
  • I've seen this method discussed in the missing data literature, e.g. the reference in [this answer](https://stats.stackexchange.com/a/135890/17230). It's really appropriate for "not applicable" rather than "unknown". – Scortchi - Reinstate Monica Jun 21 '17 at 13:20

1 Answers1

1

It's better to model the online purchases and, offline purchases separately. Also try to get in more columns in the data, like, time of purchase, if he knows about what the membership program provides and more variables that you think effects the decision.

Let's try to see what filling in the missing data for the online purchases mean. For people who did become a member, you are looking at the time a person can wait for a given amount of purchase. For a person who hasn't become a member, the time a person if had waited for a given amount of purchase wouldn't have become a member. I don't think this is what you want your model to do. So, filling those based on the offline purchases doesn't make sense

Gopick
  • 91
  • 7
  • Actually both groups of costumers may have much in common and it may be worth building model for both groups together as e.g. suggested by @Scortchi and in the link he referred to https://stats.stackexchange.com/q/6563/35989 – Tim Jun 17 '17 at 09:07