ICD10 (categorical) encoding

Question

I am trying to figure out how best to encode ICD10 codes for input into a machine learning model.

It isn't ordinal by any means, however, there is a sort of logic you can apply to just the labels that will tell you which ones can be grouped together without any additional knowledge. In some ways, it's closer to an interval data type in that the difference between consecutive codes frequently (but by no means always) has some kind of meaning.

This makes me think that binary encoding may be more effective than one-hot, but are there other options I should be considering?

for example:

I71.1 Thoracic aortic aneurysm, ruptured

I71.2 Thoracic aortic aneurysm, without mention of rupture

I71.3 Abdominal aortic aneurysm, ruptured

I71.4 Abdominal aortic aneurysm, without mention of rupture

J12.2 Parainfluenza virus pneumonia

J12.3 Human metapneumovirus pneumonia

J12.8 Other viral pneumonia

J12.9 Viral pneumonia, unspecified

Can you explain the difference between binary and one-hot for your application? It seems that one-hot coding, which used to be called dummy indicators, is probably your best option. — Todd D, Nov 22 '18 at 00:09
for binary I mean that I would assign a number to each label e.g. J12.3=0, J12.8=1, J12.9=3 and then binary encode these 000, 001, 010 etc. with the idea that this may outperform purely 1-hot because then all the J1 labels would have the higher order digits the same and all the I71 labels would have higher order digits the same as well, allowing groupings to be learned as well as just the actual label - if I predict 'other viral pneumonia' when it should have been one of the other choices, this is still better than many other predictions despite not being 100% correct — gkennos, Nov 22 '18 at 00:14
Also I am intending to drop digits to the right of the decimal point to improve on sparsity, but included all here for example — gkennos, Nov 22 '18 at 00:15
Are you training a model to cluster the codes or using the codes as explanatory features/variables? — Todd D, Nov 22 '18 at 00:22

score 1 · Answer 1 · answered Dec 01 '18 at 19:34

You should encode as dummy variables ("one-hot"). I don't understand how you could use your binary encoding, you would have to explain. But you say In some ways, it's closer to an interval data type in that the difference between consecutive codes frequently (but by no means always) has some kind of meaning. This is interesting, but probably difficult to encode information. Maybe there is an Bayesian way. But this could still be exploited, and you say one goal (additional) could be to learn that some labels are close to other labels in effect.

One method which could achieve that, is fused lasso, search this site or look at Principled way of collapsing categorical variables with many levels? or Continuous dependent variable with ordinal independent variable

ICD10 (categorical) encoding

1 Answers1