There are lots of discussions and research about tips and tricks that are helpful for convolutional networks training. (Like in this paper). The new architectures/optimizers/layers emerge very often, especially in the fields of computer vision and NLP. I wonder is there something similar for the fully connected networks applied to the tabular/structured data classification?
The most widely used approach I see is to use embeddings layers, concatenate them with numerical input, and pass it through the stack of fully-connected layers (probably with dropouts and batch normalization). Are there any other recommendations or state-of-the-art methods applied for the tabular data, any custom architectures suitable for this task? Or the "simple" approach outlined above is the best available solution?
I understand that people mostly use boosted trees or linear models in this case but I would like to investigate possible solutions available in the Deep Learning world.
