2

I want to use the encoder of my autoencoder for feature extraction in an image anomaly detection framework.

For that reason, I thought that pretraining the autoencoder on a large dataset and then fine-tuning it on my target dataset would be a good idea. This idea crossed my mind because many anomaly detection approaches use CNN architectures like VGG, ResNet etc. as a feature extractor, which are pretrained on ImageNet.

I did not find papers regarding this matter and therefore my question if transfer learning is even really used on autoencoders?

mar_ey
  • 93
  • 8

2 Answers2

2

That is certainly possible to do, but somewhat different that the other image recognition approaches you mentioned. With those approaches, the encoding that you use in transfer learning has been optimized for some image recognition task, so the it likely has features that are beneficial for that task. Fine tuning just takes a previously optimized set of features and adjusts them slightly to the new task.

An autoencoder is trained to create an embedding that can reproduce the original data space - an unsupervised machine learning problem - but there is no reason that embedding is useful for another task. It might be, but it wasn't trained for that purpose, so it might not have the right structure for a supervised machine learning problem.

KirkD_CO
  • 1,013
  • 1
  • 6
  • 17
  • But shouldnt the extracted features of an encoder also be beneficial for that task, since the decoder uses these to reconstruct the image and therefore these features should contain good information about the input-image? – mar_ey Oct 14 '20 at 23:09
  • 1
    Yes, it is possible they are beneficial, but since they were derived in an unsupervised manner, the only task they are certain to be useful for is to efficiently encode the original data. The advantage that the image recognition embeddings have is that the features were found to be useful for a problem that is similar to how they will be applied later in transfer learning. I don't want to suggest the encodings are guaranteed not to be useful, but they are not guaranteed to be useful either. – KirkD_CO Oct 15 '20 at 04:09
2

This approach often used in NLP models, so called via tuning and it is what you called transfer learning. A recent debate on this so called Foundation models actually revolves around this concept.

transfer learning is even really used on autoencoders?

Yes, it is possible, as these new foundation models approach training a large vision models or autoencoders. However, VGG or Resnet are very tiny models to be used as a foundational model compare to language models like GPT-3 or similar in size and training data coverage.

msuzen
  • 1,709
  • 6
  • 27