Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [transfer-learning]

A setting in machine learning when a model trained in one context/domain should then be applied to a different (but related) context/domain.

126 questions
33
votes
6 answers

Why study convex optimization for theoretical machine learning?

I am working on theoretical machine learning — on transfer learning, to be specific — for my Ph.D. Out of curiosity, why should I take a course on convex optimization? What take-aways from convex optimization can I use in my research on…
Upendra01
  • 1,566
  • 4
  • 18
  • 28
29
votes
5 answers

What is difference between 'transfer learning' and 'domain adaptation'?

Is there any difference between 'transfer learning' and 'domain adaptation'? I don't know about context, but my understanding is that we have some dataset 1 and train on it, after which we have another dataset 2 for which we want to adapt our model…
mrgloom
  • 1,687
  • 4
  • 25
  • 33
28
votes
2 answers

What's the intuition behind contrastive learning or approach?

Maybe a noobs query, but recently I have seen a surge of papers w.r.t contrastive learning (a subset of semi-supervised learning). Some of the prominent and recent research papers which I read, which detailed this approach are: Representation…
CATALUNA84
  • 498
  • 1
  • 4
  • 11
17
votes
2 answers

Fine Tuning vs Joint Training vs Feature Extraction

I am reading this paper http://zli115.web.engr.illinois.edu/wp-content/uploads/2016/10/0479.pdf It distinguishes between feature extraction and fine tuning in deep learning. I am not getting the difference as feature extraction is just the same as…
Rafael
  • 1,109
  • 1
  • 13
  • 30
12
votes
1 answer

How should I standardize input when fine-tuning a CNN?

I am working on a model for binary classification of skin samples from https://www.isic-archive.com as either benign or malignant. I want to use the VGG16 model pre-trained on ImageNet and fine-tune some layers to my dataset. The VGG16 paper…
fabiomaia
  • 579
  • 3
  • 14
12
votes
3 answers

Difference between multitask learning and transfer learning

I am reading Caruana (1997) Multitask learning (pdf). In the definition of multi task learning, the author states that; Usually, we do not care how well extra tasks are learned; their sole purpose is to help the main task be learned better. This…
Rafael
  • 1,109
  • 1
  • 13
  • 30
10
votes
2 answers

Where to find pre-trained models for transfer learning

I am new to the machine learning field, but I wanted to try and implement a simple classification algorithm with Keras. Unfortunately, I have a very small set of data, so I thought to try to apply transfer learning to the problem; however, I…
Eggman
  • 103
  • 1
  • 8
9
votes
1 answer

Fine Tuning vs. Transferlearning vs. Learning from scratch

In my master thesis, I am researching on transfer learning on a specific use Case, a traffic sign detector implemented as a Single Shot Detector with a VGG16 base network for classification. The Research focuses on the problem of having a detector…
Jürgen Zornig
  • 215
  • 1
  • 2
  • 6
8
votes
1 answer

Transfer learning: How and why retrain only final layers of a network?

In this video, Prof. Andrew Ng says regarding transfer learning: Depending on how much data you have, you might just retrain the new layers of the network, or maybe you could retrain even more layers of this neural network. The new layers he is…
Tom Hale
  • 2,231
  • 3
  • 13
  • 31
7
votes
0 answers

Transfer Learning on generative adversarial networks (GANs)

Is it possible to apply transfer learning to GANs and if so, what are some examples of someone having tried this?
Alberto MQ
  • 71
  • 1
  • 4
7
votes
0 answers

Change image input size of a pre-trained convnet

maybe this question will sound a bit as a newbie one but I'd like to have some clarification. I'm using a VGG16-like convnet, pre-trained with VGG16 weights and edited top layers to work with my classification problem; specifically I removed the…
matteodv
  • 171
  • 2
  • 5
6
votes
1 answer

Transfer Learning: data in the source domain and the target domain are required to be independent and identically distributed

In instance-based transfer learning, it is said that data in the source domain and the target domain are required to be independent and identically distributed. When it says that the data "are required to be independent and identically distributed",…
The Pointer
  • 1,064
  • 13
  • 35
6
votes
2 answers

Python package that allows to train a CRF on two datasets

I am looking for a Python package that allows to train a conditional random field (CRF) on two datasets. For example: I have two datasets, dataset A and dataset B. I want to train a conditional random field on dataset A, then train the CRF on…
Franck Dernoncourt
  • 42,093
  • 30
  • 155
  • 271
5
votes
3 answers

Confusion about the training procedure while using transfer learning

Suppose that we have a trained CNN, there is 5 conv layers and 3 fully connected layers. We take the first 5 conv layers as it is (with their parameter settings: like kernel size, activation etc) and their weights and biases which are trained before…
Mas A
  • 175
  • 9
5
votes
3 answers

Extending a neural network to classify new objects

Suppose a model M classifies apples and oranges. Can M be extended to classify a third class of objects, e.g., pears, such that the new images for 'retraining' only have pears annotated and apples and oranges ignored? That is, since M already…
John M.
  • 161
  • 1
1
2 3
…
8 9