-1

Siamese network consists of two identical networks. Networks share the same weights. The general workflow is as follows (taken from here):

enter image description here

Suppose that I have 10 images of apples, 10 images of bananas, 10 images of strawberries, and 3 of oranges. I want to create a system that can understand if a given image is an orange or not, a binary classification problem. However, the number of orange images is not enough. Therefore I use a Siamese network that can learn a feature vector to map similar images (oranges) closer (1) and differents far away (0).

Therefore to train my siamese network, I give a pair of 2 images at each time. For example:

apple1 and orange1 
apple1 and orange2 
apple1 and orange3 
orange1 and orange1 
orange2 and orange3
orange1 and orange3 
...

And suppose that know a siamese network is trained and has high performance. But how can I use it in inference? What should I do to obtain my binary orange or not output? Do I need to use this network as a feature extractor? I was confused at that point.

Mas A
  • 175
  • 9

1 Answers1

0

The purpose of Siamese and triplet networks is to produce a vector representation of the input. The vector representation can be used later for other tasks, such as classification, but you’ll need a model to do so.

To make such a model, train a binary network where the features are the vectors obtained from the Siamese network and the labels are the class labels. This is like any other classifier where you have a feature extraction module, except here that module is also a neural network

However, because the vectors are designed to encode class information, and have small dimension, these classifiers probably don’t need to be very ornate!

Sycorax
  • 76,417
  • 20
  • 189
  • 313
  • The idea is not very clear to me. The vector representation is created to deal with the differences between orange, apple, banana, etc. The network does not reflect the individual characteristics of each class but instead, it creates a feature vector reflecting their differences, right? How can I use it for classification purposes? For the apple or not-apple problem, Do I give the input vector to the trained siamese network and then get the feature vector. Then I use it to train my binary new classification network? – Mas A Jan 17 '22 at 10:57
  • Train a binary network where the features are the vectors obtained from the Siamese network and the labels are the class labels. This is like any other classifier where you have a feature extraction module, except here that module is also a neural network. – Sycorax Jan 17 '22 at 14:20
  • At that time, suppose that I have 1x data from class A, and 99x data from class not-A. I got the embeddings from the siamese network for each of my 100x data and I use them to train a binary classification model. How this helps with the unbalanced datasets. At the end of the day, my classification model only sees 1x embeddings from class A, and 99x from the other class. – Mas A Jan 17 '22 at 14:25
  • Training a classifier using vectors obtained from Siamese networks doesn’t solve the class imbalance problem (to the extent imbalance really is a problem at all), but that’s also not a question you asked. You can ask a new question about it. – Sycorax Jan 17 '22 at 14:26