6

I am currently researching in the facerecognition field.

And I can not understand how the facenet algorithm handels a new image

They use an euclidean space for image representation. Which means that the elements in this space represent the images (faces). To create this space they use triplet's of images (1 Anchor, 1 Image of the same person (ancher) and a foreign image). These triplets are generated via a data-mining method.

In the end result images of the same person are close together and seperated via a margin. Which means that all images outside this margin belong to a foreign person. So similarity is measured by distance.

If we now want to recognize or verify a new image/face, what exactly is done?

It seems we will need again triplets, so a single image is not even possible? And what happens then? This triplet must be compared somehow again to all existing triplets to find a place for this new image in the euclidean space? If it is whithin a margin that this new image got verified, otherwise its unkown?

Sycorax
  • 76,417
  • 20
  • 189
  • 313
Roman
  • 163
  • 6
  • 1
    The basic idea is to compute the feature vector for your image as provided by the prediction function of the neural net. If you have similar information for a reference set of images, just find the image(s) in that reference set that are closest to your image regarding these features (nearest neighbour). – Michael M Oct 21 '16 at 05:50
  • without reading the paper, wouldn't recognition just be, compute similarity measure and if it's further away than the margin then it's a foreign image? Is there anything wrong with that solution? – Charlie Parker May 26 '19 at 20:16

1 Answers1

4

After the network is trained, we can throw away the loss layer. Actually the facenet (and many other networks for facial recognition) is trained for extracting features, that is to represent the image by a fixed length vector (embedding).

The triplet loss basically says, the distance between feature vectors of the same person should be small, and the distance between different persons should be large.

enter image description here

After training, for each given image, we take the output of the second last layer as its feature vector. Thereafter we can do verification (to tell whether two images are of the same person) or clustering based on the features and some distance function (e.g. Euclidean distance).

dontloo
  • 13,692
  • 7
  • 51
  • 80
  • So basicly verification can be only done with images, that are part of the dataset. It will not work with images that are not in the dataset? – Roman Oct 25 '16 at 10:01
  • @Roman no, after the network is trained, we can feed any image $x$ to the network and get the output of the L2 layer $v$ as its feature vector. so in order to compare the similarity of two images $x_1$ and $x_2$ we can just compare $v_1$ and $v_2$. we don't need the triplet layer anymore after it is trained. – dontloo Oct 25 '16 at 10:22
  • Ah okay, now I have an idea what happens. Right at beginning all images are turned to vectors with values by using the NN to extract features. And the triplets are used to organize the given dataset, so the distances are adjusted. – Roman Oct 25 '16 at 15:14
  • 1
    @Roman yes, the triplets are only used in training to select some proper samples for the loss function. you can take a look at this paper http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2014_5416.pdf, it's one of the first that uses distance between pairs as part of the loss for face verification tasks. – dontloo Oct 25 '16 at 15:31
  • 1
    @dontloo The paper you linked to does not exist. Can you please re-link it, or perhaps provide the name of the paper? – Ankur Roy Chowdhury May 23 '18 at 13:40
  • @AnkurRoyChowdhury hi I think it's this one https://arxiv.org/abs/1406.4773 Deep Learning Face Representation by Joint Identification-Verification – dontloo May 23 '18 at 13:43