4

In this paper by Gabrielsson, Nelson, et al. the authors "present a differentiable topology layer that can, among other things, construct a loss on the output of a deep generative network to incorporate topological priors".

I only have a basic understanding of topology and it's causing me some confusion. To summarize the context for my question, the authors state this in the introduction (emphasis is my own):

In many deep learning settings there is a natural topological perspective. This is true both for images and for 3D data such as point clouds or voxel spaces. In fact, many of the failure cases of generative models are topological in nature [32, 18]. We show how topological priors can be used to improve such models.

For example, later on in 3.1, the authors describe an example on MNIST:

We show how one can encourage the formation of lines, clusters, or holes in a set of points using geometric filtrations

As far as I can tell, the "geometric filtrations" are applied as part of the loss function, and they express the kind of topological prior described in the first quote.

So my question: is the topological prior learned by the topological layer, or is the prior is imposed by the human who's training the network?

To put it in terms of the example, does the topological layer learn to "encourage the formation of lines, clusters, or holes," or is that prior information supplied by the human by properly specifying the regularizing loss function term?

kdbanman
  • 560
  • 1
  • 5
  • 11

2 Answers2

3

The topological prior is provided by the human. In the paper, they have an expression for the loss in equation 2: E(p, q, i_0; PD). The topological prior is essentially determined by 4 parameters.

p and q are the exponents to the two terms in the loss. The first term, with exponent p, relates to the length of the bars in the persistent homology barcode diagram. The second term, with exponent q, relates to the position of the centre of the bar. Clearly, knowing the centre and length of the bar determines its beginning and end points (ie. birth and death filtrations).

The i_0 term is an integer relates to the summation in equation 2 - I think it can be thought of as determining the number of the relevant topological features you want. The PD term is also determined by an integer and relates to the dimensionality of the persistence term we're talking about.

James_Clough
  • 46
  • 2
  • This is exactly the kind of information I was looking for - thank you! – kdbanman Jun 14 '19 at 05:33
  • I have a follow up question that is hopefully not too large for just a comment: How generic of a prior can one provide with that loss term? That is, are there some kinds of shapes/objects for which a prior cannot be described with just those 4 parameters? – kdbanman Jun 14 '19 at 05:42
  • There are definitely some kinds of shapes that you could not uniquely specify in this framework. Persistent homology only really measures the Betti numbers of a shape (or set of shapes). So shapes that are topologically identical can't be distinguished in this framework (ie. a circle and a square are the same thing). Furthermore the Betti numbers don't uniquely determine the topology of an object either. For example, two blobs each with one hole in have the Betti numbers 2, 2. But a blob with two holes plus a blob with zero holes would also have the Betti numbers 2,2. – James_Clough Jul 26 '19 at 14:53
2
from topologylayer.nn import AlphaLayer, BarcodePolyFeature
import torch, numpy as np, matplotlib.pyplot as plt

# random pointcloud
np.random.seed(0)
data = np.random.rand(100, 2)

# optimization to increase size of holes
layer = AlphaLayer(maxdim=1)
x = torch.autograd.Variable(torch.tensor(data).type(torch.float), requires_grad=True)
f1 = BarcodePolyFeature(1,2,0)
optimizer = torch.optim.Adam([x], lr=1e-2)
for i in range(100):
    optimizer.zero_grad()
    loss = -f1(layer(x))
    loss.backward()
    optimizer.step()

The above code from their repo (https://github.com/bruel-gabrielsson/TopologyLayer) would to me suggest the topology is chosen by the human

Aidan Connelly
  • 91
  • 8