Let's say you've got a neural network which takes in a vector of real numbers as input. Additionally, let's say you're uncertain about the values of some components of the vector, and your level of certainty is a number between $0$ and $1$. It's a promising idea to map the $(\text{certainty}, y)$ values to $(u,v)$ values in such a way that when $\text{certainty} = 0$, $f(\text{certainty}, y)$ doesn't change with $y$.
Two ideas:
Idea 1: $f(\text{certainty}, y) = (\text{certainty} \times\cos(y),\text{certainty}\times\sin(y))$. The closer $\text{certainty}$ is to $0$, the closer the points in the image will be to each other. It seems to me that this idea works best when the space of $y$ values is topologically a circle.
Idea 2: $f(\text{certainty}, y) = (\text{certainty}, \text{certainty} \times y)$. Again, degenerate when $certainty = 0$, as we want it to be. Works best when the space of $y$ values is topologically the same as $\mathbb R$.
- Has this been tried? I would be very surprised if it hasn't.
- Is it a good way to deal with missing values in time series analysis?
- Any relationship to dropout?
