0

I have recently stumbled across a paper about learning arbitrary dynamical systems in a spiking neural network. The paper assumes an underlying dynamical system of the form $\dot{x}=f(x)+c(t)$ where $c$ is just a random input and $f$ can be a non-linear function.

The paper also states that the dynamical system must be well-behaved. I could not find anything understandable on the web. Can somebody explain this to me?

enter image description here

user47808
  • 1
  • 1

1 Answers1

1

I'm not sure there's a concrete definition. I find it mildly amusing that it's in a neural net paper, because it's one of those common sense things that can't easily be encoded for AI, with or without neural nets.

Mostly I'd call a system "well behaved" if I could get it to do what I need it to do without pulling my hair out. But if I had to give it criteria I'd probably consider a dynamic system to be well behaved if it is stable or easily stabilizable, if it doesn't have any strong resonances that can't easily be damped, and if it isn't too "stiff"; i.e. if the Jacobian of $f(x)$ doesn't change greatly as x is varied (or even more formally, if the Hessian of $f(x)$ isn't too big).

TimWescott
  • 7,510
  • 1
  • 6
  • 17
  • I was wondering if I could describe a temporal version of XOR in the form $\dot{x}=f(x)+c(t)$ where $x$ will be either a positive or negative response depending on the input $c$, which would be just two "bumps" indicating true or false. If this would be the case, one could learn XOR using the approach in the paper, but I don't think it is a dynamical system at all, right? – user47808 Feb 21 '20 at 14:57
  • That's practically a question in itself -- an ideal XOR is memoryless, it doesn't fit the author's form (the more general form is $\dot x = f(x, u, t), y = h(x, u, t)$, where u is an input vector), and from the perspective of "no sudden changes in the Jacobian" it kinda flunks out. – TimWescott Feb 21 '20 at 15:01
  • What if I were to smooth it according to the image I have posted? The upper image is the input and the lower image is the desired response. – user47808 Feb 21 '20 at 15:29
  • Could you incorporate your description of the system into your question? The short story, though, is that the equation you give is for an autonomous stochastic system, and your XOR example, because it is driven by external inputs, is not autonomous. – TimWescott Feb 21 '20 at 20:05
  • Does autonomous in this sense mean that the change of the output variable only depends non-linearly on the output variable itself, but not on the input? If the input would also be part of the parameters for $f$ then this would be possible, right? But then it feels like $f$ could be any (recurrent) neural network and the presented work could learn any mapping. – user47808 Feb 24 '20 at 08:17
  • "Autonomous" means that the system acts by itself, without input. As expressed, the model you give is a bit ambiguous (is it non-autonomous with a random input, or is it autonomous with an internal random process?), but compared to the *distinctly* non-autonomous system you want to analyze, I'm calling it autonomous. – TimWescott Feb 24 '20 at 15:22