I'm assuming this is pretty commonly discussed but haven't found it yet.
For example, if you start with a binary (0-1 binomial), conjugate prior is beta distribution. Beta distribution has some un-named conjugate prior. You can take the conjugate prior of that.
Is there anything interesting coming of this? Anything useful? Looking for links and references mostly.
No, there is no end to this process, and you always end up with a distribution having one additional parameter.
Copied from my answer here:
Let's say that you have a distribution $F$ in the exponential family with density \begin{align} \newcommand{\mbx}{\mathbf x} \newcommand{\btheta}{\boldsymbol{\theta}} f(\mbx \mid \btheta) &= \exp\bigl(\eta(\btheta) \cdot T(\mbx) - g(\btheta) + h(\mbx)\bigr) \end{align}
Given independent realizations $\{x_1, x_2, \dotsc, x_n\}$ of $F$ (with unknown parameter $\theta$), then the distribution over $\theta$, $F'$, is the conjugate prior of $F$. The density of $F'$ is \begin{align} f(\btheta \mid \boldsymbol\phi) = L(\btheta \mid \mbx_1, \dotsc, \mbx_n) &= f(\mbx_1, \dotsc, \mbx_n \mid \btheta) \\\\ &\propto \prod_i f(\mbx_i\mid \btheta) \\\\ &= \textstyle\prod_i\exp\Bigl(\eta(\btheta) \cdot \textstyle T\left(\mbx_i\right) - g(\btheta) + h(\mbx_i)\Bigr) \\\\ &\propto \textstyle\prod_i\exp\Bigl(\eta(\btheta) \cdot \textstyle T\left(\mbx_i\right) - g(\btheta)\Bigr) \\\\ &= \textstyle\exp\Bigl(\eta(\btheta) \cdot \bigl(\textstyle\sum_iT\left(\mbx_i\right)\bigr) - ng(\btheta)\Bigr) \\\\ &= \exp\bigl(\eta'(\boldsymbol \phi) \cdot T'(\btheta)\bigr) \end{align} where \begin{align} \eta'(\boldsymbol\phi) &= \begin{bmatrix} \sum_iT_1(\mbx_i) \\\\ \vdots \\\\ \sum_iT_k(\mbx_i) \\\\ -\sum_i1 \end{bmatrix} & T'(\btheta) &= \begin{bmatrix} \eta_1(\btheta) \\\\ \vdots \\\\ \eta_k(\btheta) \\\\ g(\btheta) \end{bmatrix}. \end{align} Thus, $F'$ is also in the exponential family ($T'$ replaced $\eta$ and $\eta'$ replaced $T$ since this distribution is over $\theta$ the parameter of the distribution over $x$.)
Interestingly, $\boldsymbol\phi$ has exactly one more parameter than $\btheta$ except in the rare case where natural parameter $\phi_{k+1}$ is redundant, but such a distribution would be very weird (it would mean that the number of observations $\mbx$, that is, $n$, tells you nothing about $\btheta$.)
So, to answer your question, with each conjugate prior you get exactly one more hyperparameter.
