(Much of this answer is copied from another answer of mine here; I do so without further specific attribution of quoted or paraphrased material.)
A good way to proceed here is to conduct your analysis within the imprecise probability framework (see esp. Walley 1991, Walley 2000). In this framework the prior belief is represented by a set of probability distributions, and this leads to a corresponding set of posterior distributions. Below I will show you how this works for your specific example of binomial data.
Before getting to the implementation of this method, the first thing to note here is that the most extreme form of ignorance possible would be to take an imprecise prior distribution composed of the set of all possible priors on the parameter range. This would lead to an imprecise posterior with is the set of all possible posteriors, so your inference would then be vacuous. That is, total ignorance going in leads to total ignorance coming out. Consequently, if we want a useful inference at all, we must frame our "ignorance" in some way that restricts the imprecise prior to a reasonable range of priors from which a small set of posteriors can be formed. This can be done by allowing the prior expectation of the parameter to vary over all possible values in its range, but restricting the prior variance either to a single value or a small range to get a useful posterior inference.
Application to the binomial model: Suppose we observe data $X_1,...,X_n | \theta \sim \text{IID Bern}(\theta)$ where $\theta$ is the unknown parameter of interest. Usually we would use a beta density as the prior (both the Jeffrey's prior and reference prior are of this form). We can specify this form of prior density in terms of the prior mean $\mu$ and another parameter $\kappa > 1$ as:
$$\begin{equation} \begin{aligned}
\pi_0(\theta | \mu, \kappa) = \text{Beta}(\theta | \mu, \kappa) = \text{Beta} \Big( \theta \Big| \alpha = \mu (\kappa - 1), \beta = (1-\mu) (\kappa - 1) \Big).
\end{aligned} \end{equation}$$
(This form gives prior moments $\mathbb{E}(\theta) = \mu$ and $\mathbb{V}(\theta) = \mu(1-\mu) / \kappa$.) Now, in an imprecise model we could set the prior to consist of the set of all these prior distributions over all possible expected values, but with the other parameter fixed to control the precision over the range of mean values. For example, we might use the set of priors:
$$\mathscr{P}_0
\equiv \Big\{ \text{Beta}(\mu, \kappa) \Big| 0 \leqslant \mu \leqslant 1 \Big\}. \quad \quad \quad \quad \quad$$
Suppose we observe $s = \sum_{i=1}^n x_i$ positive indicators in the data. Then, using the updating rule for the Bernoulli-beta model, the corresponding posterior set is:
$$\mathscr{P}_\mathbf{x} = \Big\{ \text{Beta}\Big( \tfrac{s + \mu(\kappa-1)}{n + \kappa -1}, n+\kappa \Big) \Big| 0 \leqslant \mu \leqslant 1 \Big\}.$$
The range of possible values for the posterior expectation is:
$$\frac{s}{n + \kappa-1} \leqslant \mathbb{E}(\theta | \mathbb{x}) \leqslant \frac{s + \kappa-1}{n + \kappa-1}.$$
What is important here is that even though we started with a model that was "uninformative" with respect to the expected value of the parameter (the prior expectation ranged over all possible values), we nonetheless end up with posterior inferences that are informative with respect to the posterior expectation of the parameter (they now range over a narrower set of values). As $n \rightarrow \infty$ this range of values is squeezed down to a single point, which is the true value of $\theta$.
