Suppose you have a random vector $\boldsymbol X=(X_1,X_2,\ldots,X_p)$.
Consider the linear regression models
$$X_1=X_{1\cdot 34\ldots p}+ \varepsilon_{1\cdot 34\ldots p}$$
and
$$X_2=X_{2\cdot 34\ldots p}+ \varepsilon_{2\cdot 34\ldots p}$$
Here $X_{i\cdot 34\ldots p}$ is the part of $X_i$ explained by $(X_3,\ldots,X_p)$ and $\varepsilon_{i\cdot 34\ldots p}$ is the unexplained error, $i=1,2$. Unknown parameters in $X_{i\cdot 34\ldots p}$ are found subject to minimization of $E(\varepsilon_{i\cdot 34\ldots p}^2)$.
If $e_{i.34\ldots p}$ are the residuals corresponding to the models above, the (population) partial correlation between $X_1$ and $X_2$, eliminating the linear effect of $X_3,\ldots,X_p$, is defined to be
$$\rho_{12\cdot 34\ldots p}=\operatorname{Corr}(e_{1.34\ldots p},e_{2.34\ldots p}) $$
For some distributions like the multivariate normal, this correlation coincides with the correlation between $X_1$ and $X_2$, conditioned on $X_3,\ldots,X_p$. In fact, your linked document does assume multivariate normality of $\boldsymbol X$. To quote Wikipedia, "The partial correlation coincides with the conditional correlation if the random variables are jointly distributed as the multivariate normal, other elliptical, multivariate hypergeometric, multivariate negative hypergeometric, multinomial or Dirichlet distribution, but not in general otherwise."
Hence in these specific situations only, one can say
$$
\rho_{12\cdot 34\ldots p}=\rho_{(X_1,X_2)\mid X_3,\ldots,X_p}=\operatorname{Corr}((X_1,X_2) \mid X_3,\ldots,X_p) \tag{$\star$}
$$
When $\boldsymbol X$ is multivariate normal, this conditional correlation (i.e. the conditional covariance and the conditional variances) does not depend on $X_3,\ldots,X_p$. Note that the conditional distribution of $(X_1,X_2)$ given $X_3,\ldots,X_p$ is bivariate normal. And you can see here that the dispersion matrix of this conditional distribution is free of $X_3,\ldots,X_p$ (hence non-random). Hence in this case, there is no ambiguity in the formula in your post.
The partial correlation is of course a scalar by definition. In fact, it is entirely based on the entries of the correlation matrix (or equivalently, the dispersion matrix) of $\boldsymbol X$.
Specifically, if $R=((\rho_{ij}))$ is the correlation matrix of $\boldsymbol X$, one can show that
$$\rho_{12\cdot 34\ldots p}=-\frac{R_{12}}{\sqrt{R_{11}}\sqrt{R_{22}}}\,,$$
where $R_{ij}$ is the cofactor of $\rho_{ij}$.
For $p=3$ (say), this reduces to
$$\rho_{12\cdot 3}=\frac{\rho_{12}-\rho_{13}\rho_{23}}{\sqrt{1-\rho_{13}^2}\sqrt{1-\rho_{23}^2}}$$
Related: Derivation of the formula for partial correlation coefficient of second order.
