Discrete Pareto Distribution vs Zipf Distribution and Power Law vs Zipf Law

Question

I need to get a simple, but clear idea of Discrete Pareto Distribution vs Zipf Distribution and Power Law vs Zipf Law. (Are they similar/ how they relate to each other.) Wikipedia definitions do not address my issue. If graphical explanation is possible, would be clearer.

Answer 1

[In relation to the relationship between the Zipf and the zeta distributions, the Wikipedia definitions absolutely address your main question. It's possible that you didn't understand what was there.]

I'm going to use Wikipedia's definitions of these distributions; its references are explicit, so we at least know where they're coming form.

1. Let us start with a zeta distribution. This is a pmf proportional to $x^{-s}$, $s=1,2,...$. We see that this is akin to the Pareto in that it has a density of essentially the same power-law form. (Why power-law? Because it is in the form of a constant times a power of $x$.)

   On this basis it's a candidate for a discrete equivalent of a Pareto. On the other hand, the correspondence is less direct if we're focused on the survival function of the Pareto $1-F(x) = S(x) \propto x^{-\alpha}$; that's also in power-law form but the survival function of the zeta is not (though it's an increasingly good approximation to the tail).

2. Let us then discuss the Zipf. It's of the same form as the zeta, but the difference is it's over a finite range, not a semi-infinite range; that is, it only assigns probability to $x=1,2,...,N$ and this alters the normalizing constant on the pmf. Then for a given index (negative-power), $s$, the probability associated with each outcome up to $N$ must be higher (because there's no probability associated with any outcomes $>N$).

   It's a (right-) truncated zeta and would be a candidate for the discrete equivalent of a right-truncated Pareto.

   In terms of pictures/graphs, the wikipedia articles already give log-log graphs of the pmfs for both zeta and Zipf and the effect of the truncation is obvious in those graphs; I see little point in reproducing them; if they didn't help you there, they could be no more help here. However, I will put in a plot of an example of an unlogged pmf for each:

   
   The left plot shows the first 16 probabilities in a zeta(2) distribution; in fact they continue off to the right without limit. The right plot is a corresponding Zipf truncated at 10; the open circles are the values of the zeta to the left. You can see on the first probability or two that the open circles are lower (because the filled-circle Zipf probabilities are pushed up due to the omission of the right tail).

3. Now let us consider what could be meant by a "discrete Pareto". In this case we must take some property of the Pareto which we try to preserve when we move to a discrete distribution.

   We saw an example in the zeta, where it was the basic form of the pdf as proportional to a (negative) power of the argument that was preserved in moving to a pmf. However, many people focus on the survival function when defining what makes for a power-law and in that case defining a discrete Pareto directly in terms of $S(x)\propto x^{-\alpha}\,,\: x=1,2,3,...$ would give a different pmf from the zeta.

   Note further that the zeta is always defined on the positive integers, while the Pareto has its left-limit as a parameter, so there's a whole class of potential discrete Paretos that have different left-limits.

   Indeed, rather than preserve the functional form in either the pdf or the survival function one might proceed by directly associating a section of the density in $(u,u+1]$ with the probability mass at $\lfloor u+1\rfloor$ -- i.e. by specifying how to 'round' the probability in the interval (in a general sense), you will get yet another discrete Pareto (or rather, an entire class of them).

   All of these (and likely more besides) are entirely valid candidates for being called a "discrete Pareto". It's up to the user to figure out what properties they need in such an object and choose appropriately.

---

On the use of the word "Law":

"Law" in science is a pretty general thing (usually referring to some specific hypothesis, but sometimes to a commonly used model or observed regularity), but in relation to probability distributions, it's more specifically a reference to a functional form (or class of functional forms) for the distribution (whether expressed as a pdf/pmf, as a cdf, as a survival function or in some other way). For example you may see reference to "the normal law" or "the Poisson law", and generally the intent there is to refer specifically to the distribution.

So which are we dealing with here?

Zipf's law and the Pareto principle are both "scientific laws" in the "observed regular behavior" sense (at least up to the usual approximation involved in scientific models in general):

https://en.wikipedia.org/wiki/Empirical_statistical_laws#Examples

But at the same time, those observed laws lead directly to the distributional models, so they're also sometimes a direct reference to the distributional form in the sense of a probability law.