Suppose I made a random number generator that's supposed to return a number 1-10, but I made it always return 4, and didn't tell you.
How would you know with 100% certainty it wasn't random?
Even if you generated 4 so many times that the odds of such are less than atoms in the observable universe, would you ever be able to actually know?
An obligatory Dilbert comic:
If you have a random number generator that at random generates "4" with probability $p$, the probability of observing it $n$ times in a row is $p^n$, assuming that the draws are independent. Notice that the more times you observe "4", the smaller the probability gets, but it will never go down to zero. There will always be a slight chance of observing one more "4" in a row. So you can never be 100% certain.
how would you know with 100% certainty it wasn't random?
You wouldn't. This gets into why there are many different probability interpretations.
All answers seem to focus on the nature of random while the question concerns truly the nature of knowledge. What is it to know something?
You seem to implicitly allude to an unattainable and impractical notion of knowledge, some God-like insight into matters. We're human, for us knowledge is not an absolute state of clarity and vision. We know that it's Friday today, that milk is white and the Winter is coming etc.
Unfortunately the subject is outside the field of statistics. Hence, my terse answer: if your RNG keeps returning 4, then you will know that it's not random after a few trials.
You and I know that the Sun will rise tomorrow. If someone doesn't then they should see a therapist to deal with anxiety, maybe take some pills etc. The point is that it's not the subject of statistics in this case.
Checking for the randomness can be viewed as "discover patterns".
In your random generator example, we can calculate the probability of certain events (for example consecutive 4 for 10 times) and conducting experiment to verify our assumption.
For example, we know certain thing is very less likely to happen and it is happening all the time (say hitting the jackpot all the time). Then we are suspecting the problem of the random generator.
Of course we cannot sure, but we can say, it is highly likely (say 99.9999999%) the data is not from random. And In real world we dot not need to have a black or white answer, we just simply do not trust the random generator and do not use it.
In addition to the correct point that @Tim and @HaitaoDu make, there may be additional reasons to distrust a supposed random number generator that "always return[s] 4".
Suppose I know that the numbers are produced by a deterministic algorithm running in a computer (not, say, incorporating inputs from a quantum mechanical device), even if I don't know that the algorithm is actually return 4. Every deterministic pseudorandom number generating algorithm has a finite period--i.e. after outputting a certain quantity of "random" numbers, the sequence of returned numbers must repeat. So if I keep drawing more and more numbers from your "random number" program, at some point the quantity of numbers that have been produced will be close to the largest period that any known random number generating algorithm would produce. The fact that I have only seen the number 4 through all of those draws, and am now close to exceeding any reasonably possible period--or even close to half of such a period--would be evidence that what is producing the numbers is not a legitimate attempt at a pseudorandom number generating algorithm. Granted, it may take a lot of time and computing power to get to this point. (The largest pseudorandom number algorithm period that I have heard of is that of a standard Mersenne Twister, $2^{ 219937−1}$.)
Furthermore, any non-horrible random number generating algorithm is supposed to try to approximate a uniform distribution with independent draws--of 1 through 10, in this case. There have been many algorithms proposed that are not good, and even very bad, because they failed to come sufficiently close to this goal. I am thinking of algorithms that are not truly horrible. return 4 is beyond truly horrible, and that will be apparent after many draws: if a non-horrible pseudorandom number generator has produced enough numbers that a moderately significant percentage of its period has been used up, those numbers can't all be the same, because even a bad, but not truly horrible algorithm would superficially seem to return numbers that are independent and uniformly distributed. A very long sequence of 4's that take up a significant percentage of the possible period of a psedurorandom number generating algorithm does not have that appearance, even superficially, so the algorithm that produces that long sequence of 4's must be truly horrible--at the very least.
