Discrete Inversion (Generating Random Sequences XII)

July 30, 2019

While this sounds something like a shameful family secret, discrete inversion is only the finite-valued variation on the method of inversion for the generation of random numbers with a given distribution (as I’ve discussed quite a while ago here). The case we’ll consider here is a random variable with few possible outcomes, each with different odds

Read the rest of this entry »


The Zero Frequency Problem (Part I)

September 23, 2014

In many occasions, we have to estimate a probability distribution from observations because we do not know the probability and we do not have enough a priori knowledge to infer it. An example is text analysis. We may want to know the probability distribution of letters conditional to the few preceding letters. Say, what is the probability of having ‘a’ after ‘prob’? To know the probability, we start by estimating frequencies using a large set of observations—say, the whole Project Gutenberg ASCII-coded English corpus. So we scan the corpus, count frequencies, and observe that, despite the size of the corpus, some letter combinations weren’t observed. Should the corresponding probability be zero?

Read the rest of this entry »


Suggested Reading: The Drunkard’s Walk: How Randomness Rules our Lives

August 29, 2009

Leonard Mlodinow — The Drunkard’s Walk: How Randomness Rules our Lives — Vintage, 2008, 252 pp. ISBN 978-0-307-27517-2

(Buy at Amazon.com)

(Buy at Amazon.com)

For those who are interested in (but not already familiar with) probabilities and statistics, I would most certainly recommand this book. Mlodinow presents the basic concepts of probability and statistics by concrete everyday examples—and visually, whenever possibile, rather than through classical mathematical notation. He discusses psychological factors that make us so bad at estimating probabilities and understanding statistics. The concepts he presents are deep but the style is fluid and makes The Drunkard’s Walk an easy read.

[…]the human understanding, once it has adopted an opinion, collects any instances that confirms it, and though the contrary instances may be more numerous and more weighty, it either does not notice them or else rejects them, in order that this opinion will remain unshaken.

Francis Bacon, 1620