Powers of 2

by Nathaniel Virgo

The relationship between probability and information is interesting and fun.  The table below is a work in progress, but I think it’s kind of cool already.  The idea is to compare information quantities with the probabilities of unlikely events.  For instance, if I flipped a coin for every bit on my hard drive and they all came up heads, it would be pretty unlikely.  But what else would be that improbable?

The leftmost column has quantities of information, ranging from one bit upwards.  The next column is the same thing, but in Joules per Kelvin, the historical physical unit of entropy.  Then there’s an illustrative example from the world of computing.  The next column is a probability.  This can be thought of as the probability of flipping heads on N coins, where N is the number of bits, though if you’ve studied information theory you’ll know the connection between information and probability is somewhat deeper than that.  Finally, there’s a bunch of examples of other things that have a similar probability.

It’d be pretty cool to make this table more complete.  I’d quite like to have every power of two on the left-most column, all the way from one bit up to around 10^{26}, and even beyond.  The main problem is thinking of interesting rare events with the appropriate probability – so if you think of any, let me know in the comments and I’ll see if I can fit them into a later version of this table.  (You don’t have to do the maths, though if you can it would help.)

Bits Physical units (JK^{-1}) Computer storage capacity Probability Event with approximately the same probability
1 bit 9.5\times 10^{-24} 1 bit 1/2 flip heads on a coin.
2 bits 1.9\times 10^{-23} 1/4 flip heads on 2 coins, or roll four on a 4-sided die
8 bits
(1 byte)
7.6\times 10^{-23} One ASCII character 1/256 flip heads on 8 coins (1/256), roll three sixes on three dice (1/216), or die in an accident* (1/215)
16 bits
(2 bytes)
1.5\times 10^{-22} A 16-bit integer 1/65536 die by being bitten by a mammal other than a dog* (1/54794), match 5 numbers in the UK national lottery, which wins about £1,500 (1/55491), or be dealt a straight flush (1/64974)
32 bits
(4 bytes)
3\times 10^{-22} A 32-bit floating point number 1/4,294,967,296 get dealt two straight flushes in a row** (1/4,198,262,436), be the next person in the world to blink (1/6,800,000,000)
64 bits
(8 bytes)
6\times 10^{-22} A 64-bit double-precision number \approx 5\times10^{-20} You win the UK national lottery and your house gets hit by a meteorite in the same week (\approx 10^{13} \times 1.4 \times 10^{7} / 7 = 5\times 10^{-20})
160 bits 1.6\times 10^{-21} An SHA-1 hash \approx 7\times10^{-49} An SHA-1 collision occurs between a given pair of files; or get struck by lightning on your birthday 6 years in a row***; or get dealt 10 straight flushes in a row.
8192 bits
(1 kilobyte)
8.5\times 10^{-20} 1/16th of a Sinclair ZX’s Spectrum’s RAM  \approx 10^{-2466} Win the UK national lottery every week for six and a half years \approx 10^{-2415}****
10^{12} bits
(125 gigabytes)
9.5\times 10^{-9} Small-ish modern hard drive \approx 10^{-10^{11}} A fully functioning E. coli cell assembles by chance in an inert mixture of chemical elements with no energy supply*****
10^{123} bits (so big there isn’t a word for it) 10^{100} If this were ever to be built it would take up a sizeable fraction of the visible Universe \approx 10^{-10^{123}} Roger Penrose’s (probably rather silly) estimate of the probability of a universe being created with similar physical properties to ours.5
10^{124} bits 10^{101} The information capacity of our universe – this would take up the entire visible universe.  It’s as big as you can get, probably. \approx 10^{-10^{124}} The probability of finding yourself in our exact universe, as opposed to one of the many others that are compatible with the laws of physics as we currently understand them.6

* cause of death probabilities are based on the 2009 totals from this UK data set, and should be taken with a large pinch of salt.  So should everything else on this page, in fact!

** more accurately, this is the probability of drawing two straight flushes if you’re dealt exactly two hands, which is less probable than just getting two in a row if you were playing poker all night (though that’s still pretty astronomically unlikely).

*** this site estimates the odds of being struck by lightning to be roughly 1 in 280000 per year.  I divided this by 365 and raised to the power of 6.  This is a bit of a sketchy calculation because there are quite a few things I didn’t take into account, including 1. you’re quite likely to die if you get struck by lightning, reducing your chances of being struck again (though my grandfather said he survived being hit three times); 2. some people are more susceptible to being struck than others (my Grandad was a farmer, hence he was often the highest point in a field); and 3. storms are more likely at some times of the year than others, which increases the probability of this happening.  I’m hoping these all roughly cancel each other out.

**** of course, 10^{-2415} is actually 51 times smaller than 10^{-2466} — but these numbers are so incredibly small that for many purposes even a few dozen orders of magnitude can be neglected.

***** life didn’t actually start this way, of course – it probably started with something quite a lot simpler than a cell, and there most certainly was an energy supply – so the actual probability of life forming under the conditions of the early Earth was almost certainly a lot higher.  I reckon it was probably pretty much 1.  The probability of spontaneous formation of an E. coli cell under these unlikely circumstances was estimated as \approx 10^{-10^{11}} by Morowitz, 1968.

5 Penrose’s calculation would make much more sense if it was supposed to be the probability of a Universe like ours forming due to a thermal fluctuation in an equilibrium Universe that lacks a low-entropy past.  The probability is so small that it hardly makes a difference whether you put a 10 or a 2 at the beginning.

6 a reasonable limit on the information capacity of the Universe is the entropy of the de Sitter horizon, which is an event horizon that surrounds the observable universe like a giant inside-out black hole.  Blindly throwing the area of the de Sitter horizon into the black hole entropy formula gives an information capacity of the order 10^{124} bits, and that’s what I’m taking as the maximum number of bits that could fit into the Universe.  (But note that this number is actually increasing – it must do so more-or-less linearly as the universe expands exponentially due to “dark energy”.  It’d be interesting to know the rate of its increase, but I haven’t worked that out yet.)

About these ads

18 Responses to “Powers of 2”

  1. I think that this is a decent one: http://www.creationofuniverse.com/html/equilibrium03.html

    Penrose’s estimate of our universe having the physical characteristics it does = 10^10^123 : 1

    (ignore the rest of the page it is rubbish, just gives the figure prominance as I can’t find the paper yet)

    • Blimey, Roger Penrose doesn’t half think some odd things. I’ve added it to the table anyway, along with an even smaller one – the probability of our exact universe, rather than merely a similar one.

    • (the calculation is in the book ‘The Emperor’s New Mind’ – I looked it up briefly to check what he was actually calculating, since most of what I could find about on the internet was misquotations by creationists.)

  2. Am not sure but you may need to change a minus sign on the last two entries in the second column

  3. The internet also says the odds of a meteor landing on your house are 182,138,880,000,000 to 1

    • I don’t believe that one for a second. Houses /have/ been struck by meteorites in the past, and there are a lot less than 10^{14} houses in the world.

      • The Earth’s surface area is about 5\times 10^{14}\, m, so if your house has a floor area of 60 m, the odds of a given meteor hitting it are about 1 in 2.5\times 10^{12}. Wikipedia (http://en.wikipedia.org/wiki/Impact_event) says about 500 meteorites hit the ground every year (other sites give much higher estimates), and your house will probably be around for a century or so, which makes the odds of one of those meteorites hitting it in that time about 1 in 50,000,000 – not that much less likely than winning the lottery jackpot (1 in 14,000,000).

      • On the other hand, I I’m being pedantic, the probability of a *meteor* hitting your house is zero, since it’s only called a meteor if it burns up in the atmosphere, otherwise it’s a meteorite.

      • Oops, I have to correct myself, I made a mistake due to an integer rounding error. If your house has a floor area of 50 square metres, the odds of a given meteorite hitting it are about 1 in 10^{13}. It makes the overall odds of your house being hit about 1 in 200,000,000, not 1 in 50,000,000.

      • I’ve added an extra entry for the probability of your house being struck by a meteorite and winning the lottery in the same week, corresponding to about 64 bits.

  4. Nice comparisons :) what could be an approximation of 160 bits (the probability of sha1 collision)?

  5. Out of interest is the E coli chanceever or per second or in some other unit of cell building?

    • Good question. I suppose it’s actually the chance that, if you were to look at the system at any given time, you would find a cell there. If you wanted to work out the probability per unit time you might have to multiply or divide it by 1000000 or so — but the figure is so astronomically small that this would barely make a dfference (10^{-10^{11}} \approx 10^{-10^{11}+6}).

    • Also, the calculation as Morowitz performs it assumes that the system you’re looking at is the size of an E. coli cell, about 10^{-16}\, kg. If you wanted to calculate the probability of a cell forming anywhere in the Earth’s oceans (while still assuming there’s no energy supply, i.e. no sunlight or geological processes), you’d have to multiply the probability by around 10^{18}/10^{-16} = 10^{34}, but even that wouldn’t make a noticeable difference to the figure.

      (Also worth noting is that, since we’re assuming there’s no energy supply, any cell that did form would die almost straight away, and then decay back into its constituent matter – so it would be very, very difficult to know if it had happened.)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 483 other followers

%d bloggers like this: