This is basically a summary of Friston’s Entropy paper (available here). A friend of jellymatter was instrumental in its production, and for this reason I am fairly confident that my summary is going in the right direction, even if I have not emphasised exactly the same things as Friston.

I’ve made a point of writing this without any maths, and I have highlighted what I consider to be the main assumptions of the paper and maked them with a P.

Optimisation Theories

The basic structure of Fristons theory is not particularly unusual, it is one of many theories that basically work by assuming that something is optimised. In this case, a “free energy” (at bit like the physics one). Much of modern physics is based on some kind of optimisation: in mechanics one minimises an action, and in thermodynamics one minimises the (thermodynamic) free energy. Further afield it is found in economic decision making: where a risk is minimised; and in population dynamics: where, depending on ones interpretation, fitness is maximised.

In fact, all theories, right or wrong, can be formulated in terms of an optimisation. The theory: “Summer is warmer than winter” could be expressed as: “physics works so as to maximise summer temperature minus winter temperature”. Though usually, we do not speak in this way all the time.

Formulating a problems in this was always raises a question: “why should this quantity be optimised?” and there are two distinct but non-exclusive responses. (1) Some other theory suggests that it should be optimised; and (2) it is an elegant summary of other models.

Risk and fitness optimisation are of the first type: in economics, the notion of utility justifies* the minimisation risk and the notion of replicators is often used motivate the maximisation of fitness. Contrariwise, the action principles of physics are of the second type, they unify various principles in one coherent and elegant framework.

The argument that Friston provides addresses both. In the first case, it is motivated by a notion that agents are “coherent” in some sense (active systems). In the second, it generalises a number of concepts in machine learning and statistical inference.

Here, I will not worry about it’s ability to generalise mathematical theorems, but attempt to restate the argument for it following from biological principles. Friston’s presentation is usually aimed at those who prefer overarching, general, mathematical theories, but this seems to me the source of many of the difficulties that people have when trying to understand it.

Importantly, it is the justification according to other biological theories that is matters to the non-theoretician. It is this which matters to them when deciding whether or not to attempt an understanding of the mathematical details.

Agent and Environment

The model beings with a sensory-motor feedback loop. The agent affects its environment and the environment affects the agent – these are modeled as physical systems. In the environment there is “added noise”, and this noise motivates us to talk about probabilities and information.

P1: The internal states of an organism react deterministically on random (but correlated) sensory information from environment.

Because of the noise implicit in the environment, the environment behaves randomly. And because of this the agent – who’s state is affected by the environment – behaves randomly in a corresponding fashion. Which then affects the environment with (delayed and transformed) noise, which affects the agent… etc. etc. The consequence of all of this is that there is a now a probability distribution over the possible physical states of the agent and the environment.

P2: Organisms act against the environment’s randomness, towards being in a definite state.

The stated motivation for this is homeostasis: we act so as to not decay into disordered molecules, by eating, avoiding danger, not exploding, and what have you.

The most obvious (but not the only) way of measuring the amount of randomness is by talking about entropy, or in this case, the surprisal which for the purposes here we can consider to be the same thing as entropy. The surprisal measures the number of states that the system can be in. If it is low, the system stays in one of very few states, if it is high, the system is in one of many. So, the self-maintaining-ness/homeostaticity/distinctness is measured by the surprisal, which, according to Friston, should be minimised.

At first glance it may seem that one should apply this measure to the organisms internal states, but it turns out that this doesn’t work. For example: A rock would have very definite states and a low surprisal. Instead, the proposed solution is for the organism to instead minimise the surprisal associated with the external world* – he calls systems that do this active systems.

An active system acts as to make its sensory inputs as predicable and unsurprising as possible. This means we can make a modification to P2:

P2b: Organisms act against the environment’s randomness, towards obtaining sensory evidence that suggests that they are in a well defined definite state.

Doing this solves the rock problem. And because the randomness of the inputs affects the internal state, it is measuring a very similar thing.

Depending on ones philosophy P2b may be either a refinement or a change to P2. Ones opinion about this is crucial for deciding if the notion of homeostasis is a justification for this theory. Either way, with P2b we still have a problem much like the rock example from before. Sitting in a dark room with your fingers in your ears would be an excellent way of minimising surprisal – and we obviously don’t do that. Much.

The optimality of sensory deprivation can be seen as a motivational problem for free energy, but first I must go though some stuff about inference.

Best Inference

P3: Organisms make good inferences.

Organisms can be considered to make inferences, and making good inferences has different requirements to the notion of surprise as described above.

To discuss inference we must introduce some more probability distributions. Let’s say that the probability of sensory inputs are determined by things that we can’t directly observe. If we observe a coin landing on heads five times and tails five times, then we can make inferences about some hidden parameter that has a value somewhere around one half – a statistician would say the outcome of a fair coin flip is drawn from a Bernoulli distribution with a parameter of 1/2. The more we observe the coin landing, the more sure we can be about the parameter.

Of course in this example the choice of parameter has no obvious physical basis: one could easily choose another, related parameter – say by adding one, squaring it, taking the logarithm etc – and have it describe the same coin but in a different way. The choice of parameter is kind of arbitrary; and it is for this reason that Friston describes them as fictive. We can choose them how we like as long as they describe the same thing.

The fictive parameters are used to model the world. Consider a brick. This brick could have parameters width, height and depth. Associated with the parameters there would a confidence in each paramter: there is a probability that the brick is between 9 and 10cm long, 8 and 9 cm, 7 and 8, as well as for any other pair of numbers, 8.51234… and 8.51333… or whatever. But, we needn’t have chosen length, width, and height – we could have equally chosen surface area, volume and perimeter, we could still describe the same brick, and with these there would be a corresponding probability distribution which you could work out from the former.

It is both the power and the failure of information theory that it talks about probabilities with complete indifference for what they are.

Because the parameters don’t necessarily have any specific meaning or interpretation, at this point we simply forget about trying to work out what they are or what they mean – all we care about is the thing they describe. Friston argues, that whatever they happen to be we can still talk about them abstractly. The mathematical tools he uses are then chosen so as to make this aspect of his probabilities unproblematic (measure invariance).

The probability of the fictive parameters comes in two main flavors. One, the probability of the parameters as determined by the state of the sensory system (an “objective” probability in some sense), related to Friston’s “generative model”. The other, the probability of the parameters as determined by an internal model (the subjects probability), which Friston calls a “proposal density”. The former, the world as it is best described, the latter, the product of attempt for an organism to describe it.

The main idea of making inferences is that the organism tries to find the probability distribution (proposal density) that best matches the “real” probability distribution (generative model). The better the internal model is the better it matches the world. The better the model model matches the world, the better it is for making predictions about it.

Choosing to minimise the difference between probability distributions entails lots of things that people want from inferential systems, such as maximum entropy principle**. To measure this difference, Friston uses a standard tool usual: the Kullback-Leibler divergence. There are plenty of other measures that he could have used, but this one is usually preferred by information theory and Bayesian types.

Conflict Between Surprise and Inference

The next step is to acknowledge that both inferential ability and surprisal minimisation are “good”, and define a quantity that is measures the goodness. Depending on how you go about making this quantity, you end up with different things. But the quantity that is nice to work with once one has settled on the surprise and the Kullback-Liebler divergence is Friston’s free energy. This basically just adds the two together. The result is certainly elegant, but there is no motivation for this particular form beyond mathematical tractability (the reason for the mathematical niceness is the subject of information geometry).

Importantly, when optimal inference and minimising surprisal are mutually exclusive, minimising the free energy minimises both of them. This is an “all things being equal” justification:

P4: If it were possible, an organism would minimise both the surprisal and and the inferential “error” of their prediction independently of each other.

But no things are ever equal. In practice, the two quantities are not mutually exclusive. This is because the “subject” probabilty must have some physical basis in the internal states of the organism, and is thereby constrained by this physicality. This is essentially the idea that the brain represents probabilities, and is what Friston calls entailment. The internal states are the same thing as the subjects probability distribution but viewed, as it were, through a different lens. The exact manner in which the brain states are mapped to probabilities is not discussed directly, but there is an implicit notion that the brain cannot represent just any old probability distribution.

The usual consequence of entailment is that it is no longer possible to simultaneously minimise surprise and maximise inference, and instead, there is a trade off between the two:

P5: Due to the physical nature of organisms, maximising inferential ability and minimising surprise are in conflict with each other.

This motivates the need for free energy as one singular quantity, rather than two separate ones. It is also how one solves the “dark room with fingers in ears” problem, though for a slightly technical reason: Implicit in the formalisation of maximised inferential ability is the notion that making the best inferences about lots of things is better than making the best inferences about less things. Whilst in the state of sensory deprivation I mentioned one can make rather good inferences about you see, but one cannot make inferences about other things that you would if you opened your eyes and took your fingers out of your ears***.

The relationship between the internal states and the internal/subject probabilities is of fundamental importance. It is the very heart of what Friston calls the free energy principle. In this he elaborates on the nature of the physical constraint on the probability distribution it encodes. Basically, the constraint is simply the number of states that the brain can be in. So, we have a motivation for P5

P6: The world is very big, the brain is relatively small. The brain just does not have the capacity to match the complexity of the real world as provided in sensation.

Between them P1-P6 are the main motivations for the free energy in what I called type (1) terms – those that do not appeal to the generality of the theory. I hope they provide a good outline of the reasons for using free energy.

I will skip the applications sections of the paper as they are of the other kind. As I said, whilst these are important for those theoreticians who already use those techniques, it is not my concern here.

Summary

So, to summarise the notion of free energy: it is one way that one may quantify the trade off between making ones environment predictable, and the ability to make predictions where you cannot. The quantity is chosen so as to fit easily with established formalisms in information theory and Bayesian probability.

Throughout this summary I have omitted the mathematical assumptions such as additivity and the relevance of the surprisal and KL-divergence as I do not think including these helps with readability. But we must not forget that they are fundamental to how the particular form of the free energy is formulated. The wordy argument I have presented applies equally to all other free-energy like formulations.

Footnotes

* However, the internal and external surprisals are related because the state of the environment determines the internal state of the agent.

** Interestingly, the same version of the maximum entropy principle is also a property of Friston’s free energy. One should probably view Friston’s corollary 3 as showing that using free energy does not break this property of the KL divergence. Also, there are some very interesting results in information geometry along exactly these lines.

*** This raises obvious questions about what fictive variables are valid, but I shall skip over this problem mentioning only that it is a problem that may potentially be solved empirically.

Generally, when we look at a face we look at the eyes first. These days it is pretty easy to track where people look, the equipment is cheap and easily available – one simply uses a camera to look at the pupil and then calculates where the subject is looking. A preference for beginning with the eyes is a widely observed phenomenon (‘eyes are special’).

English readers, like readers in most languages, scan left to right when reading. With reading we constantly train ourselves to prefer moving left to right, something which leads to a phenomenon often called a readers bias (see this). It is not only during reading that the direction from left to right is preferred.

So, it is not surprising that we should think that smilies with eyes on the left are correct: left to right is preferred for reading, and eyes to mouth is preferred for viewing faces.

Here is a litte piece of computer-generated music that I created yesterday:

As Twitter user @DanieleTatti noted, it sounds like a sort of Scottish raga. But what I wanted to post about was the algorithm used to generate that ever changing sequence of pitches and warbles. It’s quite a simple idea – simple enough in fact that the whole piece is generated by the following 140 characters of SuperCollider code:

play{ar(r=RLPF,Saw.ar([200,302]).mean,5**(n={LFNoise1.kr(1/8)})*1e3,0.6)
+r.ar(Saw ar:Amplitude.kr(3**n*3e3*InFeedback.ar(0)+1,4,4),1e3)/5!2}

Lately, I’ve been reading about cybernetics (specifically, I’m reading the really rather excellent book The Cybernetic Brain by Andrew Pickering), and it’s got me thinking about homeostasis and feedback-based control systems. This code defines a drone consisting of two filtered sawtooth oscillators at fixed frequencies, as well as another filtered sawtooth oscillator whose frequency can vary. (I’ll call this one “the oscillator.”) The oscillator’s frequency is chosen by listening to the output sound (that is, the drone combined with the oscillator’s output), and choosing its frequency based on the overall amplitude of this signal. The amplitude-following mechanism (which is built into SuperCollider) has a bit of a time lag in it, so what happens is that when the drone and the oscillator are out of phase, the oscillator’s frequency will decrease over time, whereas when they’re in phase (and hence louder), the frequency increases.

For reasons I have some intuitive understanding of but would have a hard time showing mathematically, this tends to result in the system falling into stable states where the frequency of the drone and the oscillator are in a nice integer ratio. The phenomenon is presumably closely related to oscillator entrainment, which I’ve also experimented with musically, but which does’t tend to sound as good for some reason.

The human perception of sound is such that we find integer ratios to sound consonant, particularly when the integers involved are small. For example, a frequency ratio of 3/2 is a perfect fifth, and Pythagoras’ discovery of this was in many ways the genesis of physics as a numerical science.

There are several sources of variation in this code, which prevent the oscillator from remaining in the same frequency all the time. The output of the amplitude tracker is multiplied by a slowly (pseudo-randomly) changing value before being sent to the oscillator’s frequency; but also the drone has some variation in it, and the system is sensitive to this as well. The drone has a slowly changing filter, and it consists of two tones at frequencies 200 and 302 Hz, which is slightly wider than a perfect fifth, so there’s a beat frequency of 2 Hz. Sometimes you can hear the oscillator doing a slow vibrato as it responds to these beat frequencies. It’s quite satisfying to listen to this piece knowing that the oscillator is “listening” to and responding to the sound of its output. This is very rarely the case with computer-generated music.

You can hear the oscillator’s frequency dropping to zero at the end. This is a response to me turning the volume down, but if I hadn’t done that the piece would have gone on forever.

It was quite a lot of work to fit this idea into 140 characters. I originally had a 200-ish character one that was a lot more complicated, but in the end I think I prefer this one.

As you might have guessed, creating 140-character pieces of music is a hobby of mine. I post them at unpredictable intervals on Twitter as @headcube. I’ve only recently started posting audio clips of them, so if you want to listen to the full back catalogue (or to hear more than two minutes of this piece), you’ll have to install SuperCollider. It’s available for every platform, and if you’re on linux you should be able to get it via your package manager.

Just look at it.

“Well,” you say to yourself, “it doesn’t matter, they’re both the same,” so you pick one at random. Let’s say it’s the one on the left. But now I ask you if you want to change your mind.

“Well,” you might say to yourself, “let x be the amount of money in the envelope I’m holding. This envelope has a 50% chance of being the Prize, in which case the other envelope contains 2x. On the other hand, there’s a 50% chance that this is the Special Bonus Prize, in which case the other envelope contains 0.5x. But still, the expected value of the other envelope is 0.5*2x + 0.5*0.5x = 1.25x. So on the balance of probabilities I should definitely switch.” But then I offer to let you switch again, and again, and again, and every time you go through the same reasoning, never managing to settle on a particular envelope because each one seems like it should contain more money than the other.  Clearly something is wrong with this reasoning, but what is it?

In this post, I’ll solve this problem in what I consider to be the proper Bayesian way, pinpointing exactly where the problem is.  You might want to think about the question for a bit and come up with your own idea of its solution before reading on.

One thing to note before we start is that the problem goes away if I tell you how much money the Prize is worth. For example, if the Prize is £10 and the Bonus Prize is £20, the reasoning goes like this: If the envelope in my hand is the Prize then the other envelope is worth £20, otherwise the other envelope is worth £10, so its expectation is £15. That’s the same as the expected value of the envelope in your hand, so there’s no problem. So whatever the difficulty is, it has something to do with the idea that the value of the Prize is unknown.

How can we model this problem using probability theory? I decided to use three jointly distributed random variables: A, which represents the value of the envelope on the left (call it envelope A); B, which represents the amount of money in the other envelope; and E, which can take on two values, a or b. The variable E represents whether envelope A or envelope B contains the Special Bonus Prize.

We have to come up with a prior distribution for these three variables. In order to make them represent the things they’re supposed to represent, this distribution must have the following properties:

• The marginal distributions must be the same for the amount of money in the two envelopes. That is, , for every x.
• Each envelope should have a 50/50 chance of containing the Bonus Prize: .
• The Bonus envelope should contain twice as much money as the Prize envelope: and .
• The joint distribution shouldn’t change if you swap the two envelopes: .

These aren’t necessarily independent, since the last one implies the first one and possibly the first three imply the last one (I’m not sure), but they’ll all be used below.

Now, with this notation, let’s go through the paradoxical argument again. I’ve marked where the problem is.

1. Envelope A has a 50% probability of being the Special Bonus envelope. , or .
2. (The incorrect step.) Let the contents of A be x. Then, from (1), envelope B contains 2x with probability 0.5, and 0.5x with probability 0.5. That is, and , for any given x.
3. Therefore the expected value of B is equal to , where x is the value of envelope A.
4. Therefore the expected value of B is greater than the expected value of A, and I should switch
5. By symmetry I should also switch if I choose B, so by induction I should keep switching forever and become infinitely rich.

Step 2 is the problem. It’s true that , but step 2 (and hence the whole argument) hinges on  claiming that this is still the case when conditioned on the actual value of A. That is,

for every x. But in fact is given by

whereas is given by

Thus can only be equal to if , for every x. That is, step 2 only works if A is conditionally independent of E.

In other words, it only works if knowing whether A is the bonus prize tells you nothing about how much money is in it. This should raise a red flag already — if I told you that the envelope you’d selected was, in fact, the bonus prize, it would be quite strange if you didn’t then expect it to contain more money.

But an argument one sometimes hears in regards to this thought experiment is that if you know nothing — literally nothing — about the value of A, then this will in fact be true. Let’s try to codify this idea mathematically and see where it leads us.

Now, it’s quite difficult to say, a priori, what it really means to “know nothing” about something in probability theory. There’s a whole theory of so-called ignorance priors in Bayesian probability theory, but they’re quite fiddly and subtle things, so I’m not going to start out by trying to construct one. Instead I’ll just accept the claim in the previous paragraph (that knowing nothing means A is conditionally independent of E) and see where it leads.

Now, with this assumption of conditional independence, we have that

            (i)

But . By similar reasoning to (i), this is also equal to , and by exchange of A and B this is equal to .

So we have that this particular notion of “literally not knowing anything” implies that the marginal prior for A has the property that

for every x. You can construct various fancy priors that have this property, such as , but the one that looks most like an ignorance prior is the uniform prior. Uniform priors for unbounded quantities are a bit odd and have a few formal subtleties, but you can deal with them. They’re improper priors, meaning that you can’t normalise them, but essentially, this ignorance prior assigns the same infinitesimal probability density to every value of x.

Using such a uniform prior for an amount of money is a bit weird. It means, for example, that the probability that the envelope contains £10 is the same as the probability that it contains £, which is the same as the probability that it contains £. But more than that, it means that the expected amount of money in the envelope is infinite. (This is also true of the “fancy” priors mentioned above.)

So now, finally, we can fully explain the paradox. If you really really “didn’t know anything” about the amount of money in the envelopes, you might be justified in assigning a uniform marginal prior to the value of each envelope’s contents. Then when you select envelope A, you can ask yourself “what is the expected amount of money in this envelope?” The answer is infinity. Should you switch? Well, the expected amount of money in envelope B is infinity, which is equal to infinity*5/4. So there’s no paradox. It doesn’t matter if you switch or not, because you’ll expect to become infinitely rich in any case.

But of course, if this was a real situation then the uniform prior would be a rather silly one. The contents of my envelopes cannot be less than 1p, and they can’t be more than the total amount of money in existence, which I guess is in the trillions of pounds. (Of course, you’re free to pick a smaller upper bound if you want.) Any prior you can come up with that fulfils these constraints will have finite expectations, and won’t allow you to conclude step 2 from step 1 in the argument above.

So to conclude, the two envelopes paradox is not a paradox at all, but just an intuitively reasonable argument that has a hard-to-spot error. (Confusing a conditional distribution with an unconditional one.) If you work through the problem in a proper Bayesian fashion, you realise that you can’t avoid considering your prior knowledge of the envelopes’ contents. As long as you choose a sensible prior, the problem evaporates.

So, last time, I stressed the point that you can’t interpret a P-value as a probability or frequency of anything, unless you say “given that the null hypothesis is true”. Most misinterpretations, e.g. “the probability that you would accept the null hypothesis if you tried the experiment again”, make this error. But there is one common interpretation that is less obviously false: “A P-value is the probability that the data would deviate as or more strongly from the null hypothesis in another experiment, than they did in the current experiment, given that the null hypothesis is true”. This is something that you might think is a more careful statement, but the problem is that in fact when we calculate P values we take into account aspects of the data not necessarily related to how strongly they deviate from the prediction of the null hypothesis. This could be misleading, so we’ll build it up more precisely in this post.

To begin with, let’s make up some symbolic language. “The data” are taken to be a random variable , and the particular instance of the data observed in the current experiment . This evidence itself is likely to consists of a number of observations, e.g. a collection of real values etc. The null hypothesis will specify a value for some parameter of interest, call it . For example:

Generally, in order to investigate this parameter, we find a “sufficient statistic”, that is, a function(al) that operates on the observed data to produce (usually) a single real number that contains all the relevant information that allows the data to discriminate between different possible values of the parameter of interest. We’ll call this , so for example we might use – the arithmetic mean of the data values produced by an experiment (if the parameter of interest is the expectation value of the process that is being investigated). This statistic has a “sampling distribution”, i.e. let be a random variable representing the randomness of the statistic rather than the data itself.

Now we need to say what “deviate as or more strongly” means. For this, assume there is a distance function which describes how far the statistic if from the expectation under the null hypothesis. So if the null hypothesis predicts a zero population mean or expectation and is the sample mean from the experiment, we could have:

(i.e. the absolute value of the statistic). The null hypothesis says that the parameter is zero, so it implies that the statistic (in this case an estimator of the parameter) is also (expected to be) zero – we just take how far the statistic is from zero as a measure of deviation. Then, we can encode the above interpretation of the P-value symbolically:

“The probability that the distance of the statistic from the prediction is greater than the distance observed, given that the null hypothesis is true”.

By the law of large numbers, this could be read as the relative frequency with which the statistic would take a more extreme value than the one observed if we repeat the experiment many times on a process where applies. This might look fairly reasonable, so what is wrong? In fact, for many tests, it might be right, but I’ll borrow an example from Fisher of a case where it’s not.

Suppose we are doing a linear regression on two Gaussian random variables. We take a model with and as two random variables:

The parameter represents an offset which is more or less irrelevant for our purposes. The “slope” is the strength of the relation, and will be the subject of our hypothesis test. There is also some random deviation such that is distributed marginally (for any given value of ) as a Gaussian with standard deviation .

We do an experiment and obtain pairs of values, . Presenting the regression in the same way as Fisher, let’s define three intermediate statistics:

Say that the null hypothesis says again – the data are uncorrelated. A suitable estimator for is:

Of course the null hypothesis predicts , so a reasonable distance value is . According to the “misinterpretation”, the P value is

Where is the estimator statistic as a random variable (i.e. representing the values the estimator would take if you sampled over and over from a system where the two variables were uncorrelated).

But in fact, the usual way to test this null hypothesis is to do a t-test. This involves first getting the estimator for :

Then calculating the t-value:

And the P-value is the probability of the observed or one more extreme given that null hypothesis is correct:

Where is the cumulative density function of the t-distribution with degrees of freedom. We multiply by 2 because we want a “two-tailed” test. But because t is calculated using the actual value of (observed in the experiment), it is dependent on this actual value, even though it would not be constant across all experiments. That is, the value represents the probability that the test statistic would take a more extreme value under the null hypothesis:

We can see how this is different from the probability that the data deviate more strongly from the null with a little computational experiment (Python code). First, generate N data points using a Gaussian random number generator (so X and Y are genuinely two independent random variables). Then calculate the and values as above. Then, if we re-run the process (generate lots, say 500, of sets of N uncorrelated data), the original (mis)interpretation of the P value says that P should be the proportion of the time that takes a larger absolute value in one of the re-runs than it did in the original experiment.

If we repeat this process a few times, we can plot the P values obtained against the proportion of the time that b took a greater value in one of the re-runs than it did in the original experiment:

If it was the case that the P value was the proportion of the time that the data would be observed to deviate more strongly from the prediction under the null hypothesis, the points should lie very close to the diagonal – but they don’t. I’ve also coloured the points blue if in the original experiment, was very low (less than 14) and red if it was very high (more than 27 – the values were chosen empirically). Notice that the blue values tend to lie below the diagonal, and the red values above it – so you can see that the probability of observing more extreme data is dependent not only on the P value, but on the value of in the original experiment.

So to sum up, we have a few different kinds of statistics:

• The estimator for the relevant parameter – i.e. a sufficient statistic for the parameter of interest in the hypothesis
• Ancillary statistics – give information about parameters not relevant to the null hypothesis (e.g. the variance in the linear regression model)
• Test statistics – make use of the relevant statistic and potentially ancillary statistics.

So the P value is in fact conditional not only on the null hypothesis, but on the ancillary statistics:

P = Pr( data more extreme than observed | , ancillary statistics as observed )

Fisher points out that of course this can’t reasonably be interpreted as any mechanical procedure of repeated sampling, since it would imply that you would have to select for those samples where the ancillary statistics were exactly the same as the one you observed in the original test (which is never going to happen).

I’m not going to draw any grand conclusions from this. I’m not even quite sure what to make of it at the moment. However, it does seem to me like quite an important point if you want to understand what a P value is.

RCTs are the best way of determining whether a policy or intervention is working.

It’s not hard to find opinion pieces backing up the report’s central idea, and the thesis that RCTs are the best way to “find things out”. Here’s one by Tim Harford, a writer who covers economics; a similar argument made by Paul Johnson who is the director of an economics research group, the Institute for Fiscal Studies; and Prateek Buch, who is a research scientist. A phrase that keeps popping up is “gold standard”. RCTs are “the gold standard in evidence”, says Johnson, or the “gold-standard for showing that medical interventions are effective” according to Buch. Mark Henderson’s book, “The Geek Manifesto” says that the RCT is “commonly considered the ‘gold standard’ for medical research because it seeks systematically to minimise potential bias through a series of simple safeguards”. What exactly does all this mean? I think it’s a question worth asking, since not all science involves RCTs. The Higgs boson for example, was recently “discovered” (if that’s the word) without (as far as I can tell) the need to randomise test subjects. So are RCTs in fact the “gold standard”?

A little searching reveals I’m far from the first person to ask this question. An article by Nancy Cartwright: “Are RCTs the Gold Standard?”, addresses exactly this, as does to an extent her book “Hunting Causes and Using Them”. I think Cartwright makes some good points, so I’ll use these as a jumping off point. First, though, there’s one obvious respect in which RCTs can be considered the “gold standard” which Cartwright doesn’t directly address (perhaps because it’s too obvious), which is simply that RCTs are the gold standard because everyone already agrees they are. In medical research (as Buch says), RCTs are used all the time. But everybody agreeing isn’t really the kind of reason for using RCTs that I’m looking for.

For Cartwright, the reason RCTs are interesting is that they tell us about causes. One way of putting this would be that an RCT helps to address the question (if we’re thinking about testing a pill) “the patient got better after taking the pill, but would not have got better had they not taken the pill”. That is, rather than simply talking about correlations (the patients that took the pill got better, the ones that didn’t, didn’t), RCTs can potentially answer “counterfactual” questions about what would have happened if patients who (in reality) took the pill had not done so (hypothetically, in imaginationland). This can sound complicated, but I think it looks like a reasonable definition of “causation” to most people (Cartwright doesn’t use this definition, but I think it’s intuitive and it will do for our purposes).

Now, RCTs (according to Cartwright) guarantee that certain causal conclusions follow deductively given the assumptions of the RCT design and a positive result in the trial – the conclusion (i.e. that say, a pill “works” in the counterfactual sense discussed above) follows logically, you can probably intuit roughly what the logic is if you read “Test, Learn, Adapt”. This makes RCTs, as a method of testing causal claims, “clinchers”, rather than “vouchers” – that would be bits of evidence often considered “softer” that might be consistent with a particular causal claim, but don’t necessarily disprove any others.

There are two main issues that Cartwright points out with saying that RCTs are the gold standard. One is that, generally, there are a lot of possible “clinchers” that aren’t RCTs. Anything where the causal conclusion follows deductively from the assumptions and the test result. This not only applies to the “ideal RCT” but to many other methods (Cartwright gives a list, I might go in to it in another post). Of course to make the causal conclusion we have to be sure that the assumptions are met, and one could argue that RCTs offer a way of ensuring this. But they certainly don’t guarantee it.

Secondly, “vouchers” have a certain advantage over “clinchers”, namely that of external validity – it is hard (even with an RCT) to be sure that a conclusion obtained on a test population is correct for another target population. If you test a pill on some people who are alive today, how do you know that it will still work on people who are alive tomorrow? Or in a year, a decade or a century? Our understanding of human physiology suggests we aren’t going to be tricked by a pill that only works on Tuesdays, but what about testing government policies (such as back-to-work schemes which are discussed in the report I originally cited)? How do we know people will react to being put on a given scheme the same way this year as next? The extrapolation seems much more shaky – you can’t just assume that even as economic and social factors change, everyone’s behaviour stays the same, but equally you can’t do a new RCT on every policy every year, since that would be, well, silly. RCTs have no (intrinsic) way of dealing with this problem (which is not to say it is not something that cannot be analysed quantitatively, it’s just that the RCT methodology doesn’t fix it for you).

RCTs are nice, they make a lot of sense, and they are useful in many settings. But to say they are the “gold standard” smacks of received wisdom. What this report and the many comments on it ignore (or gloss over) is that RCTs rely on satisfying assumptions (internal validity) and extrapolating from limited evidence (external validity) just as much as anything else in science. That doesn’t make RCTs bad (far from it), it’s just that the “gold standard” claim seems to be based on the assumption that RCTs automatically produce the “right” result, when that only happens if you are careful about how you carry out the experiment, and the conclusions you draw from it. Cartwright again:

But this introduces expert judgment into the assessment of internal validity, which RCT advocates tend to despise. Without expert judgment, however, the claims that the requisite assumptions for the RCT to be internally valid are met depend on fallible mechanical procedures. Expert judgments are naturally fallible too, but to rely on mechanics without experts to watch for where failures occur makes the entire proceeding unnecessarily dicey.

The italics are in the original, and Cartwright has a point about expert judgment, and RCT advocates’ views towards it. But I don’t think anyone is suggesting you can run RCTs without expert oversight, as “Test, Learn, Adapt” says:

RCTs in their simplest form are very straightforward to run. However there are some hidden pitfalls which mean that some expert support is advisable at the outset. [Emphasis added]

Personally, I think part of the reason why you can’t really avoid some kind of expert judgment is partly because causal questions tend to involve thinking about what would have happened “in imaginationland”, i.e. the counterfactual question I used to define cause above. Its hard to know what imaginationland looks like in any given context, so we inevitably end up just asking an “expert”. But RCT advocates don’t want us to talk to experts (apart from RCT experts). “Test, Learn, Adapt” encourages us to read “The Black Swan“[1] by Nassim Nicholas Taleb, and gives this interpretation:

Similarly, such thinkers tend to be sceptical about the ability of even the wisest experts and leaders to offer a comprehensive strategy or masterplan detailing ʻtheʼ best practice or answer on the ground (certainly on a universal basis). Instead they urge the deliberate nurturing of variation coupled with systems, or dynamics, that squeeze out less effective variations and reward and expand those variations that seem to work better.

Ok, when I said it was “plain-English”, I might have been being generous in places.

[1] Having read it I can safely say that Taleb really doesn’t like experts. There’s the odd good idea in this book (though they are mostly borrowed from Benoit Mandelbrot among others), but it’s hard work getting past the author’s ego, baseless attacks on anyone who ever does any maths or who wears clothes Taleb doesn’t approve of, and constant reminders that the author is oh so clever. I don’t recommend it.

In a previous post I discussed autocatalysis, the mechanism by which a bunch of different molecules can react with each other in such a way that they end up producing more of themselves, at the cost of using something else up.  The ideas in that post don’t only apply to chemistry – you can use them to think about just about any kind of physical process.  In this post I’ll talk about how to think about the economy as a whole in autocatalytic terms. But let’s start with something on a smaller scale, the process of baking bread:

(You can click on the images for bigger versions.) This is a variation on the diagrams I used in my previous post, except that I’ve changed the conventions a bit: I’m now using boxes to represent “stuff”, because it’s more traditional and because it’s easier to fit text into them, and I’m using hexagons to represent processes, because hexagons are cool.

Arrows always either go from a box (stuff) to a hexagon (process), meaning the process uses up the stuff, or the other way around, meaning the process creates stuff. The above diagram means that the process of baking takes in water, fuel, flour and human labour to produce bread.  It also uses up oxygen and gives off carbon dioxide, mostly because baking involves burning the fuel to create heat. I haven’t drawn the heat on the diagram, but it could be drawn just like the other products. In chemistry terms, the heat given off per unit bread produced would be called the enthalpy of the baking reaction.

The baker’s shop itself acts as a catalyst – it doesn’t get created or used up by the baking process, but it’s necessary in order for the process to occur.  I’m assuming the baker grows her own yeast, so she always ends up with a bit more yeast than she started with.  This means that, according to the definition in my previous post, the yeast is autocatalytic. This shouldn’t be surprising, because making more of themselves is what organisms do.

If I was being rigorous I would put numbers on all of the arrows, indicating how much of each of these things gets used or created, per unit of bread. In chemistry these would be called the “stoichiometric coefficients” of the baking reaction, and the coefficient on the outgoing yeast arrow would be slightly higher than the one on the incoming arrow. Since I don’t know the numbers and don’t want to clutter the diagram, I’ve drawn the outgoing arrow as a double one as a kind of shorthand for this.

This picture is a huge simplification of course.  In reality baking consists of many processes, including making the dough, allowing it to rise, kneading it, the burning of the fuel (which might take place miles away in a power station), and the actual conversion of dough into bread in the oven.  But it’s OK to lump all this together into single process as long as we remember that it’s really composed of lots of separate ones. (Chemists do this all the time.)

But instead of zooming in deeper into the details of the baking process, let’s zoom out a bit and look at what happens to the bread after it’s made:

I’ve added a second process to the diagram, representing the bread being eaten by people.  This uses up oxygen and produces carbon dioxide, because breathing is an integral part of how we extract energy from food. It also uses up some other food groups (otherwise you’d get rickets) and produces sewage, which must be disposed of.

I’ve also drawn it as producing “human labour”, representing the fact that once people have been fed they are able to work. Housing acts as a catalyst because it’s very difficult for people to eat and work if they don’t have somewhere to live.

People also reproduce, of course, and we need more things than these basics in order to live and work.  As in the case of baking, a more detailed diagram could include these things, but this one gets the point across well enough for now.

The double arrow from eating to labour represents the fact that you get more labour out of the combined baking/eating process than you put in, otherwise everyone would spend all of their time making bread and no-one would be able to do any other jobs. (It would be possible to work this out from the stoichiometric coefficients of the two reactions if they were included in the diagram.)  This results in an autocatalytic cycle, highlighted below:

More bread leads to more eating leads to more labour leads to more baking leads to more bread – as long as there’s a big enough supply of fuel, water, flour, oxygen, etc.

Here’s the big point I want to make in this post: the way the economy grows is through this type of autocatalytic cycle.  Economic growth is all about the physical self-reproduction of physical stuff.  It’s about more stuff leading to more stuff leading to more stuff, and when we say something contributes to the economy, we really mean it plays a role in this self-reproduction of stuff.  All this might sound terribly materialistic, but it’s not meant to be – the goal here is to understand economic growth, but I’m not saying it’s always necessarily a good thing.  Things like a high quality of living, high employment and a healthy environment are important, and in an ideal world the economy would be organised to maximise these things instead of, or as well as, growth.

At this point you might be thinking I’ve neglected something that’s usually considered very important when it comes to the study of economics: money.  You’d be right, but I think one of the biggest mistakes you can make when trying to understand economics is to think it’s all about money.  Money is an important tool by which the physical processes of the “real” economy are organised, but if we lived in a money-less command economy the processes in the diagram above would still happen, they’d just happen for different reasons.  They also happen if you bake your own bread at home. In that case no money changes hands, but the process still produces bread, still produces more capacity for labour, and thus still contributes to the economy.  Although money is terribly important in the general scheme of things, and it’s certainly important to understand it, I’ve chosen to try and understand the physical basis of the real economy first.

So anyway, let’s expand out even further from the diagrams above, and try to see where some of those processes’ other inputs come from.

I’ve included farming and the mining of fossil fuels as well as the economically hugely important production of nitrogen fertiliser through an industrial process called the Haber-Bosch process, which uses up quite a lot of energy. (Mind-blowing stat: globally, we fix more nitrogen per year through this industrial process than the entire biosphere put together.)

Ultimately everything on this diagram is powered by two sources: sunlight and buried fossil fuels.  One of these will last for a few more billion years but the other won’t, so this picture clearly represents a very temporary state of affairs. (The availability of fresh water is also ultimately due to the sun via the water cycle, but I haven’t put that on the diagram. You have to stop somewhere.)

A lot of the power from sunlight enters the system indirectly through the production of oxygen by natural ecosystems. One of the things I like about this approach is how quickly you have to deal with the fact that the economy can’t be seen as a separate system from the natural environment.

This diagram is a massive over-simplification of the real picture, and I’m sure I’ve left off some important arrows, but it’s already become very complicated.  Studying it for a while reveals that it’s absolutely chock full of autocatalytic cycles.  Most of them involve the food-human interaction.  More farming means more grain means more bread means more humans means more farming.  Or more fossil fuel burning means more in the atmosphere means higher crop productivity means more humans means more fossil fuel mining means more fossil fuel gets burnt. (As you might expect from the diagram, carbon dioxide acts as a fertiliser.)

There’s also at least one autocatalytic cycle that doesn’t involve the creation of more humans: oil mining uses fuel but produces more fuel than it uses up (otherwise it wouldn’t be worthwhile). In general the growth of a business is autocatalytic, as well as the growth of the economy as a whole.  A business invests in capital, which then (hopefully) produces a return larger than the investment, allowing the purchase or construction of more capital.  It’s interesting to think about what these cycles look like for different types of business, but I’ll leave that for another time.

For now I’ve got to the point I wanted to get to with this post, where I’ve shown that the economy can be seen as a huge network of interrelated physical processes, some of which can be thought of as being like chemical reactions and others of which (like the Haber-Bosch process) actually are chemical reactions. I’ll leave you with the thought that other place where you’re likely to find big, complicated diagrams mapping interlinked networks of chemical processes is molecular biology, where these diagrams map out the metabolism of an organism. It’s an interesting comparison to think about.

Learning isn’t something we normally associate with plants, of course. We don’t normally think of plants as behaving at all – but of course they do, it’s just that they generally do it much more slowly than animals. Anyone who’s ever pitched a tent for a few days at a time will have noticed that the grass underneath grows tall and thin and pale. It does this in an attempt to find some light. If the tent were instead a small rock or fallen tree then the grass would be able to grow its way out from underneath and get a source of energy again.

But is this learning? No – it’s an evolutionarily in-built response, equivalent to what in animals we could call an instinctive or reflex response. You could say it’s a kind of learning that’s taken place on an evolutionary time scale (some the grass’ ancestors grew in this way an survived, so grass as a whole “learnt” to do it), but we’re interested in learning on the time scale of an individual.

So what makes a behaviour “learning”? There are probably a lot of different definitions depending on exactly what question you want to ask, but the one I’m interested in here is classical, or Pavlovian, conditioning. Pavlov noticed that dogs salivate in preparation for eating, and discovered that if he rang a bell every time he fed them, they would salivate whenever he rang the bell. The point is that the bell is otherwise an arbitrary thing – it doesn’t cause salivation unless the dog has learned to associate it with food.

So could we do the same thing with plants? It seems to me that it wouldn’t be too hard. Plants don’t salivate, but they do have their own set of “reflex” responses, including the response to low light that I mentioned above. Another is the response to grazing by animals, which usually involves growing tougher leaves, and perhaps producing some bitter-tasting or toxic compounds in order to deter further grazing.

The basic idea of the experiment would be something like this: we grow a whole load of plants at the same time, under identical conditions. These plants are divided into a test group and a control group. For the test group, at some randomly determined time we expose them to some arbitrary stimulus. This has to be something the plant is able to detect, but it shouldn’t be something that normally produces a strong response – I guess it could be something like a small change in soil pH, or blocking out part of the light for a while. In classical conditioning terms, this is called the “conditioned stimulus” (CS).  Then, some time later (perhaps the next day) we apply a different stimulus, one that the plant will respond to (but which will not kill it). For example, we could block out all the light for a couple of days, or we could clip the leaves to simulate grazing. This is called the “unconditioned stimulus” (US). Some time later we do this again – apply the conditioned stimulus, then apply the unconditioned stimulus. This is repeated several times. Finally, we apply the conditioned stimulus but not the unconditioned one and observe the results.

For the control group we apply both stimuli, but we apply both stimuli at randomly determined times. So the plants have been exposed to the same conditions overall, except that there’s no point in learning an association between the conditioned stimulus and the unconditioned stimulus, because they’re uncorrelated to one another.

A successful result from this experiment would be that the test group responds to the conditioned stimulus as if it were the unconditioned one. So we change the pH slightly and the plants start to grow tall and thin, or we block out part of the light and they produce anti-grazing toxins. But crucially, it’s not a successful result unless the control group doesn’t react in this way. This shows us that the response to the US is a learned one, and not just something that all plants of that species would do anyway.

Of course there are a lot of subtleties involved in running such an experiment, most of which I’m probably not aware of. An unsuccessful result wouldn’t be very interesting – it would only show that we were using stimuli that that species of plant can’t respond to. If plants can learn, some are probably better at it than others. The experiment would be easier to perform with a fast-growing weed (I was considering trying it myself with watercress at one point), but I suspect that trees are probably more likely to be smart, since they have more invested in staying alive for a long time.

This would be a very interesting experiment to try. Although it might seem unlikely to be successful, it probably isn’t all that hard to carry out, on the general scale of plant experiments. If any experimental plant biologist ever reads this who would like to give it a try, please feel free to get in touch!

Horgan seems to be particularly infuriated by string theorist Edward Witten. String theory, by the sounds of things, is the ultimate ironic science. The purpose of string theory is to unify Einstein’s general relativity with quantum mechanics, which it does by positing the existence of new underlying physical laws. The new laws only really exist in the mathematical world, and don’t add anything to quantum mechanics and relativity that empirical scientists could ever hope to find. Since most physicists are quite happy to use the quantum mechanical and/or relativistic foundations with no obvious loss of modelling capability, string theory seems to be in a sense pointless. The likes of Witten are only intrigued by it because they are actually mathematicians more than physicists, and in the mathematical domain it does actually represent a significant body of new findings. (Incidentally, I know nothing about string theory, I’m just summarising Horgan’s argument).

I think the “ironic” label can easily be applied to authors like Chemero (and there are many others who make similar arguments). Chemero’s position centres around an apparent dichotomy between viewing cognitive systems as computers and viewing them as “dynamical systems” (the latter being the “radical embodied” stance). From the computationalist viewpoint, our brains are like computers than act on symbols that “represent” meaningful properties of the world we live in – the states of neurons encode such representations, and the responses from other neurons perform manipulations, computations, and eventually encode an output state (such as motor activation). In the radical embodied paradigm, our brains are simply part of a larger physical system comprised of the brain itself, its body, and the environment it finds itself in. In much the same way as a tennis ball traces a trajectory in space as it is hit by rackets and falls at some point to earth, our neurons (as a collective) are simply tracing out a trajectory though the space of all their possible activation states in accordance with physical laws (obviously, brains obey much more complex trajectories than something passive like a ball, but the modelling approach is essentially the same).

It’s worth mentioning that there is in fact a “middle ground” which Chemero calls (non-radical) embodied cognition, which considers the role of the body in cognition as important but does not absolutely refuse to accept the existence of any type of computational symbol manipulation.

A problem for people like Chemero is that they are often interpreted as trying to argue that “the brain is a dynamical system”. Such a statement would be trivial almost to the point of absurdity – a dynamical system is really just a model, and the dynamical systems modelling approach can be applied to all sorts of physical systems. Stating that the brain “is” a dynamical system is nothing more than saying that the brain is made of physical-stuff, a viewpoint with which arch computationalists would surely agree.

To really drive the point home, not only is it trivial that a brain is a dynamical system, it is also trivially true that any computer (such as the one you are reading this on) is a dynamical system, if that is how you choose to view it.

I would also argue that it’s pretty much just as valid for any dynamical system to be viewed as a computer, if that’s what you want to do. So in exactly the same way, the statement “the brain is a computer” is again true, but trivially so.

Since both statements are obviously true, debating which one is more correct seems like a waste of time, and hence a candidate for the “ironic” science label. However, the real crux of Chemero’s argument is not that the brain “is” a dynamical system, but that it is “best viewed as” a dynamical system. A simple experiment to do is to hold out your arms so that your forearms are pointed upwards, bent at the elbow, then wave your forearms side to side as if they were windscreen wipers (both left at the same time then both right at the same time). Try and do this faster and faster and you should find that it becomes increasingly difficult to keep your arms in sync in the windscreen wiper-way – in fact, they are likely to sync up the opposite way, with both pointing inwards together then pointing outwards together. This phenomenon has been experimentally observed in a lot of similar scenarios and explained using a dynamical systems model. Chemero’s point is that when you think of the behaviour as the result of a dynamical system, it’s actually quite easily to model it and thus gain some understanding of how your brain works. If you wanted to take a computational approach, where you insist that the brain must be working on some internal representation of the movement of your arms, it actually seems quite convoluted and unnecessarily difficult to describe this phenomenon in any meaningful detail.

This is an “is best viewed as” argument, and it’s inherently pragmatic – the whole point is that thinking of the brain in a different way is supposed to make real, empirical science easier to do. However, the “is best viewed as” argument is still ironic in the sense that it, itself, is completely untestable. Clearly whether something “is best viewed as” can’t be measured using scientific instruments, it’s a value judgement as to what is the most productive approach for scientists.

So really, what I wanted to say, is that “ironic” science is not really as wasteful as Horgan makes out. Perhaps string theory goes to extremes in terms of mathematical obfuscation and is really too separated from reality. However, science that involves making an “is best viewed as” argument does, at least potentially, have a lot of value. I also think that (perhaps) you could even say that the things that Horgan clearly does view as genuine scientific contributions are actually themselves ironic “is best viewed as” arguements. General relativity, for example, views gravity as a property of a four dimensional “spacetime”, not because the existence of spacetime can be empirically tested, but because doing so generates models of physical systems which can be tested understood (at least by people sufficiently familiar with the theory), and thus it is pragmatically useful.