Wednesday, 5 April 2017

The Ecological Approach, Explained to an 8 Year Old

About 3 weeks ago I got an email from a person who had found our blog via Robert Epstein's piece 'The Empty Brain'. The email said
I've had a good read this afternoon, and it has been informative to some degree, however ...
I have an 8 year old son, and due to questions we both have, we have had some very interesting laypeople's conversations about the nature of experience and "the mind" (is it a thing, a physical thing, a process?) as well as such things as memory, embodiment and perception.
It seems it would be really helpful for us (and by extension, possibly many others?) if you could summarise the broad strokes of your theory in some way in which an intelligent 8 year old (and his father!) could understand.
Would this be possible?
Ed Yong has taught me that good science communication doesn't have to be dumbed down, it just has to be pitched right, and while I am no Ed Yong, I say, challenge accepted! Let me know how it goes!

What Does Cognitive Science Want to Know?
The basic question in cognitive science (psychology plus related fields like linguistics, computer science, neuroscience, philosophy and more) boils down to this: "Why did that person do the thing they just did, in the way that they did it?" There are two basic answers to this question; the computational and the ecological approaches.

The Computational Approach
If you look at what goes into the person from the environment (sensations) and look at what comes out (behaviours), they don't look anything alike. Sensations (input) clearly have to be taken, altered, processed until a behaviour (output) can be generated. 

The main thing sitting in between sensation and behaviour is the brain, and it's clearly built in ways that enable it to do this kind of processing, transforming work. The mainstream view in cognitive science is therefore that people do what they do because the brain makes us. Cognitive science therefore needs a way to talk as precisely (read: mathematically) as possible about those brain-based processes.

The ideas that makes this possible come from computer science. Pioneers like Claude Shannon and Alan Turing invented the maths that made it possible to take (almost) any input and process it into (almost) any output. All the maths that make it possible for your computer to take some input (e.g. a mouse click) and transform that into an action (e.g. taking you from one web page to another) were now available to carefully describe the processes of the brain.

We therefore end up with this basic hypothesis: the form of our behaviour is caused by computational mental and neural processes that transform input into that output, and we need all this because our perceptual contact with the world (through sensations) is not rich enough to explain the form of the behaviours we can get up to.

James J Gibson
The Ecological, Embodied Approach
It turns out, however, that there was a psychologist (James J Gibson) who spent the better part of 40 years figuring out that our perceptual contact with the world is, in fact, amazingly rich and detailed. People realised that Gibson's theory might offer a way to think about perception that could replace the need for mental representations and computations; suddenly there was another option. 

The key to the ecological approach is that the brain is not the only place where the good stuff happens. Our environments offer some opportunities for action and not others (Gibson called these 'affordances'), our bodies enable us to do some things and not others, and the way our bodies perceive and act in their environments allows all this to change in interesting, complex but not random ways. 

As soon as it becomes possible that the form of our behaviour can be caused by something other than just the brain, everything changes. The job of the brain changes from "process input into output" to "link body and environment through perception and action". The brain is no longer necessarily a computer, because computation is no longer necessarily the thing that has to happen to get to behaviour. The difference between the two approaches to behaviour is captured by the slogan, 'Ask not what is inside your head; ask what your head is inside of'. 

A Simple Example
How does a baseball outfielder catch a fly ball? In terms we've been using so far, we want to know the form of the actions the outfielder takes and how that relates to various ways of solving the problem.

The computational strategy is to detect the initial motion off the ball from the bat and then to predict where it will land. You can technically do this because the physics of projectile motion (motion caused by an initial force and then left to run without any more help) is fairly straight forward and you can predict where the ball will go pretty quickly. This account predicts that the form of the outfielder's behaviour will be to run in a straight line (the shortest distance) from where they are to where they need to be.

There are two ecological strategies. Fly balls follow a curved path and they change speed (they slow to a stop at the top, then speed back up as they fall). The two strategies are to move so as to cancel out one aspect of this motion. Optical Acceleration Cancellation predicts that the outfielder's behaviour will be to run with varying speed that tries to offset the acceleration of the ball. Linear Optical Trajectory predicts that the outfielder's behaviour will be to run along a curved path that tries to cancel out the curvature of the ball's path. Weirdly, it turns out that if you succeed at either of these, you will arrive in the right place at the right time to catch the ball.

The data unambiguously support the ecological strategies. Outfielders never simply run to the predicted landing location; instead they run along curved paths at varying speed in various combinations of the two perception based strategies. The form of their behaviour maps directly onto the form of the perception of the environment. 

Whenever the details of the perceptual coupling to the environment have been worked out and tested, human behaviour always shows the various tell-tale signatures of online perceptual control, rather than mental prediction. While there are many tasks still to solve, so far so good. 

I hope that helps; comments and questions welcome, I'm keen to fine tune this as much as possible!


  1. Dear psych S.
    Couldn't the ecological approach (example baseball) be the result of repeated, and endless, re-computations (corrections to initial prediction)?

    1. No. The behaviour dos not look like this is what is going on; no straight line running

    2. I disagree. For many problems, there is more than one way to do it (different algorithms for the same task). You start and end in the same places but the path may differ. Some ways of doing stuff may be more/less useful depending on circumstances. Look up all the sorting algorithms for an example.

      By showing ball catchers not running in straight line you don't dismiss computation as such, you only show that humans do not use "precompute-and-move-to-the-destination algorithm". We use a different way to catch a ball - one that is easier for our monkey brains & bodies. More "ecologically suitable".

      Repeated, endless re-computations and corrections is exactly what is taking place.

      The point of ecological approach is that we rely on (quick) feedback from the environment quite a lot while doing our computations.

    3. Repeated, endless re-computations and corrections is exactly what is taking place.
      What is being computed?

    4. Where and how to move to catch the ball.

      How else would you engage muscles if your central nervous system wouldn't compute it first?

    5. Why compute when you can perceive?

    6. Isn't perception of the environment one of the inputs for computation? I mean what drives the catcher to the end goal of motion if not based upon information about speed, height, trajectory (perception) and experience (LT memory)?

    7. @Andrew
      Unless you claim that you can always link sensations to actions exactly 1:1 without any interaction between signals whatsoever, you'll end up with some kind of computation (transforming input to output that is different from the input). Or "signal processing" if you dislike the word "computation" - I guess that our notions of it are a bit different.

      For me, the point of ecological/embodied approach is that these computations might be quite straightforward because we can use neat algorithms that are enabled and supported by the properties of the environment and our bodies.

      In short: the ecological approach doesn't dismiss the computational approach. It refines it by trying to explain out HOW the computing is done.

    8. Unless you claim that you can always link sensations to actions exactly 1:1 without any interaction between signals whatsoever, you'll end up with some kind of computation (transforming input to output that is different from the input).
      One of the overlooked findings in all the perception-action research is that once you identify the right information variables, the action tends to unfold in a way that matches how the information unfolds. In essence, as a general rule in the perceptual control of action, the primary job of the nervous system seems to be to preserve the spatiotemporal structure of the information variable so that it shows up in behaviour. See work by Audrey van der Meer, for example.

      So what exactly are the required transformations that the brain is adding to the mix?

  2. I'm not an 8-year-older but I really needed this.

    1. I will take that as a compliment and a hint that I hit the right spot :)

  3. Jonathan Led Larsen6 April 2017 at 06:40

    Well - I suppose the trajectory of a baseball is somewhat dependent on the air it flies through (temperature, wind) and how this interacts with details - which are not easy to read - of the ball's movement (such as spin). This means that the trajectory is not so easily computed from only the angle and speed of the ball - environmental condition and the interaction with these needs to be taken into account as well and they only unfold as the ball traverses it's course. Thus you need an ongoing perceptual activity to correct your position.
    A different example - billiard - is perhaps not so straight forward? Getting ready to make a shot typically involves moving round the table, taking in different perspectives, but it also involves considering the exact angle and power you wish to hit the ball with.
    Wouldn't a dual theory better account for this? Where you have both the embeddedness of moving round the table and a more theoretical reasoning about strength and angle? I don't suppose ecological psychology would claim, that we culturally can't develop and learn to use our brains to something like holding a representation of something in our imagination? Certainly that is what much of school is about. Of course the representation would be reminiscent of actual motor-perceptual experiences but it would nevertheless be another way of planning action than the in-the-moment of running to catch a ball?

    Good post, by the way - great you take the time :-)

    1. The problem with the dual model is twofold:

      1. There's no evidence for it. Whenever computational and ecological solutions are tested directly against one another, the computational solution is no where in sight in the data.

      2. You are proposing that the nervous system implements two radically different kinds of solution. This is actually quite a big claim, and comes with problems like "when do I use this one vs that one?". So while I get the urge to combine, it's important to remember it is not a trivial thing to do.

      Billiards; I bet if you looked at eye movements and the kinds of warm up movements that happen as people heft their cues etc, you would see a lot of action designed to parameterise and calibrate an upcoming action. I don't see huge trouble here for the ecological approach.

      Thanks for the questions, glad you enjoyed the post! :)

  4. Jonathan Led Larsen6 April 2017 at 12:46

    Thanks for the answer. I feel the need for a follow-up comment, though :-) First 2), then 1) and finally billiards :-)

    2) Perhaps we could call them 'ends of a continuum' instead of opposites? I think you sort of sidestep the issue I try to raise: The humans I've talked to generally agree that there are situations where their cognition is more dependent on movement than at other times. Planning how to solve a math puzzle, before putting the pencil to the paper, is an example from one end of the spectrum. Catching a ball on a windy day is from the other end of that spectrum. It is intuitively difficult to see these two situations as intrinsically similar and entailing the exact same processes.

    1) There are situations in which movement is severely restricted but where cognition is intact. A strong example is the locked-in-syndrome where people are unable to move but still able to function intellectually.

    Billiards: Such a 'locked in person' could in theory successfully play a game of virtual billiard by adjusting angle and strength of a virtual cue using a brain-computer interface. Perhaps disregarding minimal eye movement this would eliminate the use of warm up movements etc.

    If you could accommodate the explanation you started out with, so that it explicitly tackles situations which are not absolutely dependent on movement, I think it would sink in better... with me at least... and perhaps other 8 year olds as well ;-)

    1. No one is saying brains don't do things. The trick is they are not doing everything. Much of the form of our behaviour comes directly from the task and our perceptual contact with it.

      Playing billiards is not the same task as virtual billiards, and there is literally no way to make them the same task. So the fact you can rig up a virtual game here tells me nothing about performance in the real version.

      Notice that absence of movement does massively alter the things you can do. If you could test action control in such a person, you would find all kinds of calibration problems, etc that are the consequence of not being able to move. So sure, some stuff can happen without overt movement; but it's unstable and limited.

      Don't get too caught up on the outfielder problem. It's just an example to clarify the different approaches.

  5. Nice post Andrew. How would you explain the differences between Behavior Analysis (B. F. Skinner's Radical Behaviorism) and Gibson's Ecological Approach? I'm aware the explanation of the outfielder's behavior would be quite different from a Skinnerian approach, but how would you describe the philosophical, epistemological differences, as you did between the Computational Approach and the Ecological Approach?

    Thank you!

    1. First, I love and respect Skinner's work. The man was not quite right but by god he was onto something and he was not wrong about so many things.

      What Skinner lacked is dynamics and ecological information; he was mapping out structure and contingencies in the environment and showing how that all showed up in and accounted for behaviour. But he was expressing his description of the environment in inadequate terms (he needed dynamics) and he didn't have a theory of perception (he needed information).

      Basically he was sufficiently right in his belief that the structure of behaviour often comes from the structure of the environment that he was able to generate reliable effects. He had the right idea but an inadequate formal vocabulary.

      Gibson was very much at the behaviourist end of things; he thought of himself as a molar behaviourist like Tolman. He just went a few steps further into the mechanistic details of how it all actually worked.

  6. First of all, the 8 (now 9 year old) and his Dad were very pleased to see their request answered.

    Secondly, it did make sense - all of it (though some explaining of the language was needed for the 9 year old).

    The questions which now come up are:

    1) Where does thought come from? Is it just a stimulus response?

    2) What about imagination / dreams / daydreams / emotions?

    3) Could you give us some more examples of where " the details of the perceptual coupling to the environment have been worked out and tested"?

    And thanks once again for this post - it's made for a very interesting afternoon's conversation!

    1. Glad it helped!

      (1) and (2); no idea. They are big questions and the ecological approach has not tackled everything. We have a first draft idea of what might be going on in the brain here, although it's still first draft and also a little technical. The hypothesis is that we think you can build more abstract thought than required for catching a fly ball out of interactions with information and a nervous system handling that interaction. There are MANY devilish details to be worked out!

      (3); check the rough guide for the posts on coordinated rhythmic movement, especially about Geoff Bingham's perception action model (we summarised a lot of this in this paper). I did my PhD on this with Geoff and we've done a ton of empirical work mapping this all out in a lot of detail.

    2. Also, if you have the time; what language didn't work? My kids are 3 and 5 so I didn't have an 8yr old to run things by :)

  7. I loved this and I will be coming back here.

    It seems to me that, in each of a series of moments, the outfielder is doing whatever brings him/her closer to a desired feeling -- maybe it could be called "the feeling of what's worked in the past in this context" or
    "the feeling of being closer -- vs further away -- from the moment of catching this ball." I'm wondering if, instant by instant, behaviour depends on fluctuations in that feeling.

    I wouldn't think that physical movement is necessary; when I was enthralled by algebra in high school I got good at resolving equations quickly because I had a feeling, based (I assume) on past experience of similar feelings and actions, that a particular step had a good chance of bringing me closer to a desired state (the feeling of satisfaction / achievement of having resolved the equation.)

    But it does seem likely that a body is necessary; otherwise where does the input of "feels good" or "feels bad" come from?

    Is there overlap here with "implicit learning" and "tacit knowing"? Or are these just labels referring to "things that are hard to explain"?

  8. I always look forward to your blog.
    How does the ecological versus non-ecological contrast differ from dead-reckoning versus trial and error?
    Also, your outfield baseball example does not allow enough for uncertainty. Considering the interplay between a pitcher and a batter would force the analyst to consider the indetermiancy of inter-brain interaction as well as probabilities based on past experience and differential ability. There is an enormous amount of historical data that could be mined to test alternative analyses against a large set of variables.

  9. Hiya! I also think your explanation was very clear and very winning. In reading some of the comments, I think what might need more rhetorical emphasis is that the strategy of MOVING in relation to perceptual data is fundamentally different from COMPUTING. Then, when we watch fielders actually do this, it looks very much like they are using the former. This seems quite coherent to me vis-a-vis the initial contrast between models of cognition, and I wonder if, ironically, resistance to embodied cognition might be the old cartesian dualism making a curious comeback. That is, have we swallowed the bitter pill that mind is a product of evolution, but now are refusing to honor the feet? I'm reminded of Pinker's claim that there will never be a Decade of the Pancreas. But hasn't the stomach recently been called a "second brain"?