How Every Movie & Video Game Tricks Your Brain

Thank you to Wren for supporting PBS.

Hey smart people, Joe here. In the  1820s, there was a dinner party.  

Mathematician Charles Babbage was there with  astronomer William Herschel. You might know  

Babbage as the father of computing and  Herschel as the guy who found Uranus. It  

sounds like a fun dinner party. Well, after  dinner, Herschel asked Babbage a question,  

"How can you see two sides of a coin at  the same time?" Well, Babbage's answer,  

"Just look at its reflection in the  mirror," pretty creative solution,  

but Herschel had an even better answer. He spun  the coin on the table, like magic both sides of  

the coin seemed to blend together as if they  were seeing them at the same time. Now, this  

story got around until an Irish doctor heard it  and it gave him an idea. He put two pictures on  

either side of a disc. And when he spun them  between the string, the two images became one.

This became known as the thaumatrope.  

This was one of the most popular toys of the 19th  century, but it was eventually mostly forgotten,  

except that it wasn't. This and the toys that it  inspired are the reason you can watch this video,  

it's all why movies exist and cartoons and video  games. Every moving image we view on screens,  

big or small, can trace its origin to this  simple illusion, one that can take still  

images flashed in sequence and fool our brain into  perceiving motion. So what makes this all work?

Spinning discs are one of the simplest ways  to create illusions that blend images into  

one. And in the 19th century, they inspired a  whole range of so-called philosophical toys,  

including this one, a spinning disc with several  slits cut around it. Now, when viewed in a mirror,  

a series of still images on one side  appear as one moving image. The illusion  

of apparent motion. These inspired a series of  Victorian cinematic toys that were commercialized  

throughout the night 19th century, selling by the  thousands and captivating people around the world.  

These cinematic toys became more and more  complex, eventually giving birth to the era  

of the moving picture. Despite the fact  that no one quite knew how they worked.  

Ultimately, those toys directly led  to this invention: the film projector.

To create the illusion of movement, a sequence  of still images is fed between a light source  

and a set of lenses, projecting a series of  pictures on a screen, but simply projecting  

a scrolling stream of still images alone is  not enough to create the illusion of motion,  

we merely see an indecipherable blur.  This illusion is about what you don't see.  

What actually happens in a projector is one  still frame is projected on screen, then the  

screen goes black while the film is in advanced  one full step and the next still image is shown.  

The film doesn't move continuously, but instead is  advanced frame by frame many times every second.  

Inside the projector is a carefully timed shutter  

that blocks the light so that whenever the film is  moving, you only see black. When these images are  

flashed on the screen at a fast enough rate,  we perceive a moving picture. Before around  

2010, it's likely that any movie that you  saw in a theater was projected like this.

Most of the time, the movie wasn't actually  moving and you spent half your time in the theater  

completely in the dark. Now today, of course,  films and other moving pictures are projected  

digitally or displayed directly on digital  screens. As you can see in these incredible clips  

from our friends The Slow Mo Guys, modern screens  no longer flicker to black between still images,  

instead millions of individual pixels  are refreshed dozens of times per second,  

but effect is still the same. What's getting  beamed into your eyes is a series of still  

images and that's true on screens big  or small. Thanks to this technology,  

in the last century and a half or so, billions of  minds have been tricked into seeing moving images  

that aren't really there. Many of  us spend hours every day staring at  

these illusions and never even think twice  about it and it all traces back to these.

These were the first forms of moving picture  entertainment. Though, there are some hints from  

caves and artifacts that prehistoric cultures  may have made versions of their own long before  

as well, which is pretty cool. It's been said  that our understanding of vision was changed  

as much by these toys as the field of biology  was changed by the invention of the microscope.  

Literal toys inspired fundamental  questions about how our brains work,  

how we perceive the world and how we construct  

reality itself and scientists today are still  using these illusions to tackle those questions.

I made something that I have to show you  right off the bat. I hope this works.  

It's my Eagleman trope.

Yes, that works surprisingly  well even over video. I like it.

Why does this work? I mean, this  is like a toy that people played  

with in the 1830s or something? What is happening.

So why does that work? It's because  when the brain sees something,  

even if it's very rapid, your brain's unable to  turn it on and off that quickly, so your brain  

sees things for longer than your eye does.  This is what's called persistence of vision.

The idea of some kind of visual persistence  goes back long before Babbage's dinner party,  

even to ancient Greece and Egypt. Early  philosophers noticed that streaks from  

lightning seem to stick around for a  split second after the flash or that  

the sun stayed in their vision after they  looked away. Don't stare at the sun ever,  

okay. The guy who invented this did an experiment  where he stared at the sun and he went blind.  

Just don't do it, people. It's bad for you.  In the 11th century, an Arab mathematician and  

philosopher noted that a flickering flame seemed  to appear where it was a moment before. DaVinci  

and Newton even devoted time to these mysteries.  All of those early explanations centered on the  

idea that the light that enters our peepers  temporarily burns a scene into our eyes,  

then it's basically wiped clean, our eyes refresh  and then the next scene is burned in and so on.

In 1765, an Irish mathematician did  this experiment with glowing embers.  

He calculated the velocity you'd have to spin to  see the streak of light make a complete circle  

and estimated the supposed refresh rate the  eye to be 130 milliseconds. You can try this  

experiment for yourself. If you rapidly move  a point of light, it appears to leave trails  

behind. These illusions are not simply artifacts  of my camera or the shutter speed, you can see  

them if you try this in person. This theory that  the eye acts like a camera with some refresh rate,  

capturing slices of the world in single frames  stuck around for a long time, but it's wrong.

That's right. It has to do with what's going on  in the brain. There are many things that happen  

in the retina and that's just the first stage  and then you go back into the brain to an area  

of the visual thalamus and then the visual cortex  and then all these areas of the visual cortex.  

All of these things are interplaying.

The effects in these toys and movies and screens  today create an illusion neuroscientists call  

apparent motion. When you see a series of still  images appear to move, it's your brain that's been  

fooled, not your eye, and how it really works  is pretty freaky. Look at this arrangement of  

dots and here's a slightly different arrangement  and another. These dots are just dots, right?  

But played rapidly in succession, at some point  these dots become something else. And depending on  

just how those dots are arranged relative to  each other, as they move our brains can write  

very different stories about what we're  seeing, even invent different characters.  

Now, look at these two dots. Does it look like  one is chasing the other? Or perhaps they've  

switched? Well, none of those are true. One dot  is simply moving erratically around the other,  

but when you add a moving background,  suddenly you perceive something else.

That may feel like a trick, but there's no reason  to think dots chase each other. They're just  

dots. The position of one of  the dots never even changes,  

you made it do that. It's a story invented by your  brain. How about this? Did it move or do they just  

blink on and off? So why are our brains inventing  stories that don't exist? Well, these little white  

lies our brains tell us are an important part of  understanding the visual information in our world.

If I see a bird flying and  it goes behind the tree,  

then a moment later I see the bird emerge on  the other side of the trunk and for my motion  

detectors, that's perfectly fine. I say, "Oh  great, there was motion there, smooth motion."

The bird didn't disappear. Birds  just don't do that in the real world.

Exactly, exactly. And this is  this notion of object permanence.  

Your brain is wanting to say, "I want to  still hang on to that bird wherever it went."

I played a lot of Mario Brothers  growing up on the old Nintendo.  

And actually, when you look at what's happening  frame by frame, like you got a little plumber man  

standing there and then all of a  sudden you have like a plumber guy  

with his fist up a few pixels away and there's  nothing in between that. But when you play the  

game, you have this sensation that this  character smoothly jumped up in the air.

That's exactly right and that's what  apparent motion is about. Your brain  

does all kinds of computations and says, "Oh wait,  

here's Mario and there was a Mario over here.  It must be the same Mario and he has moved."

Your brain makes an unconscious decision  that images seen at different places  

and at different times represent the same  object. This is called correspondence. The  

snake in the classic game isn't moving,  it's simply a series of blocky shapes,  

but our brains object permanence interprets  the shape as the same snake between frames.  

And based on our understanding of how  snakes usually move in the real world,  

we assemble those shapes into motion. Even in  modern video games while the picture's quality  

has definitely improved and the technology has  increased the number of images played each second,  

what we see as continuous motion  is just a series of still images.

What's really interesting is that the brain can  only do this in retrospect. So in other words,  

it collects up the data from frame one, then from  frame two and then it retrospectively says, "Ah,  

it must have moved smoothly between those  things." And part of the way we know this  

is from a visual illusion called the colorify  effect, which is if I show, let's say, a red dot  

and then I show a blue dot, you will think that  it moved from one position to the other, but you  

will also have the impression that it changed  color halfway between, and that's only possible  

after the second dot has appeared, even though  it feels like, "Oh, I'm seeing this thing and  

it changes color to blue," you can only know  the position and the color after it's over.

So your brain is going back and writing a story  that never happened in between those two things.

That's exactly right. And actually, I coined the  term for this some years ago called postdiction.  

It's the opposite of prediction, which is to  say your brain collects up all the information  

about a scene before it retrospectively says  what it thinks it saw. You're living about...  

probably about half a second in the past.  So when you think the moment now occurs,  

it's already happened a long time ago, your  brain's collecting up all this information,  

including what's coming through your eyes  and your ears and your fingertips and  

your toes and everything, which comes in at  different speeds. Your brain has to collect  

all that up, stitch it together and then  say, "Okay, here's what I think happened."

I'm really starting to question reality more and  more as we talk here. Here's another example of  

this. There are cells in your eyes, even in the  visual processing centers of your brain that  

respond fast enough to detect rapidly flickering  light, but what's really interesting is there's  

a threshold where your brain flips, it decides  to ignore that flickering and you suddenly see a  

constant light source. When the rate of flicker  is higher than 35 flashes per second or so,  

your brain says, "I'll just smooth this out  to look like real life," even though your eyes  

and parts of your brain are sensing flickering  light. Movie cameras record 24 images per second,  

but if you played that back at 24 flickers  per second, it would look like a strobe light.  

Film projector shutters flicker two or three times  for each still image. Digital displays refresh two  

or three times before the image changes. It's over  that threshold, so your brain ignores the flicker.

When you think about this, that your brain  is taking in all of this information from  

your senses, it's comparing it with your past  experience about how things in the universe  

should behave, then your brain goes back and  writes the story of what it thinks you saw,  

it brings up a lot of interesting questions,  like how did our brains end up this way? I mean,  

our species has only been staring at illusions  like these for a couple of centuries, but  

our brains are way older than that. We didn't  evolve with cartoons and TV and video games around  

or even spinning picture discs, but being able  to sense motion, even the illusion of motion,  

has been a big part of our specie's survival.  Maybe 99 times out of a hundred that wasn't  

a tiger in the grass, it was just an illusion  of motion, but our only ancestors who survived  

are the ones whose motion detectors  got it right the hundredth time too.

To me, what I think it illustrates  is that your whole brain, your whole  

perceptual world is built out of cells that  are just trying to do the best job they can.  

Essentially, we found a little loophole,  a trick that we can play on these cells,  

but we certainly didn't need  to evolve to see movies.

I'm glad we did though. Otherwise,  

we wouldn't be able to talk here and no one  would watch YouTube. I like this reality.

I've always been attracted to visual illusions  because we open our eyes and there's the world  

and we take it to be reality and so it's  very interesting to see how the brain  

constructs this reality. It's like if you were  a fish in water and you were asked to describe  

water, you wouldn't be able to do it. But  if you see a bubble that comes up past you,  

you might think, "Whoa, what is that thing?" Well,  that's exactly what visual illusions are to us.

You almost have to break the brain or find  its shortcomings to figure out how it actually  

works. It's doing the best that it  can, I guess, with limited information.

Well, we wouldn't know what we're missing,  

so we think it's doing a good job, but  maybe in a hundred years when we all  

wear glasses for detecting infrared and  ultraviolet and the rest of the electromagnetic  

spectrum, maybe we'll say, "Wow, it's actually  been doing a terrible job this whole time."

I look forward to living like  a bee, that sounds amazing.  

So why do videos work or movies or games? Well,  the old idea that our eyes work like a camera,  

that motion is burned into our eyes the way light  hits film, that turns out to be wrong or at least  

incomplete. Your eyes are really good at  sensing the universe and they do sense the  

individual still images you are watching and  send that information to your brain, but our  

brains know that isn't how the universe really  works so it fills in the gaps and blends this  

with something else, something that isn't  really there. Imagine if you lived in a black,  

silent room, cut off from the outside world. The  only descriptions of the outside world that you  

get are periodic notes passed under the door.  Your image of reality will be a story you write  

based on that limited information. That's how  it is for your brain. In something as normal  

as watching TV or a movie, it isn't the actors  that are creating the story, it's your brain.

So what does this all mean? Well, it  means that this video is an illusion,  

but it's one that you can trust. Stay curious.

Thank you to Wren for supporting PBS. Wren is a  website where you calculate your carbon footprint  

and then offset it by funding projects that plant  trees, protect rainforests and suck carbon from  

the sky. By answering a few questions about your  lifestyle, you can find out your carbon footprint  

and how you can reduce it. Now, no one  can reduce their carbon footprint to zero,  

so you can offset what you have left after  reducing. Subscribers receive monthly updates  

from the tree planting, rainforest protection  and carbon removal projects that they support.  

You get to see the trees you planted  and what your money is spent on.  

You can learn more about Wren's climate projects  at wren.co. And as always, a big thank you to  

everyone who supports the show on Patreon.  We could not make these videos without you.

If you'd like to learn more about  how you can support the show,  

just click the link down in the description  or watch this rapidly spinning disc.  

You're getting very sleepy.  You want to click the button  

and support the show on Patreon. of computing  and Herschel as the guy who found Uranus.