The Simulation Narrative

Most of the millions of people lining up to see the latest blockbuster film know that the mind-boggling effects they are about to see on the big screen are made “with computers”. Big movies can cost hundreds of millions to make, and typically less than half of the budget is spent on marketing the premiere, or paying the actors upfront. Plenty of money left over to buy a big computer and press the ‘make effects’ button, right? Except that these movies close with 5-10 minutes of rolling credits, and about half of them are names of people working in visual effects, not computers. (Seems like a tentpole movie crew these days has more TDs (Technical Directors) than any other kind of directors combined.) [If you think making cool computer effects sounds easy, just download the open source tool blender, and create whatever your mind can imagine …]

Computer simulations are no longer just for engineering and science, they can be used as extensions of our imagination. A simplified set of rules and initial conditions are input, then a few quick test renders are made with low resolution. Twiddling the knobs, how many particles, viscosity, damping, scattering, and finding the correct lighting and camera angles, etc., iterating until you are happy or (more likely) forget what you were trying to accomplish.

Selene Endymion MAN Napoli Inv9240

Before computers, before visual effects and film, people had to use their own imagination to make entertaining simulations. The most light-weight technology to accomplish that was storytelling, guided narrative with characters and settings. The rules of the simulation were the rules of plausibility within the world of the story. The storyteller created the events, but the listeners enacted them in their imaginations. The storyteller received immediate feedback from his audience if the story became too implausible.

But once the audience “buys in” to the characters and their narratives, they become emotionally invested in them as if they were real people. Fictional characters, today protected by copyrights and corporate trademarks, can still suffer unexpected fates, and new audiences to a fictional world often demand to be protected from “spoilers” that would make it difficult for them to simulate them in their imaginations. Real people do not know what their future is, and to ‘play’ the role convincingly and without foreshadowing, it is best to live with incomplete information.

If I start to read a book of fiction, written many decades ago, when is the simulation of the characters happening?  Does every reader simulate the main characters each time they read the book, or did the author execute the simulation, and the readers are only verifying that the story is plausible? Certainly I feel like I am imagining the phenomenal ‘qualia’ that the characters in the book are experiencing, but at the same time I know that I am just reading a story that was finished a long time ago. Am I lending parts of my consciousness to these paper zombies?

In a well-known book of genre-defining fantasy, after hundreds of pages of detailed world-building, two characters are beyond weary, in the darkest trenches of seemingly unending war, when one of them starts to wonder if they shall

“[…] ever be put into songs or tales. We’re in one, of course; but I mean: put into words, you know, told by the fireside, or read out of a great big book with red and black letters, years and years afterwards.”

It’s not a bad way to put it, but even for myself at age 12, characters in a book discussing the possibility of them being characters in a book was just too self-referential to be plausible, and pushed me ‘out’ of the story for a moment. (A bit like characters in a Tex Avery cartoon running so fast they run outside the film. We get the joke, but let’s keep the “fourth wall” were it is for now.)

Since the book was written long ago, and has not been edited since, it can be argued that none of its characters have free will. The reluctant hero makes friends, sacrifices comforts, has unexpected encounters and adventures, all while trying to get rid of the “MacGuffin” that has fallen into his hands. When at last he arrives at the portal of Chaos where the artifact was forged, does his determination to destroy it falter, or will something totally unpredictable happen? To have any enjoyment in the unfolding of the story, the readers must believe that the actions of the characters have significance, and play their roles in our minds as if they had free will.

There are also professional actors, people who take to the stage night after night, repeating familiar lines and reacting to the events of the screenplay as if they were happening for the first time:

“For Hecuba! What’s Hecuba to him, or he to Hecuba, That he should weep for her?”

A good performance can evoke both the immediacy and intimacy of a real emotional reaction, but the audience still needs to participate in the act of imagining the events as actual, to understand at some emotional level “what it is like” for the characters in the play to have their prescribed experiences.

What to me really sells a scene is the interplay of the actors, not so much how photorealistic the visual effects happen to be. A painted canopy plays the part of a majestic sky, or a sterile promontory becomes earth for the gravedigger, if all the actors act as though it were so.

As convincing as our simulations can be, the point of fiction is that we enter it knowing that it is fiction, that we can always put the book down, or step outside of the theater. Fiction is not realtime, and it always requires audiences to imagine some parts of it (for example, what happens between scenes?). We choose not to pay attention to the man behind the curtain, or analyze the plot too much, when we want to immerse ourselves for a moment.

[Having said that, I don’t mean to imply that it is impossible to become lost inside made-up stories, and confuse them into reality in a quixotic manner, but that is usually not the intention of the storyteller (though it could be useful to the intentions  of a shrewd marketer, politician or cult leader).]

Time inside the simulation is independent from time in the real world. In addition to pausing the simulation, monolithic or pre-computed simulations can be executed in different sequential order from the assumed order inside the simulated world. This is used to great effect in some books, which describe the same event multiple times in different chapters, but from the point of view of different characters. Each perspective usually gives the reader some extra information, something that no character in the simulation can have. Viewing from outside the simulation, the audience gets an almost god-like view of the situation, sometimes even enhanced with indexes and bookmarks so they can page back and review the events in previous chapter (but not forward, since it would “spoil” the freshness of the simulated experience).

Pre-written narrative simulations, movies and plays, edit out the parts that are though to be uninteresting. This is a careful balancing act, because editing out too much leaves the characters and their actions too distant, and harder to relate to. Leaving in too much unnecessary details on the other hand can appear gratuitous and put off many viewers and readers, who will surely find better things to occupy their time.

Computer simulations today almost always consist of time-steps. A numerical approximation of some evolution equation uses the results of the previous steps to compute the next step of the simulation. The smaller the time interval used, the closer the approximation is to the real solution [in the mathematical sense, for example a piecewise linear line approximating a smooth curve], and the longer it takes to compute. If the simulation is pre-computed, the audience need not view every individual step, to make use of the simulation. [Note: In terms of the blender software, the physics timestep is independent of the framerate of the animation, and changing either will affect the needed baking and/or rendering time.]

When played at sufficient number of frames per second, our mediocre senses are fooled to interpret a sequence of still images as moving pictures [interestingly, while higher framerates increase the realism, and are technically possible today, moviegoers still prefer the “cinematic” 24 fps for big screen cinema, or even less for dream-like sequences]. But it would be naive to think that the timeline of the physical world also consists of individual static states, progressing in infinitesimally short step transitions across the whole Universe. Such ideas of motion were debated already hundreds of years BC by the Eleatic school, most famously with the paradoxes of Zenon. [Note: Unfortunately, even logical and analytical philosophy usually implicitly assumes that there is always a “world at time t” snapshot, at any chosen time t, with defined entities and properties. But that is a topic for another post.]

Stories can be used used to simulate, or theorize, about the minds of others. By vicariously identifying with characters, we can sometimes glimpse across the inter-subjective gap. Even a child can understand that a character in the story does not have all the information, that Red Riding Hood is asking questions because she doesn’t know who she is speaking with. But even voluntary participation in a simulative narrative can reveal hidden agendas in the audience, through transference: For example, did Shakespeare put baits in his Scottish play to “catch the conscience” of king James, or just harmless references to the new king’s known interests?

Some stories contain such powerful or virulent motivations for their characters, that the audience starts doubt their own volition as participants of the simulation. [Note: This could to be related to hypnosis, which can also be induced using only words and signs, and makes subjects doubt their own volition to some degree.] Being part of something larger, even if it is just a simulation, is a recognizable desire in the human psyche. Experience, of real life, as well as other kinds of stories, can also recontextualize previous narratives in new light, and help reframe them with a different viewpoint. [An example of this could be a new parent realizing: “maybe my parents were just as clueless as I am now?”; a kind of subject-object relationship reversal, in the psychoanalytical sense.]

In this transitional state between the simulator and the simulated, we might also strive to theorize on the motivations of a possible future ‘superintelligence’. Why would it spend so much effort to compute realistic ‘ancestor simulations’, extrapolating scenarios from its vast collections of historical data, as in the simulation argument by Nick Bostrom? Perhaps the motivations are the same as when we try to understand our ancestors from the knowledge that we have: If you don’t understand history, you are doomed to repeat it, over and over. Just as intelligence does not imply wisdom, superintelligence certainly does not imply ‘superwisdom’.

Collection and storage of ever more detailed data today gives another perspective to simulation as a way of stepping outside of time. If we can store as much information about the state of the world today as we can, and build a simulated snapshot state of it at a point in time, the transhumanist proposition is that with enough data (how much?) this would be indistinguishable from the real thing, at least for the simulated subjects inside the simulated snapshot. The idea is identical to [and identically depressing as] the afterlife scenarios of many religions and cults. No release for Sisyphos, or holodeck Moriarty from his “Ship in a Bottle”.

The problem of using statistics or probability to determine the ontological status of your consciousness is problematic for many reasons, among them the transitory nature of conscious experience. For the total tally of fictional consciousnesses, do you count the number of different characters in all the scripts, all the actors, or all the audience members? Does it matter if all scripted characters are actually “played” to any audience with phenomenal consciousness? Does a simulation character need to have phenomenal consciousness all the time, or just during some key scenes, zoning out as sleepwalking zombies for most of the time? Since there can be no definite multiplicity for such an ill-defined entity as a consciousness separate from its substrate, counting of probabilities from a statistical point of view is as meaningful as arguing about the number of angels that can dance on the point of a needle.

I don’t consider myself to be any technological “luddite”, on the contrary I believe that technological progress has the potential to create many more breakthroughs in the future, or even an unprecedented ‘flowering’ of the kind that rarely happens in the evolution of life on Earth (for example, the appearance of fruit-bearing plants  (a.k.a. flowers) during the Early Cretaceous). I do dislike the word “singularity” in this context though; especially since it originally means an attempt at extrapolation beyond the limits of the current working model. (For example, the financial “singularity” of 2008, when loan derivatives cast off the surly bonds of fundaments, and left the global markets in free fall.) All flowers have their roots in the earth, and in the past, which they must grow from. No ‘sky-hook’, or ‘future-hook’ can seed the present.

The Pilot of Consciousness

We do not know in detail how human consciousness arose, and we only have direct evidence of our own consciousness. But it is common sense to assume that most normally behaving people are conscious to some degree, and that consciousness is a result of biological processes of the body, and its organs, even though we cannot see it directly, the way we appear to see our own consciousness.

Assumptions about the level of consciousness of other people affect modern society at a deep level. A conscious mind is thought to have free will, to a greater degree than simpler animals or machines do. In criminal law, a person can only be judged on the actions they take consciously, but not while for example sleepwalking or hallucinating.

The old metaphor is that consciousness is to the body like a pilot is to the ship. The pilot of a ship needs information and feedback to do his job, but he does not have direct access to it. Instead, he gets his information secondhand from the lookouts, and the various instruments for measuring speed with knots, the compass, and elsewhere. The pilot also does not row the oars himself, or stoke the engines, he just sends instructions below and assumes they will be carried out. The pilot does nothing directly, but all vital information must flow timely through him. Neither is steersman the same role as the captain; piloting work means reacting to the currents and the winds as they happen, not long-term goal-setting or strategic planning.

Ship procession fresco, part 4, Akrotiri, Greece

[Note: The old greek word for pilot is kubernetes, which is the etymological root word for both ‘cyber-‘ and ‘govern-‘ words.]

Piloting a ship is not always hectic, at times the ship can be safely moored at harbour, or the sailing can be so smooth that the pilot can take a nap. But when the ship is at strange seas, risking greatest danger from outside forces, pilot-consciousness kicks in fully, alerting all lookouts and powering the engines to full reserve power, ready to react to whatever happens. When the outside forces show their full might, the pilot is more worried about the ship surviving the next wave, than getting to the destination on time.

The state or level of consciousness is often associated with some feelings of anticipation, alertness, even worry or anxiety; such feelings can even prevent dialing down the level of consciousness to restful sleep, and thereby cause more stress the next day. Pain can only be felt when conscious, hence the cliché of pinching yourself to check if you are dreaming or not. Pathos [the root word for ‘-pathy’ words, like empathy or psychopathy], in all its meanings, is a strong catalyst to rouse consciousness. Only humans are thought to be capable of becoming truly conscious of their own mortality, the conscious mind thus becoming aware of the limits of its own existence.

When the pilot takes over and commandeers a vehicle, the flexibility of consciousness allows him to extend his notion of self to include the vessel. For example, an experienced driver can experience his car sliding on a slick patch of road as a tactile sensation, as if a part of himself was touching the road, and not the tires. In the same way, human consciousness naturally tends to identify itself as the whole individual. Sigmund Freud named the normal conscious part of the mind ‘ego’, which is ‘self’ in latin. His key observation was that the mind is much more than the ego, and that true self-knowledge requires careful study, which he called psycho-analysis.

Introspection is an imperfect tool for studying one’s own mind, due to the many literal and metaphorical blind spots involved. The ego is very capable of fooling itself. This is why it is not considered safe to try doing psycho-analysis by yourself, you should have guidance from someone who has gone through the process. Same applies for some methods of controlling consciousness through meditation.

There are methods of self-discovery that are less dangerous, such as the various personality tests. To extend the metaphor, different pilots have their own favorite places on the ‘bridge’, their habitual ways of operating the ship, or specific feelings associated with its operations. Your ‘center’ may not be in the same place as someone else’s. For example, a procrastinator waits until the last possible moment to make a decision; it could be that only the imminence and finality of a deadline makes their choices feel ‘right’ or ‘real’ enough to commit to. Another example is risk-seeking/aversion: some people only feel alive when in some amount of danger; others do their utmost to pass risks and responsibilities to other people.

Most pilots become habituated to a specific level of stress when operating the self-ship, and cannot function well without it; the types and levels of preferred stress can vary much between individuals. Too much stress however can break the pilot and damage the ship. This is also variable between individuals. Hans Eysenck theorized that sensitivity of an individual to be easily traumatized is correlated to intraversion, or that extraversion could be even redefined in terms of tough-mindedness; but there are other models as well, such as psychological ‘resilience‘, which supposedly can be trained as a ‘life skill’.

Habits are also something that can be consciously trained, and paying attention to our own habits is very healthy in the long run. Consciousness is tuned to fairly limited range of timescales; changes that happen too fast or too slowly do not enter consciousness. Daily habits creep slowly, and without photographs it would be hard to believe how much we change over time. Almost all of the atoms and molecules in our bodies are swapped to new ones every few years, yet our sense of identity remains continuous.

Heraclitus says that “a man’s character is his destiny”, and to know thyself means knowing your weaknesses, as well as strengths. Multitasking is a typical weakness that the pilot often confuses for a strength. Consciousness appears to be the stage where all experience terminates, but the real multitasking happens at the edges; the decision of which of the competing stimuli enter consciousness is never a completely conscious decision. The same applies to commands outgoing, unfortunately. Completeness of control can be an illusion, a form of magical thinking.

Many philosophers have also been fascinated with the true nature of the biggest ‘blind spot’ of consciousness: consciousness itself. There have been various efforts to formalize the ‘contents’ of consciousness, or to model consciousness in terms of ‘properties’ that some entity may or may not ‘have’. There are inherent limitations with these approaches, they should be taken in the original context of phaneroscopy, without drawing any metaphysical conclusions from them.

Not many deny that life, and consciousness, is a process, and human viewpoint is one of moving inexorably forward through Time. The ‘contents’ of consciousness form an unstoppable stream, moving in relation to our self-identity. It seems to us that our mind is anchored to something unmoving and unchanging, with the world changing around it. Yet we identify no specific ‘qualia’ for change or motion, or atomic perceptions of time passing. [There are some thresholds to when we begin recognizing a rhythm, though.]

The true nature of subjective experience may be a ‘hard problem’, but no harder than explaining the true nature of Time. The human condition is to flow from an unchangeable past, inexorably and continuously forward, towards an unknown future, and to only ever be able to act in the present. The pilot role is necessary exactly because the flow that powers all flows cannot be stopped, it can only be navigated.