Most of the millions of people lining up to see the latest blockbuster film know that the mind-boggling effects they are about to see on the big screen are made “with computers”. Big movies can cost hundreds of millions to make, and typically less than half of the budget is spent on marketing the premiere, or paying the actors upfront. Plenty of money left over to buy a big computer and press the ‘make effects’ button, right? Except that these movies close with 5-10 minutes of rolling credits, and about half of them are names of people working in visual effects, not computers. (Seems like a tentpole movie crew these days has more TDs (Technical Directors) than any other kind of directors combined.) [If you think making cool computer effects sounds easy, just download the open source tool blender, and create whatever your mind can imagine …]
Computer simulations are no longer just for engineering and science, they can be used as extensions of our imagination. A simplified set of rules and initial conditions are input, then a few quick test renders are made with low resolution. Twiddling the knobs, how many particles, viscosity, damping, scattering, and finding the correct lighting and camera angles, etc., iterating until you are happy or (more likely) forget what you were trying to accomplish.
Before computers, before visual effects and film, people had to use their own imagination to make entertaining simulations. The most light-weight technology to accomplish that was storytelling, guided narrative with characters and settings. The rules of the simulation were the rules of plausibility within the world of the story. The storyteller created the events, but the listeners enacted them in their imaginations. The storyteller received immediate feedback from his audience if the story became too implausible.
But once the audience “buys in” to the characters and their narratives, they become emotionally invested in them as if they were real people. Fictional characters, today protected by copyrights and corporate trademarks, can still suffer unexpected fates, and new audiences to a fictional world often demand to be protected from “spoilers” that would make it difficult for them to simulate them in their imaginations. Real people do not know what their future is, and to ‘play’ the role convincingly and without foreshadowing, it is best to live with incomplete information.
If I start to read a book of fiction, written many decades ago, when is the simulation of the characters happening? Does every reader simulate the main characters each time they read the book, or did the author execute the simulation, and the readers are only verifying that the story is plausible? Certainly I feel like I am imagining the phenomenal ‘qualia’ that the characters in the book are experiencing, but at the same time I know that I am just reading a story that was finished a long time ago. Am I lending parts of my consciousness to these paper zombies?
In a well-known book of genre-defining fantasy, after hundreds of pages of detailed world-building, two characters are beyond weary, in the darkest trenches of seemingly unending war, when one of them starts to wonder if they shall
“[…] ever be put into songs or tales. We’re in one, of course; but I mean: put into words, you know, told by the fireside, or read out of a great big book with red and black letters, years and years afterwards.”
It’s not a bad way to put it, but even for myself at age 12, characters in a book discussing the possibility of them being characters in a book was just too self-referential to be plausible, and pushed me ‘out’ of the story for a moment. (A bit like characters in a Tex Avery cartoon running so fast they run outside the film. We get the joke, but let’s keep the “fourth wall” were it is for now.)
Since the book was written long ago, and has not been edited since, it can be argued that none of its characters have free will. The reluctant hero makes friends, sacrifices comforts, has unexpected encounters and adventures, all while trying to get rid of the “MacGuffin” that has fallen into his hands. When at last he arrives at the portal of Chaos where the artifact was forged, does his determination to destroy it falter, or will something totally unpredictable happen? To have any enjoyment in the unfolding of the story, the readers must believe that the actions of the characters have significance, and play their roles in our minds as if they had free will.
There are also professional actors, people who take to the stage night after night, repeating familiar lines and reacting to the events of the screenplay as if they were happening for the first time:
“For Hecuba! What’s Hecuba to him, or he to Hecuba, That he should weep for her?”
A good performance can evoke both the immediacy and intimacy of a real emotional reaction, but the audience still needs to participate in the act of imagining the events as actual, to understand at some emotional level “what it is like” for the characters in the play to have their prescribed experiences.
What to me really sells a scene is the interplay of the actors, not so much how photorealistic the visual effects happen to be. A painted canopy plays the part of a majestic sky, or a sterile promontory becomes earth for the gravedigger, if all the actors act as though it were so.
As convincing as our simulations can be, the point of fiction is that we enter it knowing that it is fiction, that we can always put the book down, or step outside of the theater. Fiction is not realtime, and it always requires audiences to imagine some parts of it (for example, what happens between scenes?). We choose not to pay attention to the man behind the curtain, or analyze the plot too much, when we want to immerse ourselves for a moment.
[Having said that, I don’t mean to imply that it is impossible to become lost inside made-up stories, and confuse them into reality in a quixotic manner, but that is usually not the intention of the storyteller (though it could be useful to the intentions of a shrewd marketer, politician or cult leader).]
Time inside the simulation is independent from time in the real world. In addition to pausing the simulation, monolithic or pre-computed simulations can be executed in different sequential order from the assumed order inside the simulated world. This is used to great effect in some books, which describe the same event multiple times in different chapters, but from the point of view of different characters. Each perspective usually gives the reader some extra information, something that no character in the simulation can have. Viewing from outside the simulation, the audience gets an almost god-like view of the situation, sometimes even enhanced with indexes and bookmarks so they can page back and review the events in previous chapter (but not forward, since it would “spoil” the freshness of the simulated experience).
Pre-written narrative simulations, movies and plays, edit out the parts that are though to be uninteresting. This is a careful balancing act, because editing out too much leaves the characters and their actions too distant, and harder to relate to. Leaving in too much unnecessary details on the other hand can appear gratuitous and put off many viewers and readers, who will surely find better things to occupy their time.
Computer simulations today almost always consist of time-steps. A numerical approximation of some evolution equation uses the results of the previous steps to compute the next step of the simulation. The smaller the time interval used, the closer the approximation is to the real solution [in the mathematical sense, for example a piecewise linear line approximating a smooth curve], and the longer it takes to compute. If the simulation is pre-computed, the audience need not view every individual step, to make use of the simulation. [Note: In terms of the blender software, the physics timestep is independent of the framerate of the animation, and changing either will affect the needed baking and/or rendering time.]
When played at sufficient number of frames per second, our mediocre senses are fooled to interpret a sequence of still images as moving pictures [interestingly, while higher framerates increase the realism, and are technically possible today, moviegoers still prefer the “cinematic” 24 fps for big screen cinema, or even less for dream-like sequences]. But it would be naive to think that the timeline of the physical world also consists of individual static states, progressing in infinitesimally short step transitions across the whole Universe. Such ideas of motion were debated already hundreds of years BC by the Eleatic school, most famously with the paradoxes of Zenon. [Note: Unfortunately, even logical and analytical philosophy usually implicitly assumes that there is always a “world at time t” snapshot, at any chosen time t, with defined entities and properties. But that is a topic for another post.]
Stories can be used used to simulate, or theorize, about the minds of others. By vicariously identifying with characters, we can sometimes glimpse across the inter-subjective gap. Even a child can understand that a character in the story does not have all the information, that Red Riding Hood is asking questions because she doesn’t know who she is speaking with. But even voluntary participation in a simulative narrative can reveal hidden agendas in the audience, through transference: For example, did Shakespeare put baits in his Scottish play to “catch the conscience” of king James, or just harmless references to the new king’s known interests?
Some stories contain such powerful or virulent motivations for their characters, that the audience starts doubt their own volition as participants of the simulation. [Note: This could to be related to hypnosis, which can also be induced using only words and signs, and makes subjects doubt their own volition to some degree.] Being part of something larger, even if it is just a simulation, is a recognizable desire in the human psyche. Experience, of real life, as well as other kinds of stories, can also recontextualize previous narratives in new light, and help reframe them with a different viewpoint. [An example of this could be a new parent realizing: “maybe my parents were just as clueless as I am now?”; a kind of subject-object relationship reversal, in the psychoanalytical sense.]
In this transitional state between the simulator and the simulated, we might also strive to theorize on the motivations of a possible future ‘superintelligence’. Why would it spend so much effort to compute realistic ‘ancestor simulations’, extrapolating scenarios from its vast collections of historical data, as in the simulation argument by Nick Bostrom? Perhaps the motivations are the same as when we try to understand our ancestors from the knowledge that we have: If you don’t understand history, you are doomed to repeat it, over and over. Just as intelligence does not imply wisdom, superintelligence certainly does not imply ‘superwisdom’.
Collection and storage of ever more detailed data today gives another perspective to simulation as a way of stepping outside of time. If we can store as much information about the state of the world today as we can, and build a simulated snapshot state of it at a point in time, the transhumanist proposition is that with enough data (how much?) this would be indistinguishable from the real thing, at least for the simulated subjects inside the simulated snapshot. The idea is identical to [and identically depressing as] the afterlife scenarios of many religions and cults. No release for Sisyphos, or holodeck Moriarty from his “Ship in a Bottle”.
The problem of using statistics or probability to determine the ontological status of your consciousness is problematic for many reasons, among them the transitory nature of conscious experience. For the total tally of fictional consciousnesses, do you count the number of different characters in all the scripts, all the actors, or all the audience members? Does it matter if all scripted characters are actually “played” to any audience with phenomenal consciousness? Does a simulation character need to have phenomenal consciousness all the time, or just during some key scenes, zoning out as sleepwalking zombies for most of the time? Since there can be no definite multiplicity for such an ill-defined entity as a consciousness separate from its substrate, counting of probabilities from a statistical point of view is as meaningful as arguing about the number of angels that can dance on the point of a needle.
I don’t consider myself to be any technological “luddite”, on the contrary I believe that technological progress has the potential to create many more breakthroughs in the future, or even an unprecedented ‘flowering’ of the kind that rarely happens in the evolution of life on Earth (for example, the appearance of fruit-bearing plants (a.k.a. flowers) during the Early Cretaceous). I do dislike the word “singularity” in this context though; especially since it originally means an attempt at extrapolation beyond the limits of the current working model. (For example, the financial “singularity” of 2008, when loan derivatives cast off the surly bonds of fundaments, and left the global markets in free fall.) All flowers have their roots in the earth, and in the past, which they must grow from. No ‘sky-hook’, or ‘future-hook’ can seed the present.