Home
Welcome to the Reactive-Mise-en-scene wiki.
This wiki documents the code / plugin in this repository, as well as the design concept and theoretical underpinnings informing it. Please note the documentation is a work-in-progress at the time of writing.
Reactive Mise-en-scène is a design concept being developed by Luke Skarth-Hayley for his PhD thesis.
The concept seeks to reconcile tensions between interactivity and narrative experiences by using tacit and implicit inputs from the player, in the form of their attention on specific objects within set locations, to drive changes in the environment/set dressing of scenes, and the narrative environment's tendency over time.
This plugin implements some of these design concepts in Unity and is available for others to explore the possibilities enabled by the system and said concepts.
Introduction Video
The Reactive Mise-en-scène Plugin consists of 3 main systems at present:
- Focus System
- Data Store
- Prefab Loader
A term from research by Benford and Fahlen [ref]. In their research an object was aware of being look at directly or not, and responded to this. This is the fundamental means of interaction in the Reactive Mise-en-scène plugin, though it has been expanded into a series of discrete states that the object can be in beyond Focus or Not Focus. These are:
- Focused
- Attended
- Onscreen
- Offscreen
And these map approximately to the following positions:
Implicit Interactions are those which occur without the direct knowledge of (certain) users. You often see this in ubiquitous computing and Internet of Things scenarios, but we might also point to things like usage tracking on webpages and other user analytics as ways the user's engagement with something is used for interactions that occur without their awareness.
Tacit Interactions are those that occur without a clear causal link between the viewer/player and that which is being interacted with. For example, the Focus-based interactions that occur in the moment via this plugin might be considered tacit: you are not pressing a button and explicitly interacting, rather the position of the camera.
This refers to a component that takes an object's position on camera from the Focus component and tracks it over time, building up an attention rating metric that is a combination of time tracked on all visible states (onscreen, attended, focused) multiplied by their attention multiplier value (which can be adjusted on the component). The attention ratings from this component feed into the attention data manager (see below), which tracks attention ratings per-tendency for each locale and globally by summing the attention ratings of all objects based on their tagged locales and tendencies.
A locale is a tag for an object to associate it with a specific area. Broadly, you'll want to use this as a means of organising a set of objects that accumulate attention over time, and then use the locale tag to get a cumulative level of attention for that grouping. It's also a useful means of thinking about which objects sit in which scene / area / piece of geometry.
A tendency is a way of tagging an object as being associated with a specific narrative tendency. This is not a branch or other means of traversing a narrative path, but rather best viewed as a narrative "flavour" or perspective.
For example, you might think of a story of good versus evil, which could be divided into 3 tendencies: good, neutral, evil. Over time as attention is accumulated on objects with each of these three tendencies, the tone of the narrative environment will shift: if evil is a stronger tendency, then buildings collapse, people become corrupted, and perhaps the story ends with an evil outcome.
Another example: the story of a romantic relationship breaking down. We could have tendencies representing a neutral dispassionate retelling, but also tendencies for each person in the relationship, and even perhaps of their closest friends and family, and each scene or area offers an opportunity to highlight a particular tendency/perspective. We try to understand how and why the relationship ended, but can only see it from a particular perspective at any given moment, which can be interesting in and of itself, or alternatively encourage people to go through the experience multiple times to try to get a bigger picture based on each individual tendency expressed.
Attention ratings are values given to objects as they are paid attention to over time, per level of attention/focus. Objects that are focused on in the centre of the screen accumulate a higher rating of attention faster than those that are in an area of attention around the focused area, which in turn accumulate attention faster than objects at the edges of the screen. Objects that are offscreen do not accumulate attention.
Objects organised under locales and tendencies have their attention ratings summed together to give a per-locale per-tendency rating, which in turn can produce global per-tendency ratings. Attention ratings are a proxy for user interest/attention over time, and are a key part of the Reactive Mise-en-scène system and how it drives changes in future locales as they are loaded.
The attention data manager is the "brain" of the system, and the key part of the Data Store. It holds records of all the objects in an experience and tracks the attention ratings given to each of them over time. From these records it can then determine how strong or weak each tendency is at a locale and global level.
The prefab loader (LoadPrefab component) is the output point of the system, and the primary means of implicit interactions it provides.