-
Notifications
You must be signed in to change notification settings - Fork 44
Plugin Development and API
The operation logic in Vizor is based around the concept of plugins. Each plugin resides in browser/plugins, with the unique name plugin_id.plugin.js
Individial plugins are registered with the system and placed in the editor context menu by adding them to the file plugins/plugins.json, which follows the syntax:
"Plugin Category": {
"Plugin Name": "plugin_id",
...
}plugin_id must be the same as the filename, but without the .plugin.js extension
Each plugin can be roughly categorized into one of three groups:
-
Generators: Plugins that only have output slots. Represents sources of data, which can be anything from pure numbers, to mathematical functions, to geometry, user input and so on.
-
Modulators: Plugins with both input and outputs. Represent operations on data from one or more sources.
-
Emitters: Plugins with only input slots. Usually provide final presentation of data which can be rendering, playback, recording or transmission to a remote receiver.
Some plugins have neither inputs or outputs. These usually serve cosmetic purposes, like the Annotation plugin, which provides a persistent on-canvas note or comment.
Generally each plugin has:
- An unique plugin id
- Input slots
- Output slots
- Event handling methods
- Internal variables and methods
Input and output slot arrays can be empty, but must be present.
Input slots take in data, output slots output data. Slots declared at creation time are termed static, and slots created at run-time are called dynamic.
Any plugin may also declare any amount of internal variables and functions to do calculation, store variables and so on, provided the names do not clash with the reserved methods or names.
Input and output slots have to have a datatype defined, or it can be 'ANY' to be decided automatically.
this.datatypes = {
FLOAT: { id: 0, name: 'Float' },
SHADER: { id: 1, name: 'Shader' },
TEXTURE: { id: 2, name: 'Texture' },
COLOR: { id: 3, name: 'Color' },
MATRIX: { id: 4, name: 'Matrix' },
VECTOR: { id: 5, name: 'Vector' },
CAMERA: { id: 6, name: 'Camera' },
BOOL: { id: 7, name: 'Boolean' },
ANY: { id: 8, name: 'Arbitrary' },
MESH: { id: 9, name: 'Mesh' },
AUDIO: { id: 10, name: 'Audio' },
SCENE: { id: 11, name: 'Scene' },
MATERIAL: { id: 12, name: 'Material' },
LIGHT: { id: 13, name: 'Light' },
DELEGATE: { id: 14, name: 'Delegate' },
TEXT: { id: 15, name: 'Text' },
VIDEO: { id: 16, name: 'Video' },
ARRAY: { id: 17, name: 'Array' },
OBJECT: { id: 18, name: 'Object' }
};- id (string): The plugin id
-
updated (boolean): Flag indicating whether any input slots have updated this frame and whether
update_state()need to be called as a consequence. - state: For storing plugin state between sessions. Can be undefined.
- isGraph (boolean): A special state-flag used to identity nested graphs.
-
always_update (boolean): If this plugin is a nested graph, when this flag is set
update_state()will be called each frame regardless of whether any input connections have changed, provided this plugin is in a graph that has ben updated this frame.
Each plugin must be registered in the E2.plugins namespace, with the unique plugin_id.
It is recommended to inherit the prototype class Plugin (browser/scripts/plugin.js), which contains many common methods shared by the plugins.
The plugin declaration should be wrapped in a function() wrapper to avoid any variables leaking outside the scope.
The plugin model is event driven. Event handlers are simply methods defined in the plugin, using reserved names. These event methods are called by the E2.Core automatically when defined.
(function() {
var ThreeTorusGeometryPlugin = E2.plugins.three_geometry_torus = function(core) {
Plugin.apply(this, arguments)
this.desc = 'THREE.js Torus Geometry'
this.input_slots = [{
name: 'radius',
dt: core.datatypes.FLOAT,
def: 1
},
{
name: 'tube',
dt: core.datatypes.FLOAT,
def: 0.4
},
{
name: 'radialSegments',
dt: core.datatypes.FLOAT,
def: 8
},
{
name: 'tubularSegments',
dt: core.datatypes.FLOAT,
def: 6
},
{
name: 'arc',
dt: core.datatypes.FLOAT,
def: Math.PI * 2
}]
this.output_slots = [{
name: 'geometry',
dt: core.datatypes.GEOMETRY
}]
this.subdivisions = 1
this.geometryDirty = true
}
ThreeTorusGeometryPlugin.prototype = Object.create(Plugin.prototype)
ThreeTorusGeometryPlugin.prototype.generateGeometry = function() {
this.geometry = new THREE.BufferGeometry().fromGeometry(
new THREE.TorusGeometry(
this.inputValues.radius,
this.inputValues.tube,
Math.floor(this.inputValues.radialSegments),
Math.floor(this.inputValues.tubularSegments),
this.inputValues.arc))
this.geometryDirty = false
}
ThreeTorusGeometryPlugin.prototype.update_input = function(slot, data) {
if (this.inputValues[slot.name] !== data) {
this.geometryDirty = true
}
Plugin.prototype.update_input.apply(this, arguments)
}
ThreeTorusGeometryPlugin.prototype.update_state = function() {
if (this.geometryDirty) {
this.generateGeometry()
}
}
ThreeTorusGeometryPlugin.prototype.update_output = function() {
return this.geometry
}
})()Vizor contains several abstract plugin classes that can be used as prototype for certain types of plugins, in order to avoid duplicating functionality.
####Plugin (browser/scripts/plugin.js)
Base class for all plugins, providing a good baseline with some common patterns. Should be used as base for almost all plugins.
####ThreeObject3dPlugin (browser/scripts/threeObject3dPlugin.js)
Extends Plugin
This is a base class for any plugins that output THREE.js Object3D's. Handles parameters like position, rotation, and scale.
####AbstractThreeMaterial (browser/scripts/abstractThreeMaterialPlugin.js)
Extends ThreeObject3dPlugin
Base class for Plugins that output a Material. Contains default slots for common material properties and updating inputs to THREE.js Material properties
####AbstractThreeMesh
(browser/scripts/abstractThreeMeshPlugin.js)
Extends ThreeObject3dPlugin
Base class for plugins that output a Mesh and take geometry and material as inputs. Handles backreferencing the Object3d, which is required for Object3d picking and manipulating in the editor.
When defining custom inputs or outputs, you can concatenate the already existing input_slots or output_slots to your plugin easily.
Example for a material plugin:
// First apply the parent class to get the input_slots and so on
AbstractThreeMaterialPlugin.apply(this, arguments)
// Then add your custom input slots
this.input_slots = [
{ name: 'color', dt: core.datatypes.COLOR, def: new THREE.Color(0xffffff) },
{ name: 'sizeAttenuation', dt: core.datatypes.BOOL, def: true },
{ name: 'lineWidth', dt: core.datatypes.FLOAT, def: 0.01 },
{ name: 'near', dt: core.datatypes.FLOAT, def: 0.01 },
{ name: 'far', dt: core.datatypes.FLOAT, def: 1000 },
{ name: 'wireframe', dt: core.datatypes.BOOL, def: false }
].concat(this.input_slots)If you don't want to include the slots of the parent class, just omit the .concat() call.
This is the list of default events that the Core propagates to all the plugins registered in the system. Plugins may choose to implement any number of these, depending on the wanted operation.
Called on plugin load, instantiation and when playback is stopped. If this method is omitted, it will not be scheduled for forced update after playback has been stopped and resumed. For the same reason, generator plugins will almost always implement this method. See also: stop().
Will be called by the Core immediately before its parent Node is destroyed along with all associated resources.
Called immediately before graph playback begins.
Called immediately after graph playback is paused.
Called immediately after graph playback is stopped. Unlike reset, it will not be called as part of plugin initialisation or deserialisation.
Called when the state of a given in- or outbound connection involving the plugin changes.
- on (boolean): True if a new connection was formed and false is an existing connection was deleted.
-
conn (connection instance): The object representing the connection that was just made ot is about to be destroyed. It has the following properties:
- src_node (node instance): The source node of the connection. If the connection is outbound, this will be equivalent to the node parameter given to our construction function when the plugin is first instantiated.
- dst_node (node instance): The destination node of the connection. If the connection is inbound, this will similarly be equivalent to the node parameter given to our construction function when the plugin is first instantiated.
-
src_slot (slot instance): The originating slot (see
update_input()below for more details). - dst_slot (slot instance): The destination slot.
- ui (connectionui instance): Only set when the plugin is on the currently active canvas and false otherwise.
- slot (slot declaration): The slot of this plugin involved in the operation. Checking slot.type for equality with E2.slot_type.input or E2.slot_type.output can be used to determine whether the changed connection is in- or outbound.
This method is typically implemented when a plugin needs to respond to disconnection from inbound sources of data, but has other uses as well. Plugins that allow connection of any data type to be made to or from them, i.e. that declares slots (whether static or dynamic) of type ANY, make use of this functionality to adapt the data type of their own slots to that of the slot they're being connected to and to reset the data type of the slot back to ANY when a connection to the relevant slot is destroyed. See also: LinkedSlotGroup in the section below.
Called whenever an inbound connection has new data to deliver. The Core guarantees that connected input slots are processed in the same order that they are declared by the plugin. No similar guarantee is made for processing of output slots.
- slot: The slot that received the data
- data: The new data value. This is guaranteed to be of the correct type and match that of the slot, although not all datatypes are guaranteed to have a specified value. For all datatypes that are legally allowed to have no value such as Textures, an undefined value will always be null.
The slot parameter is an object containing the following members:
- slot.dynamic (boolean): If set to true, indicates that this slot is a dynamic slot.
- slot.desc (string): Slot description.
- slot.dt core.datatypes reference.
- slot.index (integer): Static slot index. Equivalent to the index of the corresponding slot declaration as specified in the constructor function of the plugin.
- slot.is_connected (boolean): Indicates whether the slot is currently connected.
- slot.name (string): The slot name as show in the UI.
- slot.type (integer): The slot type. Either E2.slot_type.input or .output as appropriate.
- slot.array: (boolean): true if data coming from this input is an array (of slot.dt type data), false otherwise
- slot.uid (integer): Optional – only present if this is a dynamic slot.
To check which static slot is currently receiving input, testing for slot.name should be used instead of slot.index to avoid breaking graphs if inputs are added. The data can either be stored in plugin transient state by storing it in an arbitrary class property or be stored in the persisted plugin.state object, change UI state in response on incoming data and so on.
The default Plugin.prototype.update_input() implementation stores all input values into this.inputValues dictionary with slot.name as the key. It is recommended to override this behaviour only when needed.
If the plugin declares dynamic slots, slot.dynamic boolean can be used to differentiate between input to dynamic vs. static slots.
In general, no changes to plugin state should be done in update_input but instead the input values stored and any state changing done in update_state.
Called once every frame after all calls to update_input() has completed, if:
-
One or more of the connected input slots have changed value.
-
This plugin has no output slots.
-
This plugin has no input slots.
-
This plugin is a nested graph and doesn't have its .always_update property set to false.
-
updateContext specifies the context which any temporal calculations should be executed in. It contains attributes updateContext.abs_t and updateContext.delta_t respectively for absolute time and delta time since the last frame.
Called once for every connected output slot if update_state() was previously called this frame. Like update_input(), the slot parameter is the instance of the slot of this plugin being polled. See also: Plugin.prototype.query_output()
If implemented, the Core will call this before calling update_output() for a given connected output slot. If this method return false, data flow will be blocked on that output connection.
Called when the canvas on which the plugin resides is being switched to for editing.
jQuery is guaranteed to be available globally, so $ can be used, although excessive use of jQuery, especially for event handling is discouraged in production code for performance reasons.
Plugin implementers can create any nested set of DOM objects here, set up their own event handling and anything else they might like. When done, create_ui() is expected to return the root DOM element created, which will be dynamically shown on the surface of the plugin instance whenever visible in the editor. Always use the automatically persisted state plugin member to store UI state.
This method is called once after plugin creation or deserialization, with ui set to null. At this point the plugin should bind any event handlers that should only be added once
If the plugin declares a UI, this method will be called seperately with ui equal to the root DOM element returned by create_ui() earlier.
When this method is called the following is true:
- The state member will be deserialised and available.
- The parent node will be fully deserialised with all data structures patched up and ready for use.
Returns the string currently being used for the visible header of the node.
Adds a new dynamic slot to the current node.
- slot_type: Either E2.slot_type.input or E2.slot_type.output.
- def: Slot definition. An object equivalent of a static slot definition.
- returns: A unique integer slot id.
Removes a dynamic slot from this node.
- slot_type: Either E2.slot_type.input or E2.slot_type.output.
- suid: Unique id of slot to remove.
Returns the slot definition for a given dynamic slot.
- slot_type: Either E2.slot_type.input or E2.slot_type.output.
- suid: Unique id of slot to remove.
- returns: A slot definition or null if the slot could not be found.
Renames the specified slot.
- slot_type: Either E2.slot_type.input or E2.slot_type.output.
- suid: Unique id of slot to remove.
- name: Desired new name for the specified slot.
Changes the data type of the specified slot. Unless the new data type is ANY, existing connection to or from the specified slot are destroyed.
- slot_type: Either E2.slot_type.input or E2.slot_type.output.
- suid: Unique id of slot to remove.
- dt: Desired new data type for the specified slot.
Returns the default value for the supplied data type.
This is a support class that can be used by plugins that declare multiple slots of type ANY and wants to link them such that if one of them are connected, all the controlled slots will match the datatype of the slot the initial connection is made with. Note that this is currently only supported for static slots. It is used in the following way:
(function() {
MyPlugin = E2.plugins.my_plugin_id = function(core, node)
{
Plugin.apply(this, arguments)
this.desc = '...'
this.input_slots = [
{ name: 'in_0', dt: ..., desc: '...' },
{ name: 'in_1', dt: core.datatypes.ANY, desc: '...' }
{ name: 'in_2', dt: core.datatypes.ANY, desc: '...' }
]
this.output_slots = [
{ name: 'out_0', dt: core.datatypes.ANY, desc: '...' }
]
this.lsg = new LinkedSlotGroup(core, node, [this.input_slots[1], this.input_slots[2]], [this.output_slots[0]])
this.value = null
}
MyPlugin.prototype = Object.create(Plugin.prototype)
MyPlugin.prototype.connection_changed = function(on, conn, slot)
{
if(this.lsg.connection_changed(on, conn, slot))
this.value = this.lsg.core.get_default_value(this.lsg.dt)
}
MyPlugin.prototype.state_changed = function(ui)
{
if(!ui)
this.value = this.lsg.infer_dt() // Returns the default value for the inferred data type.
}
})()-
Performing computation in
update_output(): Since output slots can be connected to more than once receiver concurrently,update_output()will be called once for each outbound connection that’s attached when the Core detects a successful run ofupdate_state(). Thus calculations should always be performed inupdate_state()which will at most be run once per frame, and cached to be returned on request fromupdate_output().
-
WebGL rendering: Vizor uses [https://github.com/mrdoob/three.js] (https://github.com/mrdoob/three.js) heavily for
-
Computing and caching output values in
update_input(): As an exception to the above rule, it is possible is very simple cases to perform a calculation directly inupdate_input()and cache it for later emission byupdate_output(), omitting implementation ofupdate_state()which will yield slightly better performance, although this approach should only be used sparingly.
For example, we can imagine a plugin which adds 1 to an input float value implement its update_input() and update_output() as follows:
MyPlugin.prototype.update_input = function(slot, data)
{
// We have only one input, no need to mask on slot.index -- it will always be 0
// 'data' is guaranteed to be a float.
this.output_value = data + 1;
}
// No need for update_state() here...
MyPlugin.prototype.update_output = function(slot, data)
{
return this.output_value;
}-
Inhibiting normal data flow: The flow of data to any given input slot can be inhibited by decorating the slot declaration with a boolean flag with the name inactive set to true, i.e.:
this.input_slots[index].inactive = true. Similarly, the inhibition can be revoked by removing the flag again, i.e.delete this.input_slots[index].inactive. Wheneverupdate_state()is called, it’s implied that the plugin updated property is true. It is possible for a plugin implementation to abort data output based on logic inupdate_state()by setting updated to false, in which case no calls toupdate_output()orupdate_input()of the destination plugin will be made.