Bevy Editor Requirements Collection

[james7132]

Thus far there has been a sizable amount of ground work laid both within first and third-party crates to enable a general Bevy editor, but there hasn't been a cohesive collection of requirements to get us to an actual deliverable editor.

Each sub-thread in this discussion should be a high level and generally actionable (though not necessarily immediately so) requirement for what members of the community think are either required for a general purpose editor, or would be very nice to have. Try to keep this scoped to what a minimum viable product would have to avoid scope creeping the discussion, though large pie-in-the-sky ideas are fine too so long as there's a solid plan to get there.

[james7132]

Inspector

We need an accessible way to inspect the internal state of one or more entities within a scene. This will be the primary way a user edits the individual fields of components on entities. bevy_inspector_egui by @jakobhellermann basically nets us this capability, and a good chunk of it can be upstreamed. However, it currently uses egui and ideally we should be using Bevy's built-in UI solution.

• Should support querying and displaying every component and field on an Entity using reflection.
• Should support editing (mutating) fields directly using reflection.
• Should support editing asset metadata (i.e. image compression settings)
• (MVP Optional) Support editing more than one entity at a time.

[minecrawler]

All kinds of point&click games can benefit from having a main menu, toolbars and dockable windows. There's not just Call of Duty and LoL out there ;) Plus I really don't think that these are the heavy-weights in the backlog...

[viridia]

In general, game UIs and "productivity" UIs have significantly different design requirements. For game UIs, there is virtue in novelty - widgets that are heavy on special effects, audio cues, novel rendering methods (3d rendered widgets), even things like using physics for animated layout. Also, game panels (dialogs, menus, etc.) tend to make heavy use of 2D art. "Layout" is mostly a matter of making sure that the positions of the buttons and widgets matches the pixel coordinates of the background image.

Productivity UIs, on the other hand, benefit from standardization. The person writing a productivity app generally doesn't want to have to think about issues such as what color a hover highlight should be. They want a powerful set of layout primitives, multiple modes (tabbed panels and toolbars), letting the algorithm decide where things should go instead of having to manually place everything.

Also, productivity UIs tend to be dense, since there's a lot of things to be controlled and there's only so much screen space. Game UIs, on the other hand, tend to be as simple as possible, and spread out for the benefit of fat fingers / novice users.

[viridia]

Having spent the last 3 years writing a game + editor (in TypeScript), here are some thoughts:

Editing of entities/components may be useful for some kinds of games, but not all. Some games rely on pre-baked scenes - that is, a serialized asset with a bunch of renderable objects - the classic "level design" task. But there's another kind of game where the data assets represent "generators" or "factories" whose output is entities (Sim City is one example). Many games use procedural generation of game content, where the authored assets are like LEGO bricks which are assembled dynamically by the game's AI.

My editor allows editing of any serialized / reflectable asset, regardless of whether it is an entity or not. The same editor can be used to edit an actor, a dialogue tree, a particle system, or the contents of an in-game book. There is an interface for querying the set of available assets and filtering them by type; once selected, an asset's schema can be used to build a property sheet widget.

[Wcubed]

As a counterpoint to the notion that "game ui" and "productivity ui" are two separate things: Have a look at games like rollercoaster tycoon, or OpenTtd, which feature floating windows with arbitrary amounts of buttons or text in them. Or as another example: factorio has quite a complex ui in vanilla, and with mods can get very complex indeed. And indeed, 4x games can have very complex ui layout situations as well.
In fact, if I need to make a quick UI application, my go-to is the godot game engine. Not because I like the godot scripting, or because I need any of the game related stuff. But simply because it's ui system is so much quicker and user friendly to work with than, for example, Qt, or python's tk. Let alone the "enterprise" ui toolkit I have to use at work for creating plc ui.
And of course, if it does turn out I need more complex rendering capabilities, I don't need to import any other libraries, because I am already working in a game engine.
DungeonDraft is an example where someone has used godot to make a "productivity type" program, in a game engine.

Basically, in my opinion a good game engine should have a ui system that is as powerful, if not more, than ui libraries used for "productivity" applications. In part because game ui has such breath in both variety and complexity.
My hope is, that once bevy matures, it can also be used to quickly make user friendly (and good looking) productivity tools, not because we designed bevy as such, but as a side effect of being an exceptionally solid and versatile game engine 😃.

[iamabigartist]

Despite the different design direction of Game and Editor UI,
It will be so much more productive if the editor UI can be used in debug mode Game Build.

Because I often find disasters in my work that the game result in Editor and Build are different while some inspection and visualization tools can only be used in Editor XD.

[james7132]

Project Structure and Metadata

In addition to code, a Bevy project inevitably will include non-code assets and areas on disk that are reserved for intermediate artifacts.

• By default a Rust project already produces a target/ directory that provides a location for intermediate and final build artifacts to be stored. We could leverage this, or create our own directory where we save intermediate artifacts for asset builds. If we choose to use our own directory, we may need to provide a base .gitignore file to ensure users are not checking in the build artifacts into source control.
• By default a Rust project already necessitates the use of a Cargo.toml file to change the build configuration for code. We could put project metadata there or make our own file.

[JMS55]

Something else I thought of, asset import metadata (settings to convert from asset->runtime asset) is important. I dislike how Godot keeps each metadata file right next to the original asset. I would want it in a separate, non-target folder (this is important info, that should be kept in source control).

[james7132]

Does cargo clean affect the whole target directory, or just the debug/release/etc folders? If it's the latter, we could have a separate target/bevy folder, and get the best of both worlds more or less.

It affects the entire target/ directory, so using target/bevy/ with cargo clean would indeed clear the entire asset cache and require re-importing all raw assets, whether that's desirable or not, idk.

[gamecoder-nz]

Would projects be a part of the bevy engine since they are only really used for the editor?

[Fidius-jko]

Maybe create a .bevy directory?

[iamabigartist]

Better seperate the artifact immediate file and assets' config files. So that intermmediate file modifications will be purely local, and don't need to commit together with assets' modification in version control system.

[james7132]

Asset Preprocessing

TODO

[djeedai]

Asset Preprocessing: it should be possible to "preprocess" / "compile" / "crunch" assets at "development time" rather than when the game starts up. This enables offloading expensive work from deployed apps, faster asset loading, less runtime memory usage, etc.

This definition from #3972 describes to me the baking step of the Final Build feature below, not the work done on asset import.

[djeedai]

The baking takes the Editor data and make it runtime optimized. For example, this compresses textures into a mobile-friendly format (ETC) for Android use.

The asset importing takes raw assets (e.g. an .fbx file) and make them into or augment them with Editor data so that the Editor understands them (e.g. create a missing .meta file for it).

[IceSentry]

Yes, my point is more that asset preprocessing refers to both importing and baking, not just the baking part. It's not conflicting as much as it is just too wide.

[IceSentry]

But importing is still a form of preprocessing too and the editor should be able to do both anyway

[djeedai]

Agreed, although I think the requirements are quite different for both.

[james7132]

Final Build Capability

Users should be able to press a button on the UI and have the editor run all of the asset preprocessing and optimization for a specific target and compile an optimized binary to run the game and produce a directory or compressed archive of the whole game, ready for distribution on the target platform (or as close as we can get to it).

[GingerDas]

Eventually, this should have robust build options to align with the existing build flexibility. But for an MVP there should be some minimal configuration capability, probably including choosing which cargo build command to use.

[JMS55]

I don't have anything MVP-worthy to discuss for this, but I have opinions and experience packaging Rust programs (with icons, metadata, etc) for linux/windows/wasm. I can't think of anything that's necessary to figure out right now, though.

[omaskery]

A soft requirement in my opinion is that the Editor should only be able to invoke the build process (and possibly provide conveniences like showing its progress and results, etc.), and not actually become the build process. Or, to put it another way, it should be possible to trivially invoke builds of the game from command line (particularly for use in CI systems).

I appreciate that this is how things are now, and almost doesn't need calling out explicitly, but my concern is that if an Editor becomes the de-facto way of authoring Bevy projects - there will be a temptation over time to grow Editor-proprietary pre/post-processing steps around the existing build processes, slowly making the Editor unofficially-mandatory for builds.

See Unity as an example of an Editor whose build process and editor tooling are inextricably linked, generally meaning that one cannot simultaneously run a build and be working in the editor on the same machine (partially due to their licensing).

[james7132]

Distribution and Logistics

The editor will likely be a binary. We can ask users to compile it themselves, and that may work for programmers, but may not bode well when working in multidisciplinary teams. Eventually, we'll need to deliver a precompiled binary for the platforms the editor supports, and we will need distribution channels for getting that binary in the hands of developers.

• Steam and itch.io seem like good release channels with well built systems for distribution. Godot does this: https://store.steampowered.com/app/404790/Godot_Engine/.
• May need a good way to version the editor to ensure people aren't opening the project in the wrong editor.
• Produced Windwos and MacOS binaries may need to be signed with valid certs.

[viridia]

Let me propose a different scenario: There isn't going to be a "Bevy Editor" - instead there will be multiple editors built on top of Bevy foundations. If someone wants to make an editor that is specialized to building climbing and jumping games, and publish a binary of it on steam, they can. They don't have to be official members of the Bevy core team in order to do this.

I'm not interested in an out-of-the-box binary. A "generic" editor is not useful for me, and I suspect it's not useful for a lot of other developers. For example, a lot of the features of my existing editor relates to editing of puzzles, traps, quests and spoken dialogue. Many game developers won't care about this - but for those who do care, it's vital.

Instead of trying to build Unity, think about trying to build Creation Kit, which is the basis for Skyrim, Fallout and so on. It's an editor, but it's specialized to a particular flavor of game. That doesn't mean that you can only create one specific game with it, but there's an envelope. Go outside of the envelope and you're going to have pain. Stay within the envelope, and you can use the pre-built editor to build that kind of game.

The best thing that Bevy can provide for me is a robust library of components that I can use to assemble the editor that I need.

[idanarye]

The best thing that Bevy can provide for me is a robust library of components that I can use to assemble the editor that I need.

That sounds more like what I'm trying to do with Yoleck than what the Bevy leadership seems to have in mind for the official editor.

[alice-i-cecile]

Yep, can confirm. I'm going to push to make as much of our internal tools available in a public crate as possible though, in order to support those use cases too.

That said, ultimately we need an editor that can be used (and customized) by team members who cannot code, and cannot compile code. That means prebuilt binary with a VSCode/Blender style extension system. Separate process is compatible with that, but is also worth doing (IMO) to improve crash resilience.

[ksf]

Supporting plugins would require a choice of language or at least ABI, to be able to share code between plugin and game engine Rust should be supported, which pretty much points towards WASM. .sos would have both security and portability issues.

That just seems to be quite far off given that the closest bevy gets to have scripting support is dynamic_api. To me editor plugins sound quite orthogonal to editor infrastructure in general, and instead aligned with mod support. Which don't get me wrong is definitely something I'd like to have.

Maybe getting clearer on what about the editor is MVP would clear up a lot about these conversations.

...I wrote a whole another blurb but realised that it's veering way OT from "prebuilt editor binary" so I'm going to make it a new topic.

[iamabigartist]

So programmers can modify the editor and built their own just like Unreal,
meanwhile bevy can officially provide a recommended tool set editor binary via Steam for non-programmers' development without editor modification requirement, right?

[james7132]

Scene View and Gizmos

TODO

[alice-i-cecile]

For the flycamera, we should just steal directly from the scene viewer at this point.

Transform gizmo: https://github.com/ForesightMiningSoftwareCorporation/bevy_transform_gizmo

[FuaadHi]

Like Unity and blender 3d : 2d scene view needs to be inside a 3d world unlike godot inwhich 2d view dont give achance to add Z axis transforms
Cause alot of new games tend to be 2.5d.

[recatek]

I think Unity is the best usability model here to follow. The Unity editor has two tabs for displaying the scene and game:

(Image randomly pulled off of an image search from this article I didn't read).

• The Scene tab is always visible and interactable (even while the game is running) with a free-flying camera that allows you to click on and manipulate objects via handles and gizmos (again, even while the game is running). The scene camera generally doesn't have post-processing effects or specific configuration, it's just for a functional view of the world.
• The Game tab shows the view from the game's primary camera (or layered cameras) and is active primarily when the game/editor is in Play mode. This is essentially the game's main viewport when playing the game in the editor. While the mouse is captured by this view, you can play the game as normal with keyboard, mouse, and controller input. This camera is the game camera in the sense that it follows all camera configuration, post-processing, and so on.

While certainly not easy to implement, I think this approach is the strongest option for a number of reasons:

• It's invaluable to be able to view the game from multiple angles with both a top-down editor view and the actual play perspective while the game is running, without having to do things like manually exit the player pawn in a single game window.
• Similarly, it's invaluable to be able to select and manipulate objects around you from a secondary view as you're testing the game -- spawn in objects, tweak values, move things around, and so on without having to leave the primary game perspective in your game view.
• Play-in-editor in general is critical for debugging. Having everything in one process means breakpoints "just work", rather than having to hook between different standalone executables or use essentially remote debugging protocols even on your local machine. This also makes it much easier to debug the game's startup, since you don't need to worry about hooking the game early enough to catch initialization steps.

Few other engines do this (Flax comes to mind), but it really is a killer app and it's easy to miss it dearly if you move to an engine without it. Godot has a proposal for play-in-editor, but probably won't go as far as to include the full scene/game view split the way Unity does here.

[james7132]

Extensibility

TODO

[GingerDas]

VSCode and Factorio to some extent get away with it because of the dynamic nature of their languages (typescript and lua). I'm not saying it is impossible to do in rust, but we will likely run into different limitations.

Assuming the rust limitations, can we enable some of this style of extensibility just through a well-documented API? Part of the documentation could be a description of the limitations and what not to extend.

[1755]

I think it is vital to keep modular spirit of the bevy with small core and large bunch of crates and plugins which add features. Instead of building all-in-one solution.

For example:

• core editor is just a tiny launcher of a bevy game
• if someone want a scene editor, one can enables 3d curosr plugin, blender-like camera plugin, grid plugin, etc, etc
• if someone want to get an animation editor, one can enables blender-like camera, animation plugin, etc

It would be very flexible and task focused. Also, many teams may adopt the editor for their needs and pipeline easily.

We can build predefined profiles which enable needed plugins for every common use case, but we have to describe common user roles and their needs first to do useful defaults.

And it will make contribution more distributed and simple. Like for bevy itself

[SneakyBerry]

I think in case of using IPC to communicate between editor and engine we should provide and abstractions for this to work, we need to support both, editor-side and engine-side plugins for it.
If editor or engine plugins would not have an access to IPC it would not work at all.

[iamabigartist]

Besides the necessity to well support the plugins, an extensible editor should also make it easy to implement one plugin. For example, for the scene view, you can have an official implementation, but please also make all the tools you use to implement the scene view open to developers so they can easily make variations that they like.

[tqwewe]

🙌 WASM-based plugins 🙌

[james7132]

Debug Tooling

TODO

[james7132]

There's also rendering debug views that should be included (i.e. normal, specular, ambient occulusion, etc.) that should be debug filters on the scene view.

[JMS55]

The ability to "pause" a game is a must. Probably stop running all non-rendering systems, and somehow freeze the clock. Unclear about technical requirements.

"Query debugging" as a general topic would be neat. Being able to pause the game and run queries, see statistics about query usage, etc. I would love some kind of event logging integration. Being able to have something like "Hey! Draw a message on-screen overlaid on top of the running game, and then note down the event info for later viewing in an event log panel" just by calling a function in your game would be awesome. Basically, visual/gui logging integration of some sort. No specific ideas, or MVP requirements at the moment.

[alice-i-cecile]

See #1353 for discussions on built-in pause functionality. I think we need two layers of this: one where you stop all fixed time step systems, and another where you actually interrupt the app loop completely.

[DrRuhe]

A thing that I think is worth looking into: time-travel debugging with an ECS "Replay" system.
This allows the following use cases when analyzing/debugging:

• go back in time yourself and analyze
• save replay to disk and send to other devs, that can analyze the replay
• replay systems for games buitin (this is just a bonus)

With editor integration for things like:

• hot reloading: change constant in code and see effects
• debugger integration: time travel back before bug, set breakpoint and start debugging in editor

I recently saw a video from Tomorrow Corporation about their internal tools which could do all of this. Something similar can be done in elm with the elm-reactor project: https://elm-lang.org/news/time-travel-made-easy

Although this is most definitely not required for an MVP editor I think it should be something on the roadmap, as is provides tremendous value when debugging.

[ameaninglessname]

* hot reloading: change constant in code and see effects

Hot-Reload Hot-Restart and Hot-Deoptimize
Hope we would see this wonderful features in bevy!

[alice-i-cecile]

Level Editing

Being able to quickly and easily place game assets is a common need when working on content-driven games, especially with non-programmers.

However, the needs of each game are quite unique: how much should the official editor provide, and how can it be customized?

Providing level editing funcitonality is also a huge tooling task in its own right, and may be best left until well after the MVP.

[viridia]

Oh, one other key point: there are two different serializers. One for serializing an edited level, and one for serializing a saved game.

Most of the things that you edit in the editor are constants during game play. For example, the color of an actor's shirt might be editable in the editor, but it is immutable otherwise. There's no need to save this information in the save game file, as it will never change. Conversely, there are some dynamic state variables (such as the actor's current action) which you might want to save in the save game but which you wouldn't save as part of the level.

[therealbnut]

I'd love to be able to have a default set of editor plugins that give you a standard Unity-like experience, but also be able to almost completely customise the editing experience, by reducing, constraining, or validating a level. Primarily I'd like to be able to constrain the editor in a custom way, and emit a custom level format.

Consider a few examples:

• A 2d tile based game: constrains things to a grid, and a level format can be as simple as which tile is used in each grid square
• An RTS: you've got certain units, those units have some specific custom attributes you can change, they're on a very contrained terrain, the navigation and features of that terrain may need a custom ai and navigation solution generated for the level. The game state itself needs to be derived from that level, and then efficiently and deterministically synchronised across a network.
• A non-euclidean game like Antichamber or Portal: your levels don't follow traditional geometry, and you need custom navigation, unique attributes for level geometry, and unique states it can be in
• A world for an open world MMO: optimising world geometry is very important, it needs to be streamed to clients, worked on concurrently by a large team, and resilliant against clients on an old version. The level format is custom-built for this.
• An external tool like Houdini is used to make your level, and it has its own level format, you only want to use the Bevy editor for testing and debugging
• Community made levels are fun, but you want to be able to constrain them to your assets or any other game constraints. You want the levels to continue to work if you patch your game, so they need a constrained level format that is asset agnostic, or can be versioned and migrated. Ideally you'd be able to give users a cut down editor that only lets them make a level and test it.
• Procedural content that is very efficient to store parameters for, but it's very inefficient to store the generated content
• Your players load a lot, you want the game to be punishing but not with long load times... you want to optimise the level loading based on your knowledge of the game (maybe you only need to move the player, or remove decals, etc)

Thanks for reading!

[idanarye]

@therealbnut We need to be careful with this. The level format - or at least, its representations when editing and when loading - need to have some common grounds that plugins can build on. If it's flexible to the point that each game just define its own format with its own representation, it'll also have to pretty much build its own editing tools from scratch.

[therealbnut]

@idanarye I agree we need to be careful, but I think it's possible. I'm not sure if you're familiar with Unity's editing experience, but they let you edit things with gizmos and custom inspectors. On top of that you could disable or lock default components exposed to the editor. The current Bevy ecosystem works fine with custom types, I don't see why this would be any different. I don't think things would ever have to be from scratch, but I do see you might have to implement a few traits, depending on how custom things get.

Without a custom representation you might use Bevy's ReflectSerialize etc. With a custom representation you could at worst implement a custom serialize/deserialize, that might lose information that you don't want anyway, and you could have constraints and validation on the editor to ensure you can't input information that would be lost.

I don't think this is the place to discuss implementation, but I imagine a more flexible solution would have traits to provide a common api for things exposed to the editor. That would define traits for traversing the level graph, reflection, and advertise properties on types, property types can pair with widgets (i.e. select a number in a range, a vector in world space, a colour, another object of a specific type, etc.).

[djeedai]

I don't know where to put it, and it's a small quality of life thing for the user, but probably needs careful planning and integration ahead: the ability to distinguish between unsaved work on your content (assets) vs. unsaved Editor preferences.

This sounds trivial like this, but here's a few examples where it breaks:

• Blender saves some configs inside the .blend files (for example I think what was the last selected face). You can trivially open a Blender file and make no change whatsoever to the model, but still get the file reported as "unsaved" (small * next to the filename).
• Unity saves a bunch of Editor-only configs (position of windows etc.) inside the game asset itself.
• Similarly, Unreal marks some assets as changed when you move some nodes in a node graph. Moving a node is a purely visual action which has zero impact on the functionality of the Blueprint / material / whatever the graph is. Those two types of changes should not be treated the same.

This is quite frustrating, and even dangerous (I lost work yesterday reverting a file I thought had only cosmetic changes, but it had actual behavioral ones). You end up with the Editor/App telling you that there are some pending unsaved changes, but you can't remember any relevant change you made. And indeed there's none. This also shows up in any version control system, which can't tell the difference between those two types of changes since it works at the individual file level.

To be fair I have no good solution for this. It likely involves either having 2 separate markers for changes (but that won't work with VCS anyway), or having two separate files per asset (one for behavior, one for Editor-only configs), none of which particularly stand out as optimal.

[james7132]

Code Editor Integration (MVP Optional)

We probably are not going to be rolling our own provided code editor, but errors/warnings in the debug console could optionally support "Jump to Error" capabilities for popular editors (i.e. VSCode, Lapce, or IntelliJ).

[minecrawler]

Not a fan of an integrated code editor inside the game editor - I pay for IntelliJ for a reason... though both may be out of scope for a MVP.

However I wonder how easy it would be to prepare editor integration for full debugging support - in the simplest case starting the game from the code editor via the game editor (depending on how the game is integrated into the editor). A websocket bridge may help to communicate events and state between an IDE plugin and the game editor. I'd love to have the full debugging suite to analyse some strange behaviour.

For debugging, setting breakpoints, inspecting variables and line-by-line stepping are the minimum imho.

[soqb]

a nice alternative for @lewiszlw may be a third-party implementation of an editor ui using something like a vscode extensions.

[idanarye]

Neovim was designed with a client-server architecture to allow integration into GUI. And yes, Vim is not for everyone, but there are ways to make it behave like a normal editor, and that could be the default for the Bevy editor (though as a hardcore Neovim user, I'd appreciate a way to bypass that and use my own configuration instead 😉). It could also come preconfigured with rust-analyzer for code completion and linting and other niceties.

[dimas-cyriaco]

I'm a Neovim user, and would not use a builtin editor. But, if there's need for one, maybe Helix could be a good match, as it's written in Rust as well. Don't know if its easily embeded though.

[astrale-sharp]

Hi there, coming from godot, the pros of having built in code editor include integrating scripting languages (and less importantly if you want to make a coding game).

[james7132]

Source Control Integration (MVP Optional)

Allowing users to use source control from the editor via a UI can be highly beneficial when working with non-programmers. Supporting git, git LFS, SVN, Perforce, or PlasticSCM workflows from inside the editor can really help improve multidisciplinary collaboration.

[alice-i-cecile]

Strongly want, but I would not call this MVP.

[djeedai]

I'm not so sure. Having some UI for it is not MVP for sure. But making sure the Editor handles assets etc. in a way that is compatible with the use of source control (so, things like not locking asset files on disk) seems pretty critical, otherwise users will quickly get mad if the Editor gets in the way.

[JMS55]

Like Godot, I think we should make sure things meant to be checked into source control, like asset import metadata and project settings, are plain-text editable. (non-MVP) Ideally it would even be formatted properly, so that a user manually editing a file wouldn't reformat the whole file adding whitespace and stuff that would show up as a source control change.

[omaskery]

To me the most important thing to worry about regarding source control is that the editor produces relatively stable, deterministic outputs, that can be sanely edited by hand during merge conflicts. As a side note: there is some overlap between these concerns, and the concerns that would be involved in making the editor produce human reviewable changes, for reviewing changes in PRs.

Some anti-examples (things we could learn from and avoid) from Unity:

• Unity serializes runtime-derived data (as opposed to authored data) such as UI widget transforms & sizes to its data files, even when they are automatically derived by the proportions of your editor window. This results in hundreds of "imaginary" changes to scenes/prefabs.
• Unity's "ProBuilder" package (used for quickly prototyping geometry in-editor) serializes its meshes with A) non-deterministic, unstable names (resulting in churn every time they regenerate), B) serializes the mesh data directly into the scene/prefab, bloating an otherwise small text file into a huge one
• Unity's serialization files rely heavily on GUIDs. Whilst this is technically sound, it can be exceptionally difficult to manually resolve merge conflicts when you cannot understand what objects are being added/edited/removed. For example, I have often thought that it would be nice if the Unity editor injected comments into its serialization format which exist purely to advise human readers what the GUIDs resolve to.
• Whilst I believe they have improved this in recent years, Unity used to serialize its YAML file contents in a non-deterministic order. This would create spurious changes in version control where the contents of a file had simply been re-ordered.

It would be most wonderful if Bevy's tooling could avoid these kinds of issues, which ultimately give team members anxiety over what changes should or shouldn't be committed to scenes, etc.

[grilme99]

Editor Interface

The editor needs a Bevy UI-based interface! Ideally, the editor will support widgets and docking. Optional for an MVP release could also be windowing such that widgets can be dragged outside the main window (even onto other monitors).

[minecrawler]

@iamabigartist So you want to use the Bevy editor, and you want your scripts to interface with it, using IO visible in the UI? And you want to control the UI from your programming - your scripts? How about we create a programmable interface for the Editor application in order to change all of the parameters. We can just call it Application Programming Interface - because it's interfacing with an application via 3rd party programming, e.g. scripts. Or API for short :)

On a more serious note, instead of proposing workarounds it would be better to propose a proper API and software architecture. UI elements don't have an API, and when you click a button or change a slider, they should not do any custom logic, but instead update the internal state via the editor's API. The same API which can be used by the rest of the application and which might just as well be exposed to plugins. The neat part is that the industry has a streamlined process to version and document an API, and you don't need to search for UI elements by name and hope their type or name doesn't change from one update to another.

[iamabigartist]

I think this solution↑ is better than my previous reply.

[raffaeleragni]

Would it be possible for editor UI to work also in XR/VR?

[therealbnut]

Is the intention of:

The editor needs a Bevy UI-based interface

simply to mean that we should be able to extend the editor UI and create new widgets with a consistent cross-platform API?

I think that a game UI and editor UI likely have fundamentally different goals, so building it on BevyUI seems like it could be a premature optimisation. Using BevyUI might mean we have to re-create many native controls that will only ever be needed for the editor.

A game wants to be able to style the UI, be dynamic and unique, and likely be GPU rendered on top of or in the game scene, or even integrate with it. It likely doesn't need much in the way of operating system interaction. The UI needs to be extremely customisable and extensible.

An editor may wants to opt-in to showing a fully rendered game scene, but would ideally take advantage of the host operating system's UI affordances for accessibility, localisation, filesystem integrations, text entry, menubars, buttons and other controls, etc. An editor UI should focus on providing an efficient workflow for editing scenes and game properties, as well as integrating with the operating system and other programs. It should default to familiar controls so it's easy to learn, however being able to make totally unique unfamiliar controls and interactions should be limited or discouraged.

I think an ideal editor GUI would be simple, cross platform, exposed to plug-ins, use native-style controls, minimise styling and focus its API on concisely presenting the information we need in a consistent fashion. In addition to familiar native-style controls we'd expose a limited set of APIs for customisation, like Gizmos within the 3d viewport. Most of these goals can probably be built on top of other non-game Rust GUI projects, which can also allow for greater Rust ecosystem collaboration.

There are also a few suggestions here to reuse editor GUI controls in a game, and that seems like the main reason to use Bevy UI, but I don't think that will be a primary or even common use case.

[ksf]

I think the idea is to have a complete enough UI system to also do UI-heavy games, basically, at least catch up with egui (which does have a11y incl. AccesKit integration)

When it comes to control familiarity, yes, I also think radio buttons should be round and check boxes square but with the amount of convention-breaking stuff people are accosted with nowadays on the web I don't think "native look and feel" is really an argument, any more. When opening blender for the first time people's issue isn't that the buttons look different than in their word processor, the issue is the sheer irreducible complexity of the thing. Generally speaking though I think Blender is a good role model when it comes to these things.

[alice-i-cecile]

New Project Functionality

There should be a tool to create a new skeleton project very simply. https://lib.rs/crates/cargo-bavy by @TimJentzsch and other discussion in #436 should be used for inspiration.

[GingerDas]

Should the skeleton have any "hello world" content? Something to implement each of e/c/s. May be as simple as a spinning cube, but something to modify for the first elements.

[alice-i-cecile]

Yeah, my opinion is that eventually you should be able to pick e.g. "2D game", "3D game", "App", "turn-based game", and get a good set of default extensions plus a very basic skeleton of that game type that you can use to check out the editor features.

I actually think you want something about as complex as the Breakout game, to make sure you can actually quickly test the various features.

[GingerDas]

For the MVP, Breakout could be reused.

[JMS55]

I have two asks for this.

One it comes to 3d projects, we should have a sample scene suggested as the default. This scene should do the best it can to use every rendering feature we support out of the box to make things look as nice as possible. One of my major issues when I was learning Godot was that I had no idea how to do proper lighting. I'd stick some point lights down to illuminate some spheres and cubes and planes, but nothing looked good. I had no idea about indirect vs direct light, the importance of setting up an HDRI sky, etc. A default scene demonstrating all of this, with a variety of different materials on some meshes would go a long way towards teaching people imo. We don't have anything beyond direct lighting atm, but I imagine that will change by the time the editor comes out.

My other ask is in addition to "new project", in the same sceen, have a tab for all of the bevy examples. Lets make them more discoverable, and encourage people to try out different bevy features via the demos :)

[EmiOnGit]

I would even argue that the skeleton projects should be limited by one 3d and one 2d game. Default projects are hard to choose right.

Imo it makes more sense to let community develop their skeletons and then build an import tool which can fetch them as needed from their repository.
I am imagining something similar to the bevy asset side visually

The community would surely build also more specific and advanced skeletons that couldn't be provided by as default such as a platformer or a physic simulations

[zkat]

Generic Node Editor

I would be really nice for the editor to have a native graphical node editor that can be repurposed by multiple things (audio stuff, graphics pipeline stuff, scenes, archetypes), but that can also be used by third parties (I would LOVE to be able to build a node editor for Big Brain logic!)

(this might seem like a big and post-MVP issue, but I think it touches on so many core systems that it could be treated as a building block for a lot of the other essential features mentioned above)

[ameaninglessname]

There was some discussions in #6719

[djeedai]

Added to #6719 a section for node editor reflecting this. See #6719 (comment)

[wusticality]

It'd be wonderful to build something akin to ShaderGraph in Unity in the long term.

[DarkSeasonStudios]

Maybe something like this? https://github.com/setzer22/egui_node_graph

[raffaeleragni]

Would it be possible to have node editors work in XR/VR or touch environments too?

[james7132]

Documentation Integration

There should be easy links to both non-technical (i.e. game designer/general user facing) and technical (i.e. programmer facing) documentation. Unity provides a button in the header of every component that opens, in a browser, the corresponding documentation page on how to use the component (for designers), and the API reference page (for programmers). I think this is great learning experience and promotes guided experimentation, especially as the engine gets more and more complex.

Ideally every version of the engine should also be shipped with a full copy of the engine API docs and book for fast local access.

[djeedai]

Not sure this exactly fits, but "Tooltips" for contextual help is a big plus. And that thing is general non-trivial to implement, as this needs to overlay the UI without respecting any layouting rule of other UI elements. I include in "Tooltips" some more advanced ones like "preview image" for textures, not only textual tooltips.

[alice-i-cecile]

Tooltips would be fantastic, and very useful for games as well.

[djeedai]

IIRC (but my memories are fuzzy) Druid struggled a bit to add tooltips, that's why I mention it upfront.

[JMS55]

Love this idea, and talked about this already in the Inspection discussion. The artist/developer docs split is interesting.

One thing I want to note is we should have this not just for core bevy components, but also custom user components. This is essential for projects with more than one person.

[xgbwei]

What about something like Ableton Live's Info View panel? I strongly prefer this solution to floating tooltips and find it pretty underutilized since it:

• Doesn't get in the way of UI elements
• Doesn't require waiting for tooltips to appear
• Can easily be hidden
• Is generally less finicky (easier to implement?)

[IceSentry]

Undo/Redo

For any interactive UIs it's really important to support undo/redo operations.

To support it correctly, we need to make sure we have a framework for it in place as soon as possible to make sure every UI interaction uses it.

[lukors]

IMO this is mandatory for an MVP. An undo/redo system is foundational in an editor, and it affects how every editor action is implemented.

If there’s a framework but it’s not used across the board, that means the undo/redo will not be reliable, which is arguably worse than having no undo/redo at all.

[djeedai]

Linking bevyengine/rfcs#62 which has a discussion on how a data model could provide a unified undo/redo system "for free".

[IceSentry]

IMO this is mandatory for an MVP

When I wrote that, an MVP in my head was pretty much just an entity inspector using bevy_ui. I was going for very small M in MVP. I'll update the description to remove that not mandatory part.

[PROMETHIA-27]

Editor Layout

The editor is probably going to want a consistent layout in order to be both convenient to use and extensible. It would also be best if this layout is nailed from the get-go. I would vouch for a Visual Studio-like interface, in which individual windows (such as one for the inspector, one for the editor, etc) could be tiled and scaled, as well as stacked on top of each other as tabs. IIRC this is also similar to how vim's "windows" work. This layout would allow trivially adding new features while allowing users to customize the interface. This is also highly similar to how unity works iirc, but unity additionally has toolbars and such that I'm not so sure are necessary.

[djeedai]

It's definitely out of scope of this particular discussion.

Ok then yes if we're talking about what Taffy is already doing then I agree it's useful, but also not related to the Editor layout / UI design discussion here, being a mean and not an end.

On topic, docking windows a la Visual Studio is pretty much the golden path to allow UI customizing while supporting future extensions, both Bevy and end-user created.

[alice-i-cecile]

Agreed, I think docking windows (and possibly side panels) is the way to go here.

[wusticality]

I mentioned this in another thread, but I think the outermost interface should be tabbed like a web browser. Unreal Engine does this which makes it nice when you have 40 different assets open, each with their own editor specific to it. Also goes without saying that anything should be extensible so that folks can build their own custom editors.

[mhgolkar]

Agreed, I think docking windows (and possibly side panels) is the way to go here.

May I suggest also to give a chance to the Blender UI's approach to editor layout, windowing, and specially the Areas? They are like panels, somehow boosted! You can easily re-arrange areas, switch the purpose of each area, split and merge them, maximize or full-screen an area or duplicate any one into a new window, all by using the assigned key-bindings or a couple of clicks. You can also pre-define multiple different layouts in workspaces that are switchable from top-bar. Tabs are also worth a look. Finally all your customization (including layouts) may be kept and shared with the project file.

Blender is a good case to study and inspire for complex tool UI/UX design. It provides one of the most flexible and productive user experiences out there. From gizmos to customizable key-binding system (where you can assign a shortcut to almost anything).

Bevy does not need to copy any code or system from Blender directly, but it may eclectically look at the way things are done, the experience.

[mattdm]

I'd love multi-monitor support — at least for full-screen "scene view" and "game view" panels, but ideally as flexible as possible.

[Nilirad]

Editor-Game Interaction Model

We need to have at least a general idea of how the game and editor processes should interact together. This should give us a good direction to lay a strong foundation. We need to answer at least the following questions:

• How do the editor and game processes interact?
• How is the game process lifecycle managed by the editor process?
• What are the most important foundations needed to build the editor-game process model?
• How can the editor run game logic if it doesn't even depend on it?

[djeedai]

These are solutions. We're gathering requirements. We can't decide on solutions until we have listed what requirements are in scope and what are not.

Example:

How can the editor run game logic if it doesn't even depend on it?

Do we even want that? I don't know any editor which does that. Unity doesn't run any game logic in the Editor, unless you press Play and in that case the game effectively runs by itself in a separate Mono domain, which is equivalent to a separate process. And yet you can still customize the Unity rendering pipeline and the Editor UI. You can even edit game data while the game runs; this is done via reflection and serialization. Those things do not require running the game code inside the editor.

[omaskery]

Do we even want that? I don't know any editor which does that. Unity doesn't run any game logic in the Editor, unless you press Play and in that case the game effectively runs by itself in a separate Mono domain, which is equivalent to a separate process. And yet you can still customize the Unity rendering pipeline and the Editor UI. You can even edit game data while the game runs; this is done via reflection and serialization. Those things do not require running the game code inside the editor.

In case this is a useful distinction, whilst the Unity Editor is an application compiled separately to your game code and thus doesn't depend on it in the strictest sense, I'd argue that in practice there is a significant dependence on the game code.

Unity only has one Application Domain, containing all of the editor C# and the user C#, the game is not run in a separate Application Domain during play mode. At best the domain is reloaded (optionally) when changing between Play and Edit mode. As such the game isn't "equivalent to a separate process" in Unity.

The main way the Unity render pipeline is customised, Unity's Scriptable Render Pipeline (SRP), involves directly loading user code and executing it in-editor, not through reflection and serialization.

Furthermore, Unity doesn't have a clear distinction between "game logic" and "not game logic", so it's difficult to say "Unity doesn't run any game logic in the editor". Furthermore, you can mark arbitrary MonoBehaviours as [ExecuteInEditor], or schedule DOTS systems in the Editor World, and run arbitrary code in the editor, manipulating objects in the scene, how they're rendered, etc. If your C# code has any compilation errors when loading the Editor it will ask you to start it in "safe mode", disabling many features of the editor until you correct the compilation errors.

These are all various ways in which the code you write in your Unity project is depended on by the Editor (for better or worse).

[ricky26]

Every Unity project I've worked on professionally has had scripts which cross the Editor/Game boundary. A really good example is Unity's previous UI implementation, which is made of MonoBehaviours that also run at edit time (which is necessary for WYSIWYG editing of UI in their case).

For a while now, the render pipelines are the same: URP & HDRP are systems which execute at both edit & play time.

I also think that whether/how the editor should rely on the game logic is thorny. I personally like a nice bit of separation between editor & game (ala Hammer) but it's impossible to ignore what the integrated editing experience (i.e. Unity) provides.

[therealbnut]

If the game crashes, so does the editor and any potential unsaved changes will be lost.

I agree this is an issue, and I agree the game should be able to close/crash separately from the editor. However I’d love to see the editor be designed to have continuous automatic saving, including the undo stack. If we can make that work well then I think game crashes don’t need to mean lost save state.

There’s a huge advantage for reproducibility in having an object inspector and debug tools hidden in a standalone version of the game. I’d love the architecture to allow as many editor tools to be shipped with the game as possible. So even though I hope we default to a seperate process for the editor, I hope we don’t lock ourselves out of flexibility.

[therealbnut]

Sorry if I missed someone else saying it, but build times are another reason to want a seperate game process. Rust incremental building has improvements on the roadmap, but even then it’s still much faster to not have to link etc.

[gamecoder-nz]

Content Browser

I think there should be a content browser that shows the folder structure and assets of the project folder.

[enbugger]

The editor should remain an option, not a prerequisite

Nor the initial entry point for new users.

Bevy is one of the few game engines that give the freedom to think about a game as a regular program and not one big messy asset made in an editing application.

• The concept of a computer program is simple enough not to try to abstract it out. Bevy should preserve the feel of that by keeping access to the very start of game's fn main from the very start and not when the editor crashes blocking the entire process of development and urging people to investigate its internals.
• One of the main strengths of Bevy is Rust - there is no need to hide any of its parts. People are still touching Rust and choose Bevy as a nice playground to learn the language (like me). Seeing the entire game project written in Rust in IDE is way more satisfying than any editor's "put your code here" text field (even in Rust).
• Editor shouldn't exist just for the purpose of having an editor. Maybe even having a scene editor would be enough for Bevy? The source code of the Bevy-based game is so clean and declarative it does not need an editor! Just a good tool for placing objects would already be more than enough. Iteration time is already good.
• Overall it is hard to make people get out of editor and move to build the engine from the source code once they encounter problems in the former. Transition periods might end with nothing but negative ultimate experience. Just because people expected to never leave the editor.

I would see it very organic to have an editor be bunch of plugins that you link into your game, you can choose the parts that you need and throw/replace features that make you sick.

That should keep the project be called a "programmers game engine" which is cool and in no way making it less popular.

[mousetail]

"Removing features" from source isn't "easy" if the entire engine is built around debugging, running, managing plugins, editing assets etc. with the editor. Bevy needs to be designed explicitly to make it optional to make not using it viable

[iamabigartist]

Agree with the "an option, not a prerequisite" part.
The Bevy project should be able to be edited and built completely without the Bevy Editor to give the most freedom to the users. Please don't hide procedures by some trivial automation while the automation itself may still have bugs, in which case there will be no way to fix and use it by users.

To make my expression less confusing, in my opinion:

1. Making the editor UI frontend and functional packages separated will eventually imply the "an option, not a prerequisite" goal.
2. Then, this goal does not mean to say that using the editor is bad as it is no longer "programmer first", or it is not essential to have an editor for all the workers in the industry, or it should be more like a tutorial tool for beginners. To me, instead, it means the users will be able to operate on a project without the editor as a fallback when some part of the editor UI itself can't be used temporarily caused of bugs. Say if some function/process is embedded only in the editor program, and there are bugs in the frontend that call them, then it will disable the function to users as it hides the call and there is no other way to call that function, which is unnecessary. Even if the bugs are in the function, it will be harder to have to test an embedded function as its own lifecycle is completely managed by the editor which is not initially built for debugging and testing itself.

[gamecoder-nz]

Yes it should be an option and the user should be able to use the engine purely through code. But having an editor will make things so much easier, even if it is for things like a scene editing or UI editing.

[viridia]

The main purpose of a game editor is not to create code, but data. Editors are only useful for games that are data-driven, in other words, games whose behavior is determined by loadable assets. Of course, there are many different ways that data can drive behavior, and there are many possible design patterns, some which depend on an editor for everything, others which only use the editor for a narrow set of authoring tasks.

Even with an editor, there will be times when it is necessary to edit the raw files manually - either to debug a problem, or to deal with migration issues.

Developing a data-driven game engine also means that the data formats are being developed. As features are added to the game the asset formats are constantly changing. Large refactors of the code base often require refactoring the asset schemas as well. Most serialization formats can handle small changes to schemas automatically, but large changes require migrating the data so that you don't lose work.

[therealbnut]

I completely agree that an editor is fundamental to collaboration with a wider team. I think it should be the default tool for gameplay prototyping and a mature game. However there are many other use cases where you may not want it in your immediate workflow.

• Debugging game logic, where you only want to move between an IDE and the game.
• A technical prototype, where you don’t have any assets yet.
• Test driven development of a low level Bevy plug-in.
• Running tests from an IDE.
• Building and running CI on a much cheaper machine (because it doesn’t have a GPU).
• Cross-compiling binaries for CD (on a much cheaper machine without a GPU).
• Hot reloading assets without an IDE or editor.

[Barugon]

Internationalization

It seems to me that internationalization (i18n) would be a very important part of any UI/editor. How is Bevy currently dealing with i18n? Could ICU4X be usable for Bevy?

If i18n is not integrated from the beginning then it might be difficult to implement later.

[mikkelens]

This should definitely be thought about pre-MVP, but I have a hard time imagining internationalization before the editor meaningfully exists. Bevy is currently only available in one language, and I have a hard time imagining an editor UI for the engine immediately being available in other languages than the single language the codebase uses. Not to say it isn't important as soon as the engine becomes usable for non-programmers of course.

[idanarye]

Wouldn't it be easier if the infrastructure for i18n is laid out to begin with, even if it would initially only support English and other languages will arrive at an time unknown?

[DarkSeasonStudios]

Engine-Host-Editor

The current situation with Bevy is that it is being used as a library. Most of the extensibility issues that have been raised come down to the fact that you need a host application with EGUI and the ability to create an editor-out-of-the-box on top of it. An important issue is whether the host supports hot-reloading for plugins (which gameplay assembly can be) and how it stores data between sessions. In any case, I suggest the following scheme:

The host itself is not required to be well.. "Editor" (like a level editor), it can only provide basic engine layer maintenance functionality and hook plugins, run simulation, and feedback engine state via EGUI. Maybe project-files management. The editor, in turn, can vary greatly, depending on the game you are making. Are you considering making out-of-box-editor as plugin for the host?

I also strongly support the idea of unified Node-Graph-Framework, that can be base for custom graph-based needs. This is something that any modern editor need in guts. Maybe something like this?: https://github.com/setzer22/egui_node_graph

[supemeko]

Provides Bevy Plugin to help developers build their own custom editors

It is practically impossible to make editors for all kinds of games, and the best way is to provide powerful bevy plugins for developers to build their own custom editors. Includes mod editor and in-game editor.

[idanarye]

Restrictions on User Code

I know the ideal is to place no restrictions on users of the editor, but don't think it is realistic to expect for anything that's possible with code-only Bevy to automatically get an awesome visual UI when you attach the editor to it. Maybe it's doable with highly reflective languages that have powerful and safe(ish) ways to introspect and edit data and sometimes even functions at runtime - but not in Rust. Not without active help from the user code itself, at least.

I think an important part of the design is to determine what is reasonable to demand from user code, and what isn't.

For example (I'm not necessarily saying it is! This is just to give an example!):

• It's reasonable to expect that all the components (or at least all the relevant ones?) will implement Reflect.
• It's unreasonable to expect the user to add systems that manually transmit all component data to the editor.

[lgsandbox]

Package Manager

Having a graphical interface for managing packages (and by extension plugins) seems like it could be a solid requirement. Didn't see a post on it yet so thought I would put it up for discussion.

[Nilirad]

Can you please elaborate what do you mean by “package”, and why you think this should be a requirement? Otherwise some people may struggle to evaluate.

[lgrammer]

Sure np, I just meant something along the lines of a traditional GUI-based package manager, like the one Unity uses.

As far as why it has potential as a possible requirement there are a few considerations.

Dependency management: An editor may have many dependencies and a package manager can help manage them. This can include resolving dependencies, installing required packages, and updating packages to newer versions.

Discoverability: A package manager can make it easier to discover new packages and plugins that extend the functionality of the editor.

Accessibility: With an editor release, we will see an influx of non-programmer designers and artists interacting with Bevy. Imo, it could be helpful to give them a way to interact with plugins that are considered Bevy assets directly in the editor. An interface for managing packages and plugins only helps set them up for success.

Versioning: A package manager can help manage the versioning of packages, ensuring that users have access to specific versions of packages that are known to work with their version of the editor.

There could be a few other considerations to expand on this even further.

At the end of the day, whether it is actually a requirement, or whether it is best used as a requirement in a MVP or post-MVP release is a bit out of my scope and best left for more experienced users and contributors to discuss. I just wanted to bring the utility of a package manager interface in the editor up for discussion.

[idanarye]

One of Bevy's core tenets is that both the engine itself and the user code should be in Rust, so I assume editor plugins should also be in Rust. I say they should be crates on crates.io tagged with a special category (there is a closed list, but Bevy is big enough to request a category for editor plugins) so that the package manager can easily fetch the list from the crates.io API.

The crate itself should contain metadata that tells the package manager how to install that plugin - usually it'll just be the full path to a Plugin or PluginGroup, but maybe in the future some plugins will need some more options.

The package manager should be able to rebuild the editor based on the plugins' metadata.

[mrgzi]

In my opinion a package manager is not a good idea inside of editor. While using Unity, I had a lot of troubles with package manager or importing a package from asset store. Editor should not change anything inside code except metadata.

[viridia]

Overlays, Tool Modes and Handles

An important part of the editing experience is how the editor interacts with the 3D view. Creating a game level is more than just placing 3D objects in the world and editing their properties. An editor will also have numerous editing modes that allows sculpting or direct manipulation of the objects in the scene. A typical "editing tool" or "mode" will take over the handling of mouse events in the 3d window. This includes custom picking logic - for example, a terrain sculpting tool will use a different picking algorithm than one that is meant for selecting entities, or one that paints trees on the terrain. At the same time, many editing tools have similar picking and event handling logic, so it makes sense to be able to share the implementation of these.

In my own work, it has often proved useful to think of the edit mode as a "stack", such that the editing tool on the top of the stack is the one that handles all events. To understand this, think about the way rotations work in Blender: you press 'r' and that puts you into a temporary mode where the mouse controls the selected object's rotation. Hit ESC or click and it restores you to the previous mode.

In addition, many of the important elements of a game level are invisible - this includes:

• Physics colliders
• Navigation meshes
• Markers and waypoints
• Monster spawners
• Portals
• Trigger volumes and surfaces
• Actor tethers (regions which restrict random NPC movement).
• Particle emitters

During editing, the artist will want to enable translucent overlays that show the placement of these objects. At the same time, they also need to be able to disable these overlays so that the view doesn't become too cluttered.

Drag handles are a kind of overlay or gizmo which allows direct manipulation of a selected object. Rather than having to type in the coordinate to nudge an object by a short distance, you can click on the handle and drag the object directly.

The editor should provide a general framework that allows users to create their own edit modes, overlays, and handles. If you want more details on what this API might look like I can provide them.

[nicopap]

I stretched a potential implementation in the following thread: #9187

I think "editor overlay" (well currently gizmo users) should be registered by plugins and gathered by bevy, so both the editor and user code can manipulate the overlays. But it seems to have 0 traction :/

[viridia]

Evolutionary approach

"Build libraries, not frameworks" - old programming maxim.

Reading the comments here, I can see that a lot of people have different ideas about the requirements for the editor. Some people will want to heavily customize the editor for their needs, while others are more interested in a pre-built, out-of-the-box solution. This is entirely expected, I think - there's never going to be one single solution that satisfies everyone.

I think that the best way to approach this is to avoid making the editor a monolith. Instead, the editor can be built incrementally from modules using a bottom-up approach. There are a few key components that will be common to everyone (more on this below). Once those components are implemented, people can start using them in their own editors even before the rest of the "official" editor is complete. Different approaches to editing can compete in the marketplace of ideas, and the best of these incorporated into the "official" editor.

At the core of the editor, there are several critical pieces that provide the foundation, these include data binding, property editing, and undo / redo. These all fall within the realm of "state management".

Data binding is the part of framework that allows mutations within the editor to be propagated to the live preview of the game. For mutations of entities and components, this is simple, since these properties are typically polled every frame. However, many of the assets being edited will be generators, templates or prototypes. For example, editing the properties of a particle system might not have any effect on particles that have already spawned - instead (depending on the design of the particle framework), a signal may be required to re-initialize the particle system each time the particle parameters are mutated. So the data-binding system provides a way to notify the game engine when a specific resource has been changed. It's up to the engine, then, to decide which parts of the 3d view need to be re-initialized.

The data binding framework will often be coupled to an asset loading system - that is, when an object in memory is modified, the asset system will be notified that the object in memory is changed from the one on disk. However, a game may have more than one asset loader (for example, you might have one loader for large binary blobs and another for JSON-encoded game data.)

These same signals also drive the "save" functionality - editing an asset in memory does not automatically update the copy on disk, nor would you want it to (artists sometimes change their minds and quit without saving, or revert to saved). Thus, we need to know when assets are "dirty" so that the save button can be enabled, and then flush the modified ones to disk.

Property editing is an extensible framework for dynamically building a user-interface from the properties of a reflected struct. This needs to allow both standard built-in types, as well as custom types that are specific to a particular game. It should be possible to register plugins that handle editing of these custom types. It also needs to support nested types (arrays and structs).

The reflection metadata needs to be extensible for these custom data types. For example, in an asset system, you may have resource ids that represent different loadable assets. The editor will provide a "resource chooser" ui that allows you to browse the available resources and select one. However, a resource key property might only be compatible with specific asset types - you can't store the resource key for a skinned actor model in a slot intended for an inventory item, or a terrain patch. The reflection metadata will specify this, but only the resource chooser ui knows how to interpret this reflected data.

Undo / Redo is fairly simple, but needs to be done early because every other part of the system (that is, any part that is capable of mutating data) depends on it. Undo stack entries are simply a struct that has a label (so that you can display a button such as "Undo Delete") and a closure which undoes the effect of the mutation. A more advanced form allows a "coalescing key" so that many small user edits can be combined into a single undo action (there are various different approaches for this).

There are probably a few other core components I haven't thought of. However, just these elements alone are enough to get people started building their own editors.

[idanarye]

I still don't see why the Xilem integration with Bevy has to go through Vello instead of being done directly...

[bjorn3]

Vello is what Xilem uses for 2D drawing. Using bevy_render instead (or rather bevy_sprite, bevy_text, ...) would I think require rewriting every single ui element to use it in the place of Vello. Vello has already switched from it's own render api abstraction to wgpu to allow integrating it with for example bevy_render more easily.

[idanarye]

So the plan is to make Vello the default 2D renderer of Bevy?

[bjorn3]

Not that I know of. I think the experiment used Vello as 2D renderer for the Xilem side and the regular Bevy crates on the Bevy side. Maybe Bevy will start using Vello in the future and maybe not.

[IceSentry]

There are indeed no plans to use vello as the 2d renderer. Bevy will keep using it's own renderer. Using a separate renderer would introduce a lot of complexity.

From what a I understand, Xilem does support multiple backend for their View trait so it could potentially have a bevy renderer backend. The vello integration is essentially just rendering the output of vello to a bevy texture then rendering it in a bevy scene. This means that it could also be possible for xilem to keep using it's vello backend while bevy would render everything else using it's own renderer.

[viridia]

Evolutionary approach, V2

I've learned a lot in the last month since I posted the previous item, and my thoughts on editor requirements have evolved quite a bit. This is mainly a result of working on a daily basis with bevy_ui.

The general consensus seems to be that the editor will be based on bevy_ui, and that bevy_ui needs a lot of improvement before this can happen. Fair enough; however what I'd like to know is what the plans are for evolving bevy_ui, whether or not there is an active discussion / requirements gathering on this. As previously stated, I'm more interested in writing my own editor than using someone else's, but I need a good toolkit to do that.

The current state of bevy_ui is much like the HTML DOM. In fact, building UIs in Bevy is not that different from building UIs on the web by manually creating DOM elements and linking them together using parent / child relationships, and then manually subscribing to events. However, no one working in the web space actually does it this way anymore, because the developer experience is horrible; The DX of working with bevy_ui is in a similar state. Building large, complex UIs in bevy_ui is extremely labor-intensive at present.

The web evolved over the last 20 years through a sequence of paradigm shifts, from jQuery to Ember to React to Solid / Swift / Vue and so on. None of these innovations focused on making the DOM better, but rather building a reactive layer on top of the DOM, one which encompassed not only rendering, but event handling and state updates as well. Doing something similar in the Bevy engine seems like the obvious approach: Bevy UI nodes as they exist today become the low-level DOM nodes in this architecture, but the artist designing the UI doesn't concern themselves with objects at that level, they are working with higher, more abstract UI components.

There are several competing Bevy UI frameworks that take this approach. I'm working on one, but there are others as well, and they all have some good ideas. However, none of these are in what I would call a "finished" state, and I don't think this space is mature enough yet to be able to pick a winner.

At this point, I think that the best thing that we can do in the short term is to make bevy_ui be a better DOM, to make it easier to build these higher-level frameworks. That includes more options for 2d rendering (things like ninepatch images, clipping rounded corners, and so on), and event bubbling (like bevy_mod_picking does). The latter is important for making aggregate ui components that can be plugged in and re-used, because the aggregate component can handle all of the events coming from its children.

Also, I think that being able to query by traits (like bevy-trait-query) is a critical building block for building large, rich component libraries - without it, you have to register two or three systems for each individual type of widget. With trait query, you can have a single system (or two) that handles bubbled events and dispatches them to the correct widget logic for state updates.

I know reactivity is a hot topic, which a lot of people have asked for, but it's also a hard problem to solve in ECS and Rust. Reactive frameworks in JS depend heavily on getters / setters and Object.defineProperty(), features which don't exist in the Rust language. However, I don't think this is a problem that needs to be solved at the "DOM" level - rather, I think that this is something that the higher-level UI frameworks should experiment with. Which means that unless someone has a brainstorm for a brilliant solution, it's not something that the Bevy core team needs to worry about just yet.

[IceSentry]

In the context of signals based reactivity and rust, leptos and sycamore are 2 examples of rust libraries trying to make signal based reactivity work in the context of rust. I believe there's a lot of potential for going this route with bevy_ui. Like you said though, making sure bevy_ui is easy to abstract over is very important for people to experiment because ui in rust and bevy is still far from a solved problem.

On the topic of bridging the gap between ui hierarchies and the ECS, bevy_eventlistener is also something that can help a lot.

[idanarye]

Maybe the two-layers architecture should be what bridges the gap between the flat ECS and the hierarchical DOM? I said before that that we shouldn't fixate on the lower layer being ECS and the higher layer being something else, but maybe we should take this one state farther - the higher layer will be ECS, and the lower layer won't be.

The lower layer will be an hierarchical DOM with very little "attachment" to the ECS. It'll either have all it's data inside a single Resource (or small set of resources) or - if we decide we want to be able to have multiple such separate DOMs (e.g. - for rendering in multiple windows? Not sure if that's actually needed) - it can be inside a single Component that contains the entire DOM. That data structure would be perfectly usable without the ECS - the ECS only stores it and provides an entry point with a system that also gives it a buffer to render on and tells it what the user input is (or maybe the input should be left out of it, and the higher layer will listen to input from the game and decide what to do with it?)

The higher layer is the one how has ECS representation. Its data will be placed on components (plural), and multiple entities that participate in the same UI can have different parts of the data. The signals of the reactive paradigm will actually be stored in components, so that other parts of the game will be able to freely read and write them and the UI will... well... react.

As @viridia said, the ECS does not handle hierarchy well, so we don't want the higher layer to be too hierarchical. The lower DOM layer will have an hierarchy that represents the layout, and the higher layer will contain big chunks of that hierarchy inside single entity each, and only be as hierarchical as it has to be in order to represent the data itself.

What does that mean? Let's say we have something like this (I'm using HTML-like syntax for the UI layout because everyone knows HTML, but of course this does not mean the Bevy UI should use HTML)

<ul>
    <li>
        <ul>
            <li>foo</li>
            <li>bar</li>
        </ul>
    </li>
    <li>
        <ul>
            <li>baz</li>
            <li>qux</li>
        </ul>
    </li>
</ul>

To create this in the higher layer, we'd use something like this:

commands.spawn((
    MyData {
        foo: 1,
        bar: 2,
        baz: 3,
        qux: 4,
    },
    dom! {
        <ul>
            <li>
                <ul>
                    <li>{|data: This<&MyData>| data.foo}</li>
                    <li>{|data: This<&MyData>| data.bar}</li>
                </ul>
            </li>
            <li>
                <ul>
                    <li>{|data: This<&MyData>| data.baz}</li>
                    <li>{|data: This<&MyData>| data.qux}</li>
                </ul>
            </li>
        </ul>
    },
));

(This<T> is a shorthand for getting the data from the same entity, but maybe we'll want ways for getting data from other entities)

So in the DOM we have a nested list of lists, but the user code only needs to address that hierarchy inside the dom! macro - other than that it's a flat MyData component on a single entity.

[viridia]

I want to make a meta-point: this is probably not the right forum for this. I think that discussing the future of bevy_ui is important and valuable, and I think it's a prerequisite for building the editor - but it really ought to be its own topic, and not cluttering up the discussion of editor requirements.

[idanarye]

What's the scope of this thread then?

[JMS55]

Going to 2nd the fact that this is not the right place to discuss UI (do that here: #9538).

This thread is for determining high level user-facing features of the bevy editor, and not technical details.

[mattdm]

Accessibility (a11y)

Any application in 2023 should be designed with accessibility features in mind. There's a previous comment here which mentions non-programmer artists and designers, but I mean specifically assistive technologies for people who require them — screen readers, etc.

I know that Bevy has this in mind in general, which is aweome (https://bevyengine.org/news/bevy-0-10/#accesskit-integration-into-bevy-ui). I just want to make sure this support is explicit as an Editor requirement.

(I know the situation isn't great on Linux. Working to improve that!)

[ksf]

Crashes, data, gaudy databases, savefiles, oh my... VCS?

One of the arguments for a seperate-process architecture is crash resilience. I'm not a big fan of the approach in general, least of all as an MVP requirement, or as the only way to do things, let me talk a bit about another way to provide data safety and integrity that would also enable doing other desirable things.

tl;dr: As the old adage goes, a game is a database with gaudy user interface. Let's lean into that, even more than we already do.

---

"Crash-only software" is a funny name but an actual design principle, the gist of it is that you accept that crashes are ultimately unavoidable because even flawless software crashes when someone trips over the power cord and the only way to make sure that no data is lost is to aggressively ACID everything you care about, and for good measure replace all your shutdown procedures with sending yourself SIGKILL so that the ACID actually gets exercised. For an editor "everything you care about" would approximately be everything that affects the undo buffer: Do something, it gets written to disk. Want to undo something, that's equivalent to rolling back a transaction.

The neat part is that relational databases generally provide ACID guarantees, and we're already relational, anyway: Components are tables, Entities are foreign keys.

Another thing database folks have long since figured out is migrations: CREATE, DELETE, and ALTER TABLE to upgrade the schema, as well as running transactions to migrate the actual data: Say a program expects schema version 5 to run, but loads a file with version 2, that's fine it's going to run migrations 2->3, 3->4 and 4->5, or a squashed 2->5 one.

This has obvious use during development where component types might be introduced, altered, or deleted. It's also important for players, breaking save files isn't exactly a thing that makes a gamedev popular. It's also what the vast majority of games do in one way or the other, even if not necessarily that formally.

What I propose more concretely (arguing against my former self) is to move towards keeping component type definitions outside of Rust code, and not even in a single file listing all the component used, but as a series of database migrations, somewhat like this. The concrete types the game will use can be easily and quickly inferred by applying all schema changes one after the other, that can be done by a macro. This avoids having multiple, possibly conflicting, sources of truth.

Coming back to the editor: It enables an easy way to make things more flexible before a fully generic editor supporting plugins and everything is done, by allowing editor users to stub out components themselves, say create a Powerup component with type: String field, a Powerup entity prototype with Powerup, Position, and Sprite components to be able to place powerups in levels (sprites and position can easily be handled generically), and once a gameplay programmer gets around to implementing powerups everything that's already placed can be automatically migrated to e.g. change type to an Enum. It's not strictly necessary to write migration code for these things, the editor can also mark the component as loaded from the level file as not type-compatible with the current DB schema ask the level designer to migrate manually.

The game, too, not knowing (yet) what to do with that strange Powerup component, might be generic enough to already display them: It has a Sprite and a Position, if you already display wall and floor tiles it's probably just going to work.

All of that requires (at most) infrastructure for run-time dynamic components, but it doesn't require separate processes, plugins, or generally anything that IMO would be quite out of scope for an MVP Editor.

...lastly, having infrastructure in place to ACID the ECS, things like recording game state update-by-update, then playing it back, going forwards and backwards as you like become quite easy, and that kind of thing is very useful for debugging (even though your performance might tank and your SSD might begin to smoke).

Oh. As to "whether to actually use an off the shelf database for this": It's probably a good idea to throw SQLite at things like game save files and editor project files, at least in the beginning, or even as default. Engineering ACID is not exactly trivial and SQLite is going to be fast and lean enough for pretty much everyone. Specifically for the editor pijul would be a candidate: It's ACID and a VCS, making clones, rollbacks etc, very efficient (and it's written in Rust). It should be performant enough to have one patch per undo step without breaking a sweat provided bevy commits patches directly instead of having pijul do diffs to generate information bevy already has. Also it's a VCS. I know I know also not necessarily MVP territory.