Designing and implementing a migration path for porting assets to the linear pipeline

[illwieckz]

So, the linear pipeline has been merged:

• renderer: implement native sRGB support #1687

So for now the engine supports:

• rendering the old way maps done the old way (messing with colorspaces like Q3/Tremulous), also called “the naive pipeline”.
• rendering the new way maps done the new way (properly handling colorspace conversion at every step of the baking and rendering pipeline), also called “the linear pipeline”.

It means maps already built are just rendered like before, but contributors can start creating brand new content targeting the new linear pipeline without having to care about the pitfalls and workaround and weirdness and painful wizardry of the old naive pipeline.

But what we don't care about yet, is about migrating assets from the old naive pipeline to the new linear one.

For example we know existing textures meant to be used in blending operations were wrongly calibrated for the old naive and broken pipeline. When used in the new, correct and linear pipeline, they don't render properly because they were biased to workaround bugs that don't exist anymore.

We may want to provide some kind of conversion or specially crafted workaround for when a blended asset from one pipeline is used on the other.

This would help to solve two problems:

1. Modifying many assets is painful and tedious, if we can just flag either all old assets or old new assets to tell it was meant for a different computation, without actually checking and modifying assets one by one, we could port all of them the blind way just by doing some flagging and letting the engine do the magic.
2. Because we provide reusable texture packs, some textures may be used in maps that will never be ported and also be used in maps that will. So having ways to workaround the usage from one way to another would help. The Unvanquished maps (including community maps), can probably be ported at some point as we either have source or authors are still around, but that's only a hope, we cannot enforce that. Also the problem is stronger for a package like res-tremulous where some legacy maps will never be rebuilt (no source, no author around), but some of other legacy maps may be ported to the new pipeline and use those legacy assets.

[illwieckz]

Well, now that I posted it, I notice I could have posted it to the Unvanquished issue tracker instead (it's more a project management thread), but 95% of the solutions will likely be some technical engine implementation anyway.

[slipher]

I propose adding 2 mechanisms for defining colorspace-dependent shaders:

1. Stage-level enablement based on matching rendering mode

Let's say we wanted to adjust the strength of the additive stage here depending on the rendering mode. The colorspace naive stage would be disabled when using linear blending and vice versa.

textures/shared_ex/light2_red_6000
{
	qer_editorImage textures/shared_ex_src/light2_d

	q3map_surfaceLight 6000
	q3map_lightRGB 1.000 0.424 0.380

	{
		diffuseMap textures/shared_ex_src/light2_d
		normalMap textures/shared_ex_src/light2_n
		heightMap textures/shared_ex_src/light2_h
		specularMap textures/shared_ex_src/light2_s
	}
	{
		colorspace naive
		map textures/shared_ex_src/light2_a
		blend add
		red 0.6
		green 0.4
		blue 0.3
	}
	
	{
		colorspace linear
		map textures/shared_ex_src/light2_a
		blend add
		red 0.8
		green 0.6
		blue 0.45
	}
}

2. Shaders which can only be loaded in a particular rendering mode

Let's say you're running in linear mode. Then if you try to load a (3D mode) shader named textures/shared_ex/light2_red_6000, the renderer should first attempt to look up textures/shared_ex/light2_red_6000:linear, and only if that doesn't exist, try the unsuffixed textures/shared_ex/light2_red_6000. If RSF_2D was used, the suffixed form should be skipped.

textures/shared_ex/light2_red_6000:naive
{
    ...
}

textures/shared_ex/light2_red_6000:linear
{
    ...
}

In most cases, method 1 (stage-level enablement) should probably be used to minimize the amount of typing. But method 2 could come in handy in certain scenarios:

• Override shaders in another pak
• Prevent a shader from being loaded at all in the wrong colorspace
• Cleaner code for very complex shaders which can't share any stages

[illwieckz]

I like the idea of the colorspace stage keyword.

We may define that by default the colorspace is naive, because of the fact all of our assets are naive, so porting them would just mean implementing the feature, but not flagging them. And then rebuilding any legacy map with the new pipeline would make the engine use the naive way for blended textures without extra effort.

I'm proposing a similar mechanism (new shaders using new thing use a specific keyword) for q3map2 and the point/area backsplash light problem: https://gitlab.com/xonotic/netradiant/-/merge_requests/220

I also wonder if there can be a way to define a shader-wide-option, like in a special comment (to not break third-party softwares like the various Radiant editors):

// SOMEMAGICKEYWORD colorspace linear

The SOMEMAGICKEYWORD token would be be defined in a way a real comment cannot be confused with such global setting by mistake.

[slipher]

I like the idea of the colorspace stage keyword.

We may define that by default the colorspace is naive

A default of naive for the stage-level colorspace? That would be bad since then we would have to have 2 versions of every stage in every shader, for all shaders we want to use in both modes. My idea was that by default, a stage would be rendered in all modes.

I also wonder if there can be a way to define a shader-wide-option, like in a special comment (to not break third-party softwares like the various Radiant editors):

// SOMEMAGICKEYWORD colorspace linear

That's a good point that the suffixed names could break the map compiler. Though maybe we could still get that to work if instead of defining dual variants like foo:linear + foo:naive, we did it like foo + foo:naive or foo:linear + foo. I was trying to come up with a way that would easily work with our shader hash table that doesn't parse the inside of a shader until it's loaded. But with a bit more effort, we could implement a version that stores multiple variants and backs out upon seeing a forbidden colorspace. Why would it need to be a magic comment instead of a normal stage-level keyword though? Surely q3map2/radiants don't just refuse to process the shader if they see an unknown keyword?

Of course the map compiler won't understand stage-level enablement either, but that's probably OK? Can we assume that q3map2 ignores all shader stage contents, other than using the first colormap for light bouncing?

[illwieckz]

A default of naive for the stage-level colorspace? That would be bad since then we would have to have 2 versions of every stage in every shader, for all shaders we want to use in both modes. My idea was that by default, a stage would be rendered in all modes.

The idea is that operations that would be different in naive or linear pipeline would be assumed naive otherwise.

So a stage with no blending would be linear, a stage with blending would be naive, unless colorspace linear is used. So maybe another name like blendingcolorspace would be better.

I don't see the point of rendering an opaque stage the naive way if the lightmap was done the linear way, so only blended stuff would have a meaning of being naive or linear.

I also wonder if there can be a way to define a shader-wide-option, like in a special comment (to not break third-party softwares like the various Radiant editors):

// SOMEMAGICKEYWORD colorspace linear

That's a good point that the suffixed names could break the map compiler. Though maybe we could still get that to work if instead of defining dual variants like foo:linear + foo:naive, we did it like foo + foo:naive or foo:linear + foo. I was trying to come up with a way that would easily work with our shader hash table that doesn't parse the inside of a shader until it's loaded. But with a bit more effort, we could implement a version that stores multiple variants and backs out upon seeing a forbidden colorspace. Why would it need to be a magic comment instead of a normal stage-level keyword though? Surely q3map2/radiants don't just refuse to process the shader if they see an unknown keyword?

I believe q3map2 / radiant skip unknown keywords (unlike legacy renderers that stupidly stops on unknown token… we had to fix that in our engine).

Of course the map compiler won't understand stage-level enablement either, but that's probably OK? Can we assume that q3map2 ignores all shader stage contents, other than using the first colormap for light bouncing?

Radiant and Q3map2 parsing of shaders is much more simplistic than you think it is. 😁

Both Radiant and Q3map2 entirely skip the stages.

All Radiant is reading are the qer_ keywords, qer means QuakeEdRadiant, from QuakeEd the NextSTEP in-house Quake Editor by id Software, so basically qer keywords is for the editor, whatever the exact editor name. It also reads the surfaceparms but that's all.

Then all the map compiler reads are the q3map_ keywords, yet again, whatever the exact map compiler name. And it also reads the surfaceparms keywords. And it makes an exception, it also reads the qer_editorImage to sample the colors from the said texture, this is to sample something when the shader name itself is not the name of an actual file (exactly like the editor does). This is also why we can't put anything stupid in the editor image as it may be used to sample colors from light shaders or when light is bouncing on it. For example at some point we had a tool that created jpg previews for CRN files painting ping in alpha pixels for easy notice in editor, but then then q3map2 was sampling the pink color when bouncing the light on windows and grates. 🤡

Both the editor and the map compiler have no idea about what renderers do with stages. In some way it helped to keep the tools compatibles with many games while the renderers were all doing there own things without concertation. NetRadiant has some advanced stage parsing for Doom3 where it can mimic the Doom3 renderer (and parse normal maps, for example), but that's an exception. The usual way to render things in those editor is just to render the textures fullbright without any special effect, and the editor doesn't parse any blend operation, so we just workaround it with qer_trans to just give some transparency so we can actually notice in editor the surface is not opaque, and poorly emulate windows and such.

But, if I assume the map compiler and the editor skip unknown tokens (and very likely do it silently) within shaders, I'm not assuming they will skip unknown tokens outside of shaders. Any token that is outside first level braces is considered a shader name that should be followed by braces.

[VReaperV]

But, if I assume the map compiler and the editor skip unknown tokens (and very likely do it silently) within shaders, I'm not assuming they will skip unknown tokens outside of shaders. Any token that is outside first level braces is considered a shader name that should be followed by braces.

What about a special shader name that would have these settings? Something like __daemon_light_settings or whatever. Assuming editor/q3map2 won't just throw it out as unused.

[illwieckz]

I also thought about it, but editors may still uselessly process them at some point.

NetRadiant only lists shaders that starts with textures/ because the map format strips the textures/ prefix when applying them on brushes and surfaces and storing them in Q3 maps, but it still loads all shaders whatever their names because models can get any path applied on their surfaces.

Some editors like DarkRadiant may list all paths (D3 map format may have stopped strippingthe textures/ prefix, stripping that was probably inherited rom Quake 1).

[illwieckz]

Assuming editor/q3map2 won't just throw it out as unused.

Editors and q3map2 don't edit shaders. Q3map2 can generate the light style shaders, but no shader is edited by the software.

[illwieckz]

Such configuration header could be done that way:

__config
{
	__linearblend 1
}

Most tools should simply ignore that, at worst print some warnings for unknown keywords (though I doubt about it as the tools are meant to be permissive because of many forks), and uselessly list the shader in some cases.

[illwieckz]

@slipher Thanks for #1721!

But I don't like the idea of what is naive or linear being the colorspace.

The blending operation is naive or linear,

The image colorspace is sRGB or linear.

What if we extend the stage map keyword?

Like:

textures/castle/brick
{
	{
		diffuseMap sRGB textures/castle/brick
	}
	{
		colorMap linear textures/castle/dirt
		blendFunc blend
	}

}

The good thing is that keywords like diffuseMap in stages are specific to Dæmon (Doom 3 only supports them at the root of the shader, without stage blending), so we are free to extend that and we will break nothing, even not the NetRadiant DOOM 3-like renderer that already doesn't support those stage (if one day we would rely on it to get preview of the game, we would have to implement the multitextured stage to begin with).

To not break software that may render the legacy stages with map keyword, we can rely on the colorMap one instead.

So basically if the first word is sRGB or linear, then set the format then reads the texture name, otherwise it's a texture name.

[illwieckz]

Keywords that would receive this can be colorMap, diffuseMap and glowMap, I guess…

[illwieckz]

Welp, I misread the PR, I thought the colorspace keyword was for configuring the image loading, but it's for skipping a whole stage entirely. For such “skip the stage” feature then we better use some operation-related name than “colorspace”.