Add Yet Another Source Code Generator#6844
Merged
tangent-vector merged 11 commits intoshader-slang:masterfrom Apr 17, 2025
Merged
Add Yet Another Source Code Generator#6844tangent-vector merged 11 commits intoshader-slang:masterfrom
tangent-vector merged 11 commits intoshader-slang:masterfrom
Conversation
This change introduces an offline source code generation tool, provisionally called `fiddle`. More information about the design of the tool can be found in `tools/slang-fiddle/README.md`. Yes... this is yet another code generator in a project that already has too many. Yes, this could easily be a very obvious instnace of [XKCD 927](https://xkcd.com/927/). This change is part of a larger effort to change how the AST types are being serialized, and the way code generation for them is implemented. Right now, the source code for the new tool is being checked in and the relevant build step is enabled, just to make sure everything is working as intended, but please note that this change does *not* introduce any code in the repository that actually makes use of the new generator. All of the AST-related reflection information that feeds the current serialization system is still being generated using `slang-cpp-extractor`. The design of the new tool is primarily motivated by the new approach to serialization that I'm implementing, and once that new approach lands we should be able to deprecate the `slang-cpp-extractor`. In addition, the new tool should in principle be able to handle many of the kinds of code generation tasks that are currently being implemented with other tools like `slang-generate` (used for the core and glsl libraries). This tool should also be well suited to the task of generating more of the code related to the IR instructions.
Contributor
Author
|
/format |
Contributor
|
🌈 Formatted, please merge the changes from this PR |
…neration-tool Format code for PR shader-slang#6844
tangent-vector
added
pr: new feature
pr: non-breaking
Contributor
Author
|
Oh my. The new tool is crashing during the build on several of our target platforms. This will be fun to debug. |
csyonghe
reviewed
Apr 17, 2025
Collaborator
csyonghe
left a comment
csyonghe
left a comment
There was a problem hiding this comment.
We should use this to generate the .meta.slang files as well.
| break; | ||
|
|
||
| case '$': | ||
| if (isInScriptLine) |
Collaborator
There was a problem hiding this comment.
We need a way to escape, e.g. $$ means just a $.
Contributor
Author
There was a problem hiding this comment.
Yes, the $$ approach is probably the ideal one. Luckily nothing about this tool cares about $s outside of the templates themselves. It's also possible that we could change up the sigil character(s) so that they are less likely to interfere with stuff like how we do the .meta.slang files.
| if (isInScriptLine) | ||
| continue; | ||
| if (depthInSplice) | ||
| throw 99; |
Collaborator
There was a problem hiding this comment.
output a diagnostics instead of throwing?
Contributor
Author
There was a problem hiding this comment.
Yes, I was lazy at several points while developing this and did a quick throw instead of diagnosing properly.
We may still want to fail-fast on errors rather than try to soldier on, just given that the tool itself isn't 100% robust against bad input.
Collaborator
There was a problem hiding this comment.
you can do a fatal error style handling, just write to diagnostic and throw, and make sure we handle the exception in main and print out the diagnostics.
Contributor
Author
|
/format |
Contributor
|
🌈 Formatted, please merge the changes from this PR |
…neration-tool Format code for PR shader-slang#6844
szihs
pushed a commit
to szihs/slang
that referenced
this pull request
May 7, 2025
* Add Yet Another Source Code Generator This change introduces an offline source code generation tool, provisionally called `fiddle`. More information about the design of the tool can be found in `tools/slang-fiddle/README.md`. Yes... this is yet another code generator in a project that already has too many. Yes, this could easily be a very obvious instnace of [XKCD 927](https://xkcd.com/927/). This change is part of a larger effort to change how the AST types are being serialized, and the way code generation for them is implemented. Right now, the source code for the new tool is being checked in and the relevant build step is enabled, just to make sure everything is working as intended, but please note that this change does *not* introduce any code in the repository that actually makes use of the new generator. All of the AST-related reflection information that feeds the current serialization system is still being generated using `slang-cpp-extractor`. The design of the new tool is primarily motivated by the new approach to serialization that I'm implementing, and once that new approach lands we should be able to deprecate the `slang-cpp-extractor`. In addition, the new tool should in principle be able to handle many of the kinds of code generation tasks that are currently being implemented with other tools like `slang-generate` (used for the core and glsl libraries). This tool should also be well suited to the task of generating more of the code related to the IR instructions. * format code * Build fixes caught by CI * Fix another warning coming from CI * Another CI-caught fix * Change bare hrows over to more proper abort execptions * format code --------- Co-authored-by: slangbot <186143334+slangbot@users.noreply.github.com>
github-merge-queue bot
pushed a commit
that referenced
this pull request
Aug 18, 2025
The documentation added by #6844 included instructions to make sure that the Fiddle `#include` in a file comes after all the other `#include`s, but it's easy to accidentally violate this via `clang-format`, as happened for `source/slang/slang-ast-modifier.h` in #7559. This PR guards against this sort of violation by separating all Fiddle `#include`s from other `#include`s via a blank line followed by a `//` line (as we already do in most cases), and also adds a sentence about this in `tools/slang-fiddle/README.md`. As a bonus, I also enabled Markdown syntax highlighting for all the code blocks in that doc file. Co-authored-by: Ellie Hermaszewska <ellieh@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change introduces an offline source code generation tool, provisionally called
fiddle. More information about the design of the tool can be found intools/slang-fiddle/README.md.Yes... this is yet another code generator in a project that already has too many. Yes, this could easily be a very obvious instnace of XKCD 927.
This change is part of a larger effort to change how the AST types are being serialized, and the way code generation for them is implemented.
Right now, the source code for the new tool is being checked in and the relevant build step is enabled, just to make sure everything is working as intended, but please note that this change does not introduce any code in the repository that actually makes use of the new generator. All of the AST-related reflection information that feeds the current serialization system is still being generated using
slang-cpp-extractor.The design of the new tool is primarily motivated by the new approach to serialization that I'm implementing, and once that new approach lands we should be able to deprecate the
slang-cpp-extractor.In addition, the new tool should in principle be able to handle many of the kinds of code generation tasks that are currently being implemented with other tools like
slang-generate(used for the core and glsl libraries). This tool should also be well suited to the task of generating more of the code related to the IR instructions.