Skip to content

feat: add renderdoc capture#1329

Closed
doodlum wants to merge 4 commits into
devfrom
renderdoc
Closed

feat: add renderdoc capture#1329
doodlum wants to merge 4 commits into
devfrom
renderdoc

Conversation

@doodlum
Copy link
Copy Markdown
Collaborator

@doodlum doodlum commented Jul 29, 2025

  • Supports generating basic Renderdoc captures with no additional software.
  • Includes lots of quality-of-life for users to make creating and managing captures as simple as possible.
  • Heavy number of warnings to inform users as much as possible when it is enabled and how to disable it, that it should not be enabled.
  • Renderdoc GUI is automatically usable without hooking the game, accessible by using File->Attach to Running Instance under localhost.
  • Renderdoc overlay is disabled because users will not understand what's happening, replaced with CS overlay instead. Hotkeys still work.

Capture API supports adding comments which bake text in the capture. In the future we may want to add comments with important information to help us diagnose issues.

image image image

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Integrated RenderDoc graphics debugger with comprehensive API support for frame capture and debugging.
    • Added UI controls in advanced settings to toggle frame debugging, trigger captures, view capture status, disk usage, and clear captures.
    • Overlay displays frame debugging status and performance warnings when enabled.
  • Improvements

    • Replaced "Frame Annotations" option with "Frame Debugging" across UI, configuration, and internal state.
    • Initialized RenderDoc integration during plugin load when frame debugging is enabled.
    • Enhanced user feedback on RenderDoc availability and capture management.
  • Documentation

    • Added MIT License file for RenderDoc integration.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jul 29, 2025

Walkthrough

This change integrates RenderDoc frame debugging capabilities into the application. It introduces the RenderDoc API header and implements a RenderDoc singleton class for managing captures, disk usage, and UI integration. The menu and overlay UI are updated to provide frame debugging controls and status. The configuration and state management are refactored to use a frameDebugging flag instead of frameAnnotations. RenderDoc initialization is conditionally performed during plugin loading.

Changes

Cohort / File(s) Change Summary
RenderDoc API Integration
include/Renderdoc/renderdoc_app.h, src/RenderDoc.cpp, src/RenderDoc.h
Added the RenderDoc in-application API header. Implemented a RenderDoc singleton class that loads the RenderDoc DLL, manages API access, triggers captures, retrieves capture info, calculates disk usage, and clears captures. Exposes methods for availability, initialization, and frame capture control.
Menu and Overlay UI Updates
src/Menu.cpp
Replaced "Frame Annotations" with "Frame Debugging" in the advanced settings. Integrated RenderDoc status, controls, and warnings into the menu and overlay. Added capture trigger button, status messages, capture path display, disk usage info, and clear captures button. Overlay now displays frame debugging status and performance warnings when active.
State Management Refactor
src/State.cpp, src/State.h, src/FrameAnnotations.cpp
Renamed all usages of frameAnnotations to frameDebugging in state loading, saving, and runtime logic. Updated all checks and variables accordingly. No other logic changes.
RenderDoc Initialization in Plugin
src/XSEPlugin.cpp
Added RenderDoc initialization in the plugin load sequence. If frameDebugging is enabled, the RenderDoc DLL is loaded and initialized before D3D hooks.
Legal
package/SKSE/Plugins/Renderdoc/LICENSE.md
Added the MIT License for RenderDoc, including copyright and permissions.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant MenuUI
    participant RenderDoc
    participant State
    participant Overlay
    participant Plugin

    User->>MenuUI: Open Advanced Settings
    MenuUI->>State: Get frameDebugging flag
    MenuUI->>RenderDoc: Query availability/status
    MenuUI->>User: Show frame debugging controls and status

    User->>MenuUI: Click "Create Capture"
    MenuUI->>RenderDoc: TriggerCapture()
    RenderDoc->>RenderDoc: Perform capture if available

    Plugin->>State: Load()
    State->>Plugin: Return frameDebugging flag
    Plugin->>RenderDoc: Initialize() if frameDebugging enabled

    Overlay->>RenderDoc: Query status/disk usage
    Overlay->>User: Show warnings, capture info, and performance impact
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested reviewers

  • doodlum
  • alandtse

Poem

A bunny with a debugger's dream,
Now hops through frames with RenderDoc's gleam.
Click "Create Capture"—watch the graphics freeze,
Disk space tallies, overlays tease.
With warnings and widgets, the UI's anew,
Frame debugging is here—thanks to this review!
🐇✨


📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 527cfdc and 1a9a6ce.

📒 Files selected for processing (1)
  • src/RenderDoc.cpp (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/RenderDoc.cpp
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch renderdoc

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Automated formatting by clang-format, prettier, and other hooks.
See https://pre-commit.ci for details.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
src/XSEPlugin.cpp (1)

177-180: Consider adding error handling for RenderDoc initialization.

While the initialization is correctly placed and gated by the frameDebugging flag, consider handling potential initialization failures to prevent crashes if the RenderDoc DLL is missing or incompatible.

 // Initialize RenderDoc DLL for debugging, which automatically hooks the game
 if (state->frameDebugging)
-	RenderDoc::GetSingleton()->Initialize();
+	if (!RenderDoc::GetSingleton()->Initialize()) {
+		logger::warn("Failed to initialize RenderDoc - frame debugging will be disabled");
+		state->frameDebugging = false;
+	}
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d633f12 and f79ccfe.

⛔ Files ignored due to path filters (1)
  • package/SKSE/Plugins/Renderdoc/renderdoc.dll is excluded by !**/*.dll
📒 Files selected for processing (9)
  • include/Renderdoc/renderdoc_app.h (1 hunks)
  • package/SKSE/Plugins/Renderdoc/LICENSE.md (1 hunks)
  • src/FrameAnnotations.cpp (8 hunks)
  • src/Menu.cpp (6 hunks)
  • src/RenderDoc.cpp (1 hunks)
  • src/RenderDoc.h (1 hunks)
  • src/State.cpp (3 hunks)
  • src/State.h (1 hunks)
  • src/XSEPlugin.cpp (2 hunks)
🧰 Additional context used
🧠 Learnings (6)
📓 Common learnings
Learnt from: alandtse
PR: doodlum/skyrim-community-shaders#0
File: :0-0
Timestamp: 2025-06-24T07:17:36.604Z
Learning: When reviewing PRs, always clarify the scope if there are multiple related features or dependencies. WeatherPicker was a separate PR that was already merged, while this PR focuses specifically on WetnessEffects climate preset system enhancements.
Learnt from: alandtse
PR: doodlum/skyrim-community-shaders#0
File: :0-0
Timestamp: 2025-07-05T05:20:45.823Z
Learning: In the skyrim-community-shaders repository, file deletion error handling improvements that replace existence checks and try-catch blocks with std::filesystem::remove error-code-based approaches are considered bug fixes rather than refactoring, as they address inadequate error handling and misleading log messages.
package/SKSE/Plugins/Renderdoc/LICENSE.md (1)

Learnt from: alandtse
PR: doodlum/skyrim-community-shaders#0
File: :0-0
Timestamp: 2025-07-05T05:20:45.823Z
Learning: In the skyrim-community-shaders repository, file deletion error handling improvements that replace existence checks and try-catch blocks with std::filesystem::remove error-code-based approaches are considered bug fixes rather than refactoring, as they address inadequate error handling and misleading log messages.

src/State.cpp (1)

Learnt from: davo0411
PR: #1070
File: src/State.cpp:79-83
Timestamp: 2025-05-30T11:44:15.542Z
Learning: In the Skyrim Community Shaders project, the smoothDrawCalls array in the State class is declared as type double in src/State.h, which is the correct floating-point type for performing exponential moving average smoothing calculations.

src/XSEPlugin.cpp (1)

Learnt from: alandtse
PR: doodlum/skyrim-community-shaders#0
File: :0-0
Timestamp: 2025-07-01T18:01:07.079Z
Learning: In the skyrim-community-shaders project, simple scalar constants in HLSL shaders use #define (e.g., #define NTHREADS 128), while more complex constants use static const within namespaces (e.g., Math namespace in Math.hlsli). For epsilon standardization, #define is the appropriate choice since epsilon values are simple scalar constants.

src/FrameAnnotations.cpp (5)

Learnt from: alandtse
PR: #577
File: src/Features/WetnessEffects.h:36-36
Timestamp: 2025-06-08T11:25:14.536Z
Learning: In the skyrim-community-shaders project, boolean flags in C++ structs that interface with HLSL shaders use uint type instead of bool for compatibility reasons. This ensures consistent size, alignment, and cross-platform compatibility when passing data to shader constant buffers.

Learnt from: alandtse
PR: #577
File: src/Features/WetnessEffects.h:36-36
Timestamp: 2025-06-08T11:25:14.536Z
Learning: In the skyrim-community-shaders project, boolean flags in C++ structs that interface with HLSL shaders use uint type instead of bool for compatibility reasons. This ensures consistent 4-byte size, proper 16-byte alignment in constant buffers, and cross-platform compatibility when passing data between C++ and HLSL shaders.

Learnt from: alandtse
PR: doodlum/skyrim-community-shaders#0
File: :0-0
Timestamp: 2025-07-05T05:20:45.823Z
Learning: In the skyrim-community-shaders repository, file deletion error handling improvements that replace existence checks and try-catch blocks with std::filesystem::remove error-code-based approaches are considered bug fixes rather than refactoring, as they address inadequate error handling and misleading log messages.

Learnt from: alandtse
PR: doodlum/skyrim-community-shaders#0
File: :0-0
Timestamp: 2025-07-01T18:01:07.079Z
Learning: In the skyrim-community-shaders project, simple scalar constants in HLSL shaders use #define (e.g., #define NTHREADS 128), while more complex constants use static const within namespaces (e.g., Math namespace in Math.hlsli). For epsilon standardization, #define is the appropriate choice since epsilon values are simple scalar constants.

Learnt from: davo0411
PR: #1070
File: src/State.cpp:79-83
Timestamp: 2025-05-30T11:44:15.542Z
Learning: In the Skyrim Community Shaders project, the smoothDrawCalls array in the State class is declared as type double in src/State.h, which is the correct floating-point type for performing exponential moving average smoothing calculations.

src/Menu.cpp (1)

Learnt from: alandtse
PR: doodlum/skyrim-community-shaders#0
File: :0-0
Timestamp: 2025-07-05T05:20:45.823Z
Learning: In the skyrim-community-shaders repository, file deletion error handling improvements that replace existence checks and try-catch blocks with std::filesystem::remove error-code-based approaches are considered bug fixes rather than refactoring, as they address inadequate error handling and misleading log messages.

🔇 Additional comments (9)
include/Renderdoc/renderdoc_app.h (1)

1-742: LGTM! Standard RenderDoc API header.

This is the official RenderDoc in-application API header (v1.6.0) which is appropriate for integrating frame capture functionality.

src/State.h (1)

140-140: LGTM! Appropriate rename for RenderDoc integration.

The rename from frameAnnotations to frameDebugging better reflects the expanded functionality with RenderDoc integration.

package/SKSE/Plugins/Renderdoc/LICENSE.md (1)

1-26: LGTM! Proper licensing for RenderDoc integration.

The MIT license file appropriately covers the RenderDoc components with current copyright information.

src/FrameAnnotations.cpp (1)

14-14: LGTM! Consistent flag renaming throughout.

All occurrences of frameAnnotations have been correctly replaced with frameDebugging to match the State.h rename.

Also applies to: 33-33, 75-75, 153-153, 265-265, 284-284, 313-313, 927-927

src/XSEPlugin.cpp (1)

7-7: LGTM! Header include for RenderDoc integration.

src/State.cpp (1)

109-109: LGTM!

The renaming from frameAnnotations to frameDebugging is consistently applied across all usages in this file.

Also applies to: 239-239, 372-372

src/RenderDoc.h (1)

1-1: Add header guards or pragma once

The header file is missing include guards to prevent multiple inclusions.

Add this at the beginning of the file:

+#pragma once
+
 #include <d3d11.h>
⛔ Skipped due to learnings
Learnt from: soda3000
PR: doodlum/skyrim-community-shaders#1123
File: src/Menu.cpp:1407-1408
Timestamp: 2025-06-09T22:27:55.011Z
Learning: When checking for missing constants or definitions in C++ code, always verify both the source file (.cpp) and the corresponding header file (.h) before flagging missing definitions, as constants and static members are typically declared in header files.
Learnt from: alandtse
PR: doodlum/skyrim-community-shaders#0
File: :0-0
Timestamp: 2025-07-01T18:01:07.079Z
Learning: In the skyrim-community-shaders project, simple scalar constants in HLSL shaders use #define (e.g., #define NTHREADS 128), while more complex constants use static const within namespaces (e.g., Math namespace in Math.hlsli). For epsilon standardization, #define is the appropriate choice since epsilon values are simple scalar constants.
src/Menu.cpp (2)

1510-1557: Well-implemented frame debugging UI

The UI properly handles all RenderDoc states with appropriate user feedback and controls.


1723-1809: Effective performance impact warnings

The overlay warnings ensure users are aware of RenderDoc's severe performance impact in all UI states.

Comment thread src/RenderDoc.cpp Outdated
Comment thread src/RenderDoc.cpp Outdated
Comment thread src/RenderDoc.cpp
Comment thread src/RenderDoc.h
doodlum and others added 2 commits July 29, 2025 02:06
Automated formatting by clang-format, prettier, and other hooks.
See https://pre-commit.ci for details.
@alandtse
Copy link
Copy Markdown
Collaborator

Instead of jamming this into menu, which I'm in the middle of refactoring since it's unmaintainable, why not make this a core feature?

@doodlum
Copy link
Copy Markdown
Collaborator Author

doodlum commented Jul 29, 2025

Instead of jamming this into menu, which I'm in the middle of refactoring since it's unmaintainable, why not make this a core feature?

Hrmm, we can but I was actually modifying the original frame annotations setting that was placed here, this is just an extension of that.

@alandtse
Copy link
Copy Markdown
Collaborator

Just realized the CI was broken at the time. Can you pull from dev (merge is ok) and see if the ci has any issues?

@Pentalimbed
Copy link
Copy Markdown
Collaborator

Please resolve conflicts

@alandtse alandtse changed the title feat: user-facing renderdoc implementation feat: add renderdoc capture Aug 23, 2025
Copilot AI added a commit that referenced this pull request Oct 10, 2025
- Add RenderDoc wrapper class (RenderDoc.h/cpp) and renderdoc_app.h API header
- Rename frameAnnotations to frameDebugging throughout codebase
- Add RenderDoc UI integration to AdvancedSettingsRenderer
- Add RenderDoc warnings to overlay renderer
- Initialize RenderDoc in XSEPlugin when frame debugging is enabled
- Include RenderDoc binary (renderdoc.dll) and LICENSE in package
- RenderDoc captures saved to Data/SKSE/Plugins/CommunityShaders/Captures/

Co-authored-by: alandtse <7086117+alandtse@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants