Skip to content

Input Validation, Tensorboard Tweaks + Drag-n-drop support - #908

Closed
O-J1 wants to merge 76 commits into
Nerogar:masterfrom
O-J1:tensorboard-on-tweaks
Closed

Input Validation, Tensorboard Tweaks + Drag-n-drop support#908
O-J1 wants to merge 76 commits into
Nerogar:masterfrom
O-J1:tensorboard-on-tweaks

Conversation

@O-J1

@O-J1 O-J1 commented Jul 4, 2025

Copy link
Copy Markdown
Collaborator

High level summary of what this PR does:

  1. Drag-n-drop support for all path_entry widgets
  2. Drag-n-drop support for the concepts tab. Wont duplicate
  3. Merges file_entry and dir_entry into a single function
  4. Adds more extensive input validation, including to path_entry and especially model output destination
  5. Adds debouncing to all fields derived from entry including time_entry which will partially prevent the division by 0 backup bug, combined with prior fix this will make it impossible
  6. Fixes the tensorboard workspace spam when typing a new one and always on
  7. Streamlines Tensorboard management down to a single input and makes it work with cloud in Always On.

Initial checklist

  • If Always-on is True, shouldnt be terminated in training nor after
  • filter new tensorboard warnings that we have no control over and google has no current plans to solve: Deprecated pkg_resources package tensorflow/tensorboard#7003
  • Debounce the workspace dir field, so we dont get a new folder created for every character we type ([Bug]: Always-On Tensorboard create unwanted workspace directories #1028)
  • dnd files into a file_entry results in file path
  • dnd files or folders into dir_entry results in a directory path
  • parent folders are validated to exist
  • fix merge conflict with master
  • Check if Always-On tensorboard doesnt break cloud training
  • Check if Off and Train Only mode don't break cloud training nor start Tensorboard locally during training
  • Cloud training doesnt cause invalid path warnings or such

Validation specific checklist

  • User enters without extenstion, gets auto-added
  • User sets a prefix, is used for autocomplete
  • User can choose to use friendlywords instead of datetime
  • Tensorboard will use model output name.
  • Overwrite prevention works and is toggeable

Tensorboard has now changed from enable + toggle to just a dropdown

image

Drag and drop demo

drag-n-drop-pr.mp4

@BitcrushedHeart

This comment was marked as outdated.

@dxqb

This comment was marked as outdated.

@dxqb

This comment was marked as outdated.

@dxqb
dxqb self-requested a review August 20, 2025 11:10
@dxqb dxqb self-assigned this Aug 20, 2025
@O-J1
O-J1 marked this pull request as ready for review August 29, 2025 07:37
@O-J1

This comment was marked as outdated.

@O-J1

This comment was marked as outdated.

Comment thread modules/util/config/TrainConfig.py
@O-J1

O-J1 commented Jan 2, 2026

Copy link
Copy Markdown
Collaborator Author

@dxqb

  • is this now used for everything? Saves, tensorboard, samples, ...?
  • Tensorboard
  • Configs saved to workspace folder
  • sample filenames
  • saves
  • backups

Input validation - what is detected, what is rejected?

Every field has basic validation on its data type, reverting if an invalid input is provided (with a debounced delay). The only field with anything really fancy is Output model destination. That has a whole bunch of logic around ensuring the entered output matches the output format and if it will overwrite. This would be a lot easier if Nero let us split the fields.

Do you think we still need the automatic revert then? That always seemed unnatural to me (no other UI does that) but it was better than silently ignoring invalid values.

I strongly believe we still need, it will be toggeable though!. If we had more gatekeepers to getting started with OT or it had remained power user only software then yes I would agree, however that is not our reality

Comment thread modules/ui/ConfigList.py Outdated
@O-J1

O-J1 commented Feb 4, 2026

Copy link
Copy Markdown
Collaborator Author

I have now disabled DnD on linux after extensive conversations with the maintainer of TKInterdnd2 and another dev, who noted DnD does not even work with Qt on all linux distros (the gold standard).

Didnt want to do this but, no choice :(, tested again on Linux and Windows.

@O-J1

O-J1 commented Feb 4, 2026

Copy link
Copy Markdown
Collaborator Author

After discussing further with Dxq, value reversion has been removed. In its place I have added

  • Undo/Redo: Ctrl + Z/Shift+Ctrl + Z or Crtl + Y to CTKEntry fields.
  • Alert dialog pops up if you try to start training with invalid values (havent changed logic around whats invalid)
  • Added logic to prevent edge case prefix/suffix recursion.

I think this is ready for further reviews again.

@dxqb dxqb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • it still reverts, for example when I enter a string into Learning Rate. Is that intended? [see Discord - as long as it's a strict improvement it isn't a stopper, even though I prefer just the red marking with no reverting]
  • minor: "prevent overwrite" is enabled. I select a preset, "Model Output Destination" goes to default "model/model.safetensors". That file exists, so "Prevent Overwrite" is adding "_1" to the file. But the orange marking stays, even though there isn't a problem anymore
  • it complains that "File does not exist" when Base Model is a huggingface path like lodestones/Chroma1-HD. Not initially when selected a preset, but after I manually entered it
  • the "File does not exist" tooltip when it appears and then I switch to another tab like training, it blinks out of nowhere into that tab even though the error is on the model tab
  • I'm a slightly confused by all the functionality and switches affecting the overwrite-files behaviour (Auto-correct input, Prevent Overwrite, Use Friendly Names). It basically only affects the Model Output Destination, right? The only other output path_entrys I could find are directories (workspace, debug, cache), where this functionality mostly doesn't apply.
  • The "run name" is used in backups, for example Creating Backup workspace/run/backup/2026-02-06_13-17-53-**model_1**-backup-20-0-20. I think using any type of prefix/suffix in backups breaks the backup discovery when you enable "Continue from last backup". At least, I remember this to be the reason why we haven't used the "Save Prefix" for backups previously.
  • the "run name" is consistently used for saves, tensorboard and samples. However, the "Save Filename Prefix" is ignored in most cases. The "run name" is the output file name. I think this breaks how the prefix is used currently by many users, including myself: You do 3 differents runs with different parameters, and use the save prefix so you can later identify the runs in tensorboard, samples and saves. For example, you might use "LR1e-3" as a prefix. Now you only get "model_1", "model_2", and "model_3" instead, unless you anticipate that and modify the output filename accordingly.
  • review: test functionality that might affect it with cloud training

Comment thread modules/ui/ConceptTab.py
is_full_width=False,
show_toggle_button=True
show_toggle_button=True,
placeholder_text="You have no concepts defined. Either click the 'Add Concept' button or drag and drop a directory here to add a new concept."

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a check for DnD being available, so we don't tell the user to do something that doesn't work

data.append(("validation_auto_correct", True, bool, False))
data.append(("validation_show_tooltips", True, bool, False))
data.append(("use_friendly_names", False, bool, False))
data.append(("prevent_overwrite", True, bool, False))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be the default?
I don't like it personally, because it'll accumulate output files even though I never use them. I only use saves.


# validation settings
data.append(("validation_auto_correct", True, bool, False))
data.append(("validation_show_tooltips", True, bool, False))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this a setting? Isn't this always good? Why would I want to disable this?

self.bucket_ax.cla()
aspects = [str(x) for x in list(aspect_buckets.keys())]
aspect_ratios = [self.decimal_to_aspect_ratio(x) for x in list(aspect_buckets.keys())]
aspect_ratios = [self.decimal_to_aspect_ratio(x) for x in aspect_buckets]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are these code changes to aspect statistics?
Can't find anything about this in the PR description

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for self note: It was because of testing around the matplotlib TkAgg issue with DnD and I noted we were doing it inefficently

name = path.stem

# fall back to prefix if no name could be extracted
if not name:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see comment in main. save prefix is ignored in most cases.

@@ -1,56 +1,170 @@
from __future__ import annotations

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not reviewing this file - please double check before merge

from typing import Any

from customtkinter import CTk, CTkToplevel
from tkinterdnd2 import DND_FILES

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

always import, even if Linux? See the other file for conditional import

@@ -0,0 +1,482 @@
"""Input validation utilities for UI components."""

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please reconsider if we need all this code for one entry field (see main comment)


if sys.version_info < deprecated_ver:
sys.stderr.write("Warning: Deprecated Python version found. Update to %s or newer\n" % (tuple_to_str(deprecated_ver)))
sys.stderr.write("Warning: Deprecated Python version found. Its strongly recommended to update to %s or newer\n" % (tuple_to_str(deprecated_ver)))

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just for my information: why?

@O-J1 O-J1 Feb 6, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because we have had 8+ users coming in going "OMGWTF how do I update to python 3.11?!" so instead of saying you must update, instead strongly recomend. Python 3.10 support will likely be dropped by the end of this year for PT (speculation)

Comment thread requirements-global.txt
safetensors==0.5.3
tensorboard==2.19.0
tensorboard==2.20.0
StrEnum==0.4.15; python_version < "3.11" # For tensoboardMode on 3.10

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo

@O-J1 O-J1 mentioned this pull request Feb 7, 2026
@O-J1

O-J1 commented Feb 7, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as spliting up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feat]: Enable dragging a file from Windows Explorer to a file input field [Feat]: Drag'n Drop into Concept window

4 participants