List additions and improvements

[AIrjen]

Missing something from a list? Found a typo? Got something amazing that needs to get added?
Or even got a good suggestion for a template?

post them here, and I’ll look into adding it into the main build.

[VirusShell]

[#44] fun fact: I got lazy and the lists are deduplicated in the code 😅

Oh, ok! As I went thru things I figured the dupes happened as you went along and added to them. Especially in the longer lists where many of the tokens are sorta grouped together since they share some common theme. It's not many, but I guess I don't have to worry about removing those then.

I don't believe that gets run on the custom userfiles, correct? Had a few instances where I had the same LoRA get applied twice because I missed that I had it in the list twice. Tho if it does then I guess it was just random chance that it happened. Also, I did want to ask how random the actual selection is from those files. I swear I see some tokens/tags get used more frequently than others. If I randomize the list order then eventually I see a different group of things being more frequent. It's hard to pin it down, and maybe it's more empirical than anything

===
Overall, there are very few spelling errors throughout the csv's. Tho I saw a few in the descriptions that get shown in the webui (and some in the code comments but those don't really matter), but that's all pretty easy to get through. I'll see about making a PR for the spelling errors I've come across in places, as that might be the easier way to do it (I've sorted most of mine lexicographically, made everything lowercase, and trimmed things out that I didn't want, so I'll just make a new clone and start from scratch.). Never done a PR before, but let me know what works for you and I'll see what I can do when I have some spare time!

As for any additions/changes to the lists (or other files), I can add them to the PR as well, or make a separate one. But I'd like to get your input on a couple of things. This is your project after all, and it does an excellent job covering so many different ranges of things and stuff that can go into prompts. So I don't want to make changes based on my personal tastes. (That's what the userfiles are for. And again, thank you for having that as an option for people!)

I only just started going thru the templates.csv mostly to see what all is in there. It served as a good reminder for why I stopped doing what I would frequently see others doing for their prompts! One thing I quickly noticed is the amount of fluff and vomit that can get added that have little to no actual effect beyond being a placebo. Using a phrase or word like "midjourney" will have some effect, but that specific token isn't going to suddenly generate an image with midjourney-like qualities, you know? So is it ok to trim things along those lines out?

In other places, some tokens have a higher chance to affect things in unintended ways. Most of these are fine and work well enough, but having "Honey" as a haircolor choice tends to lead to a sticky mess lol. For things like that, should I add comments at those spots to let you decide, or would it be ok to either change them to something else or remove them if needed?

===
For LoRA/LyCo/LoCon/etc, those can be added to prompts without needing to have commas between them. The webui parses the prompt and pulls out those and other additional characters (like token weighting or other prompt editing features) so they don't negatively impact the image generation.

With the prompt generation from the script it adds commas between each, meaning the prompt then has those comma tokens left over at the end. In my own testing with prompts generated from the script, their impact is so minimal that it's not really an issue since they are at the end anyways. And since styles and TIs can be interspersed with LoRAs at the end of the complete prompt, cleaning it up efficiently would mean needing to ensure that LoRA/LyCO/etc are always moved to the end.

For now, a quick and dirty fix is to just have the cleanup steps in build_dynamic_prompt.py handle it:

completeprompt = re.sub('>, <', '> <', completeprompt)
completeprompt = re.sub(', <', ' <', completeprompt)

===
I don't know how many hours I've spent using Stable Diffusion, but I've been tinkering with it literally every day since late last September. I like using this script because it can rapidly create intriguing images that often show different approaches to making certain characteristics appear. And, in the scope of what this script aims to accomplish, I'd say it does a great job of showcasing generating prompts to people with little to no experience doing so!

As everyone knows or eventually learns, there are many factors that can significantly alter the resultant image. Changing just the order of phrases or positions of words can change things in many different ways. Given the right combination of things, many models fine-tuned for an anime or cartoonish aesthetic can still output a photoreal image. But that's a different, seemingly endless, rabbit hole for another time.

I've been wondering about the script's approach to how things within the completed prompt get ordered. From what I can see, it kinda puts things out in blocks: Image Type(s) -> Subject -> Additional Descriptors -> Vomit -> Lora/TI/Styles. (I don't personally use artists as I like seeing what the AI will do on its own, so thank you so much for having options for that!! I believe artists either get added near the front or towards the end)

Side note: word vomit

I think there's some variance to this, and correct me if I'm wrong, but I was wondering how much variance there is and if there's a way to randomize pieces more? Granted, the way it works now is actually probably the better way to handle it to account for all the different models that are out there. So maybe having an optional and somewhat randomized token/phrase weighting component in the script would be interesting see.?

===
This one is more of a quirk that I found interesting, but if you have any styles selected in the dropdown box (before loading them into the prompt), they end up being added to the end of the generated prompts from the script. Not really a bug as it's technically working as intended from Automatic1111.

Well, I ended up writing out way more than I intended lol! There's a few other things, but there's probably more than enough for now as is.

[AIrjen]

Where to begin! Thanks for the long read.

Had a few instances where I had the same LoRA get applied twice because I missed that I had it in the list twice.

Yes, this can happen. During choosing from a list, it currently doesn't erase that record from the list. Meaning the same thing can be chosen twice. I've been debating with myself wether or not to implement this. It's fairly easy (there is a python function for this), but complication comes from really small lists and how to handle if the list is already empty. Maybe its only needed for really specific lists
such as the TI and LoRA lists.

I swear I see some tokens/tags get used more frequently than others
Yup, but it's not. See: https://en.wikipedia.org/wiki/Apophenia

However, computers can't really do true randomness: https://docs.python.org/3/library/random.html

But you can create one from lava lamps: https://en.wikipedia.org/wiki/Lavarand

Never done a PR before, but let me know what works for you and I'll see what I can do when I have some spare time!
Here is some help (there are other sources out there)
https://jarv.is/notes/how-to-pull-request-fork-github/

But the main advice I can give you is the following:

• First, back up your own changes somewhere.
• Two, just try things to create the PR. You can't break anything here, without me approving it.
• Three, if its too complicated, just attach the CSV files here.

So I don't want to make changes based on my personal tastes.
Most things should be fine to include. I do try to keep things decent.

This is open source, so anyone can have their input.

cleaning it up efficiently would mean needing to ensure that LoRA/LyCO/etc are always moved to the end.

Will be put on the todo list, to make the <> parts moved to the end of the prompt. Should be something smart that can be done here!

From what I can see, it kinda puts things out in blocks: Image Type(s) -> Subject -> Additional Descriptors -> Vomit -> Lora/TI/Styles

Yup, but I am thinking on how to approach more randomness here. First step is moving everything to wildcards (which is almost complete) and then see if there are options for reordering.

Another idea I have in this space is partial templates, kind of like lego bricks of prompt parts binding together.

Currently, I don't have a concrete idea yet, just some general idea. I just have to wait until inspiration hits.

word vomit
There is a reason I started this list as vomit at the start. I first wanted to call this extension "prompt vomit" :D

And yes, random weightings has occured to me as well. Vomit and colorschemes are good examples of prompt parts that could use this.

styles

Yes, styles being applied is something natural from WebUI, nothing I can do about that.

I hope that answers some of your questions!

[AIrjen]

Just an update here as well:

• The movement of the LoRA to the end of prompt has been added in somewhere last week.
• New prompt generations methods have been added! These will help with getting more random prompts. Full documentation here: https://github.com/AIrjen/OneButtonPrompt/blob/main/user_guides/prompt_generation_modes.md

[VirusShell]

Excellent to hear, I'll give it a look. Currently going thru the lists (finally have the time!)

And thanks for the earlier response. Meant to reply sooner.

...complication comes from really small lists and how to handle if the list is already empty. Maybe its only needed for really specific lists
such as the TI and LoRA lists.

That would only be if someone has added to that list. Tho I keep adding things to my own just to mix things up more and more now. Probably small to mid-size lists tho, as the chance of a dupe selection lowers as the list gets longer

The movement of the LoRA to the end of prompt has been added in somewhere last week.
New prompt generations methods have been added! These will help with getting more random prompts.

That was fast! Thanks!!! Time to see how I can break that lol. I'll look over the docs to see how it works. Looks like you've been busy with it!

random weightings has occurred to me as well

Hoping there'd be a catch for items on a list that are already weighted. There are some/many tokens/TI/LoRA/etc that work better for general use at certain weights. Tho it'd also be interesting to be able to put a range of valid weights in the csv

[AIrjen]

Hi VirusShell, the deduplication thing is in and working. Should work for almost everything (it should at least).

[VirusShell]

I'll give it a look! Thanks for letting me know

[SirMathelot]

I noticed that there's very many modes for generating styles and artists but not much for other types of prompts like settings (e.g. outside in a picturesque Greek village) or subject actions (e.g. running, smiling). I wonder if you could make modes like these or add it in some other way. I'm using humanoid subject type btw. Thanks!

[VirusShell]

Mood stuff is pretty easy to get, so long as it's not too obscure and that the model (checkpoint) understands the concept well enough. Same for simple or basic actions, and just having the right words gets things close enough. However things like "where the concept is placed in the prompt", "the amount and/or type of other descriptors", "the weight and strength variance of other tokens", or the inherent context a token was trained on can often alter things regardless of where they're placed in the prompt.

For example, let's say I've written a prompt for a scene that I want to be set at night but instead it keeps coming out in daytime. Something in the prompt —whether alone or by combination— is more strongly associated with daytime, making it difficult for the model to match what I intended and I don't want to add a bunch of other fluff that might heavily affect things. In this case, usually I can put something along the lines of "nighttime in -location- at night" and that does the job. For other cases it's a matter of simplifying, rewording/restructuring, or adding weights to tokens. And if those fail, LoRA's, TI's, and Hypernets come in handy to make up for the difference.

With this extension, adjusting the Complexity and Randomness slider shows the correlation of prompt intentions vs generated result accuracy pretty well. (Or in other words, expectations vs reality.) In general, lower values more closely match prompt intentions, meaning a higher chance of prompt accuracy. Conversely, higher values tend to make a lot of tokens seem superfluous, meaning lower result accuracy. (mine's usually set to 10!)

So, in order for it to show more specific things like that accurately, I'd imagine some of the internals would have be restructured. Like splitting up some of the lists into more specific concepts? Or have lists similar to the artists_and_category.csv or templates.csv and construct phrases along those lines?

I dunno, just some quick idle/random thoughts before sleep lol.

[AIrjen]

I am unsure where you are going with this?

Example you mention:
Something in the prompt —whether alone or by combination— is more strongly associated with daytime, making it difficult for the model to match what I intended and I don't want to add a bunch of other fluff that might heavily affect things.

So generating someone with sunglasses, in a sundress at a beach with a palm tree has indeed a high tendency so be sunny, even if we do add "nighttime" to the prompt. This is more on how the neural net actually works than this extension.
Or is the expectation that we somehow know this, and then add more weight to the "nighttime" part of the prompt? That seems a bit much. It's just a text generator.

Anyway, maybe this all needs to lands a but first, to see how some ideas can be transferred.

[VirusShell]

lol sorry. I was really tired when writing that, so I can understand the confusion. I was trying to use it as an example that with the way the extension works, things would have to generated in a more focused way in order to have the intended effect. To make it a little easier for the user to accomplish something more specific than that, what about adding a text entry for the middle of a prompt?

Altho as it is now, using the workshop mode kind of already allows that to happen. Random Prompt(s) get generated -> Select one to move to prompting area -> Edit if desired.

The last part of what I was saying was where I was trying to go with an idea. It would allow for some form of categorization of certain concepts/actions with a loose template structure.

[AIrjen]

There is already logic that can insert things in the middle of the prompt. It now goes via a custom csv file. But yeah, adding another field in between prompt prefix and prompt suffix is doable. I had another suggestion that was about giving a list of random words that have a chance to be inserted into the prompt. Which is sort of a similar idea. I am unsure of adding even more fields. I already see people getting confused (one button prompt but there is so many fields!). But that is also on me, the UI is horrible (because gradio is horrible!).

I am also for adding just another cool addition list, for adding things in the middle of the prompt.

For categorization, there is one high on the todolist right now, which is male/female. About 99% of output of One Button Prompt I see online is females. Maybe it is time to set my intentions aside and add this in.

[VirusShell]

HA!

AIrjen;idiot

[Shimanogov]

That wood be sweet to have settings to increase the strength of the generated object, i.e. it should always appear as ((object)). Sometimes, added modifiers completely remove the chosen object from the image.

[AIrjen]

Hi! This is a good suggestion. While not opposed to a new setting, I think I can do something automatically based on the lenght of the prompt.
However, I can place the brackets at different places. Some examples:

_masterpiece, soft focus of a Chic Delightful portly British ((Bosmer)), Geisha, Adventure pose, Samurai Mask, Smartwatch, Braces, at The Andes Mountains, deep focus, Detailed illustration, Accent lighting, UHD, [ (art by David Aronson:1.2) , (art by Hashimoto Gahō:1.2) ::4]

masterpiece, soft focus of a ((Chic Delightful portly British Bosmer, Geisha)), Adventure pose, Samurai Mask, Smartwatch, Braces, at The Andes Mountains, deep focus, Detailed illustration, Accent lighting, UHD, [ (art by David Aronson:1.2) , (art by Hashimoto Gahō:1.2) ::4]

masterpiece, soft focus of a ((Chic Delightful portly British Bosmer, Geisha, Adventure pose, Samurai Mask, Smartwatch, Braces)), at The Andes Mountains, deep focus, Detailed illustration, Accent lighting, UHD, [ (art by David Aronson:1.2) , (art by Hashimoto Gahō:1.2) ::4]_

So, what would you pick? I can then look at some way of implementation

[Shimanogov]

I was thinking about the first variant to increase object weight against all modifiers.

[AIrjen]

Hi, I was busy converting this to ComfyUI. But I had some time now. So it's in. It now automatically adds more weight to the subject as the prompt gets longer, and longer. :)

[VirusShell]

Do you have anything to cap it if it's used with SDXL? I think I read somewhere that weight values work differently with SDXL 1.0, and it's better to lower weights in places than it is to raise them. But if they get raised, 1.2 is considered pretty high now.

[AIrjen]

It is capped at 1.3, it needs a pretty large prompt to actually get there. I think it needs more as 450 characters in the prompt to do this. For SD XL, smaller prompts seem to do just fine.
so unless it turn the insanitylevel up to 9 or 10, you wouldn’t even see this feature.

I can always finetune it some more.

[SirMathelot]

I know there's an option to generate a random aspect ratio from three options - portrait, square, or landscape but it'd be cool to make it have more customization options, for example a random aspect ratio between 1.5:1 and 1.7:1. Also it'd be cool if there was an option to randomize the CFG value, also from some specified range. Also also it'd be even cooler to randomize the checkpoint. Basically to have options to randomize things you can find in the XYZ plot.

[AIrjen]

Hi! Thank you for the feedback. For aspect ratio’s there should be an option for “wild” which does randomisation tricks with the aspect ratio.
Random checkpoints is in as well, just choose “all” as your model/checkpoint.

CFG randomisation is currently not in.

[SirMathelot]

When I'm using One Button Run and Upscale the ADetailer extention isn't working even though it's enabled. This doesn't happen if I generate images using the normal Generate button.

[AIrjen]

Hi! Yes, that doesn’t work with One button run and upscale, only when using One Button Prompt during normal generation.

when using the run and upscale method it runs outside of the UI. So it doesn’t take into account other extensions. Only support for ultimate sd upscale, controlnet tile resample and image qualification have been built in.

[darkninjavn]

If it can add to UI like controlnet instead of using script slot. It will be great.

[AIrjen]

Yes! This is not the first time this has been requested. If I move over to a 2.0 version it will be changed to an extension level instead of a script.