feat: simplified invocation context #5491

psychedelicious · 2024-01-13T13:32:42Z

What type of PR is this? (check all applicable)

Have you discussed this change with the InvokeAI team?

Yes
No, because:

Have you updated all relevant documentation?

Yes
No

Description

This PR iterates on the rough draft in #4910, taking into consideration feedback from @RyanJDick. It's written from scratch.

I'm sure there are a few methods that node authors will want that are not yet included - I imagine we'll need more boards-related methods. The new context is easily extensible.

The app is should be fully functional in this PR. I've converted everything.

See invocation_context.py for the meat and potatoes. It's well-commented.

There are a crapload of changes, of course - every invocation needed to be updated. And I had to shuffle files around to deal with circular imports.

Some of the feedback from the earlier PR was that it would be nice to redo the BaseInvocation API at the same time. We should have some discussion around that. Maybe we can get a revised invocation context (this PR or a different design if this doesn't work out) and revised BaseInvocation for v3.7.0.

Related Tickets & Documents

QA Instructions, Screenshots, Recordings

Note that I have not touched onnx.py. It's been broken for a while so I'm just ignoring it.

The app should run just fine.

Merge Plan

This PR needs thorough review, testing and pondering. If it is approved, I will handle the actual merging.

Added/updated tests?

Yes
No : please replace this line with details on why tests
have not been included

[optional] Are there any post deployment tasks we need to perform?

This will be a breaking change for node authors. We'll need to provide solid documentation.

psychedelicious · 2024-01-15T00:01:59Z

I've added the old InvocationContext attrs to the new version and added deprecation warnings.

While this makes it easier for custom nodes to migrate, the file structure changes needed to avoid circular references mean that custom nodes need to be revised anyways, so I'm not sure it makes sense to handle deprecation.

I've added an invocation_api module (thanks for this suggestion Ryan!) at invokeai.invocation_api and added this to the package. Custom nodes can import everything they may need from this file. There may be some classes I've missed in this file.

invokeai/app/invocations/onnx.py

invokeai/app/invocations/compel.py

invokeai/app/services/shared/invocation_context.py

… `WithMetadata`/`WithBoard`-provided values

- The default is to not delete on startup - feels safer. - The two services using this class _do_ delete on startup. - The class has "ephemeral" removed from its name. - Tests & app updated for this change.

…jectSerializerDisk Replace `delete_on_startup: bool` & associated logic with `ephemeral: bool` and `TemporaryDirectory`. The temp dir is created inside of `output_dir`. For example, if `output_dir` is `invokeai/outputs/tensors/`, then the temp dir might be `invokeai/outputs/tensors/tmpvj35ht7b/`. The temp dir is cleaned up when the service is stopped, or when it is GC'd if not properly stopped. In the event of a catastrophic crash where the temp files are not cleaned up, the user can delete the tempdir themselves. This situation may not occur in normal use, but if you kill the process, python cannot clean up the temp dir itself. This includes running the app in a debugger and killing the debugger process - something I do relatively often. Tests updated.

- LatentsOutput.build - NoiseOutput.build - Noise.width, Noise.height multiple_of

We have two different classes named `ModelInfo` which might need to be used by API consumers. We need to export both but have to deal with this naming collision. The `ModelInfo` I've renamed here is the one that is returned when a model is loaded. It's the object least likely to be used by API consumers.

This was named inaccurately.

Crawled through a few custom nodes to figure out what I had missed.

psychedelicious · 2024-02-19T10:59:48Z

Superseded by next branch

This class works the same way as `WithMetadata` - it simply adds a `board` field to the node. The context wrapper function is able to pull the board id from this. This allows image-outputting nodes to get a board field "for free", and have their outputs automatically saved to it. This is a breaking change for node authors who may have a field called `board`, because it makes `board` a reserved field name. I'll look into how to avoid this - maybe by naming this invoke-managed field `_board` to avoid collisions? Supporting changes: - `WithBoard` is added to all image-outputting nodes, giving them the ability to save to board. - Unused, duplicate `WithMetadata` and `WithWorkflow` classes are deleted from `baseinvocation.py`. The "real" versions are in `fields.py`. - Remove `LinearUIOutputInvocation`. Now that all nodes that output images also have a `board` field by default, this node is no longer necessary. See comment here for context: #5491 (comment) - Without `LinearUIOutputInvocation`, the `ImagesInferface.update` method is no longer needed, and removed. Note: This commit does not bump all node versions. I will ensure that is done correctly before merging the PR of which this commit is a part. Note: A followup commit will implement the frontend changes to support this change.

See this comment for context: #5491 (comment) - Remove this now-unnecessary node from all graphs - Update graphs' terminal image-outputting nodes' `is_intermediate` and `board` fields appropriately - Add util function to prepare the `board` field, tidy the utils - Update `socketInvocationComplete` listener to work correctly with this change I've manually tested all graph permutations that were changed (I think this is all...) to ensure images go to the gallery as expected: - ad-hoc upscaling - t2i w/ sd1.5 - t2i w/ sd1.5 & hrf - t2i w/ sdxl - t2i w/ sdxl + refiner - i2i w/ sd1.5 - i2i w/ sdxl - i2i w/ sdxl + refiner - canvas t2i w/ sd1.5 - canvas t2i w/ sdxl - canvas t2i w/ sdxl + refiner - canvas i2i w/ sd1.5 - canvas i2i w/ sdxl - canvas i2i w/ sdxl + refiner - canvas inpaint w/ sd1.5 - canvas inpaint w/ sdxl - canvas inpaint w/ sdxl + refiner - canvas outpaint w/ sd1.5 - canvas outpaint w/ sdxl - canvas outpaint w/ sdxl + refiner

This class works the same way as `WithMetadata` - it simply adds a `board` field to the node. The context wrapper function is able to pull the board id from this. This allows image-outputting nodes to get a board field "for free", and have their outputs automatically saved to it. This is a breaking change for node authors who may have a field called `board`, because it makes `board` a reserved field name. I'll look into how to avoid this - maybe by naming this invoke-managed field `_board` to avoid collisions? Supporting changes: - `WithBoard` is added to all image-outputting nodes, giving them the ability to save to board. - Unused, duplicate `WithMetadata` and `WithWorkflow` classes are deleted from `baseinvocation.py`. The "real" versions are in `fields.py`. - Remove `LinearUIOutputInvocation`. Now that all nodes that output images also have a `board` field by default, this node is no longer necessary. See comment here for context: #5491 (comment) - Without `LinearUIOutputInvocation`, the `ImagesInferface.update` method is no longer needed, and removed. Note: This commit does not bump all node versions. I will ensure that is done correctly before merging the PR of which this commit is a part. Note: A followup commit will implement the frontend changes to support this change.

See this comment for context: #5491 (comment) - Remove this now-unnecessary node from all graphs - Update graphs' terminal image-outputting nodes' `is_intermediate` and `board` fields appropriately - Add util function to prepare the `board` field, tidy the utils - Update `socketInvocationComplete` listener to work correctly with this change I've manually tested all graph permutations that were changed (I think this is all...) to ensure images go to the gallery as expected: - ad-hoc upscaling - t2i w/ sd1.5 - t2i w/ sd1.5 & hrf - t2i w/ sdxl - t2i w/ sdxl + refiner - i2i w/ sd1.5 - i2i w/ sdxl - i2i w/ sdxl + refiner - canvas t2i w/ sd1.5 - canvas t2i w/ sdxl - canvas t2i w/ sdxl + refiner - canvas i2i w/ sd1.5 - canvas i2i w/ sdxl - canvas i2i w/ sdxl + refiner - canvas inpaint w/ sd1.5 - canvas inpaint w/ sdxl - canvas inpaint w/ sdxl + refiner - canvas outpaint w/ sd1.5 - canvas outpaint w/ sdxl - canvas outpaint w/ sdxl + refiner

This class works the same way as `WithMetadata` - it simply adds a `board` field to the node. The context wrapper function is able to pull the board id from this. This allows image-outputting nodes to get a board field "for free", and have their outputs automatically saved to it. This is a breaking change for node authors who may have a field called `board`, because it makes `board` a reserved field name. I'll look into how to avoid this - maybe by naming this invoke-managed field `_board` to avoid collisions? Supporting changes: - `WithBoard` is added to all image-outputting nodes, giving them the ability to save to board. - Unused, duplicate `WithMetadata` and `WithWorkflow` classes are deleted from `baseinvocation.py`. The "real" versions are in `fields.py`. - Remove `LinearUIOutputInvocation`. Now that all nodes that output images also have a `board` field by default, this node is no longer necessary. See comment here for context: #5491 (comment) - Without `LinearUIOutputInvocation`, the `ImagesInferface.update` method is no longer needed, and removed. Note: This commit does not bump all node versions. I will ensure that is done correctly before merging the PR of which this commit is a part. Note: A followup commit will implement the frontend changes to support this change.

See this comment for context: #5491 (comment) - Remove this now-unnecessary node from all graphs - Update graphs' terminal image-outputting nodes' `is_intermediate` and `board` fields appropriately - Add util function to prepare the `board` field, tidy the utils - Update `socketInvocationComplete` listener to work correctly with this change I've manually tested all graph permutations that were changed (I think this is all...) to ensure images go to the gallery as expected: - ad-hoc upscaling - t2i w/ sd1.5 - t2i w/ sd1.5 & hrf - t2i w/ sdxl - t2i w/ sdxl + refiner - i2i w/ sd1.5 - i2i w/ sdxl - i2i w/ sdxl + refiner - canvas t2i w/ sd1.5 - canvas t2i w/ sdxl - canvas t2i w/ sdxl + refiner - canvas i2i w/ sd1.5 - canvas i2i w/ sdxl - canvas i2i w/ sdxl + refiner - canvas inpaint w/ sd1.5 - canvas inpaint w/ sdxl - canvas inpaint w/ sdxl + refiner - canvas outpaint w/ sd1.5 - canvas outpaint w/ sdxl - canvas outpaint w/ sdxl + refiner

This class works the same way as `WithMetadata` - it simply adds a `board` field to the node. The context wrapper function is able to pull the board id from this. This allows image-outputting nodes to get a board field "for free", and have their outputs automatically saved to it. This is a breaking change for node authors who may have a field called `board`, because it makes `board` a reserved field name. I'll look into how to avoid this - maybe by naming this invoke-managed field `_board` to avoid collisions? Supporting changes: - `WithBoard` is added to all image-outputting nodes, giving them the ability to save to board. - Unused, duplicate `WithMetadata` and `WithWorkflow` classes are deleted from `baseinvocation.py`. The "real" versions are in `fields.py`. - Remove `LinearUIOutputInvocation`. Now that all nodes that output images also have a `board` field by default, this node is no longer necessary. See comment here for context: #5491 (comment) - Without `LinearUIOutputInvocation`, the `ImagesInferface.update` method is no longer needed, and removed. Note: This commit does not bump all node versions. I will ensure that is done correctly before merging the PR of which this commit is a part. Note: A followup commit will implement the frontend changes to support this change.

See this comment for context: #5491 (comment) - Remove this now-unnecessary node from all graphs - Update graphs' terminal image-outputting nodes' `is_intermediate` and `board` fields appropriately - Add util function to prepare the `board` field, tidy the utils - Update `socketInvocationComplete` listener to work correctly with this change I've manually tested all graph permutations that were changed (I think this is all...) to ensure images go to the gallery as expected: - ad-hoc upscaling - t2i w/ sd1.5 - t2i w/ sd1.5 & hrf - t2i w/ sdxl - t2i w/ sdxl + refiner - i2i w/ sd1.5 - i2i w/ sdxl - i2i w/ sdxl + refiner - canvas t2i w/ sd1.5 - canvas t2i w/ sdxl - canvas t2i w/ sdxl + refiner - canvas i2i w/ sd1.5 - canvas i2i w/ sdxl - canvas i2i w/ sdxl + refiner - canvas inpaint w/ sd1.5 - canvas inpaint w/ sdxl - canvas inpaint w/ sdxl + refiner - canvas outpaint w/ sd1.5 - canvas outpaint w/ sdxl - canvas outpaint w/ sdxl + refiner

psychedelicious requested review from Kyle0654, Millu, blessedcoolant, brandonrising, hipsterusername and lstein as code owners January 13, 2024 13:32

psychedelicious mentioned this pull request Jan 13, 2024

feat(nodes): WIP restricted invocation context #4910

Closed

8 tasks

psychedelicious requested review from GreggHelt2, RyanJDick, StAlKeR7779 and damian0815 as code owners January 14, 2024 23:51

psychedelicious force-pushed the feat/invocation-context branch from db7a776 to 06c74f5 Compare January 14, 2024 23:52

brandonrising reviewed Jan 15, 2024

View reviewed changes

invokeai/app/invocations/onnx.py Outdated Show resolved Hide resolved

psychedelicious commented Jan 15, 2024

View reviewed changes

invokeai/app/invocations/onnx.py Outdated Show resolved Hide resolved

psychedelicious force-pushed the feat/invocation-context branch from 06c74f5 to 22c9e0c Compare February 5, 2024 06:17

github-actions bot added documentation Improvements or additions to documentation api python PRs that change python files Root PythonDeps invocations PRs that change invocations backend PRs that change backend files services PRs that change app services labels Feb 5, 2024

psychedelicious force-pushed the feat/invocation-context branch from 9b16b13 to 5c22fa9 Compare February 5, 2024 07:30

hipsterusername requested a review from brandonrising February 5, 2024 14:23

brandonrising reviewed Feb 6, 2024

View reviewed changes

invokeai/app/invocations/compel.py Show resolved Hide resolved

brandonrising reviewed Feb 6, 2024

View reviewed changes

invokeai/app/services/shared/invocation_context.py Outdated Show resolved Hide resolved

brandonrising reviewed Feb 6, 2024

View reviewed changes

invokeai/app/services/shared/invocation_context.py Outdated Show resolved Hide resolved

psychedelicious added 17 commits February 15, 2024 17:30

tidy(nodes): clarify comment

cb0b389

fix(nodes): use metadata/board_id if provided by user, overriding…

ed772a7

… `WithMetadata`/`WithBoard`-provided values

feat(nodes): make delete on startup configurable for obj serializer

c58f8c3

- The default is to not delete on startup - feels safer. - The two services using this class _do_ delete on startup. - The class has "ephemeral" removed from its name. - Tests & app updated for this change.

tidy(nodes): do not store unnecessarily store invoker

ff249a2

tidy(nodes): "latents" -> "obj"

c1e5cd5

chore(nodes): fix pyright ignore

6bb2dda

chore(nodes): update ObjectSerializerForwardCache docstring

a7207ed

tests: test ObjectSerializerDisk class name extraction

199ddd6

feat(nodes): extract LATENT_SCALE_FACTOR to constants.py

ba7b1b2

feat(nodes): use LATENT_SCALE_FACTOR in primitives.py, noise.py

2005411

- LatentsOutput.build - NoiseOutput.build - Noise.width, Noise.height multiple_of

chore(nodes): export model-related objects from invocation_api

8fb77e4

chore(nodes): remove deprecation logic for nodes API

321b939

chore(nodes): "SAMPLER_NAME_VALUES" -> "SCHEDULER_NAME_VALUES"

1bbd13e

This was named inaccurately.

feat(nodes): add more missing exports to invocation_api

6c4eeaa

Crawled through a few custom nodes to figure out what I had missed.

chore(ui): regen types

c76a6bd

psychedelicious force-pushed the feat/invocation-context branch from 2bb57ee to c76a6bd Compare February 15, 2024 06:30

psychedelicious mentioned this pull request Feb 15, 2024

v4.0.0 #5724

Closed

psychedelicious closed this Feb 19, 2024

fieldOfView mentioned this pull request Apr 8, 2024

[enhancement]: add a method to get the path of an image from the invocation context #6175

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: simplified invocation context #5491

feat: simplified invocation context #5491

Uh oh!

psychedelicious commented Jan 13, 2024

Uh oh!

psychedelicious commented Jan 15, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

psychedelicious commented Feb 19, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

feat: simplified invocation context #5491

feat: simplified invocation context #5491

Uh oh!

Conversation

psychedelicious commented Jan 13, 2024

What type of PR is this? (check all applicable)

Have you discussed this change with the InvokeAI team?

Have you updated all relevant documentation?

Description

Related Tickets & Documents

QA Instructions, Screenshots, Recordings

Merge Plan

Added/updated tests?

[optional] Are there any post deployment tasks we need to perform?

Uh oh!

psychedelicious commented Jan 15, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

psychedelicious commented Feb 19, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants