FreeInit Error- Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

[kendinitto]

Getting the following error when using FreeInit Iteration Options, if I do not use FreeInit everything works perfectly. Everything else in ComfyUI is working normally except this one thing. It used to work but after updating to the latest it no longer works.

[AnimateDiffEvo] - INFO - Sliding context window activated - latents passed in (32) greater than context_length 16.
[AnimateDiffEvo] - INFO - Using motion module AnimateLCM_sd15_t2v.ckpt:v2.
[AnimateDiffEvo] - INFO - Iteration 1/2
!!! Exception during processing!!!
Traceback (most recent call last):
File "I:\AI\ComfyUI\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack\inspire\a1111_compat.py", line 229, in sample
latent = inspire_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise, noise_mode, incremental_seed_mode=batch_seed_mode,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack\inspire\a1111_compat.py", line 96, in inspire_ksampler
raise e
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack\inspire\a1111_compat.py", line 84, in inspire_ksampler
samples = common.impact_sampling(
^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack\inspire\libs\common.py", line 14, in impact_sampling
return nodes.NODE_CLASS_MAPPINGS['RegionalSampler'].separated_sample(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\special_samplers.py", line 408, in separated_sample
return separated_sample(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_sampling.py", line 214, in separated_sample
res = sample_with_custom_noise(model, add_noise, seed, cfg, positive, negative, impact_sampler, sigmas, latent_image, noise=noise, callback=callback)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\impact_sampling.py", line 158, in sample_with_custom_noise
samples = comfy.sample.sample_custom(model, noise, cfg, sampler, sigmas, positive, negative, latent_image,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 536, in motion_sample
latents, noise = iter_opts.preprocess_latents(curr_i=curr_i, model=model, latents=latents, noise=noise,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "I:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sample_settings.py", line 475, in preprocess_latents
noised_latents = latents * sqrt_alpha_prod + noise * sqrt_one_minus_alpha_prod
~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Prompt executed in 2.48 seconds

[Kosinkadink]

Thank you for the report, I'll look into it

[kendinitto]

BTW, I am running with an RTX 4090 and 32GB of internal Ram so I got plenty of juice.

[Kosinkadink]

I've been unable to replicate your issue, could you create the simplest workflow you can that has the problem? And can you try to use the vanilla KSamplers from ComfyUI? (you can start off with this txt2vid example in the readme: https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved?tab=readme-ov-file#txt2vid)

[kendinitto]

Could it be related to my torch version?

Total VRAM 24564 MB, total RAM 32583 MB
pytorch version: 2.4.0+cu121
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4090 : cudaMallocAsync
Using pytorch cross attention`

The only other time i've ever seen an error similar to that in the past , it was related to torch

[Kosinkadink]

Did you try it with a normal ksampler?

[kendinitto]

Ok so I just tried the normal ksampler and that does in fact work, but when I use KSamplerPipe //Inspire then I get the errors so the problem seems related to that node.

[Kosinkadink]

Cool, sometime I'll try out that specific node that gives you trouble and I'll see if there's a workaround I can add in my code to just make FreeInit work with it. Something is just not on the right device when it gets there when using that other KSampler.

[Kosinkadink]

But for now you can just use other KSamplers to workaround it

[kendinitto]

I just did git pull of the latest inspire pack and get the same error

[kendinitto]

Ya ill have to switch ksampler to get around it thanks.

[kendinitto]

One more update , I tried ksampler pipe from impakt as well as non pipe versions and none of them work, I also did not see a way to get the AYS SD1 scheduler option with the standard working ksampler which I have found that one to be noticeably the best scheduler to use for animatediff.

[kendinitto]

FYI, as another work around Ksampler from efficient does work and that has AYS SD1

[Kosinkadink]

Once the rework-modelpatcher PR is merged after my ComfyUI hook PR gets merged, this issue should go away entirely. #498