Custom Node Testing

• I have tried disabling custom nodes and the issue persists (see how to disable custom nodes if you need help)

Expected Behavior

(Please first read the Other section.)

When clip 1 input of the CLipMergeSimple node is connected to the original CLIP, setting ratio to 0.0 produces the same result as the case when the original CLIP is connected to the KSampler node directly.

That is, direct.png and ratio 0.png should be identical.

Actual Behavior

(Please first read the Other section.)

When clip 1 input is connected to the original CLIP, setting ratio to 0.0 produces some different result from the case when the original CLIP is connected to KSampler directly, and setting ratio to 1.0 produces the same result.

That is, direct.png and ratio 0.png are different, and direct.png and ratio 1.png are same.

Steps to Reproduce

Use the workflows embedded in the three sample images to produce the images, and compare the resulting images.

direct.png:

ratio 0.png:

ratio 1.png:

Debug Logs

got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load SDXLClipModel
loaded completely 12178.8 1560.802734375 True
Requested to load SDXL
loaded completely 10550.872177886962 4897.0483474731445 True
100%|██████████| 20/20 [00:05<00:00,  3.64it/s]
Requested to load AutoencoderKL
loaded completely 2108.9759979248047 159.55708122253418 True
Prompt executed in 10.67 seconds
got prompt
CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
Requested to load SDXLClipModel
loaded completely 7041.021658706665 1560.802734375 True
100%|██████████| 20/20 [00:03<00:00,  5.16it/s]
Prompt executed in 5.83 seconds
got prompt
Requested to load SDXLClipModel
loaded completely 7041.021658706665 1560.802734375 True
100%|██████████| 20/20 [00:03<00:00,  5.08it/s]
Prompt executed in 5.52 seconds

Other

The documentation for the ClipMergeSimple node says:

A ratio of 1.0 means fully adopting the second model's features,
white 0.0 retains only the first model's features.

However, it actually behaves in reverse direction. That is, the same features as the first model is output when setting the ratio 1.0. I have not examined the python codes, but the implementing code should be corrected to match the behavior to the specification.

I generated three sample images with the same KSampler parameters (including the seed) under the following CLIP setup:

• direct.png is the image when the original CLIP is directly connected to KSampler.
• ratio 0.png is the image when the ratio is set to 0.0. (clip 1 is connected to the original CLIP.)
• ratio 1.png is the image when the ratio is set to 1.0. (clip 1 is connected to the original CLIP.)

Per the specification, direct.png and ratio 0.png should be identical, but actually direct.png and ratio 1.png are identical.

.safetensors files are downloaded from civitai.com. In particular:

• sd_xl_base_1.0.safetensors: https://civitai.com/models/101055?modelVersionId=126601
• balancedCLIPUpdated_ponyCLIPL.safetensors: https://civitai.com/models/2002147?modelVersionId=2266196

The debug log pasted above is from the session to first produce direct.png, then ratio 0.png, then ratio 1.png.

Observed/tested on the following versions of the programs:

Python version: 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
ComfyUI version: 0.3.67
ComfyUI frontend version: 1.28.8