Ultimate SD Upscale does not work anymore after updating ControlNet

[paolobesser]

Hi. Yesterday (jun 08th, 2023) I decided to check for A1111 extensions updates and something went terribly wrong. It broke both Dreambooth (but I can live without it) and affected ControlNet's ability to upscale images using the script above (which is more important to me). Every time I try to upscale any image, no matter the image size, no matter the upscaler I choose, no matter the tile size, at the third tile the script fails with this error:

RuntimeError: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 192, 192] to have 3 channels, but got 4 channels instead

I have removed Dreambooth at all, I have removed ControlNet as well, I downloaded ControlNet from scratch and then added all models again. I have also git updated the whole A1111 and started it with no parameters (even no xformers), including downloading Ultimate SD Upscale again. Nothing fixed the issue. I am running SD on Windows 11, 64 bit and it used to work like a charm.

2023-06-09 10:04:57,111 - ControlNet - INFO - Pixel Perfect Computation:
2023-06-09 10:04:57,112 - ControlNet - INFO - resize_mode = ResizeMode.INNER_FIT
2023-06-09 10:04:57,112 - ControlNet - INFO - raw_H = 512
2023-06-09 10:04:57,112 - ControlNet - INFO - raw_W = 512
2023-06-09 10:04:57,112 - ControlNet - INFO - target_H = 512
2023-06-09 10:04:57,112 - ControlNet - INFO - target_W = 512
2023-06-09 10:04:57,112 - ControlNet - INFO - estimation = 512.0
2023-06-09 10:04:57,112 - ControlNet - INFO - Preview Resolution = 512
2023-06-09 10:05:16,759 - ControlNet - INFO - Pixel Perfect Computation:
2023-06-09 10:05:16,760 - ControlNet - INFO - resize_mode = ResizeMode.INNER_FIT
2023-06-09 10:05:16,760 - ControlNet - INFO - raw_H = 512
2023-06-09 10:05:16,760 - ControlNet - INFO - raw_W = 512
2023-06-09 10:05:16,760 - ControlNet - INFO - target_H = 512
2023-06-09 10:05:16,760 - ControlNet - INFO - target_W = 512
2023-06-09 10:05:16,760 - ControlNet - INFO - estimation = 512.0
2023-06-09 10:05:16,761 - ControlNet - INFO - Preview Resolution = 512
Canva size: 2048x2048
Image size: 512x512
Scale factor: 4
Upscaling iteration 1 with scale factor 4
Tile 1/9
Tile 2/9
Tile 3/9
Tile 4/9
Tile 5/9
Tile 6/9
Tile 7/9
Tile 8/9
Tile 9/9
Tile size: 512x512
Tiles amount: 16
Grid: 4x4
Redraw enabled: True
Seams fix mode: NONE
2023-06-09 10:05:22,802 - ControlNet - INFO - Loading model: control_v11f1e_sd15_tile [a371b31b]
2023-06-09 10:05:23,666 - ControlNet - INFO - Loaded state_dict from [D:\automatic1111\webui\extensions\sd-webui-controlnet\models\control_v11f1e_sd15_tile.pth]
2023-06-09 10:05:23,667 - ControlNet - INFO - Loading config: D:\automatic1111\webui\extensions\sd-webui-controlnet\models\control_v11f1e_sd15_tile.yaml
2023-06-09 10:05:25,805 - ControlNet - INFO - ControlNet model control_v11f1e_sd15_tile [a371b31b] loaded.
2023-06-09 10:05:38,376 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-06-09 10:05:38,376 - ControlNet - INFO - Pixel Perfect Computation:
2023-06-09 10:05:38,377 - ControlNet - INFO - resize_mode = ResizeMode.INNER_FIT
2023-06-09 10:05:38,377 - ControlNet - INFO - raw_H = 576
2023-06-09 10:05:38,377 - ControlNet - INFO - raw_W = 576
2023-06-09 10:05:38,377 - ControlNet - INFO - target_H = 576
2023-06-09 10:05:38,377 - ControlNet - INFO - target_W = 576
2023-06-09 10:05:38,377 - ControlNet - INFO - estimation = 576.0
2023-06-09 10:05:38,377 - ControlNet - INFO - preprocessor resolution = 576
100%|████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:03<00:00, 2.01it/s]
2023-06-09 10:05:45,367 - ControlNet - INFO - Loading model from cache: control_v11f1e_sd15_tile [a371b31b], 29.50s/it]
2023-06-09 10:05:45,514 - ControlNet - INFO - Loading preprocessor: tile_resample
2023-06-09 10:05:45,514 - ControlNet - INFO - Pixel Perfect Computation:
2023-06-09 10:05:45,514 - ControlNet - INFO - resize_mode = ResizeMode.INNER_FIT
2023-06-09 10:05:45,514 - ControlNet - INFO - raw_H = 576
2023-06-09 10:05:45,515 - ControlNet - INFO - raw_W = 576
2023-06-09 10:05:45,515 - ControlNet - INFO - target_H = 576
2023-06-09 10:05:45,515 - ControlNet - INFO - target_W = 576
2023-06-09 10:05:45,515 - ControlNet - INFO - estimation = 576.0
2023-06-09 10:05:45,515 - ControlNet - INFO - preprocessor resolution = 576
100%|████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:03<00:00, 1.94it/s]
Error completing request█████████████████████████████████████▋ | 70/112 [22:57<02:03, 2.95s/it]
Arguments: ('task(7laf7bi8yemfz1u)', 0, '', '', [], <PIL.Image.Image image mode=RGBA size=512x512 at 0x21FAD355D50>, None, None, None, None, None, None, 20, 0, 4, 0, 1, False, False, 1, 1, 7, 1.5, 0.33, -1.0, -1.0, 0, 0, 0, False, 0, 512, 512, 1, 0, 0, 32, 0, '', '', '', [], 10, False, 'MultiDiffusion', False, True, 1024, 1024, 96, 96, 48, 4, 'None', 2, False, 10, 1, 1, 64, False, False, False, False, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 0.4, 0.4, 0.2, 0.2, '', '', 'Background', 0.2, -1.0, False, 1536, 96, True, True, True, False, False, '', 0, <scripts.controlnet_ui.controlnet_ui_group.UiControlNetUnit object at 0x0000021FED8408B0>, '

\n
• CFG Scale should be 2 or lower.
\n

\n', True, True, '', '', True, 50, True, 1, 0, False, 4, 0.5, 'Linear', 'None', '

Recommended settings: Sampling Steps: 80-100, Sampler: Euler a, Denoising strength: 0.8

', 128, 8, ['left', 'right', 'up', 'down'], 1, 0.05, 128, 4, 0, ['left', 'right', 'up', 'down'], False, False, 'positive', 'comma', 0, False, False, '', '

Will upscale the image by the selected scale factor; use width and height sliders to set tile size

', 64, 0, 2, 1, '', [], 0, '', [], 0, '', [], True, False, False, False, 0, None, None, False, 50, '

Will upscale the image depending on the selected target size type

', 512, 0, 8, 32, 64, 0.35, 32, 6, True, 0, False, 8, 0, 2, 2048, 2048, 4) {}
Traceback (most recent call last):
File "D:\automatic1111\webui\modules\call_queue.py", line 57, in f
res = list(func(*args, **kwargs))
File "D:\automatic1111\webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "D:\automatic1111\webui\modules\img2img.py", line 176, in img2img
processed = modules.scripts.scripts_img2img.run(p, *args)
File "D:\automatic1111\webui\modules\scripts.py", line 441, in run
processed = script.run(p, *script_args)
File "D:\automatic1111\webui\extensions\ultimate-upscale-for-automatic1111\scripts\ultimate-upscale.py", line 553, in run
upscaler.process()
File "D:\automatic1111\webui\extensions\ultimate-upscale-for-automatic1111\scripts\ultimate-upscale.py", line 136, in process
self.image = self.redraw.start(self.p, self.image, self.rows, self.cols)
File "D:\automatic1111\webui\extensions\ultimate-upscale-for-automatic1111\scripts\ultimate-upscale.py", line 243, in start
return self.linear_process(p, image, rows, cols)
File "D:\automatic1111\webui\extensions\ultimate-upscale-for-automatic1111\scripts\ultimate-upscale.py", line 178, in linear_process
processed = processing.process_images(p)
File "D:\automatic1111\webui\modules\processing.py", line 610, in process_images
res = process_images_inner(p)
File "D:\automatic1111\webui\extensions\sd-webui-controlnet\scripts\batch_hijack.py", line 42, in processing_process_images_hijack
return getattr(processing, '__controlnet_original_process_images_inner')(p, *args, **kwargs)
File "D:\automatic1111\webui\modules\processing.py", line 775, in process_images_inner
image = apply_overlay(image, p.paste_to, i, p.overlay_images)
File "D:\automatic1111\webui\modules\processing.py", line 70, in apply_overlay
image = images.resize_image(1, image, w, h)
File "D:\automatic1111\webui\modules\images.py", line 288, in resize_image
resized = resize(im, src_w, src_h)
File "D:\automatic1111\webui\modules\images.py", line 271, in resize
im = upscaler.scaler.upscale(im, scale, upscaler.data_path)
File "D:\automatic1111\webui\modules\upscaler.py", line 62, in upscale
img = self.do_upscale(img, selected_model)
File "D:\automatic1111\webui\modules\esrgan_model.py", line 150, in do_upscale
img = esrgan_upscale(model, img)
File "D:\automatic1111\webui\modules\esrgan_model.py", line 224, in esrgan_upscale
output = upscale_without_tiling(model, tile)
File "D:\automatic1111\webui\modules\esrgan_model.py", line 203, in upscale_without_tiling
output = model(img)
File "D:\automatic1111\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "D:\automatic1111\webui\modules\esrgan_model_arch.py", line 61, in forward
return self.model(feat)
File "D:\automatic1111\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "D:\automatic1111\system\python\lib\site-packages\torch\nn\modules\container.py", line 217, in forward
input = module(input)
File "D:\automatic1111\system\python\lib\site-packages\torch\nn\modules\module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "D:\automatic1111\webui\extensions-builtin\Lora\lora.py", line 415, in lora_Conv2d_forward
return torch.nn.Conv2d_forward_before_lora(self, input)
File "D:\automatic1111\webui\extensions\a1111-sd-webui-lycoris\lycoris.py", line 753, in lyco_Conv2d_forward
return torch.nn.Conv2d_forward_before_lyco(self, input)
File "D:\automatic1111\system\python\lib\site-packages\torch\nn\modules\conv.py", line 463, in forward
return self._conv_forward(input, self.weight, self.bias)
File "D:\automatic1111\system\python\lib\site-packages\torch\nn\modules\conv.py", line 459, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 192, 192] to have 3 channels, but got 4 channels instead