fix flux controlnet bug

[free001style]

What does this PR do?

Before this if txt_ids was 3d tensor, line with txt_ids[:1] concat txt_ids by batch dim. Now we first check that txt_ids is 2d tensor (or take first batch element) and then concat by token dim

[lordsoffallen]

Still getting the error even with your fix:

RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 1 but got size 2 for tensor number 1 in the list.

I am not sure where the issue lies but controlnet inpainting is not working at all...

[free001style]

RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 1 but got size 2 for tensor number 1 in the list.

Are you getting error in this line? txt_ids = torch.cat([txt_ids[:1], txt_ids], dim=0)
Because I didn't face with this issues(((

[lordsoffallen]

I am getting it in this line: encoder_hidden_states = torch.cat([controlnet_mode_emb, encoder_hidden_states], dim=1)

It happens when I use multicontrol code but only have one control defined. Reverting to use single control i made it work. I don't really know where the error comes from honestly, too many different stuff going on.

[hlky]

Hi @free001style. Is there a case where txt_ids is still being passed as 3d?

@lordsoffallen Can you create an issue for the multicontrolnet case with a reproducer?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[free001style]

@hlky both txt_ids and img_ids can be 3d (if user feed them as 3d in forward) but in that case I leave only first batch element. I don't know why original implementation need only 2d tensors (I think it's better to fix this in order to use 3d tensors).

[hlky]

@free001style I meant from a pipeline, looks like they're all using 2d already. This should be fixed for 3d case where a user is directly using FluxControlNetModel though, even if it is deprecated, so thanks!