Colab speed and user experience improvement

[ackvf]

Hey guys, I forked your notebook and improved it. It was confusing for me and I needed a way to better manage my models, so I tweaked it a bit, but I don't understand python and SD at all, so I was wondering if you could take a look, give me some feedback, and perhaps imported some of the tweaks back into your notebook too?

The main difference is that I improved some comments and documentation (such as documenting ControlNets), streamlined some flows (xFormers setting) and split the notebook into two parts:

• A1111 notebook
  https://colab.research.google.com/drive/15y0-QZM32atrnO05GVOGVP7ryPZQrl1x?usp=sharing

• Companion model downloader
  https://colab.research.google.com/drive/1eZbKtuP4XbC6wOfxmu_2_UtjLmbq0Z6a?usp=sharing

Having it in a separate file makes it more flexible as you can continue running your A1111 notebook while downloading and experimenting with models and extensions.

This also allows to place several model downloads into a queue to be executed one after another, since the companion colab provides multitude of execution cells (meaning I can add another model download, while one download is already in progress).

[i-huzaifa-arshad]

Nice stuff. Glad someone came with better approach than mine

[wordbrew]

To be honest, Fast A1111 has become incredibly slow and buggy for me lately. Not sure what's going on, but as a Colab pro member, it hangs so often in the Start Stable Diffusion cell that I'm considering moving away from this repo entirely. Hangs even on brand new installs. It's gotten a bit unusable for me and I have no idea what caused it.

[TheLastBen]

don't blame the repo, blame colab, starting sd is slow because of installed extensions, gdrive speed limitation and slow colab CPU, if you want faster SD, use my Runpod notebooks or my Paperspace notebooks, find them in the main page of the repo

[ackvf]

Interesting, I have never experienced hangs during startup. I wonder what the cause might be. Using Colab Pro, but also free on another account.

[ackvf]

@TheLastBen I wonder if it would help to first copy the gdrive to colab workspace disk, then launch the app. Same for models. It's especially noticeable when changing models. Chaching them to colab disk would be much faster.

[TheLastBen]

@ackvf copying from gdrive to the workspace would take a lot of time

[TheLastBen]

it's great work, but for unexperienced users it seems a bit complicated, I like to keep it as simple as possible to avoid complications.

[TheLastBen]

If a user wants to turn off xformers that means they are a bit experienced and know what they're doing, I take care of the necessary arguments and implements them in a way the webui functions the best.

[i-huzaifa-arshad]

One thing I love the most is reverse engineering your code. It's a great way of learning. I want to run InvokeAi webui on colab. Just wanted to use their outpainting just for once. Hopefully I will manage to make a colab to run it. Or you can do this @TheLastBen ?

[ackvf]

@TheLastBen That is not true actually. When I started, I was upset and confused that running the same prompt will not yield same result. I searched and at one point someone suggested to check my xFormers setting, which I had no idea what they were talking about, so I spent more hours figuring that out, finally finding that in the command line of your notebook, which led into more hours to learn how to do python and colab notebooks.

[ackvf]

Ideally I would like to control xFormers from the webui itself. IIUC it allows to generate images faster.

My idea is that during experimenting, I would enable xFormers and once I settle on a preferred composition and design, I will disable xFormers and fine tune the prompt and settings. - does that sound legit or is my understanding wrong?

[ackvf]

Honestly, civitai is full of confused people asking why they were not able to reproduce the art on display, where they are usually followed by a suggestion to turn off "xFormers". I don't think at all that this is a common knowledge, but it's such a common issue that it should be brought to the attention right from the UI or the Colab - which will then incentive people to search for the meaning of it.

[anonderpling]

@ackvf

you can continue running your A1111 notebook while downloading and experimenting with models and extensions.

You mean like batchlinks webui extension does? Your solution requires uploading files to drive from a separate colab, meaning your other colab instance won't have immediate local access to them; if you use batchlinks they should be available immediately from the cache, and it downloads the model previews as well.

I use batchlinks mainly for SD models (and occasionally for loras), as they're the largest files; smaller files also get downloaded directly in the same colab with the unofficial Civitai helper extension (Civitai, github); the advantage here is that while the Civitai helper doesn't download as fast as the batchlinks extension does, it downloads other data that's useful as well (for example, it grabs the prompts for the preview images). After downloading files, I tell the civitai helper to fetch metadata for those files as well, so I have the prompts for the previews. After this, I also download all the previews to my personal server, and set up a redirect extension so the previews don't time out my connection¹².

Alternatively, just use a download manager³ for your drive.

Footnotes

1. I use Redirector firefox extension (choice of extension mainly because it works for firefox mobile) with these settings. You'll have to update the domain name in that config. If you don't have a server of your own, you'll need a secondary tunnel (the whole point of this tunnel is to avoid slowing down your main A1111 tunnel); if you have an SSH client installed, I recommend using localhost.run, as they don't require registration and it needs nothing else. Alternatively, cloudflare tunnel or localtunnel will also work. ↩

2. It's a bit more complicated than this, since I have it automated. I use git lfs from huggingface to keep a copy of your models directory, and you can do a sparse checkout without the actual LFS tracked files, resulting in a small-ish ~300mb directory of just the previews, which I then recompress with imagemagick; the resulting directory is what I actually redirect to, however my server has very little bandwidth available, so I'm reluctant to make it public... ↩

3. I made this (source) using the same aria2 used by batchlinks webui extension, and ziahamza/webui-aria2 for pasting links in easier; the default path is the A1111 models/Stable-diffusion directory. You will have to choose the directory the files get downloaded in for other types, as I mainly used it for models. To make that simpler, you could create symlinks at /models or some such to use shorter paths. ↩