Improving experience for new users

[Nolendil]

Hello,

After reading #186, I'd like to add to this topic as a Windows user with little space on their C drive.
I've used a few other generative AI tools (e.g. https://github.com/oobabooga/text-generation-webui, https://github.com/lllyasviel/Fooocus/, https://github.com/lllyasviel/stable-diffusion-webui-forge) and expected a similar experience with this one.
I really appreciate the one-click installers and having the whole tool and dependencies being contained in their own folder.

However, the experience was slightly different with this tool:

During the install, pip was downloading and building packages in my temp folder (on C:). I had enough space left but it used about 10 GB and that could have been an issue. I don't think other AI tools did that (but I may have not noticed). Opting to install all the dependencies at once probably amplified the storage required.
If there's a way to tell pip to use a specific working folder and set that to a temp folder inside tts-generation-webui's folder, that would be great.

So far, I've only tried Bark and MAGNet and mostly used the React UI.
I figured that Bark models go in the "voices" folder but I got a bit confused when trying to use MAGNeT. The React UI or the terminal didn't provide any info when I first attempted to generate something with MAGNeT, until it suddenly started to download files (which was only visible on the terminal) and eventually generated what I had requested. I think some feedback was missing there.

Also, while I expected the downloaded files to be put somewhere inside the tts-generation-webui's folder (just like "voices"), I eventually realized they were created on my C: drive and found the HuggingFace cache folder.
This lead me to find #186 and wonder what would be the best way to change the environment variables.
Just now, I've found that the Gradio interface allows setting the environment variables and hope that it will solve my problem. I still have to figure the format to make it a relative path. I didn't find that setting in the React UI.
Even though it can be set later, I think the cache locations is something that should be asked when launching the tool for the first time.

Proposed solutions to improve the first launch experience:

• Make pip's working folder reside in the application folder, if possible.
• Warn that pip may (temporarily) use a lot of space when offering to install all dependencies at once.
• During installation, ask the user where to put the various extra folders (Hugging Face Cache, Torch Hub Cache, others?) and suggest reasonable choices (e.g. either a location to keep the application self-contained, the default location in the home folder or a custom folder)
• Create a proper page to explain the various cache folders (purpose, default location and how to change them) and link it in the Installation section of the ReadMe. The ReadMe currently links to Downloading models and stuff #186 but it's mentioned quite far at the bottom and it's unlikely people would notice it or know that they should read it until they face the problem. Also while Downloading models and stuff #186 provides some information, it's very bare bones and requires more research from the user to find what action they actually need to do. Mentioning the Settings page on the Gradio interface would help.
• Having two different UIs is very confusing for a new user. The Gradio one seems more feature complete, while the React one seems more streamlined. This should be explained in the ReadMe too, I think.
• Explaining how to add models and where. Mentioning what is automatically downloaded and what needs to be manually downloaded.

Anyway, thanks a lot for your work on this project, it's very appreciated.

[rsxdalv]

During the install, pip was downloading and building packages in my temp folder (on C:). I had enough space left but it used about 10 GB and that could have been an issue. I don't think other AI tools did that (but I may have not noticed). Opting to install all the dependencies at once probably amplified the storage required.
If there's a way to tell pip to use a specific working folder and set that to a temp folder inside tts-generation-webui's folder, that would be great.

Actually it is supposed to do that. If conda and pip are working correctly, all of the pip dependencies during install should be within the tts-webui folder, not outside. However, some features always get made that use some 'user' cache, like C:/Users/user/.cache or app_data etc etc and you only realize that by checking the internal code. I.e., the official and original bark uses .cache due to huggingface's 'download' implementation.
I am interested if you could share any more details about the temp folder, since I might have missed that on my own system.

So far, I've only tried Bark and MAGNet and mostly used the React UI.
I figured that Bark models go in the "voices" folder but I got a bit confused when trying to use MAGNeT. The React UI or the terminal didn't provide any info when I first attempted to generate something with MAGNeT, until it suddenly started to download files (which was only visible on the terminal) and eventually generated what I had requested. I think some feedback was missing there.

Thanks, I will make an issue for adding more feedback on gradio/React about what is visible in the console: #488

Also, while I expected the downloaded files to be put somewhere inside the tts-generation-webui's folder (just like "voices"), I eventually realized they were created on my C: drive and found the HuggingFace cache folder.
This lead me to find #186 and wonder what would be the best way to change the environment variables.
Just now, I've found that the Gradio interface allows setting the environment variables and hope that it will solve my problem. I still have to figure the format to make it a relative path. I didn't find that setting in the React UI.
Even though it can be set later, I think the cache locations is something that should be asked when launching the tool for the first time.

Yes, at this point I think I have the capacity to improve the UX for changing the cache location.

One note - for better or worse, most projects such as Audiocraft, Bark, Tortoise etc have adapted this pattern for the last 2+ years. I think as a webui I should explore ways of 'patching' these features.

I will review more later.

[Nolendil]

I am interested if you could share any more details about the temp folder, since I might have missed that on my own system.

I'm not sure what details I can provide. The temp folders were deleted after installation.
I know it used C:\Users\<username>\AppData\Local\Temp but I didn't think to check ~/.cache during the installation.
There were up to 450-500 folders in AppData\Local\Temp with names like pip-build-xxxx and similar (pip-tmp-xxx?). The amount of folders didn't go down at any point except once it was completely done.
Is there any other specific information I could provide?

Yes, at this point I think I have the capacity to improve the UX for changing the cache location.

Sounds great. Thanks!

most projects [...] have adapted this pattern for the last 2+ years. I think as a webui I should explore ways of 'patching' these features.

If changing cache location is made easy, I don't mind it as much. "Patching" all those projects might be very time consuming.

[rsxdalv]

Ok, I'm not sure I will be able to solve that but I will create an issue so that the knowledge about it does not disappear. I might be able to reproduce it. #490

[rsxdalv]

By the way - one note - SD webui does download some transformers to HF Cache last I checked.

[Nolendil]

Checking my .cache folder, its only content is from tts-generation-webui judging by the file dates and file names (plus one file from yt-dlp). It's possible that I haven't used a function of SD webui that triggers using the HF Cache yet (or that I forgot that it happened, I haven't used it in a while).
Thanks for the info, that might help if I encounter similar issues with other applications :)

[rsxdalv]

I recommend using TreeSizeFree (portable) for scanning the .cache and other folders. I have seen many shenanigans such as 24gb sized .txt log file from avast, 4-6gb of NVidia Driver caches, 3gb .npm .pip .yarn .gradle etc caches and so on.
I'd say the average ML/AI project is server grade, meaning 20gb here, 40gb there, 20 system dependencies and everything inbetween as well. But it's pulling into consumer-grade (where devs need to invest a lot more time so that the software installation is devs' problem, not end users problem as it is with Dev Ops whose job it is to install stuff)

[rsxdalv]

I'm trying to see how I can implement the model download warnings and info, but sometimes the model code does not make it easy. For example, the new chatterbox model does the model download internally when you try to use it. Therefore, I have to modify the original model code to rectify this.

[Nolendil]

Is it related to #488 ?
Modifying the original code sounds scary for maintenance, but the alternatives that come to my mind may actually be worse (e.g. parsing the terminal output, checking for the presence of model files...) 😅
Maybe you could display a message when the user launches a problematic models, that says that model files may (or may not) be downloaded? And have it displayed only once per session.

[rsxdalv]

Yes, that's the gist of it. In fact, the default mode (which you can now see with chatterbox) is that the model gets downloaded somewhere (huggingface cache). So I usually try to override as much as I can to keep the models local to tts-webui, and to have control over the checkpoint selection.

I think I will get to a solution, it's just that right now there's a lot of stuff going on. I think soon enough there might be a quieter period, when I'm not working on:

• Quantization
• Training models
• Youtube
• File sharing site
• APIs for other projects
• Upgrading Pytorch again because RTX 50xx needs CUDA 12.8 and PyTorch 2.7.0

[rsxdalv]

During installation, ask the user where to put the various extra folders (Hugging Face Cache, Torch Hub Cache, others?) and suggest reasonable choices (e.g. either a location to keep the application self-contained, the default location in the home folder or a custom folder)

I really like this idea - here's an issue to track it: #489

[rsxdalv]

Having two different UIs is very confusing for a new user. The Gradio one seems more feature complete, while the React one seems more streamlined. This should be explained in the ReadMe too, I think.

I think I had some info on it, but originally I wanted to convert everything to React. React is meant to be the definitive while gradio is meant to be the uncomfortable pro barebones mode.

[Nolendil]

I mostly used the React UI because it is more appealing, so you're on the right track :)
I didn't find any settings on it, though, while Gradio has that.
(As a side note, I really like the HuggingFace Cache and GPU Info tools on React.)

[Nolendil]

Thanks for taking my feedback in account.

One extra thing:
I saw that the default path for the "tts-generation-webui Directory" in the Gradio interface was ./tts-generation-webui/data/models/.
However, the application folder is called tts-generation-webui-main on my drive (because that's its name inside the installation zip file), so I thought I should rename it to tts-generation-webui. Doing so made the application fail to start with the following output in the terminal:

Transcript started, output file is D:\<mypath>\tts-generation-webui\installer_scripts\output.log
"Checking if the Micromamba/Conda environment is already installed."
"Micromamba/Conda environment already installed in D:\<mypath>\tts-generation-webui\installer_files\env, skipping installation"
vswhere is installed, checking for Visual Studio Build Tools installation...
Visual Studio Build Tools is installed, continuing...
The system cannot find the path specified.
'node' is not recognized as an internal or external command,
operable program or batch file.
Failed to init the app, exiting...

The "output.log" file doesn't seem to give more details but I can provide it if needed.
Renaming the folder back solved the issue but it seems there's something (node.js?) that prevents the application from being portable (which would be great).
I assume the path displayed for "tts-generation-webui Directory" is slightly wrong (and I suppose it's hard-coded since it can't be changed) and it should actually display ./data/models/.

[rsxdalv]

It's a conda thing:

conda/conda#1845

Basically, conda installs with the absolute path rather than the relative path. This also tends to be a problem for people who just want to move the folder from say C: to D:
There are some manual fixes for it, the most painful one probably being to delete installer_files and redownload 4-6gb of python packages.

Because it only checks if installer_files/env exists it does not realize that it fails to activate, this is a bug and should be fixed:
#491

Before the first install, you can place the files anywhere with any folder name, but after installing it cannot be moved (unless some day conda stops using absolute paths, which they do not want to yet).

[Nolendil]

I see, thanks for all the information. Not much you can do on your side, indeed, since it's a long-lasting Conda issue.

Otherwise, I relocated all my cache folders (HF, Torch, etc) and it works fine :)

[rsxdalv]

It's been a busy week but I finally got around to implementing one change - info bubbles for when models are being loaded/unloaded.
My current strategy is to try to utilize gr.Info more so that the Gradio UI is as informative as the console.