Skip to content

Commit 8f0c9df

Browse files
authored
Merge branch 'main' into video
2 parents ddab9b4 + 28ef016 commit 8f0c9df

File tree

56 files changed

+3011
-787
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+3011
-787
lines changed

.github/workflows/benchmark.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ jobs:
2323
runs-on:
2424
group: aws-g6-4xlarge-plus
2525
container:
26-
image: diffusers/diffusers-pytorch-compile-cuda
26+
image: diffusers/diffusers-pytorch-cuda
2727
options: --shm-size "16gb" --ipc host --gpus 0
2828
steps:
2929
- name: Checkout diffusers

.github/workflows/build_docker_images.yml

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -38,9 +38,16 @@ jobs:
3838
token: ${{ secrets.GITHUB_TOKEN }}
3939

4040
- name: Build Changed Docker Images
41+
env:
42+
CHANGED_FILES: ${{ steps.file_changes.outputs.all }}
4143
run: |
42-
CHANGED_FILES="${{ steps.file_changes.outputs.all }}"
43-
for FILE in $CHANGED_FILES; do
44+
echo "$CHANGED_FILES"
45+
for FILE in $CHANGED_FILES; do
46+
# skip anything that isn't still on disk
47+
if [[ ! -f "$FILE" ]]; then
48+
echo "Skipping removed file $FILE"
49+
continue
50+
fi
4451
if [[ "$FILE" == docker/*Dockerfile ]]; then
4552
DOCKER_PATH="${FILE%/Dockerfile}"
4653
DOCKER_TAG=$(basename "$DOCKER_PATH")
@@ -65,7 +72,7 @@ jobs:
6572
image-name:
6673
- diffusers-pytorch-cpu
6774
- diffusers-pytorch-cuda
68-
- diffusers-pytorch-compile-cuda
75+
- diffusers-pytorch-cuda
6976
- diffusers-pytorch-xformers-cuda
7077
- diffusers-pytorch-minimum-cuda
7178
- diffusers-flax-cpu

.github/workflows/nightly_tests.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,7 @@ jobs:
188188
group: aws-g4dn-2xlarge
189189

190190
container:
191-
image: diffusers/diffusers-pytorch-compile-cuda
191+
image: diffusers/diffusers-pytorch-cuda
192192
options: --gpus 0 --shm-size "16gb" --ipc host
193193

194194
steps:

.github/workflows/push_tests.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -262,7 +262,7 @@ jobs:
262262
group: aws-g4dn-2xlarge
263263

264264
container:
265-
image: diffusers/diffusers-pytorch-compile-cuda
265+
image: diffusers/diffusers-pytorch-cuda
266266
options: --gpus 0 --shm-size "16gb" --ipc host
267267

268268
steps:

.github/workflows/release_tests_fast.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -316,7 +316,7 @@ jobs:
316316
group: aws-g4dn-2xlarge
317317

318318
container:
319-
image: diffusers/diffusers-pytorch-compile-cuda
319+
image: diffusers/diffusers-pytorch-cuda
320320
options: --gpus 0 --shm-size "16gb" --ipc host
321321

322322
steps:

docker/diffusers-pytorch-compile-cuda/Dockerfile

Lines changed: 0 additions & 50 deletions
This file was deleted.

docs/source/en/_toctree.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -206,7 +206,7 @@
206206
- local: optimization/mps
207207
title: Metal Performance Shaders (MPS)
208208
- local: optimization/habana
209-
title: Habana Gaudi
209+
title: Intel Gaudi
210210
- local: optimization/neuron
211211
title: AWS Neuron
212212
title: Optimized hardware

docs/source/en/api/pipelines/sana_sprint.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,12 +88,46 @@ image.save("sana.png")
8888

8989
Users can tweak the `max_timesteps` value for experimenting with the visual quality of the generated outputs. The default `max_timesteps` value was obtained with an inference-time search process. For more details about it, check out the paper.
9090

91+
## Image to Image
92+
93+
The [`SanaSprintImg2ImgPipeline`] is a pipeline for image-to-image generation. It takes an input image and a prompt, and generates a new image based on the input image and the prompt.
94+
95+
```py
96+
import torch
97+
from diffusers import SanaSprintImg2ImgPipeline
98+
from diffusers.utils.loading_utils import load_image
99+
100+
image = load_image(
101+
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/penguin.png"
102+
)
103+
104+
pipe = SanaSprintImg2ImgPipeline.from_pretrained(
105+
"Efficient-Large-Model/Sana_Sprint_1.6B_1024px_diffusers",
106+
torch_dtype=torch.bfloat16)
107+
pipe.to("cuda")
108+
109+
image = pipe(
110+
prompt="a cute pink bear",
111+
image=image,
112+
strength=0.5,
113+
height=832,
114+
width=480
115+
).images[0]
116+
image[0].save("output.png")
117+
```
118+
91119
## SanaSprintPipeline
92120

93121
[[autodoc]] SanaSprintPipeline
94122
- all
95123
- __call__
96124

125+
## SanaSprintImg2ImgPipeline
126+
127+
[[autodoc]] SanaSprintImg2ImgPipeline
128+
- all
129+
- __call__
130+
97131

98132
## SanaPipelineOutput
99133

docs/source/en/optimization/habana.md

Lines changed: 12 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -10,67 +10,22 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o
1010
specific language governing permissions and limitations under the License.
1111
-->
1212

13-
# Habana Gaudi
13+
# Intel Gaudi
1414

15-
🤗 Diffusers is compatible with Habana Gaudi through 🤗 [Optimum](https://huggingface.co/docs/optimum/habana/usage_guides/stable_diffusion). Follow the [installation](https://docs.habana.ai/en/latest/Installation_Guide/index.html) guide to install the SynapseAI and Gaudi drivers, and then install Optimum Habana:
15+
The Intel Gaudi AI accelerator family includes [Intel Gaudi 1](https://habana.ai/products/gaudi/), [Intel Gaudi 2](https://habana.ai/products/gaudi2/), and [Intel Gaudi 3](https://habana.ai/products/gaudi3/). Each server is equipped with 8 devices, known as Habana Processing Units (HPUs), providing 128GB of memory on Gaudi 3, 96GB on Gaudi 2, and 32GB on the first-gen Gaudi. For more details on the underlying hardware architecture, check out the [Gaudi Architecture](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html) overview.
1616

17-
```bash
18-
python -m pip install --upgrade-strategy eager optimum[habana]
19-
```
20-
21-
To generate images with Stable Diffusion 1 and 2 on Gaudi, you need to instantiate two instances:
22-
23-
- [`~optimum.habana.diffusers.GaudiStableDiffusionPipeline`], a pipeline for text-to-image generation.
24-
- [`~optimum.habana.diffusers.GaudiDDIMScheduler`], a Gaudi-optimized scheduler.
25-
26-
When you initialize the pipeline, you have to specify `use_habana=True` to deploy it on HPUs and to get the fastest possible generation, you should enable **HPU graphs** with `use_hpu_graphs=True`.
17+
Diffusers pipelines can take advantage of HPU acceleration, even if a pipeline hasn't been added to [Optimum for Intel Gaudi](https://huggingface.co/docs/optimum/main/en/habana/index) yet, with the [GPU Migration Toolkit](https://docs.habana.ai/en/latest/PyTorch/PyTorch_Model_Porting/GPU_Migration_Toolkit/GPU_Migration_Toolkit.html).
2718

28-
Finally, specify a [`~optimum.habana.GaudiConfig`] which can be downloaded from the [Habana](https://huggingface.co/Habana) organization on the Hub.
29-
30-
```python
31-
from optimum.habana import GaudiConfig
32-
from optimum.habana.diffusers import GaudiDDIMScheduler, GaudiStableDiffusionPipeline
33-
34-
model_name = "stabilityai/stable-diffusion-2-base"
35-
scheduler = GaudiDDIMScheduler.from_pretrained(model_name, subfolder="scheduler")
36-
pipeline = GaudiStableDiffusionPipeline.from_pretrained(
37-
model_name,
38-
scheduler=scheduler,
39-
use_habana=True,
40-
use_hpu_graphs=True,
41-
gaudi_config="Habana/stable-diffusion-2",
42-
)
43-
```
19+
Call `.to("hpu")` on your pipeline to move it to a HPU device as shown below for Flux:
20+
```py
21+
import torch
22+
from diffusers import DiffusionPipeline
4423

45-
Now you can call the pipeline to generate images by batches from one or several prompts:
24+
pipeline = DiffusionPipeline.from_pretrained("black-forest-labs/FLUX.1-schnell", torch_dtype=torch.bfloat16)
25+
pipeline.to("hpu")
4626

47-
```python
48-
outputs = pipeline(
49-
prompt=[
50-
"High quality photo of an astronaut riding a horse in space",
51-
"Face of a yellow cat, high resolution, sitting on a park bench",
52-
],
53-
num_images_per_prompt=10,
54-
batch_size=4,
55-
)
27+
image = pipeline("An image of a squirrel in Picasso style").images[0]
5628
```
5729

58-
For more information, check out 🤗 Optimum Habana's [documentation](https://huggingface.co/docs/optimum/habana/usage_guides/stable_diffusion) and the [example](https://github.com/huggingface/optimum-habana/tree/main/examples/stable-diffusion) provided in the official GitHub repository.
59-
60-
## Benchmark
61-
62-
We benchmarked Habana's first-generation Gaudi and Gaudi2 with the [Habana/stable-diffusion](https://huggingface.co/Habana/stable-diffusion) and [Habana/stable-diffusion-2](https://huggingface.co/Habana/stable-diffusion-2) Gaudi configurations (mixed precision bf16/fp32) to demonstrate their performance.
63-
64-
For [Stable Diffusion v1.5](https://huggingface.co/stable-diffusion-v1-5/stable-diffusion-v1-5) on 512x512 images:
65-
66-
| | Latency (batch size = 1) | Throughput |
67-
| ---------------------- |:------------------------:|:---------------------------:|
68-
| first-generation Gaudi | 3.80s | 0.308 images/s (batch size = 8) |
69-
| Gaudi2 | 1.33s | 1.081 images/s (batch size = 8) |
70-
71-
For [Stable Diffusion v2.1](https://huggingface.co/stabilityai/stable-diffusion-2-1) on 768x768 images:
72-
73-
| | Latency (batch size = 1) | Throughput |
74-
| ---------------------- |:------------------------:|:-------------------------------:|
75-
| first-generation Gaudi | 10.2s | 0.108 images/s (batch size = 4) |
76-
| Gaudi2 | 3.17s | 0.379 images/s (batch size = 8) |
30+
> [!TIP]
31+
> For Gaudi-optimized diffusion pipeline implementations, we recommend using [Optimum for Intel Gaudi](https://huggingface.co/docs/optimum/main/en/habana/index).

0 commit comments

Comments
 (0)