update stepvideo, support mindspore with 2.6.0/2.7.0 #1374
Conversation
Summary of ChangesHello @zhanghuiyao, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request updates the Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the Step-Video-T2V example, adding support for MindSpore versions 2.6.0 and 2.7.0, and enhances documentation and security. The changes are beneficial, but I've identified a few areas for improvement in the README for clarity and correctness, as well as a minor but potentially confusing comment error in the server code. My review includes suggestions to address these points.
| ) | ||
| # ==================== We comment the follow code, because security risks. ====================== | ||
| # =========== You need to manually decomment it before running. ============= | ||
| # ======== The first place. ========== |
There was a problem hiding this comment.
This comment appears to be a copy-paste from the VAEapi class above. To avoid confusion for users following the README, this should be identified as the "second place" to uncomment code.
| # ======== The first place. ========== | |
| # ======== The second place. ========== |
| 1. [Introduction](#1-introduction) | ||
| 2. [Model Download](#2-model-download) | ||
| 3. [Model Usage](#3-model-usage) | ||
| 4. [Acknowledgement](#4-ackownledgement) |
There was a problem hiding this comment.
There's a typo in "Acknowledgement". It's spelled "ackownledgement" in the link target. This should be corrected to "acknowledgement" to match the section header at the end of the file and fix the broken link.
| 4. [Acknowledgement](#4-ackownledgement) | |
| 4. [Acknowledgement](#4-acknowledgement) |
| **Step-Video-T2V**, a state-of-the-art (SoTA) text-to-video pre-trained model with 30 billion parameters and the capability to generate videos up to 204 frames. To enhance efficiency, stepfun-ai propose a deep compression VAE for videos, achieving 16x16 spatial and 8x temporal compression ratios. Direct Preference Optimization (DPO) is applied in the final stage to further enhance the visual quality of the generated videos. Step-Video-T2V's performance is evaluated on a novel video generation benchmark, **Step-Video-T2V-Eval**, demonstrating its SoTA text-to-video quality compared to both open-source and commercial engines. | ||
| **[Step-Video-T2V](https://github.com/stepfun-ai/Step-Video-T2V)**, a state-of-the-art (SoTA) text-to-video pre-trained model with 30 billion parameters and the capability to generate videos up to 204 frames. To enhance efficiency, stepfun-ai propose a deep compression VAE for videos, achieving 16x16 spatial and 8x temporal compression ratios. Direct Preference Optimization (DPO) is applied in the final stage to further enhance the visual quality of the generated videos. Step-Video-T2V's performance is evaluated on a novel video generation benchmark, **Step-Video-T2V-Eval**, demonstrating its SoTA text-to-video quality compared to both open-source and commercial engines. | ||
|
|
||
| ## 2. Weight Download |
There was a problem hiding this comment.
This section title was changed from "Model Download" to "Weight Download". Please also update the corresponding entry in the Table of Contents (line 21) to maintain consistency and ensure the link works correctly.
| Step-Video-T2V exhibits robust performance in inference settings, consistently generating high-fidelity and dynamic videos. However, our experiments reveal that variations in inference hyperparameters can have a substantial effect on the trade-off between video fidelity and dynamics. To achieve optimal results, we recommend the following best practices for tuning inference parameters: | ||
| ### 📖 3.4 Inference Scripts (Multi-Cards Parallel Deployment) | ||
|
|
||
| - ⚠️ **First**, confirm the security risks from `pickle`, then manually decomment **two places** of code in file `api/call_remote_server.py` that involve security risks. |
There was a problem hiding this comment.
The term "decomment" is non-standard. For better clarity, please use the standard term "uncomment".
| - ⚠️ **First**, confirm the security risks from `pickle`, then manually decomment **two places** of code in file `api/call_remote_server.py` that involve security risks. | |
| - ⚠️ **First**, confirm the security risks from `pickle`, then manually uncomment **two places** of code in file `api/call_remote_server.py` that involve security risks. |
|
|
||
| raise NotImplementedError( | ||
| "There are some security risks from pickle here. \n" | ||
| "You need to confirm it and manually decomment the code above before running them." |
There was a problem hiding this comment.
The term "decomment" is non-standard. For better clarity, please use the standard term "uncomment".
| "You need to confirm it and manually decomment the code above before running them." | |
| "You need to confirm it and manually uncomment the code above before running them." |
3 participants
What does this PR do?
update stepvideo, support mindspore with 2.6.0/2.7.0
Before submitting
What's New. Here are thedocumentation guidelines
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@xxx