Open-Source Audio-Driven Portrait Animation Projects
1. Sonic
- Description: An open-source audio-driven portrait animation project by Tencent, suitable for long video generation. It performs well in lip synchronization, expressions, and head movements.
- Demo: Details and demonstration
2. AniPortrait
- Description: An open-source AI digital human tool that can generate animated-style dynamic videos based on user-uploaded photos and corresponding audio files.
- Demo: Details and demonstration
3. JoyHallo
- Description: An open-source Mandarin digital human project by JD, featuring smooth and natural lip expression.
- Demo: Details and demonstration
4. TANGO
- Description: An open-source lip-sync model specifically designed to synchronize character dialogue and gestures.
- Demo: Details and demonstration
5. EchoMimicV2
- Description: An open-source digital human video generation project by Alipay, generating half-body human animations compared to the V1 version.
- Demo: Details and demonstration
6. Loopy
- Description: Released by ByteDance, Loopy controls the expressions and actions of a character avatar through audio but is not open source.
- Demo: Details and demonstration
7. OmniHuman-1
- Description: An end-to-end multimodal conditional human video generation framework launched by ByteDance. It can generate human videos based on a single human image and motion signals but is not open source.
- Demo: Details and demonstration
8. PersonaTalk
- Description: An audio-driven visual dubbing framework by ByteDance, creating lip-sync videos with dubbing while preserving personal speaking style and facial details. Not open source.
- Demo: Details and demonstration
9. JoyVASA
- Description: An open-source audio-generated portrait and animal image animation project by JD Health and Zhejiang University.
- Demo: Details and demonstration
10. FLOAT
- Description: An audio-driven talking portrait video generation tool that enhances speech-driven emotional movements. Currently not open source.
- Demo: Details and demonstration
11. INFP
- Description: An audio-driven interactive head generation tool launched by ByteDance, enabling real-time voice communication between two digital humans. Not open source.
- Demo: Details and demonstration
12. Hallo3
- Description: The third version of Hallo, open-sourced by Baidu, generates videos of characters speaking the corresponding voice when inputting audio and character images, including lip sync, expressions, and head movements.
- Demo: Details and demonstration
