DreamTalk
ali-vilab/dreamtalkReviewed by AI
Checked by RepoPicks Radar on
What is DreamTalk?
DreamTalk is the code from an Alibaba research paper. Given one front-facing photo and an audio file, it generates a video of that face speaking, copying the speaking style and head movement of a reference clip. It runs from the command line and the result is a 256x256 video. The model files are no longer offered for download: the README asks you to email the author and says the method is for academic research only. You build a conda environment with Python 3.7, PyTorch, ffmpeg and dlib; a graphics card is advised, with a slower CPU option.
DreamTalk is free and open source (MIT) and can replace HeyGen and D-ID.
How easy is it to install DreamTalk?
E · developers only · Windows
- Windows E
- macOS E
- Linux E
The same E everywhere, and Linux is the realistic route: you create a conda environment with Python 3.7, install PyTorch 1.8 with CUDA 11.1 plus dlib and ffmpeg, then run a long python command with paths to audio, a 3DMM style file and a pose file. On top of that the model files are not downloadable, so even a developer cannot get a result without first emailing the author.
Coming soon: install in 1 click with the RepoPicks app.
DreamTalk on GitHub
- Stars
- 1,788on Nov 10, 2026
- Downloads
- —all releases
- Growth
- —no gain in the last reading
Stars over time
Facts
- Runs on
- Windows, macOS, Linux
- You need to install
- Python, FFmpeg
- To build from source
- Python, FFmpeg
- Disk space
- 160 MB–420 MB · estimate
- Accounts or keys
- None
- Cost
- Free · No price. The code is MIT, but the README's disclaimer says the method is for "RESEARCH/NON-COMMERCIAL USE ONLY".
- License
- MIT (Permissive)
- Last change on GitHub
- Jan 6, 2024
DreamTalk risk signals (6)
- AttentionDependencies with known vulnerabilities
Pinned dependencies with known vulnerabilities (OSV): scipy: PYSEC-2023-102, PYSEC-2023-114; scikit-learn: GHSA-jw8x-6495-233v, PYSEC-2024-110; pillow: GHSA-22hg-r758-rrw9, GHSA-24rc-8hj2-v98c, GHSA-254f-3936-288h; numpy: GHSA-fpfv-jqm9-f5jm; opencv-python: GHSA-qr4w-53vh-m672, PYSEC-2023-183
automatic check
- Attentionmodel files not publicly available
README "Download Checkpoints": "In light of the social impact, we have ceased public download access to checkpoints. If you want to obtain the checkpoints, please request it by emailing [email protected]" — without them the code produces nothing.
AI review
- Attentionresearch and non commercial use only
README Disclaimer: "This method is intended for RESEARCH/NON-COMMERCIAL USE ONLY", and the email request "implies your consent to use the provided method solely for academic research purposes", even though the code file says MIT.
AI review
- Attentionmakes a real persons face say anything
The tool animates "out-of-domain portraits" from any audio, including "speech in multiple languages", so one photo plus a voice recording produces a video of a person saying words they never said; the authors stopped the model download citing "social impact".
AI review
- Infoabandoned and built on end of life python
Last commit 2024-01-06 (1034 days before the check) with 46 open issues, and the install line is "conda create -n dreamtalk python=3.7.0" with PyTorch 1.8 — versions that are no longer supported.
AI review
- Infoextra steps to use your own video as reference
README: style and pose files are "3DMM parameter sequences extracted from reference videos", and to use your own you must follow PIRenderer and the FOMM video preprocessing scripts, with video at 25 FPS cropped to 256x256.
AI review
Evidence is shown in its original language.
What Radar could not check about DreamTalk
- Whether the author still answers the email and sends the model files
- How the result looks with your own photo and voice, and whether the mouth really matches
- Whether the Python 3.7 and PyTorch 1.8 environment still installs on a current computer
- How long one video takes on a CPU, the only option without an NVIDIA card
- Real disk use, since the model files are not published and cannot be measured