HierVST Voice Cloning | NVIDIA Perfusion | Meta's AudioCraft

08/03/23 • 11 min

Welcome to AI Daily! Join hosts Farb, Ethan, and Conner as they explore three groundbreaking AI stories First up, HierVST Voice Cloning - Experience zero-shot voice cloning with impressive accuracy using just one audio clip. Next, NVIDIA Perfusion - a small, powerful personalization model for text images, using key locking to maintain consistency. Lastly, Meta's AudioCraft - the fusion of music generation, audio generation, and codecs into one open-source code base, creating high-fidelity outputs.

Quick Points

1️⃣ HierVST Voice Cloning

Zero-shot voice cloning system achieves accurate outputs with just one audio clip.

Uses hierarchical models for long and short-term generation understanding.

Potential challenges in handling longer clips and need for further fine-tuning.

2️⃣ NVIDIA Perfusion

Personalization model for text images with key locking for subject consistency.

Only 100 kilobytes, trains in four minutes, and outperforms other models.

Open-source codebase, but may need improvements for human subjects.

3️⃣ Meta’s AudioCraft

Audio generation, music gen, and codecs combined into an open-source codebase.

High-fidelity outputs, 30 seconds of sounds, compressing audio files efficiently.