DIA is an open-source text-to-dialogue model that generates ultra-realistic dialogue with full control over scripts and voices. Built by a small team of two people with no funding, DIA rivals industry-leading models like Notebook LM and 11 Labs Studio.

Compared to 11 Labs and Sesame, DIA demonstrates superior emotional tone, dialogue flow, and nonverbal realism. The pauses, vocal tonality, and emotional expression in DIA's voices are significantly more natural and human-like.

Even this conversation was AI-generated using DIA, showcasing its impressive capabilities. The team behind DIA has made it available for free on GitHub and Hugging Face, allowing anyone to test and use the model.

One of the key advantages of DIA is that it can be run locally without requiring powerful GPUs. The model only needs around 10GB of VRAM, making it accessible to a wide range of users. DIA can generate audio in real-time, with an output of 40 tokens per second on an A4000 GPU.

The team is continuously improving DIA and has plans to further enhance its capabilities. With its open-source nature and impressive performance, DIA is a game-changer in the world of text-to-speech AI. Try it out now and experience the future of AI-generated voices.

Dia is an open-source text-to-dialogue (TTS) model that has been developed by a small team of two people with no funding. Despite the challenges, they have managed to create a model that rivals industry giants like 11 Labs and Sesame.

The key features of Dia include:

• Ultra-Realistic Dialogue: Dia generates dialogue with natural pauses, vocal tonality, and emotional expression, making it sound more like a real human conversation.
• Full Control over Scripts and Voices: Users can have full control over the scripts and voices used in the generated dialogue.
• Open-Source and Free to Use: Dia is available on GitHub and Hugging Face, allowing anyone to access and use the model for free.

When compared to 11 Labs and Sesame, Dia consistently outperforms in terms of dialogue flow, emotional tone, and nonverbal realism. The examples provided in the transcript showcase Dia's superior performance, with the generated dialogue sounding more natural and engaging.

The story behind Dia's development is equally impressive. The founders, who were not initially AI experts, fell in love with Notebook LM's podcast feature and wanted more control over the voices and scripts. After facing challenges with compute power, they were able to leverage Google's TPU access and learn the necessary skills to train a 1.6 billion parameter model.

Dia's success is a testament to the power of open-source AI and the dedication of a small team. By making their model available for free, they are democratizing access to high-quality TTS technology and pushing the boundaries of what is possible with limited resources.

As you could hear from the examples, the Dia TTS model clearly outperforms both 11 Labs and Sesame in terms of emotional tone, dialogue flow, and nonverbal realism.

The Dia model was able to generate much more natural-sounding conversations, with appropriate pauses, vocal tonality, and emotional expression. In contrast, the 11 Labs and Sesame outputs sounded more monotone and robotic, lacking the chemistry and flow of the Dia examples.

Particularly in the more dramatic "fire" scenario, Dia was able to convey a sense of urgency and panic, while 11 Labs and Sesame fell flat, sounding like poor actors reading lines. Dia also handled the rap verse much more convincingly, capturing the rhythm and cadence.

The fact that Dia was able to achieve this level of performance with a tiny team and no funding, compared to the resources of 11 Labs and Sesame, is truly impressive. It's a testament to the rapid progress being made in open-source AI.

Overall, the Dia TTS model appears to be a significant step forward, offering users much more control and realism in their generated dialogue. I would highly recommend trying it out on the Hugging Face demo or downloading the open-source model to experience the difference for yourself.

So I'm going to include this link in the description below that takes you to the free hugging phase space. Here you can write anything you want. Here is the first test. Let's click on generate audio.

Hey Andy how come you're always wearing that Casio watch? You mean this timeless classic, yeah? Does it mean anything to you? Yes, it's great to check the time without getting sucked into my...

Definitely a lot of noise in the beginning. Not that great. Let's try to redo the exact same prompt.

Hey Andy how come you're always wearing that Casio watch? You mean this timeless classic, yeah? Does it mean anything to you? Yes, it's great to check the time without getting sucked into my...

It actually doesn't say the last word here, which is quite annoying, but probably can do some punctuation that will help. Maybe even an exclamation point. But then we also have the generation parameters.

New prompt. Same settings. Generate audio.

It seems like the player is bugging out a little bit, so I'm going to download the file.

Hey guys, what's up? Nothing much, I was just thinking about subscribing to the channel. That's the spirit! Click the like button too while you're at it. Already did. I love checking out new AI tools.

As you can see, the max new tokens audio length is capped at around 372. According to ChatGPT, that's around 2,300 words, which is pretty long. We also have CFG scale, guidance strength, where higher values increase adherence to the text prompt.

Let's try to maximize that, and we'll test the other side in a minute. Temperature randomness - lower values make the output more deterministic, higher values increase randomness. Let's increase the randomness. I feel like it's already too random a little bit. Maybe we can break it.

Top P, nucleus sampling filters vocabulary to most likely tokens cumulatively reaching probability P. Honestly, don't know what that means. Let's go all the way to one. Here is this CFG filter, top K. Top K filter for CFG guidance. Again, maximum speed factor as well. I think it's already quite fast. But what happens if we go all the way, and then I wonder if we can continue making it longer, just to see how a bigger sample size is.

So I'll copy and paste that as well. How does it perform with all the settings maxed out? Let's generate audio.

And my zero GPU daily quota has been exceeded. Luckily, I got a trick up my sleeve. You got one profile? Well, we got two. Just change the settings again. Generate audio. And here we go.

Hey guys, what's up? Nothing much, I was just thinking about subscribing to the channel. That's the spirit! Click the like button too while you're at it. Already did. I love checking out new AI tools. You're one of the real ones. Got any favorites so far? That voice cloning tool blew my mind. I made my cat sound like Morgan Freeman. No way! Please tell me you saved that.

Definitely a lot more randomness, a lot more of that ambient kind of hitting the microphone type thing. So what happens if we go all the way down on all the settings here? What's going to happen? Generate. The audio file doesn't look that great. Let's skip to the end.

Yeah, let's leave some guidance on. I think these are the default settings that they recommend. So I want to keep going with that. But we also have audio prompt as an optional input. So I really want to try that with speaker 1. So I'll click on this little microphone icon in the bottom and then click on record.

Hey guys, what's up? Seems like we're getting an error with the audio. Hey guys, what's up? Let's try again. Yeah, we're definitely getting an error every time. Oh, there we go. Just try to spam it a little bit, and then the audio prompt is going to work. I got errors a couple of times. Seems the player is bugging again. Maybe they have a lot of usage right now 'cause it's very popular. I'll just download the file and let's listen.

Oh, so it sounds like it's actually just skips my first sentence there. Let's check it out again.

Nothing much, I was just thinking about subscribing to the channel. That's the spirit! Click the like button too while you're at it. Already did. I love checking out new AI tools. You're one of the real ones. Got any favorites so far? That voice cloning tool blew my mind. I made my cat sound like Morgan Freeman. No way! Please tell me you saved that.

I guess not really that great on voice cloning. But there is one more thing, and that is, can we generate without speaker two, like for example, if we were to read an entire video script, or is it only for two people speaking? It seems like that was what they intended it for.

So I'm just going to use ChatGPT a little bit. Pop it in here. I'm not going to use an audio prompt here. I'm just going to generate. Oh, I used my free quota again. I just got to know. Let's log out. Sign up. Confirm you're a human. AI and ND3. Create account. Confirm email address. And we're back.

Paste the script that only has one speaker. And generate. Let's see. Can we do one speaker?

Hey, welcome back to the channel. If you're into AI tools, automation, and a bit of chaos, we're in the right place. Today, I've got something wild to show you. It's fast, it's powerful, and you won't believe what it can do. So grab your coffee, hit that like button, and let's dive in.

That was way too fast. Really wish that I had some pausing in between sentences there. It sounded really good, but not amazing. Let's go down on the speed factor to hopefully fix that. Also, I want it to not be a female voice. So I'm going to drop an audio file in here.

Hey, welcome back to the channel. Can it clone my voice and do it well? Generate. Error. Generate. Hey, welcome back to the channel. Yeah, it definitely doesn't like the audio prompt. I tried it like five times now. Let's just change the speed. Generate.

Hey, welcome back to the channel. If you're into AI tools, automation, and a bit of chaos, we're in the right place. Today, I've got something wild to show you. It's fast, it's powerful, and you won't believe what it can do. So grab your coffee, hit that like button, and let's dive.

Did it just call me daddy? Today, I've got something wild to show you. It's sure, I'll take that. We could probably fix that in the temperature settings to make it less random, but so far, I'm actually really impressed.

And what's even crazier is that you can download this model for free and run it on your computer. As you can see, they have a GitHub page where they have everything open source, open weight, so you can get access to it. All you need to get started is to clone the repo, run Gradio UI, or if you don't have the UI pre-installed, you can run this.

Note that the model was not fine-tuned on a specific voice. Hence, you'll get different voices every time you run the model. You can keep speaker consistency by either adding an audio prompt, a guide coming very soon, try it with a second example on Gradio for now, or fixing the seed.

The features you can generate dialogue via S1 or S2, generate nonverbal like laughs, coughs, etc. And they have voice cloning. They have a voice clone PI file for more information where you can upload the audio you want to clone and place its transcript before your script. Make sure the transcript follows the required format. The model will then output only the content of your script.

And if you're wondering how fast it is and if you can run it on enterprise GPUs, Dia can generate audio in real-time. On older GPUs, inference time will be slower. For reference, on an A4,000 GPU, Dia roughly generates 40 tokens a second or 86 tokens equals 1 second of audio. Torch compile will increase speeds for supported GPUs.

Now, this is the big kicker. You only need around 10 GB of VRAM to run it. My computer has 16 GB of VRAM, which makes it easy to run. They also have plans for the future to do even better work. As you can see, their popularity is going through the roof, and it's amazing to see.

I love that we have this new model now because AI voice is really cool in content creation, as well as if you have training material that you want to have in multiple languages, or you don't have a speaker, you can add that on top of your PowerPoint presentations, for example. Also, I use a Chrome extension to read out loud websites that I'm going through, and it seems like this would be a perfect fit for a web reading app. Not to mention customer support agents that can call on the phone for you, scheduling appointments, or taking care of general customer support. We need better voice to do that, and now it seems like it's here, just a little bit on the baby side yet.

So if you want to test it out, there are links in the description down below. And if you're considering using AI voice in your business, like the use cases I just talked about, you can check out AIA our community in the description down below to automate your work. Thank you so much for watching, and I'll see you in the next one.

Dia is an open-source text-to-dialogue (TTS) model that offers several impressive features and capabilities:

1. Full Control over Scripts and Voices: Dia provides users with complete control over the scripts and voices used in the generated audio. This allows for greater flexibility and customization.

2. Ultra-Realistic Dialogue Generation: Dia is capable of generating highly realistic and natural-sounding dialogue, with seamless flow and emotional tone between characters.

3. Nonverbal Realism: The model can generate nonverbal cues such as laughs, coughs, and other sounds that add to the overall realism of the dialogue.

4. Real-Time Audio Generation: Dia can generate audio in real-time, even on older GPUs, with a speed of around 40 tokens per second on an A4,000 GPU.

5. Low Hardware Requirements: The model only requires around 10GB of VRAM to run, making it accessible to a wide range of users without the need for high-end hardware.

6. Open-Source and Free to Use: Dia is an open-weights model, meaning it is open-source and available for free. Users can access the model on GitHub or Hugging Face and run it on their own computers.

7. Voice Cloning Capabilities: Dia includes a voice cloning feature that allows users to upload audio and transcript data to generate content in the style of the provided voice.

8. Customizable Generation Parameters: Dia offers various generation parameters, such as temperature, top-k, and top-p, that users can adjust to fine-tune the output and achieve their desired results.

Overall, Dia's impressive features and capabilities make it a highly compelling open-source TTS model that rivals commercial offerings from companies like Google and 11 Labs, despite being developed by a small team with limited resources.

The DIA text-to-dialogue model is an impressive open-source AI tool that can generate highly realistic and engaging conversations. Developed by a small team with no funding, DIA outperforms well-funded models like 11 Labs and Sesame in terms of emotional tone, dialogue flow, and nonverbal realism.

The ability to generate audio with full control over scripts and voices is a game-changer for content creators, language learning, customer support, and more. The model can be run on relatively modest hardware, making it accessible to a wide range of users.

While the model is not yet perfect, the rapid progress in open-source AI is truly remarkable. DIA's performance is a testament to the power of collaboration and the democratization of AI technology. As the team continues to improve the model, it will undoubtedly become an increasingly valuable tool for a wide range of applications.

Overall, DIA is an exciting development that showcases the potential of open-source AI to disrupt traditional industries and empower individuals and small teams to create high-quality content and experiences.