ElevenLabs launches AI-powered tool to generate sound effects

Voice cloning startup ElevenLabs today introduced a new tool that lets users generate sound effects via prompts. The company first announced the project in February.

The tool, available to all users starting today, allows users to enter prompts such as “waves crashing,” “metal rattling,” “birds chirping,” and “car engine race” to generate sound clips.

The sound effects tool can also generate instrumental music videos up to 22 seconds long with prompts such as guitar loops, jazz saxophone solos, and techno music loops.

Free users get 10,000 character generations per month – one soundbyte generation takes around 150 characters per request. Essentially, free tier users can generate almost 60 sound effects per month. They must also attribute the sound to “elevenlabs.io” in the title when posting any content containing the audio clip.

ElevenLabs said it used Shutterstock’s audio library of licensed tracks as a tool to train its model. The company added that users who tried the tool during the alpha testing phase included video game developers, film producers, social media content creators and marketers.

The startup noted that the tool does not allow the generation of sounds via prompts that violate its policy on prohibited content and uses, which includes topics such as self-harm, threats to child safety, and fraud.

Although only a few companies and startups are working on AI-based sound generation, ElevenLabs might find the music generation space crowded. Harmonai, supported by Stability AI, released Dance Diffusion; Google worked on MusicLM; OpenAI has Jukebox; and Meta has its AudioCraft model. TikTok and Adobe have also been experimenting with their own AI-powered generative music creation tools.

We are launching a newsletter on AI! Register here to start receiving it in your inboxes on June 5.