Former Snap AI chief launches Higgsfield to take on OpenAI’s Sora video generator

OpenAI captivated the tech world a few months ago with a generative AI model, Sora, that turns scene descriptions into original videos – no cameras or film crew required. But Sora has so far been tightly closed, and the company appears to be aimed at well-funded creatives like Hollywood directors — not necessarily amateurs or small-time marketers.

Alex Mashrabov, the former head of generative AI at Snap, sensed an opportunity. So he launched Higgsfield AI, an AI-powered video creation and editing platform designed for more personalized and tailored applications.

Powered by a custom text-to-video model, Higgsfield’s first app, Diffuse, can generate videos from scratch or take a selfie and generate a clip featuring that person.

“Our target audience is creators of all types,” Mashhabov told TechCrunch in an interview, “from regular users who want to create fun content with their friends to social content creators looking to try a new content format, in going through social media marketers who want their brand to stand out.”

Mashrabov came to Snap through AI Factory, his previous startup, which Snap acquired in 2020 for $166 million. At Snap, Mashhabov helped create products such as AR effects and filters for Snapchat, including Cameos, as well as Snapchat’s controversial MyAI sculpin.

Higgsfield – which Mashhabov co-launched several months ago with Yerzat Dulat, an AI researcher specializing in generative video – offers a curated set of pre-generated clips, a tool for downloading reference media (that’s i.e. images and videos) and a quick editor that allows users to describe the characters, actions and scenes they want to depict. With Diffuse, users can insert themselves directly into an AI-generated scene or ask their digital image to imitate elements, like dance moves, captured in other videos.

“Our model supports very realistic movements and expressions,” Mashrabov said. “We are pioneering consumer-facing ‘global templates’, which will allow us to create and edit best-in-class videos with a high level of control. »

Higgsfield isn’t the only generative video startup taking on OpenAI. Runway was one of the first to market and its tools continue to improve. There’s also Haiper, which has the backing of two DeepMind alumni and more than $13 million in venture capital.

Mashhabov says Diffuse will stand out with its mobile-first, social-first go-to-market strategy.

“By prioritizing iOS and Android apps over desktop workflows, we enable creators to create compelling social media content anytime, anywhere,” Mashhabov said. “Indeed, by leveraging mobile, we are able to prioritize ease of use and user-friendly features from day one. »

Higgsfield also operates lean. Mashrabov says the generative models underlying the platform were developed by a team of 16 people in less than nine months and trained on a cluster of 32 GPUs. (32 GPUs may seem like a lot, but given that OpenAI uses tens of thousands of them, it’s not Really.) And Higgsfield has only raised $8 million to date, most of which came from a recent round of seed funding led by Menlo Ventures.

To stay ahead of competitors, Higgsfield plans to invest in creating an improved video editor that will allow users to edit characters and objects in videos, and in training more video generation models. powerful specifically for social media use cases. In fact, Mashrabov sees social media – and social media marketing – as Higgsfield’s main lucrative niche.

Although Diffuse is currently free, Mashrabov envisions a future in which marketers pay some sort of fee or subscription for premium features, or for volume or large-scale campaigns.

“We believe Higgsfield unlocks an incredible level of realism and content production use cases for social media marketers,” he said. “CMOs and creative directors constantly tell us they need to optimize content production budgets and shorten timelines while still delivering impactful content. We therefore believe that video generative AI solutions will be a key solution to help them achieve this.

Of course, Higgsfield is not immune to the broader challenges facing generative AI startups.

It is well established that generative AI models like those that power Diffuse can “regurgitate” training data. Why is this problematic? Well, if the models were trained on copyrighted content without permission or licensing agreements, users of those models could unwittingly generate copyright infringing work, exposing them to legal action .

Mashhabov did not reveal the source of Higgsfield’s training data (other than saying it comes from “multiple publicly available locations”), nor did he say whether Higgsfield would retain user data to train future models, which might not suit some. Company customers. He noted that Diffuse users can request that their data be deleted at any time through the app.

Digital “cloning” platforms like Higgsfield are also ripe for abuse, as the uncontrollable spread of deepfakes on social media has shown in recent months.

Along the same lines, Higgsfield could make it easier to steal content from creators. For example, simply uploading a video of a person’s choreography will generate a video of themselves performing that same choreography.

I asked Mashrabov what safeguards or protections Higgsfield might use to try to prevent abuse, and while he wouldn’t go into detail, he said the platform uses a mix of automated moderation and manual.

“We decided to roll out the product gradually and test it in some markets first, so that we can monitor the risks of abuse and evolve the product if necessary,” Mashhabov added.

We’ll have to wait and see how well this works in practice.