The Unintended Consequences of Regulating Virtual Influencers

The virtual influencer market has grown rapidly in recent years, driven by the rise of AI-generated content and social media platforms’ increasing acceptance of synthetic personalities. However, as global changes to how AI content is handled on these platforms take shape, creators like Clarissa Mansbridge are worried that their business model could be upended.

Mansbridge’s experience highlights the adaptability of creatives in the digital age. A former behind-the-scenes entertainment professional, she turned to building virtual influencers after her son’s birth left her with limited time for traditional work. With the help of AI image-generation tools, she created Mia Metaverse, a “humbly wealthy, wholesome” influencer who has brought in around $6,000 a month through brand collaborations.

The EU’s Artificial Intelligence Act, which requires AI-generated or -manipulated promotional content to be clearly labeled, has left many creators feeling uncertain about their livelihoods. This legislation is a well-intentioned effort to foster “trustworthy AI” in Europe, but its language and scope are broad enough to leave plenty of room for interpretation – and confusion.

Article 50 of the Act requires providers and deployers of AI systems to disclose when audio, image, video, or text content has been artificially generated or manipulated. However, it’s unclear how this will be managed in practice. Kai Zenner, digital policy adviser to Axel Voss, notes that the Act was drafted too early and abstractly. “ChatGPT had only just come out, and there was scant knowledge among MEPs and parliamentarians about what we were regulating for,” he explains.

This lack of understanding is a major concern for virtual influencers like Mansbridge, who rely on AI tools to create content that meets the evolving demands of social media platforms. As more legislation is likely to add to the uncertainty, creators are starting to feel the pinch. “There are several revisions coming later this year and next that will force influencers not only to label their content but provide transparency on where they come from, what they want to do, and how they are financed,” explains Zenner.

However, some experts argue that the current crackdown on AI content is swinging at a problem with a sledgehammer. Matt Navarra, social media consultant and author of the weekly Geekout Newsletter, notes that “the platforms are tweaking their algorithms to push AI-generated content back down, but none of these systems are precise enough.” He believes human creators will get caught out, even if they used AI for minor reasons, like fixing up their grammar.

Research has shown that 60 percent of creators admit to breaking industry rules due to mislabeling. But with the stakes getting higher, it’s unclear whether these measures will be enough to stop the spread of synthetic spam. As Lionel Metge, a hip-hop artist and producer who created an AI artist named Zairo, notes, “the new detection systems can identify AI-generated content as soon as it’s published.”

This shift in how platforms handle synthetic content is already having an impact on creators like him, who are struggling to make a living from their work. In the end, the unintended consequences of regulating virtual influencers will depend on how policymakers and platform owners navigate the complexities of AI-generated content. Will they be able to create clear guidelines that balance the needs of human creators with the demands of synthetic personalities? Or will the regulations drive this emerging industry underground?

The future of virtual influencers hangs in the balance, caught between the promise of AI-generated content and the risks of mislabeling and detection. As Mansbridge herself notes, “the regulatory maze is a scary place to be.”