AI VTuber Neuro-sama just conquered osu and she won’t stop there

Published: 2022-12-29T03:44:51

Updated: 2022-12-29T03:45:08

Artificial intelligence (AI) is all over the internet today, acting as art generators, chatbots, and other forms of communication. Neuro-sama is one such example, with the AI ​​VTuber conquering Twitch and osu! at the same time. But what does it say about the future of gaming, streaming and its relationship with AI?

On December 19, Neuro-sama was rebooted. The AI ​​project, created by ‘Vedal987‘ originally in 2019, started the task of making the rhythm game osu! to master.

At the time, it was just a standalone program with no personality beyond the name. There was no character, and while it was streamed on Twitch to the interest of the circle click title’s devoted fandom, it hung out in its own niche.

Article continues after ad

Fast-forward four years, with AI playing a more predominant role in the online space, Neuro-sama is grabbing attention.

It is more than a socially unconscious program. Neuro-sama has a personality and a VTuber model, all created by Vedal and modified through AI. Using one of the standard VTube Studio rigs, Neuro is now on Twitch all day, playing osu beatmaps and even interacting with chat on the hottest topics of the day.

Clips of her talking about xQc, flipping back to Vedal and more have already gone viral. She’s conscious in a way ⁠ – a clip of her telling her to take a shower caused a lot of laughs. How does an AI take a shower?

Article continues after ad

However, in just 10 days, Neuro-sama has become the talk of the town in both osu! and VTubing circles for various reasons. She has already beaten the current top osu! player in the world, the Australian teen ‘mrekk’, in a tournament style 1v1. As for her Twitch viewership, she has averaged close to 2,000 viewers since the reboot, including peaks above 4,000.

Vedal is a self-described “student with a particular interest in AI”. Speaking to Dexerto, they said the idea to revive the Neuro-sama project as an AI VTuber came after talking to a friend about using chat models for streamers.

“One of my friends had the idea of ​​making an AI streamer using a big language model, and I thought it had absolute potential,” they said.

Article continues after ad

Large Language Models (LLMs) are “pre-trained on a large amount of text from the Internet, so they have a lot of knowledge,” Vedal explains. This large database allows for a better understanding of human communication ⁠— it can search trillions of responses to see how to answer specific questions or communicate in a specific way.

Vedal labeled Neuro-sama as a fun experiment. In VTubeing, however, the mere mention of AI carries a lot of baggage.

Unlike the AI ​​art programs that infiltrate space, which artists say exploit their work, Vedal’s design doesn’t really break that line. He spent a lot of time refining Neuro’s personality before sending her live on Twitch. The model he uses is fair use of VTube Studio. It’s all his work, or copyright free.

Article continues after ad

“Later I came up with the idea of ​​combining it with my osu! AI that I was already working to improve, and I especially revived it this year because that’s when I finally finished making it,” he continued.

“The VTuber aspect was an obvious choice to combine with AI because it’s much easier to control an avatar than it is to generate a video of a real person.”

AI has already triumphed in other games. AlphaStar, a program designed to beat the world’s top Starcraft 2 players, knocked out some of the game’s greats in 2018. It used a “deep neural network” to get input data from what’s on screen and then execute an in-game command, learning game theory and strategies by playing random players up the ladder.

Article continues after ad

Early reviews noted that while AlphaStar had a lower APM than some of the top players, it was more accurate and also didn’t make as many unnecessary inputs.

The same was achieved in Dota 2 with OpenAI Five. It took several months for it to consistently beat professional teams, but after two years, the AI ​​managed to beat two-time The International champion OG in April 2019.

Neuro-sama conquered osu! much faster: “It took Neuro-sama 10 days to become arguably better than the top human player at osu! since she defeated mrekk in the 1v1,” said Vedal. “AI definitely has the power to outperform humans, even in really complex games.”

All this begs the question: where does this lead to for gaming and VTubing’s relationship with artificial intelligence? Will it remain a “fun experiment”, or will our future entertainers all emerge from programs rather than a human being behind the screen?

That is as much a matter of the technology as it is of what the public wants.

“I think there might be a market for AI streamers because they have some features that streamers can’t have ⁠ – they can stream 24/7, can be better than humans at games, can theoretically read any chat message,” they said.

“I have no idea if AI streamers have a place in the future of streaming or if streamers should be concerned about their jobs. I suppose most of the attention right now is because of the novelty of it, but we’ll see.”

And as for Neuro-sama, the project doesn’t end there, as Vedal plans to expand its reach even further: “I plan to expand beyond osu! and let her play more games, and improve her intelligence (memory) and understanding of Twitch chat.

Leave a Comment