I just got sent this as a sponsor pitch for an AI service. This is creepy as fuck…
On top of the (not to be understated) disturbing part of training AI voices based on creators w/o prior consent, and ANYTHING being done using AI trained voices at all, BUT
This is yet another dive off the deep end of parasocial toxicity. YouTubers/streamers are NOT your friend. We do not need to enable people who want to feel this way by giving them a “private chat” with an AI avatar of us, least of all while trying to be MORE convincing that the AI is actually us.
This is disturbing, and completely and utterly unhealthy for any viewer who would engage with this - PLUS it’s a liability risk, as the AI is inherently going to misrepresent the creator with tons of various questions. Huge can of worms there.
This shit has to stop. The cult of personality and hyperbolic obsession with personal branding is tanking social skills, the usability of the internet, technical literacy, and breeding/encouraging incredibly unhealthy relationship expectations in viewers. It’s not okay.
Even if people had prior consent to be apart of this, I still don’t trust any LLM outside of "give me an example of this very broad or simple thing* since hallucinations are very prevalent still. I don’t know how we can fix things given the current state of the technology and how slow governments move.
Right, like there’s no scenario in which this would be an accurate presentation.
And even if it was, I don’t want people chatting with a fake version of me and thinking they’re conversing with me.
Additionally, AI generated endorsements are just gross. The fact that the AI bot mentioned specific games in the second answer is the really troubling part for me because corporations (or anybody really) can get any personality to say anything they want and manipulate it to their own benefit. We’ve seen it with MrBeast scams on social media and now we have an example of pewdiepie endorsing/promoting League of Legends and Fortnite (which the AI could easily replace with whatever game titles the publishers paid the platform to promote).
This is so not OK. The unfortunate matter is that this is only the beginning for more pervasive uses of the technology. I’m not sure what’s worse, is that people will assume they are getting legitimate ‘advice’ or creators won’t be able to keep up with devs creating nonsense like this without their permission.
Apparently this company or a similar one is going around claiming to be supported by big YouTubers like MKBHD, who have never agreed to it, too. Siiiiiigh
