AI Secrets is your weekly update on how AI is changing our everyday lives. Our experts will keep it clear and simple, so you can stay ahead of the game. This week we are focussing on the 'Curse of Recursion'.Share this with anyone you want to keep up to date.
🎵 Daisy, Daisy, give me your content please 🎶
Far away from the noisy debate over hypothetical future AIs - you know, the ones who will either destroy humanity or rescue it, depending on who you ask - a less dramatic but more urgent discussion has been bubbling away: what happens when you train AIs on their own outputs? This is a serious concern in the AI community, because large language models (LLMs) are churning out machine-generated content at an astonishing rate, and it’s becoming increasingly hard to filter the human stuff from the AI facsimiles. Last week, the answer was supplied by researchers at Cambridge University, and it seems to be good news; old-fashioned human creativity is going to be in demand for a while yet, and may even be about to become a lot more valuable.
The paper - entitled The Curse of Recursion: Training on Generated Data Makes Models Forget - confirms what many suspected: the outcome of digital inbreeding is not pretty. Training a large language model on machine-generated content leads directly to what the researchers call “model collapse”. This looks a little like the demise of HAL 9000 in 2001: A Space Odyssey; the AI forgets almost all of its training data, and is effectively rendered useless.
This means that unless something fundamental changes with - or replaces - LLMs, human content is going to continue to be the gold standard for training AI models. Cambridge researchers have noted that this is already affecting demand in the industry, citing a sudden spike in requests from AI firms for content from the Internet Archive, with its huge and reliably AI-free library of human-generated media. One of the strangest implications of this research is that as the Internet fills with LLM-generated cruft, it will become harder to effectively train AI models by simply scraping the web, and the firms who mined that precious data first may well have a substantial advantage in the future.
Speaking of the future, what of humanity’s long-term prospects? The ongoing debate about our imminent enslavement or liberation continues to rage, and this week the loudest voices have been optimistic. Yann “Godfather of AI “ LeCun has been pushing back against the critics (see AI Roundup, below) while respected web pioneer and venture capitalist Marc Andreessen has weighed into the debate, declaring that AI will “save the world”, while lambasting AI doomers as “cultists” and “bootleggers”. Is he right? To help you decide, don’t miss our detailed article examining the claims of these AI evangelists!
Beta Testers Wanted!!
SPYGAMES is the thrilling new experience where you’ll jump, climb, throw and dodge in fun immersive challenges developed with CIA experts to stretch your physical and mental agility. ‍ Click the link below to get FREE access as a beta tester.
SPYGAMES is coming to SPYSCAPE in Manhattan, Summer 2023
Meta’s Yann LeCun - one of the three 'Godfathers of AI' - has poured scorn on claims that human-level AI is a threat that needs urgent regulation, claiming we are not even close to “dog-level intelligence”
Ars Technica’s AI reporter purchased a copy of the 22 volume World Book Encyclopedia - the only English language encyclopedia that’s still available in print - to see if the old ways were better, or merely heavier.
Paul McCartney has announced AI technology is being used to clean up and flesh out poor quality demo cassettes of one of John Lennon’s lost compositions, believed to be 1978’s Now and Then.
Meta is the latest firm to release a text-to-music generative AI tool; MusicGen takes existing tracks and modifies them through prompts. Are you ready for Ravel’s hip-hop Bolero?
Former President Trump has been hitting the AI headlines again, as has 2024 Presidential nomination rival Ron DeSantis, with both sides accused of using deepfake trickery in their campaign advertising.
In 'AI applications nobody asked for' news, a congregation of over 300 parishioners in Germany attended a 40 minute sermon where the preachers were not humans, but AI avatars.
