Your weekly update on how AI is changing our lives. Our experts keep it clear and simple, so you can stay ahead of the game. Share this with anyone you want to keep up to date.
ChatGPT’s Many Prime Days Baffle Researchers 😕 ➗
How good are you at spotting prime numbers? Do you think you’d be better than an AI?
The answer to that question may surprise you, because it depends when you ask. According to an academic paper published last week entitled “How is ChatGPT’s Behavior Changing Over Time”, if you tested yourself against GPT4 when it was first launched in March, your odds would have been awful; the big beast LLM had a 97.6% accuracy rate when asked to check if a number was prime, versus GPT3.5’s far more human 7.4% accuracy. Ask the same question three months later, however, and the results are completely flipped; GPT3.5 was suddenly the prime-crunching expert (86.8%), while GPT4 could only scrape together a 2.4% success rate!
The paper’s surprising findings poured fuel on existing rumors that OpenAI have been throttling GPT4’s capabilities to help cope with high demand, an accusation that OpenAI vehemently denies. Whether true or not, it doesn’t really work as an explanation for ChatGPT’s prime troubles, not least because it doesn’t explain GPT3.5’s behavior changes, and before long writers from the AI Snake Oil blog provided a less conspiratorial explanation: After a few attempts to check the primes properly, ChatGPT quickly got bored, and instead did what it does best: it guessed at the rest of the answers, and it guessed the same thing every time. The wild inconsistency in the results is likely down to finetuning; in March, GPT.4 guessed all the numbers were prime (and they were!) while GPT3.5 was only interested in composite numbers. In June, the reverse was true, because short-term tuning of the models changed their focus.
There are many useful takeaways from this cautionary tale, not least the lesson that ChatGPT is unlikely to give you the same answer today that it did yesterday. It also highlights ongoing concerns about OpenAI’s lack of openness. Speculation about GPT is rife because OpenAI will not reveal details of how they train and tune their models, and this clandestine approach has now drawn the attention of the FTC, who have begun investigating the firm’s inner workings. These concerns are becoming a big problem for OpenAI, especially as Meta finally made LLaMA2 - their own (completely open-source) AI - available to everybody for free this week, without any of that awkward secrecy about training methods.
Elsewhere, security professionals are concerned about a different GPT; this one’s called WormGPT, and it’s the first known instance of an LLM that’s been tuned specifically for cybercrime. Fortunately, we’ve prepared an essential guide to AI security to help you stay protected from AI dangers!
Beta Testers Wanted!!
SPYGAMES is the thrilling new experience where you’ll jump, climb, throw and dodge in fun immersive challenges developed with CIA experts to stretch your physical and mental agility. Click the link below to get EARLY access as a beta tester.
SPYGAMES is coming to SPYSCAPE in Manhattan, Summer 2023
After months of speculation about Apple’s AI plans, news has leaked that they are developing their own LLM chatbot - codenamed Ajax - and their staff are already using it internally.
The AI subplot of the SAG-AFTRA strike rumbles on as Lena Hall - aka Snowpiercer star Miss Audrey - claimed the cast have already been scanned for AI reproduction, or as the studio styled it, 'special effects'.
MIT’s Melissa Heikkilä takes a very clear-eyed look at the future of AI regulation and sees a boom time coming for lawyers, but believes politicians will struggle to have a similar impact.
The UN Security Council has held its first meeting on AI, with Ambassadors being briefed by Anthropic founder Jack Clark. The meeting was positive, but one member was skeptical. Can you guess which one?
In a rare public speech, Richard Moore - the head of Britain’s foreign intelligence services at MI6 - declared he believes that AI will make human spies even more valuable, rather than replace them.
Google’s founders, Sergey Brin and Larry Page, are showing an unusual level of interest in the firm’s AI development, with Brin apparently working on their next-gen AI project, Google Gemini.
