Debates over AI benchmarking have reached Pokémon | TechCrunch
Not even Pokémon is safe from AI benchmarking controversy. Last week, a post on X went viral, claiming that Google’s latest Gemini model surpassed Anthropic’s...
Tag : benchmarks
OpenAI launches program to design new ‘domain-specific’ AI benchmarks | TechCrunch
OpenAI, like many AI labs, thinks benchmarks are broken. It says it wants to fix them through a new program. Called the OpenAI Pioneers Program,...
People are using Super Mario to benchmark AI now | TechCrunch
Thought Pokémon was a tough benchmark for AI? One group of researchers argues that Super Mario Bros. is even tougher. Hao AI Lab, a research...
Did xAI lie about Grok 3’s benchmarks? | TechCrunch
Debates over AI benchmarks — and how they’re reported by AI labs — are spilling out into public view. This week, an OpenAI employee accused...
AI isn’t very good at history, new paper finds | TechCrunch
AI might excel at certain tasks like coding or generating a podcast. But it struggles to pass a high-level history exam, a new paper has...
AI researcher François Chollet is co-founding a nonprofit to build benchmarks for AGI | TechCrunch
Former Google engineer and influential AI researcher François Chollet is co-founding a nonprofit to help develop benchmarks that’ll probe AI for “human-level” intelligence. The nonprofit,...
Will Smith eating spaghetti and other weird AI benchmarks that took off in 2024 | TechCrunch
When a company releases a new AI video generator, it’s not long before someone uses it to make a video of actor Will Smith eating...