It happened again. In today’s arXiv Jef Pauwels claims the solution of the I3322 problem: namely that the maximal violation of the I3322 inequality cannot be reached with finite-dimensional strategies, and indeed it is the limit of the Pál-Vértesi strategy. Of course, the result was found by AI. Pauwels claims he simply gave ChatGPT the problem and told it he believed it could do it. And three hours later, a problem that had been open for almost two decades was solved.
Where is the glory in that, though? Anybody could have written that prompt, and indeed one Seth Douglas, who is apparently not even a researcher, claimed the same result a month earlier. Echoing Terence Tao, Pauwels claims the glory now lies in “proof digestion”, making a readable paper out of raw LLM output. Fair enough, unlike the earlier NPA non-convergence result, his paper is actually readable. But then he says he used Claude to do it.
As you might have guessed from the title of the blog post, I disagree.
First of all, I didn’t get into science to polish LLM turds. The reason I accepted low pay, long hours, moving countries, and instability is because I love doing science. Not giving problems to a machine and giving it a pep talk. Even less trying to make sense of LLM drivel. I suspect I’m not alone here. Indeed the students nowadays don’t seem to have motivation to learn. Why bother, if there’s no scientific work left? Almost all of them are just trying to cheat their way to a degree with LLMs. And that’s a drastic change, when I was an undergrad those who paid somebody else to write their thesis were the exception, not the rule.
Secondly, I don’t think there’s anything preventing LLMs from successfully doing digestion (even if it’s not clear how much Claude helped in Pauwels’ case). Trying to find a special role that only humans can do is a fool’s errand. Tao acknowledges that, and tries to reserve for humans the role of being the “bosses” of mathematics, deciding which proofs are relevant and interesting and should be taught. Taught to whom? Relevant to whom? Who the fuck is going to care?
I think this is the death of pure research. When we did the research ourselves, we could tell society that it should pay us because we were learning something, and teaching something, and perhaps even discovering something that might in a century be actually useful. But why should society pay us to type the prompts and “digest” the results? There is no public funding for sommeliers, those who just want to appreciate wine need to pay for it themselves.1
Applied research remains, of course. Society wants a result, and pays people to get it, doesn’t matter how they do it. I just think it will get much more expensive when people no longer do it out of love.
Ramanujan, the indian genius mathematician, claimed he had gained formulas and equations from goddess Namagiri while dreaming. So, in a sense, he was the first “proof digester” in the history of science =D
And Hardy was the second, since he largely played the “boss” role envisioned by Tao, having introduced Ramanujan to the mathematical community and helped filter, polish, and publish his works.
Hi. This is Seth (the person you mention in your post). I have tremendous respect for you and others who have dedicated their lives to the pursuit of knowledge. I often think that I picked the wrong career path for me. I am a software engineer and an AI architect by trade. I think you are touching on something so incredibly important, and at the same time inevitable. I didn’t even set out to solve I3322. I am iterating on an AI automated research method and architecture in my spare time just for fun. I’ve always felt that we are missing something important in our overall understanding of reality, and as a side project I was attempting to find out what that is. My quarterback agent needed one exact, non-toy example of a limit that finite systems get arbitrarily close to but can never actually reach, because the theory it was building kept pointing at finiteness as the thing that shapes what we can know. I3322 turned out to be the smallest open problem of that kind, so that’s where it went. It produced a proof, and then I spent far longer trying to break it than it took to find.
I want to be honest that I don’t know what this means for people like you, or for the students you describe, and I don’t think “proof digestion” is a satisfying answer either. What I do believe is that the thing that made you accept the low pay and the moving is the same thing that kept me up at night on this, and I don’t believe a machine takes that away from anyone. This world is coming whether we like it or not. I just hope we figure out together what people are for in it, because I’d much rather do that with the people who love this than without them.
Of course the machine takes it away. After the problem is solved there’s no longer anything to do. Furthermore, the set of good problems is small and rapidly shrinking. Because making a “good” problem is an inherently slow process. Someone needs to pose it, other people need to study it, solve particular cases, propose variations, invent new techniques. For the problem to be interesting people need to work on it.
Imagine that Pál and Vértesi had come up with the problem today. They could just publish the problem together with its solution. None of that would have happened, the problem would have never become well-known.
Pretty much anything that’s doable will soon be done. We’re turning a fertile research field into a radioactive wasteland. No wonder students are no longer motivated to learn and do research.
Honestly, as someone who has just completed a master’s, everything related to LLM-assisted “research” just makes me depressed and demotivates me from pursuing a PhD.
Look at all the recent controversy surrounding the Navier-Stokes result from OpenAI, for instance. How am I supposed to even discuss a research problem that I am currently working on without fearing that an eavesdropper simply feeds it to an LLM and “solves” the problem using God knows how many agents and prompts?
This technology rewards academic malpractice and I worry that there is an environmental pressure for PhD students (and senior researchers) to adopt these LLMs in their research. Now I even have to read from someone who is (probably unintentionally) contributing to the problem that they have “tremendous respect for [the people] that have dedicated their lives to the pursuit of knowledge”. What a sick joke.
I think the only way to do physics or mathematics now is in complete secrecy. The culture of openness is over. It relied on academic honour, and that simply doesn’t hold when anyone can spin up an LLM and solve any problem, without caring for their reputation. I think we won’t see people publishing workable conjectures anymore, they will either solve them themselves using an LLM or keep them secret forever.
As for doing physics, you won’t be able to be competitive without an LLM. But if you’re using an LLM you’re not doing anything anymore, you’re just a proof sommelier. And as OpenAI has show, if you use an LLM they might just steal your results if they find them interesting.
If open-weight models that can run on your own hardware become competitive at least OpenAI and Anthropic won’t have us by the balls anymore.