AI leaps from math dunce to whiz

When Michael Brenner taught the graduate-level class “Applied Mathematics 201” in fall 2023, the course’s nonlinear partial differential equations were too tough for artificial intelligence. AI managed to solve just 30 to 50 percent of the problems in the first three weeks of the class. 

“It was fine, but it wasn’t that great,” said Brenner, the Catalyst Professor of Applied Mathematics and Applied Physics and of Physics at the John A. Paulson School of Engineering and Applied Sciences.

But when he taught the same course this past spring, everything had changed. The same AI models that had stumbled on the easiest problems now aced the hardest ones. Brenner was shocked. “This actually calls into question the entire way the class is taught,” he said. 

The course taken by many graduate students has a reputation for being tough. Brenner has taught it for more than two decades, and he’d always given students take-home exams; he wanted them to have time to wrestle with difficult questions without the stress of a ticking clock. But if ChatGPT could take the exam for his students, could he trust what they turned in? 

He had two choices: Ban AI completely, or embrace it and redesign the class that has been taught at Harvard since before he was born. 

He redesigned the class. 

Forms of artificial intelligence have been used in mathematics for decades. But in the last few years, advances in machine learning and the exponential improvements of publicly available large language models have begun to reshape the discipline. While some mathematicians expect modest tools that can automate unglamorous parts of the job, others see a wholesale reimagining of the discipline and a rapid acceleration of what’s possible. 

Michael Brenner.

Stephanie Mitchell/Harvard Staff Photographer

And the notion that AI is uniquely bad at math? Brenner says: Simply not true.

“If you take your favorite large language model from two years ago and you ask it to please add 372 and 476, and it gives you the wrong number, then you say it’s bad at math,” Brenner said. “But what I would say is that’s the wrong use of a large language model. Obviously what you should do if you’re given those two numbers is you should open up a calculator.”

Evidence of AI’s mathematical capabilities is mounting. In 2024, two AI models from Google DeepMind earned a silver medal in the International Mathematical Olympiad, the largest and most prestigious competition for young mathematicians. Also in 2024, Demis Hassabis and John Jumper of DeepMind won the Nobel Prize in Chemistry for their AI model AlphaFold2, which predicted the structure of almost all 200 million known proteins. 

“It is now possible to make a computational model that leads scientifically to the extent that, within years of publication, it wins a Nobel Prize,” Brenner said. “That’s unprecedented.”

Knots and murmurations

While some mathematicians remain wary of AI’s tendency to hallucinate, specialized machine learning systems are accelerating mathematical discoveries in multiple fields. 

“Computers can deal with data sets that are too large for people to search through, and they can find patterns that people find interesting and significant,” said Michael Douglas, senior research scientist at the Center for Mathematical Sciences and Applications at Harvard. “That’s been very helpful for mathematicians whose objects of study can be assembled into data sets.”

One example: knot theory, a discipline that has applications in physics, biology, and chemistry. 

There are an infinite number of possible knots; researchers comb through databases of hundreds of millions of mathematical knots looking for relationships between them. In 2021, researchers at DeepMind, some of whom had ties to Harvard, used AI to discover new relationships between knot invariants, the numerical characteristics that define each knot’s properties. The discovery could have taken human mathematicians years to uncover through traditional methods. 

Another breakthrough came in research into elliptic curves, deceptively simple structures with big implications in both pure mathematics and cryptography. When Harvard researchers fed curve data into machine learning systems, they found that the curves’ behavior resembled murmurations — those swirling, coordinated movements of flocks of birds. 

“No mathematician ever thought to look for that before, and they were quite surprised to see it,” Douglas said. “They are now busy trying to prove it.”

But perhaps the form of AI that’s generating the most buzz in the math world is in automated theorem proving. For decades, mathematicians have used computer systems to check that theorems are logically sound. It’s an effective process, but translating human-written proofs into computer-readable formats is time-consuming, and the proofs generated by the automated provers are often very large. 

Enter generative AI. 

“The hope, and what people are starting to do now, is that we’ll use the large language models to write the proof in this way that the computer can check,” Douglas said. 

It’s solving one problem with another, Douglas said. Generative AI can almost instantaneously translate proofs into formats that automated systems can verify, while the verification process catches any AI-generated errors or hallucinations. 

Melanie Weber.

Niles Singer/Harvard Staff Photographer

Of course, just as AI is influencing mathematics, math is also influencing AI. Melanie Weber, assistant professor of applied mathematics and of computer science at SEAS, uses classical tools from geometry to build “geometric” AI models that are more efficient and transparent.

“Artificial intelligence is already revolutionizing science. However, the current models require vast data and computing resources, which can be limited in the sciences and raise sustainability concerns. Encoding geometric structures, such as symmetries arising from laws of physics, can increase the models’ efficiency by narrowing their focus to physically plausible conditions,” she said. “What that means is that the possible instances that we have to consider during training can be dramatically reduced by incorporating such structure. And that essentially means that we need less data and less computing resources to train a good model if we hard-code that structure into the model.”

The future of math 

Although the speed at which AI will change mathematics is still unclear, there’s no doubt the transformation is happening. Weber sees the technology serving both as a kind of research assistant, handling literature reviews and proof verification, and as a sounding board for ideas, helping mathematicians solve problems faster. Brenner said that kind of acceleration could be transformative.

“My hope is that we can solve problems faster and we can get more work done,” he said. “Science is infinite. There’s no limit.” 

Teaching, Brenner added, is also infinite. In the 2025 version of “Applied Mathematics 201,” he did away with traditional homework problems. Instead, students had to create their own problems, have a classmate verify them, and see if they could outsmart an AI. (“One of the good ones,” Brenner specified. “If the not-very-good models can solve your problem, it doesn’t count.”) 

By the end of the semester, the students had created nearly 700 math problems of increasing difficulty. The data on whether AI could solve them could prove useful for researchers.

“It’s just amazing, because I’m teaching math, right? And we’ve got a situation where the students are inventing math problems that are harder and harder and harder, and trying to solve them. That is the dream.” 

Brenner added, “There are problems everywhere. How to better predict the climate. How to understand what manipulations one could do to help the Earth be a more habitable place. Models that would help discover drugs. … The sky’s the limit on this, and we don’t know what is possible. But it is not a boring time to be doing this.”