Related: Will AI replace mathematicians?

About 40% of students in the ETH main library always have a Chat-GPT tab open. I soon decided to try using LLMs for my own studies (because the wisdom of the crowd is a real thing). I haven’t figured out how to best use LLMs for my coursework, but I’m experimenting with various approaches.

Getting unstuck #

For me, a big time sink is getting stuck on details. I usually go over the lecture notes after lectures, trying to work out the steps I didn’t follow with pen and paper. Ideally, I’d do this sitting next to a friend - it’s very convenient having someone whom to ask nearby. As Nate Soares put it here:

“The problem is, most of the time that I get stuck, I get stuck on something incredibly stupid. I’ve either misread something somewhere or misremembered a concept from earlier in the book. Usually, someone looking over my shoulder could correct me in ten seconds with three words. ‘Dude. Disjunction. Disjunction.’”

But studying with a friend isn’t always possible. If there’s a point of confusion I cannot resolve myself after making a reasonable effort, asking an LLM might help. Formulating a good question is always an instructive exercise. Moreover, nine times out of ten, the response is useful. Even if the LLM doesn’t entirely solve my problem, it might reference relevant concepts or serve as a sanity check. Sometimes I’ll learn that my approach was completely mistaken - and that’s certainly useful too!

Hints #

In the classic How to Solve It, George Pólya famously noted that mathematics isn’t a spectator sport. By generating hints, LLMs can be an aid in the problem-solving process too. Just have an honest attempt at the problem before consulting an LLM, and tell the LLM to not give away the entire solution. But it’s important to notice when one is stuck and ask for help. For someone like me who usually waits too long before taking hints, the ease of generating hints with Chat-GPT makes a huge difference.

The big picture #

LLMs are terrific at explaining high-level ideas. I’m a big fan of learning concepts “top-down”, starting with the big picture before getting into the details. While having more context doesn’t necessarily mean the material sticks better, I find this approach much more enjoyable. I usually ask Chat-GPT to give me the key idea before I look into the details. Apart from this, I regularly prompt Chat-GPT to give me the intuition for something or to motivate concepts. If a lecturer is pressed on time, they’ll cut the motivation bit, rather than leaving out a definition or theorem statement. For this reason, an AI-generated introduction can complement the lectured material.

Some of my favourite prompts include:

• “Why do we care about X?”
• “What is the main idea behind the proof of X?”
• “What’s the intuition for this definition?”

I also find it helpful trying to explain a concept in my own words and asking Chat-GPT to elaborate or check if my explanation is accurate.

Caveats #

All this said, I’d like to add a few caveats.

A friend or teaching assistant could help with the above tasks better than today’s LLMs. They know about your mathematical background and what conventions you’re using. When I interact with chatbots, explaining conventions and providing context adds a lot of overhead. However, this problem seems fixable. Many AI labs are already working on ways to provide more personalised responses by having the chatbot remember information across chat sessions. Students could e.g. upload lecture notes and indicate which parts they’d covered.

AI systems also make mistakes. But this isn’t that big of an issue. Most mistakes are easy to spot, especially if you ask the AI to explain steps that seem fishy. If you point out what went wrong, it will modify the argument. With human guidance, AI systems can get quite far. Also, LLMs don’t need to get all the details right in order to be useful. As Terry Tao noted in this blog post:

“Strangely, even nonsensical LLM-generated math often references relevant concepts. With effort, human experts can modify ideas that do not work as presented into a correct and original argument. The 2023-level AI can already generate suggestive hints and promising leads to a working mathematician and participate actively in the decision-making process.”

Of course, if you want to be dead certain the AI-generated argument is correct, have the AI output a formal proof in Lean.

Another fear of mine, perhaps ungrounded, is basically that LLMs will make us lazy. Learning requires a certain amount of effort, while writing a good LLM prompt is relatively easy. If we use LLMs more and more, will we remove the friction needed for learning? I don’t know whether this fear is valid or if it’s just an instance of “tech panic”. But as long as we set boundaries for our LLM usage, we need not spoil the learning experience.

Where does this leave us? #

It seems, then, as if we could overcome the problems I’ve encountered when tinkering with AIs. My experience with using LLMs as part of my studies has been positive, so I’ll continue exploring ways in which AI can assist. The one thing that LLMs can’t provide, however, is the social aspect of doing maths. Solving problems with others is infinitely more fun than coming up with LLM prompts. If anything, I think the above use cases highlight the importance of doing maths together with others.