math for dummies

A couple months ago, Scott Aaronson made a post on his blog titled Dispatches From the Possibly Last Days of Human Relevance, about a recent problem solved by an internal OpenAI model: Erdős’ Unit Distance Problem¹. I don’t care much for the post—or for the blog—but something about that title stuck with me. ‘Dispatches from the last days of human relevance’—it’s the kind of rich little phrase that rolls off the tongue, forecasting that we’re entering a truly unprecedented era. At the time I also thought that he was being pompous and generally indulging in hype-generating behaviors sprinkled in with the usual provocateur touch, but as soon as two months later it’s pretty clear in math circles, at least, that he was onto something. This is December 2019 for mathematicians. Something big is coming, and it has started.

taking the ferry to school

Until about a couple weeks ago, I was under the admittedly uninformed impression that these new LLMs were going to plateau and suffer financial failure, leaving the rest of us with glorified chatbots that could solve the occasional recurrence relation but only really do the tasks that the rest of us have really been trained to do since we were children. Many of the ‘features’ that LLMs promised appeared to me to be solutions looking for problems rather than anything new. Writing emails, that was the big one. You don’t have to write your emails! An AI can do it for you!

Something about the fact that millions of people were ‘hyped’ by this level of task to me seemed rather fishy. I don’t need an LLM to write my emails: it’s a task I can do in a few minutes, and it doesn’t even require any physical effort. It’s not like washing the dishes, where automating the task away is a huge deal—it’s annoying that every time you eat you have to wash the dishes, as well, and so it’s only natural that you can get something to do it for you. It, like many other things, is a chore. Chores are boring and dull, and distract you from the things you’d rather be doing, like hanging out with your girlfriend, or playing football, or watching TV. Or like doing the laundry, which even in its current form is one of the few truly nightmarish tasks I have to accomplish; my wardrobe is also abnormally large, having been assembled precisely to minimize the number of times I have to do laundry… if I have more clothes, then I have to wash them a lot less! And so on and so forth. Part of the reason I hated doing these chores is because these weren’t the chores I had to do as a kid; this was the stuff my parents did. The chores I did have to do—like making my bed or dusting my apartment—weren’t the kind of things that caused me any number of problems. Cleaning is fine. It can be therapeutic, even. It’s just part of my ingrained self, and the amount of effort it takes is comparable to walking. It’s just something I can do.

Writing emails is like that, or writing in general. I don’t really need an AI to do writing for me—isn’t reading and writing the essential purpose of the entire school system? It can certainly get annoying, sometimes, if you have to do a lot of it, and it feels nice to get someone to do it, occasionally, like hiring a person to do administrative tasks for you.

An LLM certainly can compose an email. But the only part of the task of an assistant that an LLM can do is the particular task of producing words on command, which is something that is second nature to me—most of the stuff I write here is first drafts produced over a couple hours, and it takes about 30 seconds to draft an email. That’s why they ingrain those things in you over primary school, so you never have to think much: start with a ‘Dear [Recipient]’, follow up with a ‘Hope you’re doing well,’ get to the point. Over the course of your lifetime you’re probably going to be writing many letters, and you need to know how to do it—not in the sense of something that you need to look up every time, but something that you throw out at command. A trivial task. What I’d like to automate away isn’t the actual letter-writing process, because that’s already automated; what I’d like to automate away is the fact that I have to deal with these things at all, and that’s why you hire the assistant.

But this is not what an LLM does. An LLM might write the mail for me, but it’s not going to exercise any kind of judgement over what is the appropriate response to the query, or whether I should answer it at all, or whether this is just spam or if it’s indeed a person suing me. The problem, of course, is exactly the fact that the LLM isn’t human. If it screws up, there’s no one I can blame. It can’t take responsibility, and saying ‘oops’ isn’t going to fix the damages it causes. And though it can write emails, it certainly can’t write the email I was going to send, with all of its idiosyncrasies and foolish questions and sign-offs that say either ‘Thanks and Regards’ or ‘Regards’ depending on how much I like the person.

Occasionally I will hear people cry about LLM-generated articles on the Guardian or on Substack or whatnot. I’ll take a look at these articles and they look like a mixture of word salad and half-baked diplomatic claims, so essentially exactly like a non-LLM generated article. But the fact that these articles so resemble non-slop articles is precisely what makes me wonder why someone would use LLMs at all. Each one of these articles has a human name attached to them. Without fail, they are either a journalist or an aspiring writer, of sorts, and the fact that this manner of person willingly chose to let an LLM do their task is astonishing to me. You’re a journalist—isn’t your entire job to regurgitate 500 words on any topic on command? Surely you’ve sat through enough deadlines that writing something acceptable in an hour isn’t difficult, but merely second nature? Surely accomplishing this task is precisely why you got a degree in the field you are now pursuing? Then why does this tool provide you with anything—after so many years, wouldn’t doing your job be as easy as breathing to you?

real jobs and fake jobs

The problem, of course, is that people can never really figure out what their job is. Unfortunately a job is not simply doing the one singular task you enjoy doing.

Like always, there’s a movie to illustrate: in The Karate Kid, after Mr. Miyagi agrees to teach Daniel karate, for the first several weeks he doesn’t teach him how to throw a single punch or a single kick. All he does is make him do chores, clean his car, culminating in the famous ‘wax off, wax on’ sequence, where it turns out that Daniel’s waxing of the car has actually contributed quite a bit to his ‘training.’ The strength, balance, and ability to do repetitive tasks has made him quite physically resilient, and the motions he’s committed to muscle memory are precisely those that make him better at karate.

Of course, that’s why people go to the gym—lifting weights is hard, but if you lift weights over and over again suddenly you will find yourself quite good at lifting weights, and somehow you will also find yourself better at sports than you were before, too, even if you’ve never played sports.

In 1963, Thomas Pynchon published V. and became Thomas Pynchon. But before he was Thomas Pynchon, he was Thomas H. Pynchon of the Bomarc Service News, where he worked as a technical writer for Boeing. Among other things, Pynchon wrote about the safety precautions necessary when airlifting the IM-99A Bomarc missile², about the Technical Manual T.O. 21-IM99A-2-2, which he referred to as the Bible, and about how ‘the pressure in the oxidizer line in the vicinity of the separation diaphragm will be approximately 35 psig.’ None of this was good or interesting work. But it was the work of one of the greatest novelists of the 20th Century, and what it did was make Thomas Pynchon better at writing. Turns out that what goes into writing Gravity’s Rainbow is technical drivel about the specifications of airplane security systems. So perhaps if you want to become Thomas Pynchon you should sit and copy down details from technical manuals every single day for several hours, until being able to write 10,000 words about the T.O. 21-IM99A-2-2 in 40 minutes is something that you can do at will. Then, perhaps, you can start writing Gravity’s Rainbow.

All that’s to say is that the way you get better at writing is by writing. It doesn’t even necessarily matter what you write, it matters whether you write or not. Similarly, getting better at mathematics involves doing mathematics. This involves similar tedious drivel of working out recurrence relations by hand and learning how to twist graphs in your head. It involves moving integrals around and being annoyed that you forgot which way the equality was pointing. There is a reason that mathematicians do this: not because it is fun, but because it is, fortunately or unfortunately, part and parcel of being a mathematician. This is how you get better at math. Turns out being forced to do thousands of addition exercises indeed makes you better at math. So does working out a bunch of word problems. And also manipulating algebraic equations. So instead of complaining about working out complicated integrals by hand, you better get used to it—because Gauss did it too. Gauss was famously a genius at performing arithmetic computations in his head and by his hand, and the way he discovered the Prime Number Theorem was by writing down 100,000 primes and graphing their growth on a piece of paper. Perhaps the skills of ‘doing mathematics’ and ‘doing annoying computations’ are not unrelated.

The job of a mathematician is not thinking about abstract problems in the head. The job of a mathematician is to do mathematics. And this involves sitting down with a pen and paper and writing down deeply annoying probabilistic calculations and bounds that you really don’t want to do but you have to because it is your job. And that’s just how it is, and there’s nothing you can do about it. Didn’t they teach you that in school?

everything machines

That mathematicians now have to work out their math by hand is not something that is true anymore, or at least has been shown to be false over the last few months. A vast scourge of preprints has arrived that purport to show the mastery of LLMs in doing mathematical tasks. And, unfortunately, those preprints are right. These machines are very good at doing math. They are often wrong, but they are only about as wrong as a human. And when they are right, they are often right because they are good at things we are bad at. They are, generally speaking, better than us at certain things. They are also probably going to take our jobs: as a friend of mine put it, the jig is up.

‘The jig is up’ is an interesting phrase to apply here, because it implies that what we were indulging in before the arrival of these things was charlatanry. But many mathematicians will indeed agree with this characterization of mathematics: they love to claim, like did GH Hardy, that their work is generally entirely useless, and they claim this with pride. They are quite happy that they can sit around and not contribute to society. Even if this were in fact the case—and it is not—to present mathematical research as a form of charlatanry blatantly forgets what the practice of indulging in mathematical research actually is. For example, in France, a mathematician that is allowed the freedom to work on ‘useless things’ can expect to earn a gross between €40,000-€80,000 depending on seniority, and even then a very large part of the job involves teaching duties. €50,000 a year is a sum that is not unreasonable, and in fact even well-warranted for a professor purely on the basis of teaching duties alone. Most earn much less. Paying a small group of people this amount of money for just teaching is hardly a jig, and much less charlatanry—the least you can expect is the freedom to do the research of your choice when being paid peanuts, and if you want to make the big bucks then you can always become a quant. ‘Swindling’ €50,000 a year from your employer by writing papers about sheaf cohomology seems to me a blatant misunderstanding of what this job is. You’re not ‘swindling’ money from anyone. You are being paid a meagre sum for educating students because that is what your job is worth.

This relationship to your job can get fairly nasty when confronted with things like LLMs, because mathematicians are both dastardly scared of being replaced while also wholeheartedly believing that they do things which are useless. They complain about administrative tasks, and teaching, and calculations, as things which ‘distract from the job,’ even as they acknowledge that the ‘job’ itself involves the useless manipulation of symbols until you feel the sudden euphoric click of things sliding into place. They believe that this is what they are providing to this world. And this is certainly the fun part of the job, sure, but the meaningless tedium of responsibilities around it is equally the job of the mathematician as the principal job of idea generation. And when you genuinely think this, you come to the rather awkward conclusion that you are a person who deserves to be replaced.

What an odd thing to think.

Mathematicians, more than anyone else, are prone to thinking that they are disembodied minds floating in some kind of biological soup, and their value lies only in the ideas that mind can produce. I have never seen a group of people who revel less in the fact that they are a human being who walks and talks and things and feels and instead wholeheartedly want to do things like upload their brains into some kind of knowledge heaven, where answers are provided to them on command. Mathematicians are the only kind of people who would ever seek to build a machine to automate their own job away.

at home, on your desk, all alone

This idea of ‘deserving’ to have your job taken is one that strikes me as rather awkward. People, by and large, don’t ‘deserve’ to have their livelihoods taken from them. But mathematicians see themselves as united in some kind of higher cause, and this causes them to make nonsensical statements. ‘This is what it’s like now.’ ‘This is what it’s all been building towards.’ I find it quite baffling to think that what mathematics has been ‘building towards’—if it has indeed been building towards something at all—is a button which can prove any theorem for you, when the part of the ‘job’ you most enjoy doing is in fact the very job that you have automated away. As far as I can tell, there’s no indication this thing can teach, and so if mathematicians will continue to have a job it will involve educating people, and doing administrative duties, and scheduling meetings, and sitting on faculty hiring positions, and all the very things that mathematicians have so hated doing for the entire time they’ve been here. So it’s rather interesting that the thing they’ve chosen to automate away is precisely the ‘useless’ part of their job that they actually enjoy.

This is the precise culmination of this line of thinking that mathematics is an activity primarily of the mind, as if it is somehow separable from other ‘human’ activities.

What mathematics provides to people is far more than just the knowledge that a theorem has been proven. Mathematics has given me a community. It has given me friends—it is not a coincidence that every single one of my best friends are mathematicians, too. It imbibes people with meaning, it gives them something to grasp for. It is also not a coincidence that many of the greatest mathematicians are—and have been—highly religious, because at its best, doing mathematics can make you feel like you are close to something greater. I’d like to think that what mathematics is not is the act of knowing that there are theorems out there which have been proven, extending towards an arbitrary notion of ‘progress.’ There are many fields of study which do naturally give themselves to this idea of progress—medicine, for instance, where it is a meaningful goal to discover the ways that human beings can live long, healthy lives, keeping themselves free of disease, or energy and materials science, where discovering better, more efficient and renewable power sources that can light the world is something the field is actively pushing towards. Mathematics is not like that, and for a long time I have found the assertion that pure mathematics can be meaningfully incorporated as part of other ‘STEM’ disciplines quite ludicrous. Certainly using mathematical techniques for various ends—like creating the aforementioned LLMs—may be tasks that have goals in sight. But I’d like to think that automating away the entire process of human discovery is certainly not one of them. Just as there is no underlying ‘goal’ to philosophical inquiry does not mean that there is a ‘goal’ underlying mathematical inquiry either. I’d like to think that we do it because we like it, nothing more.

“Wir müssen wissen—wir werden wissen!”, or, “We must know—we will know!” These are the words placed on David Hilbert’s epitaph on his tombstone in Göttingen, which he said in an dddress to the Society of German Scientists and Physicians in Königsberg. Hilbert, like many others, was a proponent of wanting to know things, and the way he proposed to know things was by attempting to fashion an algorithm that would determine whether a statement was true, purely by the process of computation. At the time this was a natural thought, built on the work of many others like Leibniz, who was on the search for an ‘instrument’ that could manipulate his logical symbols and determine if something was true all the way back in the 17th and 18th centuries. People wanted, more than anything, a machine that could think and surpass human limitations, and which could determine the truth-values of any statement that was put into it.

So much for thinking that mathematics does not mean merely knowing the truth.

And now we have this machine. I wonder what Hilbert and Leibniz thought there would be to do, anymore, if they couldn’t do mathematics.

It is interesting to me how mathematics is a solitary activity, in many ways. Sitting in a room on a desk writing things away on a piece of paper is how it works. Now you can sit on a table and pop a question into your computer, reading away the answers it generated. You can know, instantly, if your proposition is true or not. I wonder whether the prospect of knowing answers is really such a wonderful one. There is something deeply sad about sitting at your computer and consulting an oracle all alone in the darkness, waiting for it to provide you with the answers you so desire. It feels, somehow, wrong to be able to determine these answers without ever having to put in the work to find them. It seems to me that the most important knowledge is that which you need to prove yourself worthy of receiving.

making logarithm tables

Like many people, I am conflicted about these new mathematical developments. On one hand it is certainly true that an all-powerful machine to replace us is one of the oldest mathematical desires. We are too stupid to know the answers to these things—all our theories feel like houses of cards when confronted with some truly hard problems. Perhaps we have really reached the limits of human understanding, and the only way to progress is through the machine. I am reminded of a short story³ I read a while ago about a scholar-guild that has to spend all its days reading, because knowledge has advanced to the point that it takes too long to learn enough to be able to contribute; most can only contribute at the very end of their lives. Learning enough math to be able to contribute gets harder and harder. Today PhD students in certain areas of math are hardly expected to publish more than a paper or two, and maybe in a generation even that might have been too little time. We get proofs like those of the Geometric Langlands Conjecture that span a 1000 pages and stand at the edge of human comprehension. And, meanwhile, there is an entire community out there working in ways to formalize mathematical proofs so that checking their correctness goes from a matter of human judgement to something that can be settled by the rules of logic. It may very well be that the problems we have been asking are simply beyond our well and total comprehension.

On the other it seems to me equally foolhardy to believe that pressing buttons is going to provide us with any catharsis as to solving these problems. I don’t know what people envision, but the idea of LLMs generating proofs that then then formally verified and checked by another LLM seems to me little more than simply running simulations on a computer. The ‘value’ of these proofs lies merely in the fact that they exist—ultimately a proof has always been a human argument, meant to convince another human of correctness. There is ultimately no reason for proofs to have a human interpretation at all; they may merely just be, like a computer, strings of formal logic that somehow solidify into a truth or a falsity, with none of the beautiful mathematical structures that we work on arising out of them at all. Will we be able to read these proofs? Will we be able to understand them, to teach them? Will there be people who simplify them for us—people like who discovered that in fact the argument, when viewed via the lens of groups, was elementary all along?

Of the counterexamples so discovered, it is already true that we cannot really tell how some were arrived at. It really does feel like we are bleeding out at the crossroads, and just before our eyes close, the devil appears and offers to save it, but at the cost of our soul.

The human activity that ‘mathematics’ is involves a lot of back-and-forth, a lot of talking, a lot of heart, and a lot of people. Much of mathematics is talking about it with other people, just as much of it is solving elementary problems. It feels odd that people will probably still continue to do these things, but they won’t sit down and think of the problems themselves. Instead they will spend their time deciphering texts that may be too hard for them to even understand if they wanted to. But perhaps the ‘merit’ of mathematics is not in the problems themselves, but rather the desire to understand, and the desire to search. Perhaps the whole point of mathematics is to talk about mathematics. To read and think and get beautiful thoughts and ideas out of it that bring you closer to other people.

In this sense I suppose I find an aesthetic, almost literary beauty in mathematics that is almost entirely separate from that what LLMs can provide us with. I have always found that math textbooks and good papers make excellent books. Reading something like Rudin is not all that different from reading the Odyssey, for all the aesthetic pleasures it provides, and discussing the world that you get out of it is not very different from that mathematical world of Rudin’s either. Like many people, I feel the drive to read the mathematical ‘canon,’ and talk about it, and spread the word. I suppose that there is very little that can be done to take that away. But it still seems odd that the upper pantheons of mathematical research are something that are just fundamentally inaccessible. It feels odd to think that there will be no more Gauss, no more Euler. There will still be people doing and talking about math, sure, but the line that began all the way back with Pythagorus has now ended; the history of research mathematics has been written. It spanned 2500 years—we had a pretty good run, and it’s odd to come along right at the end of it. Perhaps I may be pre-empting this lament—perhaps LLMs may yet revolutionize the way we do research, and perhaps humans might carry the torch, in the end—but more than anything I have the kind of feeling that I get, ironically, while reading the final pages of Pynchon’s Inherent Vice: a kind of bittersweet reckoning, knowing that we are living at the end of a golden era and at the beginning of a darker, more cynical age, one marred by the knowledge that the ‘good old days,’ finally, have ended, and yet one in which the most profoundly human things of all—people, and children, and combing through years’-old proofs all alone in your room, and finding them delightful—these things still matter and still exist. And right now you can fly on over to the Bay and see with your own eyes the great wave of mathematics falling to the floor—

There was madness in any direction, at any hour. If not across the Bay, then up the Golden Gate or down 101 to Los Altos or La Honda. . . . You could strike sparks anywhere. There was a fantastic universal sense that whatever we were doing was right, that we were winning. . . .

And that, I think, was the handle—that sense of inevitable victory over the forces of Old and Evil. Not in any mean or military sense; we didn’t need that. Our energy would simply prevail. There was no point in fighting—on our side or theirs. We had all the momentum; we were riding the crest of a high and beautiful wave. . . .

So now, less than five years later, you can go up on a steep hill in Las Vegas and look West, and with the right kind of eyes you can almost see the high-water mark—that place where the wave finally broke and rolled back.

— Hunter S. Thompson, Fear and Loathing in Las Vegas

I am reminded of Lord Kelvin, who proclaimed in 1894 that physics was over and there was nothing new to discover, except maybe two clouds at the horizon. Writing this I feel very much like Lord Kelvin. Who knows what those clouds will be.