Exploitation and Exploration

by Daniel Bauman, 21 October 2023

A classical theme in problems ranging from decision theory and optimization to reinforcement learning and natural selection is the tradeoff between exploration and exploitation. Exploration is the process by which we learn what options and opportunities exist; exploitation is when we act on those options we have discovered. While there is plenty of interesting mathematical theory developed around understanding these problems, that’s not why I bring it up today.

I am four weeks into my first quarter as a PhD student and already notice a palpable change in graduate life compared to an undergraduate. Although my program has a greater required courseload than most, the breadth and freedom of how I am learning and exploring ideas is vastly beyond anything I have had the opportunity to experience before. My undergraduate life challenged me greatly and honed my problem-solving skills and fluency in the language of mathematics. But from both the heavy workload and inherent structure of trying to master the fundamentals, I lacked the ability to either explore or exploit much of anything.

Being a graduate student is different. Never have I felt so free to dictate my life, my thoughts, and my energy towards, well, whatever feels interesting. I have access to just about every book from the collective work of centuries of science, spend hours jotting down ideas on the scratch paper I steal from printers, and still have leftover time to complete my coursework, cook for myself, exercise, and pursue hobbies. Before, doing coursework and staying on top of all the activities I ended up doing left me burnt out at times, frequently unable to even read for fun.  

It’s as though I spent the past five years climbing the well-trodden and frequently steep pilgrimage at the center of the mathematical universe. Now, I find myself on the summit of a hill that is surrounded by topography far ruggeder and less well-explored than anything I have seen. Some subjects have been compiled into books, others are merely collections of papers to wade through, and some (most, perhaps) are still yet to be touched by humankind. (Writing this makes me wonder if there is a way to characterize the dimension of research being performed by assuming it’s a sphere in R^d and examining how long it takes one to reach the edge of the sphere).

For some people, the lack of structure to this would feel overwhelming, but so far at least I think it fits me perfectly. There is so much to explore. The very thing I am most interested in, how evolution has explored the space of possibilities of life to create the spectacular biodiversity and beauty which exists in nature, is mirrored in the path of discovery of science. Obviously, it is necessary to exploit, to find your niche and develop research that both you and the world cares about. But for now, simply getting to explore is a grand adventure.