Research, Understanding, and Attention

These are some personal thoughts on AI and research. I’m sharing them as a starting point for discussion.

Lately, a question keeps coming up in conversations with fellow researchers: as AI takes on more of the work, how does that change the value of doing research? I hear similar questions from research interns. These conversations touch on which problems are worth pursuing, what a project might contribute, and what someone might learn along the way.

One can readily foresee—or may already believe—that AI will bring lasting changes to how research is carried out and how people learn through it. This trend seems inevitable. Some changes are already visible in the case archive, which includes LLM-assisted prototypes reportedly built in one to a few weeks. Judging by the scope of some of these demonstrations, comparable projects might once have taken a team of PhD students six months to a year to go from an initial idea to a working implementation. Frontier AI models now help researchers implement their ideas much faster. For someone choosing a project now, though, there is a more immediate question: what is the aim of doing the work?

Perhaps it is useful to distinguish two main aims of research: research for performance and research for understanding. AI may help with both. When used appropriately and effectively, AI should be beneficial.

1. Two aims of research

I am thinking about research as an activity, including work that never becomes a paper. What someone gains from doing research may extend beyond what gets published.

By research for performance, I mean working toward measurable improvements or optimizations: lower reconstruction error, faster inference, a more reliable controller, or a network architecture that makes better use of a fixed computational budget. Such gains are an important part of technological progress: they make tools more useful in everyday life and expand what science and engineering can achieve.

Research for understanding might begin with a question about why a method works, which assumptions it relies on, or why an apparently reasonable idea fails. Sometimes the gain is personal: I finally understand an argument that others have known for years. Sometimes it becomes knowledge that can be shared with the field.

A project can also leave a student with a much better grasp of a subject even if it produces no publishable result. This is part of what interests me about research training. Following an argument, reproducing a result, or working out why an experiment failed can change how someone approaches the next problem. I would hope there is room to value that progress, even when it is difficult to point to a finished result. Since my first experience of research training in 2018, I have made a habit of following my curiosity and trying out lots of ideas (and, of course, getting less sleep). Quite a few of my research results have grown out of failed attempts.

This distinction may help clarify both what a project might accomplish and what someone might learn along the way. The balance may change as the project develops.

In most cases, these aims overlap. Trying to improve a method may reveal something about its behavior; understanding that behavior may suggest the next improvement. The following sections look at how researchers might use AI as a tool for each aim, starting with performance.

2. AI is a powerful tool in research for performance

Whether or not one is willing to acknowledge it, a substantial part of research involves bringing existing ideas together. Researchers adapt a technique to a new setting, combine methods, or notice that a problem in one field resembles a problem already studied elsewhere. Judgments of novelty also depend on the setting. Combining A and B within one field may look like a small improvement; bringing A from another field may give people working on B a new way to understand their problem and may be regarded as a novel contribution in that field. Below, I discuss how researchers can use AI tools appropriately and responsibly to accelerate research progress.

Literature review. Researchers can use AI to explore possible approaches and develop better ideas for improving a method. The word “research” can also be read as “re-search.” LLMs can act as powerful search engines for exploring existing knowledge, helping to find connections, compare formulations, and try different implementations. A connection suggested by a model can point to relevant literature, sometimes revealing that an apparently new idea already has a history under a different name. Used well, these tools can save researchers considerable effort by helping them avoid reinventing existing ideas. Researchers still need to review the information LLMs help gather about prior work. Once checked, a more comprehensive and well-organized account of that work can help them understand developments in their field more quickly and in greater depth.

Experimental feedback. Frontier AI models can also help researchers obtain feedback sooner in performance-oriented research. As the examples mentioned at the start suggest, researchers can now build working implementations for some tasks in one to two weeks with AI assistance, whereas comparable projects might once have taken a team of PhD students six months to a year to go from idea to implementation. A working implementation gives researchers something concrete to test and revise. Researchers can define what to improve and choose how to measure it—for example, through reconstruction error, runtime, or success on a test. AI tools can then help implement and evaluate variations, using that feedback to guide further attempts. AlphaEvolve offers a concrete example: it proposes changes to code and uses evaluator feedback to improve candidate algorithms.

Reliability. AI may help through tests, calculations, and verification, using tools such as simulators and Lean, as well as data from real-world experiments. People make mistakes, and some mistakes have consequences far beyond a paper. If an error in my work contributed to the failure of a spacecraft carrying people, I imagine that responsibility would stay with me for the rest of my life. Frontier AI models with tool-use capabilities could help researchers make such work more rigorous and catch errors they might otherwise miss. Of course, whether they actually do so still needs careful verification.

These are reasons I expect AI to become an increasingly useful tool in research for performance. Better models may provide more useful information to help researchers develop and refine novel ideas. Access to more computing power also matters: it could allow researchers to implement and test approaches more quickly with LLM assistance, obtaining feedback when evaluation is affordable. Researchers still need to judge which questions are worth pursuing and whether a measured improvement matters.

A useful result may also matter before it is fully understood. Consider a hypothetical example: someone has a form of cancer that existing treatments cannot cure, and AI helps discover a new drug. Rigorous testing provides substantial and convincing evidence of its safety and efficacy, though the evidence is still incomplete. Its mechanism is also not yet fully understood. If I were that patient, I might be willing to try it. I would care about the possibility of treatment even while the explanation remained incomplete. Others may weigh this differently, and I think it is worth discussing.

For me, this example is a reason to value a useful result even when understanding has not caught up. A discovery can benefit people while efforts to explain it continue. AI tools may make such cases more common, and understanding the results could itself become a substantial part of research.

3. AI is a powerful tool to help researchers deepen their understanding

I often hear the concern that students can use AI to obtain answers without going through the process that used to lead to them. This clearly undermines the learning process and research training, so it needs to be guarded against. But researchers and students can also use AI as a tool for deepening their understanding. I strongly believe AI's effects depend heavily on the user's motivation.

Over time, this may favor those who continue to choose understanding, even when an answer is already available. For someone who wants to get to the bottom of a question, AI can support that effort. Someone seeking a deeper understanding of a subject, or inspiration from another field, can use AI to work through papers and derivations or build a small teaching demo around a confusing point.

This semester, I am taking a course on simulation modeling. For the course project, my group chose to model heat transfer during freeze-drying in vials, a process used in preparing some vaccines. Although I am relatively familiar with simulation, numerical methods, and solvers, the physical process and its terminology were new to me. Understanding this process and the physical setup is part of deciding what the simulator should represent. AI can help me organize the background material and work through these concepts. When I am trying to understand an unfamiliar spatial process, I often ask it to “generate an interactive 3D teaching demo.” This gives me something to inspect and manipulate alongside the written explanation.

The same approach can help interns explore a research topic. In dynamics modeling, for example, an intern can use an AI-generated interactive tool to move sliders, inspect motion trajectories, and investigate where a method runs into trouble. The discussion can then focus on a particular behavior: why it is worth studying, and how the method might be improved.

Changing a parameter can prompt specific questions: Which assumption causes a particular behavior? Can the result be explained before the setup is changed again? Comparing the demonstration with the model described in the original paper or textbook is especially useful when the behavior is surprising or unclear.

If the goal is simply a working answer, the process may end once one is available. A desire to understand gives a learner a reason to keep asking questions, try a counterexample, or change the experiment. AI can make those steps easier, but access to it does not by itself supply that motivation. In mentoring, perhaps more time could go toward helping students find questions they want to pursue, alongside showing them how to use the tools.

4. The decoupling of performance and understanding

AI can support both aims of research, but an improvement in a method's performance does not always mean that the researcher understands why it works better. This is the possibility I mean by a decoupling of performance and understanding.

The gap itself is not new. I have heard of cases where researchers could not explain even the basic principles behind their own peer-reviewed papers. Forgetting details after a long time, or struggling to recall them when nervous, is understandable to me. The basic principles, though, are at the heart of a paper.

A researcher might also find a faster algorithm and only later work out which design choice made the difference. A machine-generated mathematical proof might take people much longer to check and understand than it took to produce. That gap need not be permanent. A result can become the starting point for further study, and AI can help with that study too. But, as always, making sense of an explanation and testing it against the evidence still requires someone's involvement.

Understanding also matters when deciding what an evaluation actually establishes. An evaluator may miss something a researcher cares about, or a test may pass without addressing the particular failure they are worried about. Finding those cases requires some understanding of the problem.

This raises a question about understanding: how much of a result should the person presenting it understand? A reasonable grasp of the main evidence and the conditions under which it holds makes it possible to discuss the result and respond to questions. As more of the process is automated, which parts deserve the closest attention remains an open question.

Consider someone who implements a geometric optimization method in Python or C++. During a presentation, should that person explain how the resulting machine code (binary instructions made up of 0s and 1s) runs, or focus on the optimization objective, the constraints, and how the problem is solved? Similarly, suppose someone proposes an acceleration strategy and implements it through vibe coding. Should they explain the internals of the standard Python libraries they import, or the key idea behind the speedup, why it works, and under what conditions?

5. Human attention is becoming more expensive, and there is only so much of it

Even with a strong motivation to understand, people have limited time to spend on each result. AI may make it much easier for researchers to produce information and run experiments, while the time available to examine the results is much harder to expand. In that sense, human attention is becoming more expensive: time spent carefully evaluating one result leaves less time to pursue another question or read someone else's work.

Researchers may end up with more information and experimental results than they can examine carefully. Choosing what to follow up, checking how a result was evaluated, and working out what the evidence supports all take time. Results may therefore improve faster than people can understand them, even when they want to keep up.

I expect this demand for attention to grow as researchers use AI tools for more tasks, although the pace seems likely to vary across fields. Access to useful data, experiments, and reliable feedback may make some settings easier to work in than others. Researchers may also use AI to improve the tools used to build AI, potentially speeding up search and experimentation further. How far such feedback can go remains an open question. If more of the search becomes automated, I wonder how much time people will have to explore the results.

Faster implementation and experimentation with AI tools could free up time, but whether that leads to deeper understanding depends on how researchers spend that time. Some of that time could go toward examining an unexpected result or revisiting a question that motivated the project. I find this possibility encouraging, though the changes ahead also leave me deeply concerned and anxious. I would like to keep discussing these questions with colleagues and students. The effects of today's frontier AI models on research and education are still unfolding...

Acknowledgments

Thanks to people in our lab for the discussions. I’m also grateful to everyone who has shared their perspectives on X; those discussions have given me a lot to think about. Special thanks to Clément Jambon—these thoughts grew out of our conversations over beers. 🍻