Dear Sir or Madam, Will You Read My Paper?
Kevin Bryan asks a fun little question in a nice thread over on Twitter:
Now, if you're plugged into developments in this part of the world, you might think Kevin is talking about Deep Research. But as he goes on to explain in the thread, that is "only a dollar of compute".
What he is talking about is something very different. What would it cost to set up a system involving multiple AI's that can talk to each other, in order to
a) generate ideas, or a body of research
b) Do the literature review
c) Get data, potentially from different sources, do a merge and a data cleaning exercise
d)Write the theory and/or the statistical model
e) Write the paper and all of the associated paraphernalia
Is it possible, today, to create something like this? I am thinking about primarily economics based research, but you can of course think about any academic endeavor you like.
He says yes, it is, and I agree. Certainly within the realms of possibility, and while it would not be cheap, it would cost maybe two thousand dollars of compute to get a paper out of such a system (his estimate, not mine, and it is just that - an estimate. There's no formal model there, but if anything, my guess is that this would be even cheaper, both to set up and to run).
And please note that I am not saying that quality will be uniformly high, across all of the tasks in pts. a) through e). It will vary, both for each paper, and across all papers. You may need humans in the loop for all tasks to begin with, and maybe you can never eliminate all humans in the loop, of course.
What do the economics of this look like?
What happens to the supply of quality papers written by AI in such a world?
What happens to the supply of quality paper written by humans in such a world?
What happens to the demand for quality papers (whether written by AI or by humans) in such a world?
What does this do to the paper review process in top tier journals?
What does this do to the body of research work that is going to come out in the near future?
What does this to folks who are hoping to get into research as a career?
What does this do to the future of researchers, both those currently employed in universities the world over, and to universities themselves?
Should we be teaching undergraduate and postgraduate students today to produce research, consumer research, or guide research? What weightage to each of these, and how do you decide?
Does this drive up the incentive to share research across universities, between firms, and across nations, or does this drive the incentive down?
And a question that Kevin asks and answers in his thread, please go read it: why has this not been built yet?
And if all that was too abstract for you, here, read this:
Before filing, applicants would submit their application to a certification entity that would conduct a thorough prior art search and patentability analysis. There may be a certification process – that requires the entity to meet quality metrics parallel to those used by the USPTO internally. Applications certified as meeting patentability requirements would then be eligible for highly streamlined USPTO registration process.
Although AI tools would not be a required element of this process, I expect that they would be integral to the process of both certification entities and patent applicants seeking to navigate this new system effectively. Modern AI systems, particularly those built on large language models and specialized patent examination databases, could dramatically enhance the preliminary examination process by identifying relevant prior art, conducting an analysis for both anticipation and obviousness, suggesting claim amendments, and flagging potential issues. Leaving quality aside for a moment, the benefit of the AI system is its potential cost effectiveness and its timeliness. In particular, this process could compress the examination process into a single day rather than the multi-year scenario that 95% of applicants face.
While this proposal is primarily a thought experiment, its greatest value may be in demonstrating how AI-driven preliminary analysis and immediate feedback could be implemented within the USPTO’s existing framework to improve patent examination.
Here's the author's background, if you were curious. Use AI to draft patent applications, and use AI to vet patent applications.
What will happen to the process of writing, reviewing and publishing research, do you think?
What should happen, do you think?
What should academia be doing about all this today, and why, and how?
Who is asking and answering these questions in Indian academia today?
P.S. Kevin himself has a paper on this topic: AI-Assisted Academic Writing: Adjustments in Quantitative Social Science
Exciting times, that much is for sure.