A patent-drafting experiment suggests that language models can help another AI agent revise complex professional work, but cannot yet stand in for expert review without qualification. The Vibe Patenting testbed has a drafting agent produce patent applications and a separately invoked model judge return structured feedback for repeated revision.
Across several inventions and agent configurations, judge-guided drafts improved on the judge’s own quality measures, while revisions made without guidance tended to level off. The researchers also report that a lower-reasoning, cheaper agent approached the judge-assessed performance of a substantially more expensive high-reasoning agent after iterative feedback.
That result carries an obvious risk: optimizing against an automated judge can improve the score without improving the document in ways a professional values. To test that, the team compared the model judge with an independent patent attorney. Agreement was meaningful, but depended strongly on the selected metric, and the evaluators showed systematic differences in calibration.
The study therefore supports model judges as an optimization signal, not as an unquestioned final authority. Stronger models, more reasoning and workflows tailored to patent drafting all improved judge-assessed quality, but attorney validation still exposed measurement limits. The paper is a new arXiv preprint and does not establish that generated applications meet legal requirements in real filings.