Researchers have introduced BBOWP-Bench, a benchmark for testing whether large language models can formulate black-box optimization problems from natural language. In these tasks, the objective function is not available as a clear mathematical expression, and only observed results guide the search.
That makes the benchmark different from many optimization word-problem tests. The model must infer a useful search space and choose an optimization algorithm, not merely translate a problem into equations.
Each benchmark instance includes a natural-language description, an executable evaluation environment, and a human-designed baseline formulation. That setup lets researchers compare both the model’s framing of the problem and the resulting optimization performance.
The work is early-stage research, but it targets a practical gap. Many real design, scheduling, and tuning problems are black boxes, and useful AI assistants will need to reason about how to search them rather than only solve neatly specified formulas.