Your optimization performance is only as good as the (meta)data behind it.
Your optimization performance is only as good as the (meta)data behind it.
Last week, I had the opportunity to speak at ACS Fall in Chicago, touching on a topic that can be easily overlooked.
In practice, when an optimization campaign underperforms, it’s not necessarily the surrogate model or acquisition function causing it but possibly one of these five gaps:
The reality gap. The variables in your model might only capture setpoints rather than realized conditions. If it’s only the former, the model will learn the wrong input-output relationshíp.
The traceability gap. A result exists, but the path from raw analytical output to the number that went into the model can't be reconstructed. You have a value and no way to audit it.
The reproducibility gap. The same conditions, run again, give a different answer. Sometimes that's chemistry. More often it's an unrecorded difference in execution (e.g. new processing script).
The accountability gap. When a campaign goes wrong, nobody can say which decision caused it, because the reasoning behind each round wasn't captured alongside the data.
The memory gap. Every campaign starts from zero. The last one produced knowledge that no future one can reach.
None of these are algorithm problems. They're all data and metadata problems, and they all get solved before the optimizer runs, not after.
Reliable learning is what makes data reusable.
And reuse compounds.
A campaign captured properly is a set of experiments you never have to run again.
