Hi everyone — we’re introducing Retrieval Pipeline Optimization, a new way to compare retrieval setups directly against your own data.
Choosing an embedding model or reranker is rarely one-size-fits-all. A setup that performs well on a public benchmark may behave very differently with your documents, queries, and users. Testing those combinations manually can also require a lot of repetitive setup and analysis.
Retrieval Pipeline Optimization brings that comparison into GoodMem. Select the models you want to consider, provide or generate evaluation queries, and choose how much effort to spend on the experiment. GoodMem then evaluates the options consistently and presents the strongest retrieval pipelines in a single ranked view.
The result is intended to support a decision—not merely produce another benchmark table. It shows which pipelines performed best, when multiple options are effectively close, and how much confidence the experiment provides. This makes it easier to decide whether to adopt a leading pipeline, choose between comparable options based on cost or latency, or gather more evidence before making a change.
With Retrieval Pipeline Optimization, you can:
- Compare embedders and rerankers using your own content.
- Replace one-off testing with a repeatable experiment.
- Understand when a small score difference is meaningful—and when it may not be.
- Make retrieval decisions with quality, cost, and latency in mind.
You can find the feature under Tuning → Optimize pipelines in GoodMem.
We’re continuing to improve the experience and would love to hear how it works with your data. If you try it, let us know what you found and what you would like to see next.