I’m using Weave to evaluate a model on sentence pairs generated from a long text, consisting of a reference sentence and a changing potentially connected sentence. However, I don’t want to evaluate all pairs (too expensive) I want to stop further evaluation for a reference sentence after detecting three consecutive sentences that didn’t show a connection (so I move away from the reference sentence on a sentence-by-sentence basis, evaluating each sentence on the way and ideally stopping after three turned out to not show a connection).
Is is possible to add a condition to stop the evaluation loop when that condition is met? And if not, how would you set up an Evaluation for such a use case?
We wanted to follow up with you regarding your support request as we have not heard back from you. Please let us know if we can be of further assistance or if your issue has been resolved.