Ask HN: How do I use LLMs to generate test cases for groundedness benchmarks? (opens in new tab)

(devblogs.microsoft.com)

1 pointsthis_steve_j5mo ago1 comments

1 comments

What are some ways to avoid common methological pitfalls when generating test cases for "groundedness" benchmarks with automation?

Confirmation bias is one obvious pitfall that comes to mind, but also I wonder how it is possible to achieve reproducibility when the input is stochastic.

j / k navigate · click thread line to collapse

1 comments

this_steve_jOP5mo ago

What are some ways to avoid common methological pitfalls when generating test cases for "groundedness" benchmarks with automation?

Confirmation bias is one obvious pitfall that comes to mind, but also I wonder how it is possible to achieve reproducibility when the input is stochastic.

j / k navigate · click thread line to collapse