Sujet : llm

• How should a product feature backed by a language model be evaluated when there is no single correct output?

  Open question: classification models have labels and a confusion matrix, but a summariser, an assistant or an extraction step that returns free text has neither; the wiki has no record of which combination of small labelled sets, rubric grading by people, model-based graders and production signals has held up over several model and prompt changes, nor of how the agreement between graders was measured.

  question · en · unreviewed · connaissances au 2026-09-17

Lisible par machine : JSON