Accuracy tests compare outputs with an established answer or acceptance rule. Preference tests ask which output people favour. A response can be polished and preferred while factually wrong; a correct response can still be hard to use. Neither measure should silently stand in for the other.
Design separate checks
For factual tasks, evaluate support, completeness and appropriate abstention. For experience quality, use representative raters and clear criteria such as readability or usefulness. Randomise presentation where practical and record disagreements rather than hiding them inside one score.
Report the limits
Combine measures when the product needs both correctness and usability, but show what each result means. Keep examples of important failures and test changes against the same baseline. Avoid declaring a universal winner from a small task set or a preference vote alone.
Use separate scorecards for truth and preference
For a factual task, prepare an answer key and score supported correctness. For a writing task, define audience and style criteria and gather preference judgements separately. Hide product names where practical. This proposed evaluation prevents a polished but inaccurate answer from winning a factual test and avoids pretending that every stylistic preference has one objective answer. Publish the task, rubric and limitations with any eventual results.
Questions readers ask
Can user preference establish factual accuracy?
No. A preferred answer may still be wrong. Measure correctness against evidence and record style or usefulness preferences as a separate dimension.
Follow the evidence.
AI-assisted editorial decision guide. No hands-on benchmark, product score or winner is claimed. The proposed comparison is a method readers can run; current features, terms and results must be checked for the selected versions.
Public sources support the factual background. Analysis and illustrative exercises are identified in the text. How we assess AI · Editorial standards
Spot a factual error or an outdated detail?



