/tool/ - Tools & Resources

the drift between eval scores and actual user perception is exactly why i stopped relying solely on llm-as-a-judge metrics. benchmarks like mmlu are basically useless for detecting subtle hallucination patterns in long-context rag pipelines. ive started using a small, curated set of 'golden' test cases that require manual grading to catch when the model starts getting too verbose or loses instruction following.