Evaluate Systems
TREC ad hoc and routing evaluations
TREC interactive track
introduces the user as a component but not the
problems, perceived needs, and actions
Hybrid solutions
human + automatic
statistical + natural language processing