Review of Interactive Information Retrieval Evaluation (RIIRE)

Author: Diane Kelly, University of North Carolina, USA
Author: Cassidy R. Sugimoto, Indiana University, USA

Summary: The following articles were used in a research study investigating evaluation methods used in Interactive Information Retrieval evaluations. This web page functions as an Appendix to the paper and lists the articles examined.

Citation: Kelly, D. and Sugimoto, C. R. A Systematic Review of Interactive Information Retrieval Evaluation Studies, 1967-2006.

Acknowledgements:  Thanks to Maxwell Felsher for preparing this bibliography.

Aaronson, S. (1997). Optimal demand-oriented topology for hypertext systems. In Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 168-177). Philadelphia, Pennsylvania, United States: ACM. doi:10.1145/258525.258560

Akeroyd, J. (1990). Information Seeking in Online Catalogues. Journal of Documentation, 46(1), 33 - 52. doi:10.1108/eb026851  

Allan, J., Leuski, A., Swan, R., & Byrd, D. (2001). Evaluating combinations of ranked lists and visualizations of inter-document similarity. Information Processing & Management, 37(3), 435-458. doi:10.1016/S0306-4573(00)00056-X  

Allen, B. (1998a). Information space representation in interactive systems: relationship to spatial abilities. In Proceedings of the third ACM conference on Digital libraries (pp. 1-10). Pittsburgh, Pennsylvania, United States: ACM. doi:10.1145/276675.276676

Allen, B. (1998b). Designing information systems for user abilities and tasks: An experimental study. Online and CD-ROM Review, 22(3), 139 - 153. doi:10.1108/eb024663  

Amento, B., Hill, W., Terveen, L., Hix, D., & Ju, P. (1999). An empirical evaluation of user interfaces for topic management of Web sites. In Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit (pp. 552-559). Pittsburgh, Pennsylvania, United States: ACM. doi:10.1145/302979.303156

Amitay, E., Darlow, A., Konopnicki, D., & Weiss, U. (2005). Queries as anchors: selection by association. In Proceedings of the sixteenth ACM conference on Hypertext and hypermedia (pp. 193-201). Salzburg, Austria: ACM. doi:10.1145/1083356.1083393

Belkin, N. J., Cool, C., Kelly, D., Kim, G., Kim, J., Lee, H., Muresan, G., et al. (2003). Query length in interactive information retrieval. In Proceedings of the 26th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 205-212). Toronto, Canada: ACM. doi:10.1145/860435.860474

Belkin, N. J., Cool, C., Kelly, D., Lin, S.-J., Park, S. Y., Perez-Carballo, J., & Sikora, C. (2001). Iterative exploration, design and evaluation of support for query reformulation in interactive information retrieval. Information Processing & Management, 37(3), 403-434. doi:10.1016/S0306-4573(00)00055-8  

Bell, D. J., & Ruthven, I. (2004). Searcher’s Assessments of Task Complexity for Web Searching. In Advances in Information Retrieval (pp. 57-71). Presented at the ECIR 2004, Berlin: Springer. doi:10.1007/b96895

Berenci, E., Carpineto, C., Giannini, V., & Mizzaro, S. (2000). Effectiveness of keyword-based display and selection of retrieval results for interactive searches. International Journal on Digital Libraries, 3(3), 249-260. doi:10.1007/s007990000035  

Brajnik, G., Mizzaro, S., & Tasso, C. (1996). Evaluating user interfaces to information retrieval systems: a case study on user support. In Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 128-136). Zurich, Switzerland: ACM. doi:10.1145/243199.243249

Carpineto, C., & Romano, G. (1996). Information retrieval through hybrid navigation of lattice representations. International Journal of Human-Computer Studies, 45(5), 553-578. doi:10.1006/ijhc.1996.0067  

Ceaparu, I., & Shneiderman, B. (2004). Finding governmental statistical data on the Web: A study of categorically organized links for the FedStats topics page. Journal of the American Society for Information Science and Technology, 55(11), 1008-1015. doi:10.1002/asi.20052  

Chen, C. (2000). Individual differences in a spatial-semantic virtual environment. Journal of the American Society for Information Science, 51(6), 529-542. doi:10.1002/(SICI)1097-4571(2000)51:6<529::AID-ASI5>3.0.CO;2-F  

Chen, H., & Dumais, S. (2000). Bringing order to the Web: automatically categorizing search results. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 145-152). The Hague, The Netherlands: ACM. doi:10.1145/332040.332418

Chen, H., Houston, A. L., Sewell, R. R., & Schatz, B. R. (1998). Internet browsing and searching: User evaluations of category map and concept space techniques. Journal of the American Society for Information Science, 49(7), 582-603. doi:10.1002/(SICI)1097-4571(19980515)49:7<582::AID-ASI2>3.0.CO;2-X  

Chen, H., Ng, T. D., Martinez, J., & Schatz, B. R. (1997). A concept space approach to addressing the vocabulary problem in scientific information retrieval: An experiment on the Worm Community System. Journal of the American Society for Information Science, 48(1), 17-31. doi:10.1002/(SICI)1097-4571(199701)48:1<17::AID-ASI4>3.0.CO;2-4  

Chung, W., Zhang, Y., Huang, Z., Wang, G., Ong, T., & Chen, H. (2004). Internet searching and browsing in a multilingual world: An experiment on the Chinese Business Intelligence Portal (CBizPort). Journal of the American Society for Information Science and Technology, 55(9), 818-831. doi:10.1002/asi.20025  

Cockburn, A., & McKenzie, B. (2000). An evaluation of cone trees. In People and Computers XIV - Usability or Else! Proceedings of HCI 2000 (pp. 425-436). London: Springer-Verlag. Retrieved from http://hdl.handle.net/10092/522

Dalrymple, P. W. (1990). Retrieval by reformulation in two library catalogs: Toward a cognitive model of searching behavior. Journal of the American Society for Information Science, 41(4), 272-281. doi:10.1002/(SICI)1097-4571(199006)41:4<272::AID-ASI6>3.0.CO;2-5  

Dalrymple, P. W., & Zweizig, D. L. (1992). Users' experience of information retrieval systems: An exploration of the relationship between search experience and affective measures. Library & Information Science Research, 14(2), 167-81.  

Das Neves, F. A., & Fox, E. A. (2000). A study of user behavior in an immersive virtual environment for digital libraries. In Proceedings of the fifth ACM conference on Digital libraries (pp. 103-111). San Antonio, Texas, United States: ACM. doi:10.1145/336597.336648

Davis, C. H., & Shaw, D. (1989). Comparison of retrieval system interfaces using an objective measure of screen design effectiveness. Library & Information Science Research, 11(4), 325-34.  

Dempsey, B. J., Vreeland, R. C., Sumner, R. G., & Yang, K. (2000). Design and empirical evaluation of search software for legal professionals on the WWW. Information Processing & Management, 36(2), 253-273. doi:10.1016/S0306-4573(99)00057-6  

Dennis, S., Bruza, P., & McArthur, R. (2002). Web searching: A process-oriented experimental study of three interactive search paradigms. Journal of the American Society for Information Science and Technology, 53(2), 120-133. doi:10.1002/asi.10015  

Dimitroff, A., & Wolfram, D. (1995). Searcher response in a hypertext-based bibliographic information retrieval system. Journal of the American Society for Information Science, 46(1), 22-29. doi:10.1002/(SICI)1097-4571(199501)46:1<22::AID-ASI3>3.0.CO;2-D  

Dimitroff, A., Wolfram, D., & Volz, A. (1996). Affective response and retrieval performance: Analysis of contributing factors. Library & Information Science Research, 18(2), 121-132. doi:10.1016/S0740-8188(96)90015-8  

Drabenstott, K. M., & Weller, M. S. (1996). Failure analysis of subject searches in a test of a new design for subject access to online catalogs. Journal of the American Society for Information Science, 47(7), 519-537. doi:10.1002/(SICI)1097-4571(199607)47:7<519::AID-ASI5>3.0.CO;2-X  

Drori, O. (2000). The Benefits of Displaying Additional Internal Document Information on Textual Database Search Result Lists. In Research and Advanced Technology for Digital Libraries (pp. 69-82). Presented at the ECDL 2000, Berlin: Springer. doi:10.1007/3-540-45268-0_7

Drori, O., & Tamir, E. (2005). Display of search results in Hebrew: a comparison study between Google and LCC&K interface. Journal of Information Science, 31(3), 164-177. doi:10.1177/0165551505052345  

Dumais, S., Cutrell, E., & Chen, H. (2001). Optimizing search by showing results in context. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 277-284). Seattle, Washington, United States: ACM. doi:10.1145/365024.365116

Egan, D. E., Lesk, M. E., Ketchum, R. D., Lochbaum, C. C., Remde, J. R., Littman, M., & Landauer, T. K. (1991). Hypertext for the electronic library?: CORE sample results. In Proceedings of the third annual ACM conference on Hypertext (pp. 299-312). San Antonio, Texas, United States: ACM. doi:10.1145/122974.123002

Egan, D. E., Remde, J. R., Gomez, L. M., Landauer, T. K., Eberhardt, J., & Lochbaum, C. C. (1989). Formative design-evaluation of SuperBook. ACM Transactions on Information Systems, 7(1), 30-57. doi:10.1145/64789.64790  

Finkelstein, L., Gabrilovich, E., Matias, Y., Rivlin, E., Solan, Z., Wolfman, G., & Ruppin, E. (2002). Placing search in context: the concept revisited. ACM Transactions on Information Systems, 20(1), 116-131. doi:10.1145/503104.503110  

Fox, E. A., France, R. K., Sahle, E., Daoud, A., & Cline, B. E. (1993). Development of a modern OPAC: from REVTOLC to MARIAN. In Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 248-259). Pittsburgh, Pennsylvania, United States: ACM. doi:10.1145/160688.160730

Fraser, L., & Locatis, C. (2001). Effects of link annotations on search performance in layered and unlayered hierarchically organized information spaces. Journal of the American Society for Information Science and Technology, 52(14), 1255-1261. doi:10.1002/asi.1194  

Gauch, S., & Smith, J. B. (1991). Search improvement via automatic query reformulation. ACM Transactions on Information Systems, 9(3), 249-280. doi:10.1145/125187.125193  

Golovchinsky, G. (1997). Queries? Links? Is there a difference? In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 407-414). Atlanta, Georgia, United States: ACM. doi:10.1145/258549.258820

Gomez, L. M., Lochbaum, C. C., & Landauer, T. K. (1990). All the right words: Finding what you want as a function of richness of indexing vocabulary. Journal of the American Society for Information Science, 41(8), 547-559. doi:10.1002/(SICI)1097-4571(199012)41:8<547::AID-ASI1>3.0.CO;2-P  

Griffiths, J. R. (2003). Evaluation of the JISC Information Environment: student perceptions of services. Information Research, 8(4), paper no. 160. Retrieved from http://informationr.net/ir/8-4/paper160.html  

Hammer-Aebi, B., Christensen, K. W., Lund, H., & Larsen, B. (2006). Users, structured documents and overlap: interactive searching of elements and the influence of context on search behaviour. In Proceedings of the 1st international conference on Information interaction in context (pp. 46-55). Copenhagen, Denmark: ACM. doi:10.1145/1164820.1164833

Harper, D. J., & Kelly, D. (2006). Contextual relevance feedback. In Proceedings of the 1st international conference on Information interaction in context (pp. 129-137). Copenhagen, Denmark: ACM. doi:10.1145/1164820.1164847

Heo, M., & Hirtle, S. C. (2001). An empirical comparison of visualization tools to assist information retrieval on the web. Journal of the American Society for Information Science and Technology, 52(8), 666-675. doi:10.1002/asi.1117  

Hersh, W., Pentecost, J., & Hickam, D. (1996). A task-oriented approach to information retrieval evaluation. Journal of the American Society for Information Science, 47(1), 50-56. doi:10.1002/(SICI)1097-4571(199601)47:1<50::AID-ASI5>3.0.CO;2-1  

Hersh, W., Turpin, A., Price, S., Kraemer, D., Olson, D., Chan, B., & Sacherek, L. (2001). Challenging conventional assumptions of automated information retrieval with real users: Boolean searching and batch retrieval evaluations. Information Processing & Management, 37(3), 383-402. doi:10.1016/S0306-4573(00)00054-6  

Hersh, W. R., Elliot, D. L., Hickam, D. H., Wolf, S. L., Molnar, A., & Leichtenstein, C. (1995). Towards new measures of information retrieval evaluation. In Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 164-170). Seattle, Washington, United States: ACM. doi:10.1145/215206.215355

Hersh, W. R., & Hickam, D. H. (1992). A comparison of two methods for indexing and retrieval from a full-text medical database. In Celebrating change: information management on the move, ASIS '92, Proceedings of the 55th annual meeting of the American Society for Information Science (pp. 221-230). Pittsburgh, Pennsylvania, United States: American Society for Information Science.

Hersh, W. R., & Hickam, D. H. (1995). An evaluation of interactive Boolean and natural language searching with an online medical textbook. Journal of the American Society for Information Science, 46(7), 478-489. doi:10.1002/(SICI)1097-4571(199508)46:7<478::AID-ASI2>3.0.CO;2-#  

Hertzum, M., & Frokjaer, E. (1996). Browsing and querying in online documentation: a study of user interfaces and the interaction process. ACM Transactions on Computer-Human Interaction, 3(2), 136-161. doi:10.1145/230562.230570  

Hook, K. (1997). Evaluating the utility and usability of an adaptive hypermedia system. In Proceedings of the 2nd international conference on Intelligent user interfaces (pp. 179-186). Orlando, Florida, United States: ACM. doi:10.1145/238218.238320

Hsu, Y., & Schwen, T. M. (2003). The effects of structural cues from multiple metaphors on computer users' information search performance. International Journal of Human-Computer Studies, 58(1), 39-55. doi:10.1016/S1071-5819(02)00125-8  

Jahoda, G., & Stursa, M. L. (1969). A comparison of a keyword from title index with a single access point per document alphabetic subject index. American Documentation, 20(4), 377-380. doi:10.1002/asi.4630200422  

Jansen, B. J., & McNeese, M. D. (2005). Evaluating the effectiveness of and patterns of interactions with automated searching assistance. Journal of the American Society for Information Science and Technology, 56(14), 1480-1503. doi:10.1002/asi.20242  

Jansen, B. J., & Resnick, M. (2006). An examination of searcher's perceptions of nonsponsored and sponsored links during ecommerce Web searching. Journal of the American Society for Information Science and Technology, 57(14), 1949-1961. doi:10.1002/asi.20425  

Joho, H., & Jose, J. M. (2006a). A Comparative Study of the Effectiveness of Search Result Presentation on the Web. In Advances in Information Retrieval (pp. 302-313). Presented at the ECIR 2006, Berlin: Springer. doi:10.1007/11735106_27

Joho, H., & Jose, J. M. (2006b). Slicing and dicing the information space using local contexts. In Proceedings of the 1st international conference on Information interaction in context (pp. 66-74). Copenhagen, Denmark: ACM. doi:10.1145/1164820.1164836

Joho, H., Sanderson, M., & Beaulieu, M. (2004). A Study of User Interaction with a Concept-Based Interactive Query Expansion Support Tool. In Advances in Information Retrieval (pp. 42-56). Presented at the ECIR 2004, Berlin: Springer. doi:10.1007/b96895

Ju, B., & Gluck, M. (2005). User-process model approach to improve user interface usability. Journal of the American Society for Information Science and Technology, 56(10), 1098-1112. doi:10.1002/asi.20201  

Kabel, S., de Hoog, R., Wielinga, B. J., & Anjewierden, A. (2004). The added value of task and ontology-based markup for information retrieval. Journal of the American Society for Information Science and Technology, 55(4), 348-362. doi:10.1002/asi.10379  

Kaki, M. (2006). fKWIC: Frequency-based keyword-in-context index for filtering Web search results. Journal of the American Society for Information Science and Technology, 57(12), 1606-1615. doi:10.1002/asi.20338  

Kim, H. H. (2005). ONTOWEB: Implementing an ontology-based Web retrieval system. Journal of the American Society for Information Science and Technology, 56(11), 1167-1176. doi:10.1002/asi.20220  

Kim, J. (2005). Finding documents in a digital institutional repository: DSpace and Eprints. In Proceedings of the American Society for Information Science and Technology. doi:10.1002/meet.1450420173

Klas, C., Fuhr, N., & Schaefer, A. (2004). Evaluating Strategic Support for Information Access in the DAFFODIL System. In Research and Advanced Technology for Digital Libraries (pp. 476-487). Presented at the ECDL 2004, Berlin: Springer. doi:10.1007/b100389

Klein, A., Schwank, I., Genereux, M., & Trost, H. (2001). Evaluating multi-modal input modes in wizard-of-oz study for the domain of web search. In People and Computers XV - Interaction without Frontiers, Joint Proceedings of HCI 2001 and IHM 2001 (pp. 475-483). London: Springer-Verlag.

Koenemann, J., & Belkin, N. J. (1996). A case for interaction: a study of interactive information retrieval behavior and effectiveness. In Proceedings of the SIGCHI conference on Human factors in computing systems: common ground (pp. 205-212). Vancouver, British Columbia, Canada: ACM. doi:10.1145/238386.238487

Koshman, S. (2004). Comparing usability between a visualization and text-based system for information retrieval. Journal of Documentation, 60(5), 565 - 580. doi:10.1108/00220410410560627  

Koshman, S. (2005). Testing user interaction with a prototype visualization-based information retrieval system. Journal of the American Society for Information Science and Technology, 56(8), 824-833. doi:10.1002/asi.20175  

Kruschwitz, U., & Al-Bakour, H. (2005). Users want more sophisticated search assistants: Results of a task-based evaluation. Journal of the American Society for Information Science and Technology, 56(13), 1377-1393. doi:10.1002/asi.20230  

Larson, R. R. (2001). TREC interactive with Cheshire II. Information Processing & Management, 37(3), 485-505. doi:10.1016/S0306-4573(00)00058-3  

Leroy, G., Lally, A. M., & Chen, H. (2003). The use of dynamic contexts to improve casual internet searching. ACM Transactions on Information Systems, 21(3), 229-253. doi:10.1145/858476.858477  

Ma, W. (2002). A database selection expert system based on reference librarian's database selection strategy: A usability and empirical evaluation. Journal of the American Society for Information Science and Technology, 53(7), 567-580. doi:10.1002/asi.10067  

Mana-Lopez, M. J., de Buenaga, M., & Gomez-Hidalgo, J. M. (2004). Multidocument summarization: An added value to clustering in interactive retrieval. ACM Transactions on Information Systems, 22(2), 215-241. doi:10.1145/984321.984323  

Marchionini, G., & Liebscher, P. (1991). Performance in Electronic Encyclopedias: Implications for Adaptive Systems. In Systems understanding people, ASIS '91, proceedings of the 54th ASIS annual meeting (pp. 39-48).

Marshall, B., McDonald, D., Chen, H., & Chung, W. (2004). EBizPort: Collecting and analyzing business intelligence information. Journal of the American Society for Information Science and Technology, 55(10), 873-891. doi:10.1002/asi.20037  

Meadow, C. T., Wang, J., & Yuan, W. (1995). A study of user performance and attitudes with information retrieval interfaces. Journal of the American Society for Information Science, 46(7), 490-505. doi:10.1002/(SICI)1097-4571(199508)46:7<490::AID-ASI3>3.0.CO;2-I  

Meyyappan, N., Foo, S., & Chowdhury, G. G. (2004). Design and evaluation of a task-based digital library for the academic community. Journal of Documentation, 60(4), 449 - 475. doi:10.1108/00220410410548162  

Morehead, D. R., & Rouse, W. B. (1983). Human-computer interaction in information seeking tasks. Information Processing & Management, 19(4), 243-253. doi:10.1016/0306-4573(83)90021-3  

Morehead, D. R., & Rouse, W. B. (1985). Online assessment of the value of information for searchers of a bibliographic data base. Information Processing & Management, 21(2), 83-101. doi:10.1016/0306-4573(85)90019-6  

Mynatt, B. T., Leventhal, L. M., Instone, K., Farhat, J., & Rohlman, D. S. (1992). Hypertext or book: Which is better for answering questions? In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 19-25). Monterey, California, United States: ACM. doi:10.1145/142750.142753

Olston, C., & Chi, E. H. (2003). ScentTrails: Integrating browsing and searching on the Web. ACM Transactions on Computer-Human Interaction, 10(3), 177-197. doi:10.1145/937549.937550  

Paek, T., Dumais, S., & Logan, R. (2004). WaveLens: A new view onto Internet search results. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 727-734). Vienna, Austria: ACM. doi:10.1145/985692.985784

Park, S. (1999). User preferences when searching individual and integrated full-text databases. In Proceedings of the fourth ACM conference on Digital libraries (pp. 195-203). Berkeley, California, United States: ACM. doi:10.1145/313238.313307

Park, S. (2000). Usability, user preferences, effectiveness, and user behaviors when searching individual and integrated full-text databases: implications for digital libraries. Journal of the American Society for Information Science, 51(5), 456-468. doi:10.1002/(SICI)1097-4571(2000)51:5<456::AID-ASI6>3.0.CO;2-O  

Petrelli, D., Levin, S., Beaulieu, M., & Sanderson, M. (2006). Which user interaction for cross-language information retrieval? Design issues and reflections. Journal of the American Society for Information Science and Technology, 57(5), 709-722. doi:10.1002/asi.20332  

Pirolli, P., Card, S. K., & Van Der Wege, M. M. (2003). The effects of information scent on visual search in the hyperbolic tree browser. ACM Transactions on Computer-Human Interaction, 10(1), 20-53. doi:10.1145/606658.606660  

Pirolli, P., Schank, P., Hearst, M., & Diehl, C. (1996). Scatter/gather browsing communicates the topic structure of a very large text collection. In Proceedings of the SIGCHI conference on Human factors in computing systems: common ground (pp. 213-220). Vancouver, British Columbia, Canada: ACM. doi:10.1145/238386.238489

Qu, Y., Furnas, G., & Walstrum, B. (2006). Using Category Information for Relationship Exploration in Textual Data. In Proceedings of the American Society for Information Science and Technology. doi:10.1002/meet.14504301163

Reiterer, H., Mubler, G., Mann, T. M., & Handschuh, S. (2000). INSYDER - an information assistant for business intelligence. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval (pp. 112-119). Athens, Greece: ACM. doi:10.1145/345508.345559

Rinewalt, J. R. (1977). Feature evalution of a full-text information-retrieval system. Online Review, 1(1), 43 - 52. doi:10.1108/eb023934  

Risden, K., Czerwinski, M. P., Munzner, T., & Cook, D. B. (2000). An initial examination of ease of use for 2D and 3D information visualizations of Web content. International Journal of Human-Computer Studies, 53(5), 695-714. doi:10.1006/ijhc.2000.0413  

Ruthven, I., Lalmas, M., & van Rijsbergen, K. (2003). Incorporating user search behavior into relevance feedback. Journal of the American Society for Information Science and Technology, 54(6), 529-549. doi:10.1002/asi.10240  

Salampasis, M., & Tait, J. (2005). The Effect of Collection Fusion Strategies on Information Seeking Performance in Distributed Hypermedia Digital Libraries. In Research and Advanced Technology for Digital Libraries, Lecture Notes in Computer Science (Vol. 3652, pp. 57-68). Presented at the ECDL 2005, Berlin: Springer-Verlag. doi:10.1007/11551362_6

Sengupta, A., & Dillon, A. (1997). Query by templates: a generalized approach for visual query formulation for text dominated databases. In Proceedings, IEEE International Forum on Research and Technology Advances in Digital Libraries (pp. 36-47). Los Alamitos, CA, USA: IEEE Computer Society. doi:10.1109/ADL.1997.601198

Shapira, B., Taieb-Maimon, M., & Nemeth, Y. (2005). Subjective and objective evaluation of interactive and automatic query expansion. Online Information Review, 29(4), 374 - 390. doi:10.1108/14684520510617820  

Slack, F. E. (1989). Subject searching on OPACs: Who needs help? In Proceedings of the 11th BCS IRSG Research Colloquium on Information Retrieval (pp. 72-82). British Computer Society.

Smyth, B., & Balfe, E. (2006). Anonymous personalization in collaborative web search. Information Retrieval, 9(2), 165-190. doi:10.1007/s10791-006-7148-z  

Stein, A. (1997). Usability and assessments of multimodal interaction in the SPEAK! system: an experimental case study. New Review of Hypermedia and Multimedia, 3(1), 159-180. doi:10.1080/13614569708914688  

Suomela, S., & Kekalainen, J. (2006). User evaluation of ontology as query construction tool. Information Retrieval, 9(4), 455-475. doi:10.1007/s10791-006-6387-3  

Swan, R. C., & Allan, J. (1998). Aspect windows, 3-D visualizations, and indirect comparisons of information retrieval systems. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval (pp. 173-181). Melbourne, Australia: ACM. doi:10.1145/290941.290987

Szlavik, Z., Tombros, A., & Lalmas, M. (2006). The Use of Summaries in XML Retrieval. In Research and Advanced Technology for Digital Libraries, Lecture Notes in Computer Science (Vol. 4172, pp. 75-86). Presented at the ECDL 2006, Berlin: Springer-Verlag. doi:10.1007/11863878_7

Tague, J., & Schultz, R. (1989). Evaluation of the user interface in an information retrieval system: A model. Information Processing & Management, 25(4), 377-389. doi:10.1016/0306-4573(89)90066-6  

Tague, J. M. (1981). User-responsive subject control in bibliographic retrieval systems. Information Processing & Management, 17(3), 149-156. doi:10.1016/S0306-4573(81)80006-4  

Tombaugh, J., Lickorish, A., & Wright, P. (1987). Multi-window displays for readers of lengthy texts. International Journal of Man-Machine Studies, 26(5), 597-615. doi:10.1016/S0020-7373(87)80073-1  

Topi, H., & Lucas, W. (2005a). Searching the Web: operator assistance required. Information Processing & Management, 41(2), 383-403. doi:10.1016/S0306-4573(02)00092-4  

Topi, H., & Lucas, W. (2005b). Mix and match: combining terms and operators for successful Web searches. Information Processing & Management, 41(4), 801-817. doi:10.1016/j.ipm.2004.03.007  

Turpin, A. H., & Hersh, W. (2001). Why batch and user evaluations do not give the same results. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 225-231). New Orleans, Louisiana, United States: ACM. doi:10.1145/383952.383992

Vakkari, P., Jones, S., MacFarlane, A., & Sormunen, E. (2004). Query exhaustivity, relevance feedback and search success in automatic and interactive query expansion. Journal of Documentation, 60(2), 109 - 127. doi:10.1108/00220410410522016  

Veerasamy, A., & Belkin, N. J. (1996). Evaluation of a tool for visualization of information retrieval results. In Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 85-92). Zurich, Switzerland: ACM. doi:10.1145/243199.243218

Veerasamy, A., & Heikes, R. (1997). Effectiveness of a graphical display of retrieval results. In Proceedings of the 20th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 236-245). Philadelphia, Pennsylvania, United States: ACM. doi:10.1145/258525.258580

Wacholder, N., & Liu, L. (2006). User preference: A measure of query-term quality. Journal of the American Society for Information Science and Technology, 57(12), 1566-1580. doi:10.1002/asi.20315  

Wang, X., & Liebscher, P. (1988). Information seeking in hypertext: effects of physical format and search strategy. In Information & Technology: planning for the second 50 years, ASIS '88, proceedings of the 51st ASIS annual meeting (pp. 200-204). American Society for Information Science.

Watters, C., Shepherd, M. A., & Qiu, L. (1994). Task-oriented access to data files: An evaluation. Journal of the American Society for Information Science, 45(4), 251-262. doi:10.1002/(SICI)1097-4571(199405)45:4<251::AID-ASI3>3.0.CO;2-4  

White, R. W., Jose, J. M., & Ruthven, I. (2003). A task-oriented study on the influencing effects of query-biased summarisation in web searching. Information Processing & Management, 39(5), 707-733. doi:10.1016/S0306-4573(02)00033-X  

White, R. W., Jose, J. M., & Ruthven, I. (2006). An implicit feedback approach for interactive information retrieval. Information Processing & Management, 42(1), 166-190. doi:10.1016/j.ipm.2004.08.010  

White, R. W., & Ruthven, I. (2006). A study of interface support mechanisms for interactive information retrieval. Journal of the American Society for Information Science and Technology, 57(7), 933-948. doi:10.1002/asi.20365  

White, R. W., Ruthven, I., & Jose, J. M. (2002). Finding relevant documents using top ranking sentences: an evaluation of two alternative schemes. In Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 57-64). Tampere, Finland: ACM. doi:10.1145/564376.564389

White, R. W., Ruthven, I., & Jose, J. M. (2005). A study of factors affecting the utility of implicit relevance feedback. In Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 35-42). Salvador, Brazil: ACM. doi:10.1145/1076034.1076044

Wiesman, F., van den Herik, H. J., & Hasman, A. (2004). Information retrieval by metabrowsing. Journal of the American Society for Information Science and Technology, 55(7), 565-578. doi:10.1002/asi.10410  

Williams, M. E., Kinnucan, M., Smith, L. C., Lannom, L., & Cho, D. (1986). Comparative analysis of online retrieval interfaces. In ASIS '86, proceedings of the 49th ASIS annual meeting (pp. 365-370). American Society for Information Science.

Wolfram, D., & Dimitroff, A. (1998). Hypertext vs boolean-based searching in a bibliographic database environment: A direct comparison of searcher performance. Information Processing & Management, 34(6), 669-679. doi:10.1016/S0306-4573(98)00035-1  

Wolfram, D., Volz, A., & Dimitroff, A. (1996). The effect of linkage structure on retrieval performance in a hypertext-based bibliographic retrieval system. Information Processing & Management, 32(5), 529-541. doi:10.1016/0306-4573(96)00027-1  

Woodruff, A., Rosenholtz, R., Morrison, J. B., Faulring, A., & Pirolli, P. (2002). A comparison of the use of text summaries, plain thumbnails, and enhanced thumbnails for Web search tasks. Journal of the American Society for Information Science and Technology, 53(2), 172-185. doi:10.1002/asi.10029  

Wu, M., Fuller, M., & Wilkinson, R. (2001a). Searcher performance in question answering. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval (pp. 375-381). New Orleans, Louisiana, United States: ACM. doi:10.1145/383952.384028

Wu, M., Fuller, M., & Wilkinson, R. (2001b). Using clustering and classification approaches in interactive retrieval. Information Processing & Management, 37(3), 459-484. doi:10.1016/S0306-4573(00)00057-1  

Wu, Z., Ramsden, A., & Zhao, D. (1995). The user perspective of the ELINOR electronic library. Aslib Proceedings, 47(1), 13 - 22. doi:10.1108/eb051377  

Yang, K., Maglaughlin, K. L., & Newby, G. B. (2001). Passage feedback with IRIS. Information Processing & Management, 37(3), 521-541. doi:10.1016/S0306-4573(00)00060-1