Barbara M. Wildemuth

School of Information and Library Science
University of North Carolina at Chapel Hill
100 Manning Hall, CB # 3360
Chapel Hill, NC 27599-3360

Phone: (919)962-8072 or (919)962-8366
Fax: (919)962-8071
wildem at ils.unc.edu
Complete vitae: http://ils.unc.edu/~wildem/wildemuth-vitae.pdf


Publication Abstracts

Refereed Articles, Proceedings Papers, and Chapters

Yang, S., Levy, J., Miller, K., Pomerantz, J.P., Oh, S., Wildemuth, B.M., Fox, E.A. (2008). Two approaches to enhance the education for ETDs: Developing educational modules and migrating the ETD Guide into a community wiki. Paper presented at ETD 2008: The 11th International Symposium on Electronic Theses and Dissertations, Aberdeen Scotland, June 4-7, 2008.

Two efforts have been made by the Digital Library (DL) Curriculum Development Project Group (http://curric.dlib.vt.edu) to help the ETD community. Our first activity is the preparation of multiple educational modules, which may be combined to create DL courses. In a paper presented at ETD 2007, the group identified the modules that might be most useful for scholars’ research endeavors (i.e., for ETD authors). Since then, five modules from the selected module list have been developed and a formal review by subject experts has been completed for two draft modules. In this paper, the project team will present the details of the five modules. They are: 3-b: Digitization; 4-b: Metadata; 6-b: Online information seeking behaviors and search strategies; 7-e: Web publishing (e.g., wiki, RSS, blogs); and 9-e: Intellectual property.
The second portion of this paper describes the recent migration activity of the ETD Guide (etdguide.org), which was written by several authors, with support by UNESCO, into a local wiki server. The ETD Guide has been supporting scholars, who would like to know more about ETDs, and/or utilize NDLTD systems effectively. However, there were problems such as outdated information in some sections, and the lack of easy means to update the information in the Guide. To address those problems, a wiki-based version of ETD Guide has been created with updated information (http://curric.dlib.vt.edu/wiki/index.php/ETD_Guide). Our plan is to move it into wikibooks.org so that it could be exposed to an even larger community. It will allow the ETD community to update information on the Guide as new technologies and approaches arise related to ETDs.
It is our hope that the efforts described will help with the understanding of digital libraries and of ETDs, and will promote the use of NDLTD-related systems and services.

Pearce, P. F., Williamson, J., Harrell, J. S., Wildemuth, B. M., & Solomon, P. (2007). The Childern’s Computerized Physical Activity Reporter (C-CPAR): Children as partners in the design and usability evaluation of an application for self-reporting physical activity. Computers, Informatics, Nursing, 25(2), 93-105.

The objectives of this three-phased study were to design and evaluate the usability of a computerized questionnaire, The Children's Computerized Physical Activity Reporter, designed with and for middle school children's self-report of physical activity. Study design was qualitative, descriptive, and collaborative, framed in a usability engineering model, with 22 participating children (grades 6-8; mean age, 12.5 years; range, 11-15 years) of three ethnic backgrounds. In Phase 1, children's understanding of physical activity and needs for reporting were determined, which were then translated in Phase 2 to the design features and content of the questionnaire; content validity, readability, and algorithm reliability were completed. Phase 3 involved children's evaluation of the questionnaire's usability (ease of use, efficiency, and aesthetics). The children all liked the questionnaire but identified several usability issues within instructions and reports. Working collaboratively with children was highly effective in ascertaining their understanding of physical activity and their self-reporting needs. Thus, the questionnaire's design was created from children's understanding of physical activity and their needs for recalling activities. The development of the questionnaire and its usability evaluation contribute to understanding children's physical activity and to the importance of designing for usability. Additional research is needed to ascertain reliability and validity of data derived from its use and to explore its usefulness in clinical or research venues.

Yang, S., Wildemuth, B.M., Kim, S., Murthy, U., Pomerantz, J.P., Oh, S., & Fox, E.A. (2007). Further developments of a digital library curriculum: Evaluation approaches and new tools. Asian Digital Libraries: Looking Back 10 Years and Forging New Frontiers, Lecture Notes in Computer Science, 4822, 434-443. http://dx.doi.org/10.1007/978-3-540-77094-7_55.

This paper is a follow-up to our ICADL 2006 paper, reporting on our progress over the past year in developing a digital library curriculum.  It presents and describes the current curriculum framework, which now includes ten modules and 41 sub-modules. It provides an overview of the curriculum development lifecycle, and our progress through that lifecycle. In particular, it reports on our evaluation of the modules that have been drafted. It concludes with a description of two new technologies – Superimposed Information (SI) to help resource presentation in a module and Visual User model Data Mining (VUDM) to help long-term module upgrade by visualizing the user community and its trends.

Yang, S., Oh, S., Pomerantz, J.P., Wildemuth, B.M., & Fox, E.A. (2007). Improving education and understanding of NDLTD. Paper presented at ETD 2007: The 10th International Symposium on Electronic Theses and Dissertations, Uppsala, Sweden, June 13-16, 2007. http://epc.ub.uu.se/ETD2007/files/papers/paper-39.pdf.

To understand ETDs, what NDLTD is, how it works, and the benefits of NDLTD, it is necessary to educate those involved, such as students who will create and submit their ETDs, as well as the library staff members who will be participating in NDLTD and administering their local system. To help educators prepare digital library (DL) courses supportive of their goals, our DL curriculum group has been developing educational modules and conducting field analyses since January 2006. This paper is a follow-up to our previous study of the subject distribution of ACM DL papers, JCDL papers, and D-Lib Magazine articles. In this paper, we focus on the selected DL modules that might help scholars conduct their research and share their knowledge. The contents are:
• Revised DL educational module framework: Based on our analysis of hundreds of DL papers, we identified 10 core topical areas, and 43 sub-areas. A detailed diagram is provided.
• After discussion, our team selected several DL modules which might be the most relevant to scholars’ research endeavors. Especially, those modules are important to fully utilize ETDs. The module numbers and their descriptions are presented.
• To help scholars navigate and study the corresponding ETD Guide (www.etdguide.org) sections, we mapped the selected DL modules into the Guide sections. The section numbers, titles, and starting page numbers are provided.
• In our previous study, we’ve collected DL course syllabi in the computer science (CS) and the library and information science (LIS) areas. Then the readings were retrieved from the syllabi collections. These readings were classified into the selected DL modules. They are presented here; the complete data can be accessed from our project web site.
Increased understanding about DLs might improve scholars’ research efficiency and effectiveness as well as universities’ participation in NDLTD. We invite the ETD community to assist with module development and evaluation so students, scholars, and staff will know more about DLs.

Fox, E. A., Yang, S., Wildemuth, B.M., Pomerantz, J. P. (2006). Digital library curriculum development: Enhancing education and comprehension of NDLTD. Paper presented at ETD 2006: The 9th International Symposium on Electronic Theses and Dissertations, Quebec, Canada, June 7-10, 2006. Slides.

Yang, S., Fox, E. A., Wildemuth, B. M., Pomerantz, J., & Oh, S. (2006). Interdisciplinary curriculum development for digital library education. Proceedings of the International Conference of Asian Digital Libraries, Lecture Notes in Computer Science, 4312, 61-70. http://curric.dlib.vt.edu/DLcurric/icadl06-dlcurric.pdf.

The Virginia Tech (VT) Department of Computer Science (CS) and the University of North Carolina at Chapel Hill (UNC-CH) School of Information and Library Science (LIS) are developing curricular materials for digital library (DL) education, appropriate for the CS and LIS communities. Educational modules will be designed, based on input from the project advisory board, Computing Curriculum 2001, the 5S framework, and workshop discussions. These modules will be evaluated, first through expert inspection and, second, through field testing. We are identifying and refining module definitions and scopes, collecting related resources, developing a module template, and creating example modules. These will be presented at the conference. The developed curriculum should contribute to producing well-balanced digital librarians who will graduate from CS or LIS programs.

Marchionini, G., Wildemuth, B. M., & Geisler, G. (2006). The Open Video Digital Library: A Möbius strip of research and practice. Journal of the American Society for Information Science & Technology, 57(12),1629-1643.

The Open Video Digital Library (OVDL) provides digital video files to the education and research community and is distinguished by an innovative user interface that offers multiple kinds of visual surrogates to people searching for video content. The OVDL is used by several thousand people around the world each month and part of this success is due to its user interface. This article examines the interplay between research and practice in the development of this particular digital library with an eye toward lessons for all digital libraries. We argue that theoretical and research goals blur into practical goals and practical goals raise new research questions as research and development progress—this process is akin to walking along a Möbius strip in which a locally two-sided surface is actually part of a globally one-sided world. We consider the gulf between the theories that guide current digital library research and current practice in operational digital libraries, provide a developmental history of the OVDL and the research frameworks that drove its development, illustrate how user studies informed its implementation and revision, and conclude with reflections and recommendations on the interplay between research and practice.

Marks, J. T., Campbell, M. K., Ward, D. S., Ribisl, K. M., Wildemuth, B. M., & Symons, M. J. (2006). A comparison of Web and print media for physical activity promotion among adolescent girls. Journal of Adolescent Health, 39(1), 96-104.

PURPOSE: To compare the effectiveness of a Web-based physical activity (PA) intervention with identical content delivered in a printed workbook among a sample of adolescent girls.
METHODS: Participants consisted of 319 girls with home Internet access enrolled in four middle schools within one school district. A randomized trial design was used to compare changes in PA self-efficacy and intentions after two weeks of exposure to either a Web- or print-based intervention delivered to their home. Self-reported physical activity was assessed as a secondary outcome. Analysis of covariance was conducted to determine changes between the intervention groups while controlling for baseline levels of PA constructs.
RESULTS: Both Web and print groups had significant changes in physical activity self-efficacy (Web: t[155] = 2.58, p = .01; print: t[156] = 3.11, p = .002) and intentions (Web: t[157] = 2.27, p = .02; print: t[159] = 6.32, p < or = .001). The print group demonstrated significantly greater increases in intentions compared with the Web group (F [1,315] = 13.53, p < or = .001). Self-reported physical activity increased significantly in the print group only (t[159] = 3.21, p = .002).
CONCLUSIONS: It cannot be assumed that new media technologies are superior to traditional media such as print for health communication to adolescents. These results suggest that a printed workbook was more effective than an identical website for increasing physical activity intentions and behavior among a sample of middle school girls.

Pomerantz, J., Wildemuth, B. M., Fox, E., & Yang, S. (2006). Curriculum development for digital libraries. Proceedings of the Joint Conference on Digital Libraries (Chapel Hill, NC, June 15, 2006), 175-184.

The Virginia Tech Department of Computer Science (VT CS) and the University of North Carolina at Chapel Hill School of Information and Library Science (UNC SILS) have launched a curriculum development project in the area of digital libraries. Educational resources will be developed based on the ACM/IEEE-CS Computing Curriculum 2001. Lesson plans and modules will be developed in a variety of areas (that cover the topics of papers and conference sessions in the field), evaluated by experts in those areas, and then pilot tested in CS and LIS courses. An analysis of papers on digital library-related topics from several corpora was performed, to identify the areas in which more and less work has already been performed on these topics; this analysis will guide the initial stages of this curriculum development.

Wildemuth, B. M. (2006). Evidence-based practice in search interface design. Journal of the American Society for Information Science & Technology, 57(6), 825-828.

An evidence-based practice approach to search interface design is proposed, with the goal of designing interfaces that adequately support search strategy formulation and reformulation. Relevant findings from studies of information professionals’ searching behaviors, end users’ searching of bibliographic databases, and search behaviors on the Web are highlighted. Three brief examples are presented to illustrate the ways in which findings from such studies can be used to make decisions about the design of search interfaces. If academic research can be effectively connected with design practice, we can discover which design practices truly are “best practices” and incorporate them into future search interfaces.

Bonnett, C., Wildemuth, B. M., & Sonnenwald, D. H. (2006). Interactivity between protégés and scientists in an electronic mentoring program. Instructional Science, 34(1), 21-61.

Interactivity is defined by Henri (1992) as a three-step process involving communication of information, a response to this information, and a reply to that first response. It is a key dimension of computer-mediated communication, particularly in the one-on-one communication involved in an electronic mentoring program. This report analyzes the interactivity between pairs of corporate research scientists (mentors) and university biology students (protégés) during two consecutive implementations of an electronic mentoring program. The frequency and structure of the interactions within each pair were examined to provide context: 542 messages were posted among the 20 mentors and 20 protégés. These messages were formed into 5-10 threads per pair, with 3-4 messages per thread, indicating a high level of interactivity (there were more responses posted than independent messages). Mentor-protégé pairs rated as effective by both mentors and protégés posted more messages overall, had well-structured threads, had protégé and mentor postings that were similar in topic coverage and message length, and had little overt “management” behavior by mentors. However, there appears to be no clear recipe for successful interaction. Not only are there a variety of factors at play in developing an online relationship in this context, but mentor-protégé pairs can falter at various stages in the process and in various ways.

Sutherland, L. A., Wildemuth, B. M., Campbell, M. K., & Haines, P. S. (2005). Unraveling the Web: an evaluation of the content quality, usability, and readability of nutrition web sites. Journal of Nutrition Education and Behavior, 37(6), 300-305.

OBJECTIVE: To determine the content quality, general readability, and usability characteristics of consumer nutrition information on the World Wide Web.
DESIGN: Almost 500 Web sites were identified for evaluation through 2 different approaches. Of these, 150 were included for further evaluation. Each site was rated on a 27-item tool covering content quality, readability, and usability.
ANALYSIS: Summary statistics, means, ranges, and standard deviation were calculated for each study variable. The statistical significance of differences between item means by search strategy was determined using Student's t tests.
RESULTS: Web sites identified using popular search engines scored significantly lower for content quality (P < .0001), were easier to navigate (P < .001), had better overall adherence to usability standards (P < .0001), and had lower reading levels compared with those sites identified using a government Web portal.
CONCLUSIONS AND IMPLICATIONS: Nutrition Web sites obtained using popular search engines may be aesthetically appealing and easy to use, but they often provide inaccurate nutrition information. As consumers increasingly turn to the World Wide Web for nutrition advice and education, it is imperative that the needs of diverse user populations be identified and addressed. Future nutrition education research should build on these findings by creating strategies that help users find reliable user-friendly gateways to accurate nutrition information on the Internet.

Wildemuth, B. M., & Hughes, A. (2005). Perspectives on the tasks in which information behaviors are embedded. In Fisher, K E., Erdelez, S., & McKechnie, L. (Eds.), Theories of Information Behavior. Medford, NJ: Information Today, for the American Society for Information Science & Technology, 275-279.

As Allen noted in 1996, “information needs happen to individuals who are embedded in a range of social situations.” His person-in-situation approach is consistent with Dervin’s more process-oriented view of information behaviors, in which people are engaged in making sense of their world. While Allen’s and Dervin’s work point in an appropriate direction, it is argued that we need to go further in our understanding of how the experiencing of information needs and the act of information seeking are embedded within people’s accomplishment of other tasks. In their 1999 exposition of information foraging theory, Pirolli and Card describe these as the “task environment” or “embedding task” of information seeking. This paper calls for more research into the tasks that motivate information seeking, so that we can develop more robust theories about the influence of these tasks on information behaviors.

Yang, M., Wildemuth, B. M., & Marchionini, G. (2004, October). The relative effectiveness of concept-based versus content-based video retrieval. In Proceedings of the 12th annual ACM international conference on Multimedia, 368-371. Poster. PDF.

Three video search systems were compared in the interactive search task at the TRECVID 2003 workshop: a text-only system, which searched video shots through transcripts; a features-only system, which searched video shots through 16 video content features (e.g., airplanes and people); and a combined system, which searched through both transcripts and content features. 36 participants each completed 12 video search tasks. The hypothesis that the combined system would perform better than both the text-only and the features-only systems was not supported, and large topic effects were found. Further analysis showed that concept-based video retrieval worked best for specific topics, whereas the hybrid retrieval techniques which combine both concept- and content-based video retrieval showed some advantage when searching for generic topics. The results have implications for topic/task analysis for video retrieval research, and also for the implementation of hybrid video retrieval systems.

Lobach, D. F., Arbanas, J. M., Mishra, D. D., Campbell, M., & Wildemuth, B. M. (2004). Adapting the human-computer interface for reading literacy and computer skill to facilitate collection of information directly from patients. MedInfo 2004, 11(Pt 2), 1142-1146.

Clinical information collected directly from patients is critical to the practice of medicine. Past efforts to collect this information using computers have had limited utility because these efforts required users to be facile with the computerized information collecting system. In this paper we describe the design, development, and function of a computer system that uses recent technology to overcome the limitations of previous computer-based data collection tools by adapting the human-computer interface to the native language, reading literacy, and computer skills of the user. Specifically, our system uses a numerical representation of question content, multimedia, and touch screen technology to adapt the computer interface to the native language, reading literacy, and computer literacy of the user. In addition, the system supports health literacy needs throughout the data collection session and provides contextually relevant disease-specific education to users based on their responses to the questions. The system has been successfully used in an academically affiliated family medicine clinic and in an indigent adult medicine clinic.

Boyington, A. R., Wildemuth, B. M., Dougherty, M. C., & Hall, E. P. (2004). Development of a computer-based system for continence health promotion. Nursing Outlook, 52(5), 241-247.

Computer-based systems (CBS) can provide information to help individuals analyze their health care needs and make decisions about management of health problems. This article discusses the development of a CBS that delivers an individualized educational intervention for continence health promotion. System development included cycles of prototype design, testing, analysis, and redesign. Knowledge acquisition, representation methods, and design decisions are discussed. Participants completed 4 rounds of usability testing and a pilot test, which resulted in enhancements to both the CBS and the expert system feature that produced individualized feedback. This iterative design process involved users throughout system development. User involvement resulted in a tutorial to explain navigation and other features of the CBS, graphics to enhance the written message, and clarification of continence-related content. The procedures resulted in an informative, usable product; they can be used to develop systems that provide information about symptom self-management for other health conditions.

Linnan, L. A., Wildemuth, B. M., Gollop, C., Hull, P., Silbajoris, C., & Monnig, R. (2004). Public librarians as a resource for promoting health: results from the Health for Everyone in Libraries Project (HELP) librarian survey. Health Promotion Practice, 5(2), 182-190.

Background: Public libraries are located in all communities, and are places nearly two-thirds of adults visit each year. Libraries give the public access to computers and the Internet, and librarians offer technical assistance for accessing information through these resources. Few studies have considered the self-reported interests and training needs of public librarians for assisting the public in accessing health information.
Methods: One public library from each North Carolina county was randomly selected and mailed a questionnaire to be completed by the librarian who typically answers questions at the reference desk. The survey questions asked about the health-related information services currently provided and librarians’ skills and knowledge for providing these services.
Results: Eighty-four percent (83/99) completed the questionnaire. Librarians answer more than 10 health-related questions per week, feel moderately comfortable answering many health questions, and are very interested in receiving additional training for addressing health-related questions.
Conclusions: Public libraries and librarians are uniquely positioned to assist the public in finding desired health information. Creating partnerships between public libraries and the public health community holds much promise for enhancing the ability of community members to access health information.

Hayslett, M. M., & Wildemuth, B. M. (2004). Pixels or pencils? The relative effectiveness of Web-based versus paper surveys. Library & Information Science Research, 26(1), 73-93.

A statewide survey of academic reference librarians provided data for an examination of differences in survey administration on paper and the World Wide Web, with particular attention to sampling bias, response rates, and differences in responses that are related to the medium of administration. Three methods of survey distribution were compared: paper survey distributed by mail, Web survey announced by mail, and Web survey announced by email. The study detected no sampling bias or differences in the content of responses. Response rates to the Web survey were not as high as traditional survey methods. Responses to the Web survey were gathered more quickly, but the paper instrument was not far behind. Email notices were more effective for promoting the Web survey than paper notices. Thus, it can be concluded that traditional postal surveys still hold some advantages over Web surveys. Although the advantages in cost and speed are alluring, researchers must use all the care of traditional research to match its effectiveness when using this new medium.

Wildemuth, B. M. (2004). The effects of domain knowledge on search tactic formulation. Journal of the American Society for Information Science & Technology, 55(3), 246-258.

A search tactic is a set of search moves that are temporally and semantically related. The current study examined the tactics of medical students searching a factual database in microbiology. The students answered problems and searched the database on three occasions over a nine-month period. Their search moves were analyzed in terms of the changes in search terms used from one cycle to the next, using two different analysis methods. Common patterns were found in the students’ search tactics; the most common approach was the specification of a concept, followed by the addition of one or more concepts, gradually narrowing the retrieved set before it was displayed. It was also found that the search tactics changed over time as the students’ domain knowledge changed. These results have important implications for designers in developing systems that will support users’ preferred ways of formulating searches. In addition, the research methods used (the coding scheme and the two data analysis methods—zero-order state transition matrices and maximal repeating patterns (MRP) analysis) are discussed in terms of their validity in future studies of search tactics.

Lobach, D. F., Hasselblad, V., & Wildemuth, B. M. (2003). Evaluation of a tool to categorize patients by reading literacy and computer skill to facilitate the computer-administered patient interview.  Proceedings of the American Medical Informatics Association, 391-395.

Past efforts to collect clinical information directly from patients using computers have had limited utility because these efforts required users to be literate and facile with the computerized information collecting system. In this paper we describe the creation and use of a computer-based tool designed to assess a user’s reading literacy and computer skill for the purpose of adapting the human-computer interface to fit the identified skill levels of the user. The tool is constructed from a regression model based on 4 questions that we identified in a laboratory study to be highly predictive of reading literacy and 2 questions predictive of computer skill. When used in 2 diverse clinical practices the tool categorized low literacy users so that they received appropriate support to enter data through the computer, enabling them to perform as well as high literacy users. Confirmation of the performance of the tool with a validated reading assessment instrument showed a statistically significant difference (p=0.0025) between the two levels of reading literacy defined by the tool. Our assessment tool can be administered through a computer in less than two minutes without requiring any special training or expertise making it useful for rapidly determining users’ aptitudes.

Hughes, A., Wilkens, T., Wildemuth, B., & Marchionini, G. (2003). Text or pictures? An eyetracking study of how people view digital video surrogates. Proceedings of the International Conference on Image and Video Retrieval (CIVR), University of Illinois at Urbana-Champaign, July 24-25, 2003, 271-280. PDF.

One important user-oriented facet of digital video retrieval research involves how to abstract and display digital video surrogates. This study reports on an investigation of digital video results pages that use textual and visual surrogates. Twelve subjects selected relevant video records from results lists containing ti-tles, descriptions, and three keyframes for ten different search tasks. All sub-jects were eye-tracked to determine where, when, and how long they looked at text and image surrogates. Participants looked at and fixated on titles and de-scriptions statistically reliably more than on the images. Most people used the text as an anchor from which to make judgments about the search results and the images as confirmatory evidence for their selections. No differences were found whether the layout presented text or images in left to right order.

Wilkens, T., Hughes, A., Wildemuth, B. M., & Marchionini, G. (2003). The role of narrative in understanding digital video: an exploratory analysis. ASIST 2003: Proceedings of the 66th ASIST Annual Meeting, Volume 40. Medford, NJ: Information Today, for the American Society for Information Science & Technology, 323-329. PDF.

Narrative is perhaps the oldest and most widely used form for organizing information and human experience, thus, it is not surprising that there is a significant body of research concerning narrative and its importance to comprehension and understanding. One important outcome of this research is the concept of narrative intelligence, the human tendency to fit experience into narrative form. This research is extremely relevant to information seeking in general and sense-making in particular. This paper outlines the basic principles and research supporting the concept of narrative intelligence and its applicability to the ways in which people make sense of digital video. We explore relevant theory and research in sense-making, surrogates, narrative, and narrative intelligence and then present the preliminary results of two research studies. The first clarifies and operationalizes the concept of narrative as it relates to video. The second demonstrates how narrativity can have significant effects on information seeking and sense-making in digital video. Results from these studies have implications for how syntactic form can be used as a means of indexing digital video.

Yang, M., Wildemuth, B. M., Marchionini, G., Wilkens, T., Geisler, G., Hughes, A., Gruss, R., & Webster, C. (2003). Measuring user performance during interactions with digital video collections. ASIST 2003: Proceedings of the 66th ASIST Annual Meeting, Volume 40. Medford, NJ: Information Today, for the American Society for Information Science & Technology, 3-11. PDF.

With more and more digital videos found online, video retrieval researchers have begun to create various representations or surrogates for digital videos, such as poster frames, storyboards, video skims and fast forwards. How to evaluate the effectiveness of these video surrogates has become an issue for researchers. This paper proposes two general classes of user tasks—recognition tasks and tasks requiring inference—for which performance measures were developed. The measures include graphical object recognition, textual object recognition, action recognition, free-text gist determination, multiple-choice gist determination and visual gist determination. The preliminary results from two user studies applying these six measures are also discussed in this paper.

Wildemuth, B. M., Marchionini, G., Yang, M., Geisler, G., Wilkens, T., Hughes, A., & Gruss, R. (2003). How fast is too fast? Evaluating fast forward surrogates for digital video. Paper presented at the ACM/IEEE Joint Conference on Digital Libraries, Houston, May 2003. Winner of the Vannevar Bush Award for Best Paper. PDF.

To support effective browsing, interfaces to digital video libraries should include video surrogates (i.e., smaller objects that can stand in for the videos in the collection, analogous to abstracts standing in for documents). The current study investigated four variations (i.e., speeds) of one form of video surrogate: a fast forward created by selecting every Nth frame from the full video. In addition, it tested the validity of six measures of user performance when interacting with video surrogates. Forty-five study participants interacted with all four versions of the fast forward surrogate, and completed all six performance tasks with each. Surrogate speed affected performance on four of the measures: object recognition (graphical), action recognition, linguistic gist comprehension (full text), and visual gist comprehension. Based on these results, we recommend a fast forward default speed of 1:64 of the original video keyframes. In addition, users should control the choice of fast forward speed to adjust for content characteristics and personal preferences.

Cheh, J. A., Ribisl, K. M., & Wildemuth, B. M. (2003). An assessment of the quality and usability of smoking cessation information on the internet. Health Promotion Practice, 4(3), 278-287.

Little is known about the quality and usability of online health information. This analysis evaluated smoking cessation Web sites' content quality and usability. Thirty sites were analyzed to determine their adherence to established tobacco cessation guidelines and their accessibility, usability, credibility, and currency. Most explained addiction (86.7%) and mentioned nicotine replacement therapy (NRT) (93.3%) and social support (93.3%). However, few explained potential side effects of NRT (33.3%) or which smokers should avoid using NRT (30.0%). Two sites advocated substituting smokeless tobacco or herbal cigarettes when quitting, and 16 (53.3%) provided information written at greater than an eighth-grade level. Few sites provided a search mechanism (40.0%) or offered text-only versions (30.0%), and most (83.3%) failed to indicate when content pages were last updated. Most sites adhered to established cessation guidelines. A small subset offered erroneous and potentially harmful information. Applying fundamental design principles would improve accessibility, usability, credibility, and currency.

Wildemuth, B. M., Marchionini, G., Wilkens, T., Yang, M., Geisler, G., Fowler, B., Hughes, A., & Mu, X. (2002). Alternative surrogates for video objects in a digital library: users' perspectives on their relative usability. Paper presented at the European Conference on Digital Libraries (ECDL), September 2002. PDF.

In a digital environment, it is feasible to integrate multimedia materials into a library collection with ease. However, it seems likely that non-textual surrogates for multimedia objects, e.g., videos, could effectively augment textual representations of those objects. In this study, five video surrogates were evaluated in relation to their usefulness and usability in accomplishing specific tasks. The surrogates (storyboards with text or audio keywords, slide shows with text or audio keywords, fast forward) were created for each of seven video segments. Ten participants, all of whom watch videos at least monthly and search for videos at least occasionally, viewed the surrogates for seven video segments and provided comments about the strengths and weaknesses of each. In addition, they performed a series of tasks (gist determination, object recognition, action recognition, and visual gist determination) with three surrogates selected from those available. No surrogate was universally judged "best," but the fast forward surrogate garnered the most support, particularly from experienced video users. The participants expressed their understanding of video gist as composed of three components: topicality, the story of the video, and the visual gist of the video. They identified several real-world tasks for which they regularly use video collections. The viewing compaction rates used in these surrogates supported adequate performance, but users expressed a desire for more control over surrogate speed and sequencing. Further development of these surrogates is warranted by these results, as well as the development of mechanisms for surrogate display.

Sutherland, L. A., Campbell, M., Ornstein, K., Wildemuth, B., & Lobach, D. (2001). Development of an adaptive multimedia program to collect patient health data. Journal of Preventive Medicine, 21(4), 320-324.

Background: Direct entry of personal health survey information into a computer by patients could streamline clinical data collection, improve completeness and accuracy of health information available to providers, and provide data for tailoring health education messages. Few computerized programs, however, have been developed to adapt the interface to diverse literacy levels, language, computer skills, and health literacy levels of the broad spectrum of patient populations.
Methods: To develop such a program, we conducted cognitive response interviews with a diverse sample of 21 adult participants from two North Carolina health clinics. Participants were placed into one of three interview categories: 1) low reading literacy level, low computer skills, 2) high reading literacy level, low computer skils, and 3) high reading literacy level, high computer skills. The "think-aloud" technique was employed to elicit feedback on a series of computer interface screens and health risk assessment questions.
Results: Interface-related findings showed that preference for touchscreen versus keyboard data entry was not strongly related to literacy level but was related to question type. Respondents with low literacy skills indicated that they would not click on a "help" or hyperlink option to get more inforamtion, and that frequent reminders of directions on the scrren were needed. Content-related findings showed numerous misperceptions regarding meanings of common health questions and terminology (e.g., seat belt use, IV drugs).
Conclusions: These findings have implcations for health literacy and may impact the accuracy of information obtained. Study results will be incorporated into the development of an innovative adaptive multimedia program.

Sonnenwald, D. H., Wildemuth, B. M., & Harmon, G. L. (2001). A research method to investigate information seeking using the concept of information horizons: an example from a study of lower socio-economic students' information seeking behaviour. The New Review of Information Behaviour Research, 2, 65-86.

As research questions and topics in information studies evolve, there is a continual need to seek out innovative research methods to help us investigate and address these questions. This paper presents an emerging research method, the creation and analysis of information horizon maps, and discusses the use of such maps in an ongoing research study. Sonnenwald's framework for human information behavior provides a theoretical foundation for this method. This theoretical framework suggests that within a context and situation is an "information horizon" in which we can act. Study participants are asked to describe several recent information seeking situations for a particular context, and to draw a map of their information horizon in this context, graphically representing the information resources (including people) they typically access and their preferences for these resources. The resulting graphical representations of their information horizons are analyzed in conjunction with the interview data using a variety of techniques derived from social network analysis and content analysis. In this paper, these techniques are described and illustrated using examples from an ongoing study of the information seeking behavior of lower socio-economic students. These techniques are compared to other techniques that could be used to gather data about people's information seeking behavior.

Sonnenwald, D. H., & Wildemuth, B. M. (2000). Investigating Information Seeking Behavior using the Concept of Information Horizons. SILS TR-2001-01. Chapel Hill: University of North Carolina, School of Information and Library Science, Technical Report Series. 20p. Winner of the 2001 ALISE Methodology Paper Competition. PDF.

As research questions and topics in information studies evolve, there is a continual need to seek out innovative research methods to help us investigate and address these questions. This paper presents an emerging research method, the creation and analysis of information horizon maps, and discusses the use of such maps in an ongoing research study. Sonnenwald's (1999) framework for human information behavior provides a theoretical foundation for this method. This theoretical framework suggests that within a context and situation is an 'information horizon' in which we can act. Study participants are asked to describe several recent information seeking situations for a particular context, and to draw a map of their information horizon in this context, graphically representing the information resources (including people) they typically access and their preferences for these resources. The resulting graphical representation of their information horizons are analyzed in conjunction with the interview data using a variety of techniques derived from social network analysis and content analysis. In this paper these techniques are described and illustrated using examples from an ongoing study of the information seeking behavior of lower socio-economic students. These techniques are compared to other techniques that could be used to gather data about people's information seeking behavior.

Wildemuth, B. M., Friedman, C. P., Keyes, J., & Downs, S. M. (2000). A longitudinal study of database-assisted problem solving. Information Processing & Management, 36, 445-459.

The current study examines the effects of database assistance on clinical problem solving across three cohorts of students and two database interfaces. Medical students attempted to solve problems presented as clinical scenarios, first from their personal knowledge and then, on a second pass, assisted by one of two versions of a bacteriology database. One version had a form-filling interface in which each term was typed into the form; the other had a direct manipulation interface that allowed students to select terms with a mouse. Data were collected on three assessment occasions: just before the bacteriology course, just after the course, and six months after the course. Personal Knowledge Scores varied by occasion, being highest just after and lowest just before the course. The Database-Assisted Scores were similar just before and after the course, but were higher six months after the course. The differences between the interfaces did not have any statistically significant effect on database-assisted performance. There was a marginally significant interaction between interface style and assessment occasion, suggesting that performance with the direct manipulation interface was less affected by variation in the students' personal knowledge than performance with the form-filling interface.

O'Keefe, K. M., Wildemuth, B. M., & Friedman, C. P. (1999). Medical students' confidence judgments using a factual database and personal memory: A comparison. Journal of the American Society for Information Science, 50(8), 698-708.

In order to determine whether medical students can recognize when an information need has been fulfilled and when it has not, this study examined the quality of medical students' confidence estimates in answering short-answer questions dealing with bacteriology, based upon their personal knowledge alone and what they were able to retrieve from a factual database in microbiology. Twelve students, assessed over three occasions, remained in the final sample. The results indicate that students displayed a positive relationship between their expertise in answering the questions and the amount of overconfidence they indicated (the opposite of the hard-easy effect) for the personal knowledge task using a partial credit format. For the database-assisted task using the partical credit format, students showed less overconfidence in their answers with greater expertise in using the database. For both the personal knowledge and database-asssted tasks using a binary format (all or nothing correct), the students displayed the opposite of the hard-easy effect. We conclude that the patterns of confidence estimates (in the form of Brier scores), and thus students' ability to recognize whether their information need has been fulfilled, differ with varying degrees of expertise in both the personal knowledge responses and the database-assisted responses for both the partial credit and binary formats. Taking into consideration the fact that when subjects, in this case future medical practitioners, are extremely overconfident, they stop looking for information long before they have found material that is relevant, the results have broad implications for medical practice and information seeking.

Downs, S. M., Marasigan, F., Abraham, V., Wildemuth, B., & Friedman, C. P. (1999). Scoring performance on computer-based patient simulations: beyond value of information. Journal of the American Medican Informatics Association (AMIA Fall Symposium Supplement), 520-524.

As computer based clinical case simulations become increasingly popular for training and evaluating clinicians, approaches are needed to evaluate a trainee's or examinee's solution of the simulated cases. In 1997 we developed a decision analytic approach to scoring performance on computerized patient case simulations, using expected value of information (VOI) to generate a score each time the user requested clinical information from the simulation. Although this measure has many desirable characteristics, we found that the VOI was zero for the majority of information requests. We enhanced our original algorithm to measure potential decrements in expected utility that could result from using results of information requests that have zero VOI. Like the original algorithm, the new approach uses decision models, represented as influence diagrams, to represent the diagnostic problem. The process of solving computer based patient simulations involves repeated cycles of requesting and receiving these data from the simulations. Each time the user requests clinical data from the simulation, the influence diagram is evaluated to determine the expected VOI of the requested clinical datum. The VOI is non-zero only it the requested datum has the potential to change the leading diagnosis. The VOI is zero when the data item requested does not map to any node in the influence diagram or when the item maps to a node but does not change the leading diagnosis regardless of it's value. Our new algorithm generates a score for each of these situations by modeling what would happen to the expected utility of the model if the user changes the leading diagnosis based on the results. The resulting algorithm produces a non-zero score for all information requests. The score is the VOI when the VOI is non-zero It is a negative number when the VOI is zero.

Abraham, V. A., Friedman, C. P., Wildemuth, B. M., Downs, S. M., Kantrowitz, P. J., & Robinson, E. N. (1999). Student and faculty performance in clinical simulations with access to a searchable information resource. Proceedings, AMIA Symposium, 648-652.

In this study we explore how students' use of an easily accessible and searchable database affects their performance in clinical simulations. We do this by comparing performance of students with and without database access and compare these to a sample of faculty members. The literature supports the fact that interative information resources can augment a clinician's problem solving ability in small clinical vignettes. We have taken the INQUIRER bacteriological database, containing detailed information on 63 medically important bacteria in 33 structured fields, and incorporated it into a compter-based clinical simulation. Subjects worked through the case-based clinical simulations with some having access to the INQUIRER information resource. Performance metrics were based on correct determination of the etiologic agent in the simulation and cross-tabulated with student access of the information resources; more specifically, it was determined whethr the student displayed the databse record describing the etiologic agent. Chi-square tests wshow statistical significance for this relationships (chi-square = 3.922; p=0.048). Results support the idea that students with databse access in a clinical simulation environment can perform at a higher level than their counterparts who lack access to such information, reflecting favorably on the use of information resources in training environments.

Wildemuth, B. M., Cogdill, K., & Friedman, C. P. (1999). The transition from formalized need to compromised need in the context of clinical problem solving. In Wilson, T. D., & Allen, D. K. (Eds.), Exploring the Contexts of Information Behaviour: Proceedings of the Second International Conference on Research in Information Needs, Seeking and Use in Different Contexts. 13/15 August 1998. Sheffield, UK. London: Taylor Graham, 290-303.

In 1968, Taylor argued that each information need moves along a continuum from the visceral need through the compromised need. The current study examines the transition from formalized need to compromised need. More specifically, it examines the sources from which medical students, presented with a written clinical scenario and a factual database, selected search terms; whether the term sources changed as the students' personal knowledge levels changed; and the relationship between term sources and search effectiveness. On each of three occasions (just prior to their bacteriology course, just after it, and six months after the course ended), 76 medical students solved six problems expressed as clinical scenarios in bacteriology. On their first pass through the problems, they provided solutions based on their personal knowledge. They then solved the problems, assisted by a search of INQUIRER, a factual database in bacteriology. Each term used during the INQUIRER searches was categorized based on its source. Words taken directly from the clinical scenarios were the predominant source of search terms, accounting for 52-65% of the terms used during the three assessment occasions. Statistically-significant differences across occasions were found for only three of the sources of terms: scenario words, truncated scenario words, and terms that were correctly suggested by the database. The correlations between frequency of students' use of a particular source and recall indicated that some sources of search terms were more fruitful than others. The highest positive correlations with recall were achieved using scenario words or truncated scenario words. The results of this study suggest that the students were able to successfully select words from a formalized information need (as represented in the clinical scenarios) to formulate compromised statements of their information needs.

Lipscomb, C. E., Moran, B. B., Jenkins, C. G., Cogdill, K., Friedman, C. P., Gollop, C. J., Moore, M. E., Morrison, M. L., Tibbo, H. R., & Wildemuth, B. M. (1999). Feasibility and marketing studies of health sciences librarianship education programs. Bulletin of the Medical Library Association, 87(1), 50-57.

The University of North Carolina at Chapel Hill evaluated five curricular models designed to improve education for health sciences librarianship. Three of the models enhanced existing degree and certificate programs, and two were new programs for working information professionals. Models were developed with input from experts and a Delphi study; the marketability of the models was tested through surveys of potential students and employers; and recommendations were made as a guide to implementation. The results demonstrated a demand for more specialized curricula and for retraining opportunities. Marketing data showed a strong interest from potential students in a specialized master's degree, and mid-career professionals indicated an interest in post-master's programs that provided the ability to maintain employment. The study pointed to the opportunity for a center of excellence in health sciences information education to enable health sciences librarians to repospond to their evolving roles.

Wildemuth, B. M., Friedman, C. P., & Downs, S. M. (1998). Hypertext versus Boolean access to biomedical information: A comparison of effectiveness, efficiency and user preferences. ACM Transactions on Computer-Human Interaction, 5(2), 156-183.

This study compared two modes of access to a biomedical database, in terms of their effectiveness and efficiency in supporting clinical problem solving and in terms of user preferences. Boolean access, which allowed subjects to frame their queries as combinations of keywords, was compared to hypertext access, which allowed subjects to navigate from one database node to another. The accessible biomedical data were identical across system versions. Performance data were collected from two cohorts of first-year medical students, each student randomly assigned to either the Boolean or the hypertext system. Additional attitudinal data were collected from the second cohort. At each of two research sessions (one just before and one just after their bacteriology course), subjects worked eight clinical case problems, first using only their personal knowledge and, subsequently, with aid from the database. Database retrievals enabled students to answer questions they could not answer based on personal knowledge alone. This effect was greater when personal knowledge of bacteriology was lower. There were not statistically-significant differences between the two forms of access, in terms of problem solving effectiveness or efficiency. Students preferred Boolean access over hypertext access.

Wildemuth, B. M., Crenshaw, L., Jenniches, W., & Harmes, J. C. (1997). What's everybody talking about?: Message functions and topics on electronic lists and newsgroups in information and library science. Journal of Education for Library and Information Science, 38(2), 137-156.

The use of listserv software and usenet newsgroups to support discussion among professionals in information and library science has become commonplace in the 1990's. The current research was conducted to find out which topics of discussion are dominant and which functions are performed by the messages posted to discussion groups in library and information science. In Fall 1994, 21 students in an introductory course on communication processes completed analyses of 14 different electronic lists or newsgroups as their term projects. Messages on each list or newsgroup were captured for approximately one month. Each message or partial message was described, in terms of its topic and its function, and frequencies were calculated. The topics most frequently discussed in each group were reasonably consistent with the group's stated purpose. In addition, technology topics were frequently discussed, as well as topics related to the library's clientele. The most common message functions were queries and responses of various types, and announcements. Comments made in the context of free-wheeling discussions also were common on some of the lists, but were rare on others. In addition to providing some baseline data on the topics and functions of messages in computer-mediated discussions, this study accomplished two pedagogical objectives: to introduce new students to electronic discussion groups as a channel for professional communication, and to introduce them to content analysis as a research method.

Friedman, C. P., Wildemuth, B. M., Muriuki, M., Gant, S. P., Downs, S. M., & Ruth de Bliek, R. (1996). A comparison of hypertext and boolean access to biomedical information. In Proceedings of the AMIA Fall Symposium, 2-6. Received SCAMC Best Theoretical Paper Award.

This study explored which of two modes of access to a biomedical database better supported problem solving in bacteriology. Boolean access, which allowed subjects to frame their queries as combinations of keywords, was compared to hypertext access, which allowed subjects to navigate from one database node to another. The accessible biomedical data were identical across systems. Data were collected from 42 first year medical students, each randomized to the Boolean or hypertext system, before and after their bacteriology course. Subjects worked eight clinical case problems, first using only their personal knowledge and, subsequently, with aid from the database. Database retrievals enabled students to answer questions they could not answer based on personal knowledge only. This effect was greater when personal knowledge of bacteriology was lower. The results also suggest that hypertext was superior to boolean access in helping subjects identify possible infectious agents in these clinical case problems.

Moran, B. B., Jenkins, C. G., Friedman, C. P., Lipscomb, C. E., Gollop, C. J., Moore, M. E., Morrison, M. L., Tibbo, H. R., & Wildemuth, B. M. (1996). Preparing tomorrow's health sciences librarians: Feasibility and marketing studies. Bulletin of the Medical Library Association, 84, 541-548.

The University of North Carolina at Chapel Hill is devising and evaluating five curricular models designed to improve education for health sciences librarianship. These models fit into a continual learning process from the initial professional preparation to lifelong learning opportunities. Three of them enhance existing degree and certificate programs in the School of Information and Library Science (SILS) with a health sciences specialization, and two are new programs for working information professionals. The approaches involve partnerships among SILS, the Health Sciences Library, and the Program in Medical Informatics. The planning process will study the feasibility of the proposed programs, test the marketability of the models to potential students and employers, and make recommendations about implementation.

Wildemuth, B. M., de Bliek, R., Friedman, C. P., & File, D. D. (1995). Medical students' personal knowledge, searching proficiency, and database use in problem solving. Journal of the American Society for Information Science, 46, 590-607.

The relationship between personal knowledge in a domain and searching proficiency in that domain and the relationship between searching proficiency and database-assisted problem solving performance were the foci of this study. On four assessment occasions over a two-year period, 36 medical students solved problems in three biomedical domains (bacteriology, pharmacology, and toxicology) with and without assistance from a factual database in the relevant domain. There was little evidence of any relationship between personal domain knowledge and searching proficiency (i.e., search results, selection of search terms, improvement in selection of search terms over the course of the search, and efficiency). Search results, selection of search terms, and efficiency were found to be related to database-assisted problem solving performance.

Wildemuth, B. M., & Moore, M. E. (1995). End-user search behaviors and their relationship to search effectiveness. Bulletin of the Medical Library Association, 83, 294-304.

One hundred sixty-one MEDLINE searches conducted by third year medical students were analyzed and evaluated to determine which search moves were used, whether those individual moves were effective, and whether there was a relationship between specific search behaviors and the effectiveness of the search strategy as a whole. The typical search included 14 search statements, used seven terms or "limit" commands and resulted in the display of 11 citations. The most common moves were selection of a database, entering single-word terms and free-text term phrases, and combining sets of terms. Syntactical errors were also common. Librarians judged the searches to be adequate, and students were quite satisfied with their own searches. However, librarians identified many missed opportunities in the search strategies, including underutilization of the controlled vocabulary, subheadings, and synonyms for search concepts. There were no strong relationships found between specific search behaviors and the librarians' or students' evaluations of the searches. Implications of these findings for system design and user education are discussed.

Wildemuth, B. M., & O'Neill, A. L. (1995). The 'known' in known-item searches: A pilot study. College & Research Libraries, 56, 265-281.

Traditionally, the catalog record for an item has been based on the cataloger's inspection of the item and has contained a complete description of the item. An alternative approach--user-centered design--would require the study of user behaviors and cognition related to interaction with the catalog and using the results of those studies to inform design decisions. To support known-item searches, one would need to study users' conceptions of the item being sought, what the user knows about the item, and which pieces of known information are viewed by the user as most appropriate for inclusion in a search. A pilot study was conducted to develop methods that can be applied to these questions. During the three phases of the study, 103 catalog users described 386 searches. Any written information known by the searchers was photocopied. The searchers generally knew the title, publication date, page numbers (particularly for journals), and/or the author (particularly for books). The information known by the searcher was usually accurate. Results from the study indicated that the method was feasible and valid, and provided a preliminary picture of known-item searching in one library's catalog.

de Bliek, R., Friedman, C. P., Wildemuth, B. M., Martz, J. M., Twarog, R. G., & File, D. (1994). Information retrieved from a database and the augmentation of personal knowledge. Journal of the American Medical Informatics Association, 1, 328-338.

Objective: To assess the degree to which information retrieved from a biomedical database can augment personal knowledge in addressing novel problems, and how the ability to retrieve information evolves over time.
Design: This longitudinal study comprised three assessments of two cohorts of medical students. The first assessment occurred just before student course experience in bacteriology, the second occurred just after the course, and the third occurred five months later. At each assessment the students were initially given a set of bacteriology problems to solve using their personal knowledge only. Each student was then reassigned a sample of problems he or she had answered incorrectly to work again with assistance from a database containing information about bacteria and bacteriologic concepts. The initial pass through the problems generated a "personal knowledge" score; the second pass generated a "database-assisted" score for each student at each assessment.
Results: Over two cohorts, students' personal knowledge scores were very low (~12%) at the first assessment. They rose substantially at the second assessment (~48%) but decreased six months later (~25%). By contrast, database-assisted scores rose linearly: from ~44% at the first assessment to ~57% at the second assessment, to ~75% at the third assessment.
Conclusion: The persistent increase in database-assisted scores, even when personal knowledge had attenuated, was the most remarkable finding of this study. While some of the increase may be attributed to artifacts of the design, the pattern seems to result from the retained ability to recognize problem-relevant information in a database even when it cannot be recalled.

Wildemuth, B. M., de Bliek, R., Friedman, C. P., & Miya, T. S. (1994). Information-seeking behaviors of medical students: A classification of questions asked of librarians and physicians. Bulletin of the Medical Library Association, 82, 295-304.

One hundred sixty-one MEDLINE searches conducted by third year medical students were analyzed and evaluated to determine which search moves were used, whether those individual moves were effective, and whether there was a relationship between specific search behaviors and the effectiveness of the search strategy as a whole. The typical search included 14 search statements, used seven terms or "limit" commands and resulted in the display of 11 citations. The most common moves were selection of a database, entering single-word terms and free-text term phrases, and combining sets of terms. Syntactical errors were also common. Librarians judged the searches to be adequate, and students were quite satisfied with their own searches. However, librarians identified many missed opportunities in the search strategies, including underutilization of the controlled vocabulary, subheadings, and synonyms for search concepts. There were no strong relationships found between specific search behaviors and the librarians' or students' evaluations of the searches. Implications of these findings for system design and user education are discussed.

de Bliek, R., Friedman, C. P., Wildemuth, B. M., Martz, J. M., File, D., Twarog, R. G., Reich, G. M., & Hoekstra, L. (1993). Database access and problem solving in the basic sciences. In Proceedings of the Seventeenth Annual Symposium on Computer Applications in Medical Care, A Conference of the American Medical Informatics Association: Patient-Centered Computing, October 30-November 3, 1993, Washington DC, 678-682.

This study examined the potential contribution that access to a database of biomedical information may offer in support of problem-solving exercises when personal knowledge is inadequate. Thirty-six medical students were asssessed over four occasions and three domains in the basic sciences: bacteriology, pharmacology, and toxicology. Each assessment consisted of a two-pass protocol in which students were first assessed for their personal knowledge of a domain with a short-answer problem set. Then, for a sample of problems they had missed, they were asked to use a database, INQUIRER, to respond to questions which they had been unable to address with their personal knowledge. Results indicate that for a domain in which the database is well-integrated in course activities, useful retrieval of information which augmented personal knowledge increased over three assessment occasions, even continuing to increase several months after course exposure and experience with the databse. For all domains, even at assessments prior to course exposure, students were able to moderately extend their ability to solve problems through access to the INQUIRER database.

Wildemuth, B. M. (1993). Post-positivist research: Two examples of methodological pluralism. Library Quarterly, 63, 450-468.

A post-positivist research approach advocates methodological pluralism. It is based on the assumption that the method to be applied in a particular study should be selected based on the research question being addressed. The research described here takes a post-positivist approach, applying interpretive research in two ways: in an exploratory study of end-user computing conducted prior to a positivist study, and in a study of end-user searching behaviors conducted concurrently with a positivist study.

Wildemuth, B. M., de Bliek, R., & Friedman, C. P. (1993). Measures of searcher performance: A psychometric evaluation. Information Processing & Management, 29, 533-550.

Several measures--such as recall, precision, term overlap, and efficiency--have been used to evaluate bibliographic searching (Lancaster, 1977; Saracevic, Kantor, Chamis, & Trivison, 1988). When applied to searches for specific facts in a full-text database, these measures seem less appropriate. For instance, recall is reduced to a binary measure reflecting the success or failure of the search to retrieve the desired fact; and lack of precision may simply reflect the searcher's unwillingness to expend further effort in narrowing a search (Blair, 1980). It is likely that new measures will need to be developed, and the applicability of known measures to factual searches in full-text databases needs to be evaluated. A study was conducted to evaluate 21 measures of performance on factual searches of a full-text database. The measures included two measures of recall, two measures of precision, seven measures of search term overlap, seven measures of improvement in search term overlap, and three measures of efficiency. Each of these measures was calculated for the searches performed by each of 26 first-year medical students on INQUIRER (Friedman et al., 1988; Friedman, Twarog, File, Youngblood, & de Bliek, 1990), a database of facts and concepts in microbiology. Their reliability and construct validity were investigated. Their underlying structure consisted of three factors: Process/Outcome (precision, recall, and term overlap), Improvement in term overlap, and Efficiency. One scale for each factor was constructed after eliminating seven of the original 21 variables. Each of these scales demonstrated adequate reliability for research purposes. The utility of these measures in future research on online searching is discussed.

Wildemuth, B. M., de Bliek, R., He, S., & Friedman, C. P. (1992). Search moves made by novice end users. ASIS '92 Proceedings (Pittsburgh, PA, October 26-29, 1992), 29, 154-161.

When conducting an online search, a searcher enters terms, gets a response from the system, and iteratively modifies the terms until satisfied with the results. The transaction logs of 53 medical students' searches of a factual database were analyzed in detail to determine the overall frequency of search moves; the interaction between the problem statement and the students' search strategies; the search moves selected by individual students; and the tactics (i.e., combinations of moves) used by students. Over 200 searches were conducted in response to clinical scenarios in microbiology; the searches were made up of 853 search moves. The results indicate that students used only a few distinct moves and that their selection of moves varied by individual and by search stimulus. Patterns also emerged in students' combination of search moves into tactics.

de Bliek, R., Martz, J. M., Reich, G. M., Friedman, C. P., & Wildemuth, B. M. (1992). Domain knowledge and information retrieval in bacteriology: An information science perspective. Academic Medicine, 67 (October supplement), S54-S56.

The studies reported here assessed medical students' domain knowledge and their ability to retrieve information from INQUIRER, a factual database in bacteriology, over three occasions. Thirty-nine first-year medical students participated in three assessment occasions: just before, just after, and five months after the microbiology course. On each occasion, the students responded to six problems in bacteriology, first from their personal knowledge and subsequently with assistance from INQUIRER. Personal knowledge scores increased after the course, but then decreased somewhat five months later. Database-assisted scores steadily increased over the three occasions. These results suggest that some facets of personal knowledge and database-assisted performance may be independent and evolve differently over time.

Stine, W. D., & Wildemuth, B. M. (1992). The training of microcomputer users: Insights from two disciplines. Journal of Education for Library and Information Science, 33, 100-109.

Little research has been conducted on the relative effectiveness of different methods for training users of microcomputer systems. Research in the disciplines of human-computer interaction and instructional design can provide insights pertaining to the learning and use of computers and, specifically, to the training of microcomputer users. The combined insights of the two disciplines advocate user/learner-centered training, designed to achieve the following objectives: improving the user's understanding of system operation through the use of appropriate metaphors, taking advantage of error situations to teach error recognition and recovery strategies, and encouraging users to be actively engaged in learning by incorporating job-related tasks in training exercises.

Wildemuth, B. M. (1992). An empirically grounded model of the adoption of intellectual technologies. Journal of the American Society for Information Science, 43, 210-224.

A qualitative study of end-user computing applications in five departments in three organizations was undertaken. The study was designed to develop an empirically grounded model of the adoption process for intellectual technologies--technologies that are inherently flexible, enabling the user to take on the role of developer. Detailed analysis of 43 user-developed computing applications resulted in a model consisting of five stages: Resource Acquisition, Application Development, Adoption/Renewal, Routinization/Enhancement, and External Adoption. By comparing this model with two prior models of the adoption process, aspects of the adoption process requiring further research were identified.

Wildemuth, B. M., Jacob, E. K., Fullington, A., de Bliek, R., & Friedman, C. P. (1991). A detailed analysis of end-user search behaviors. ASIS '91 Proceedings (Washington, DC, October 27-31, 1991), 28, 302-312.

Individuals searching online databases frequently begin with an initial group of terms, then interactively revise these terms to reduce or enlarge the size of the retrieval set or to increase both precision and recall. Each search statement in this revision process can be viewed as a "move" in the overall search strategy [2]. Because very little is known about how end users develop and revise their search strategies, a descriptive study was conducted to analyze the moves made in 244 database searches conducted by 26 medical students at the University of North Carolina at Chapel Hill. In preparing clinical case problems, these students voluntarily searched INQUIRER [3, 4], a database of facts and concepts in microbiology. The searches were conducted during a three-week period in spring 1990 and were recorded by the INQUIRER system. Each search statement was categorized, using Fidel's [2] online searching moves and Bates's [5] search tactics. Further analysis of the individual moves indicated that the most common moves were Browse/Specify, Select, Exhaust, Intersect, and Vary, and that selection of moves varied by student and by problem. Analysis of search tactics (i.e., combinations of moves) identified five common search approaches. The results of this study have implications for future research on search behaviors, for the design of system interfaces and database structures, and for the training of end users.

Wildemuth, B. M. (1990). A method for inducing process models from qualitative data. Library and Information Science Research, 12, 329-340.

A method for inducing a theoretical model of a multistage nondeterministic process is described. The method was developed in a study of the adoption process for end-user computing applications. Qualitative data were transformed into histories of the events occurring during each application's adoption and implementation. Each event was then coded as an agent-action-object phrase, representing the instigator of the action, the action itself, and the entity affected by the action. The agent-action-object phrases were displayed on an electronic spreadhseet, with one application's event history in each column and similar events aligned horizontally. Patterns of event sequences were then identified and interpreted as stages of the adoption process. This method can be applied in a broad range of research areas involving the observation of events occurring in an ongoing process.

Wildemuth, B. M., de Bliek, R., Friedman, C. P., & Dekker, H. (1990). Measures of success in searching a full-text fact base. ASIS '90: Proceedings of the 53rd Annual Meeting of the American Society for Information Science, Toronto, November 4-8, 1990, 27, 104-109.

As the creation and use of factual databases increases, research is needed to develop measures that can evaluate both system effectiveness and searcher proficiency. A pilot study was conducted to evaluate five measures of overall success in searching a factual database. The five measures were success/failure, precision, search term overlap, number of search cycles, and time per search (in minutes). Each of these five measures of search success was calculated for each of 19 first-year medical students on each of four searches of INQUIRER, a database of facts and concepts in microbiology. The feasibility of calculating these measures directly from INQUIRER system logs was investigated. In addition, a preliminary evaluation of the validity of each measure was undertaken by examining (1) its psychometric properties and (2) the relationship between an individual student's score for each measure and that student's score for the composite of all five measures. The results indicate that all of these measures can be calculated easily and that, with the single exception of success/facilure, these measures are corerelated with a composite search proficiency measure. It can be concluded that these measures hold promise for evaluating searches of factual databases.

Wildemuth, B. M. (1988). The management of end-user computing: Lessons from a qualitative study. ASIS '88: Proceedings of the 51st Annual Meeting of the American Society for Information Science (Atlanta, Georgia, October 23-27, 1988), 25, 41-47.

A qualitative study of end-user computing was undertaken to begin to understand the appropriate management techniques for departmental management of end-user computing. The participants includeed the staff in five departments from three different corporations. Data were collected through in-depth interviews over a nine-month period, ending in January 1988. A comparative case study approach was used to analyze the data. The results led to recommendations for managers on the management techniques appropriate to four components of their own computing environment: the technological infrastructure, support for computing, control of the computing activities, and promotion or encouragement of computer use.

Other Journal Articles

Wildemuth, B.M., Pomerantz, J., Oh, S., Yang, S., & Fox, E.A. (2008). A digital libraries curriculum: Expert review and field testing. D-Lib Magazine, 14(7/8). http://www.dlib.org/dlib/july08/07inbrief.html#WILDEMUTH.

After reviewing the status of module development within a digital libraries curriculum framework, this paper briefly describes the procedures that are being used for expert review of each module (using the project wiki) and field testing of each module.

Pomerantz, J., Oh, S., Yang, S., Fox, E.A., & Wildemuth, B.M. (2006). The core: Digital library education in library and information science programs. D-Lib Magazine, 12(11). http://www.dlib.org/dlib/november06/pomerantz/11pomerantz.html.

This paper identifies the "state of the art" in digital library education in Library and Information Science programs, by identifying the readings that are assigned in digital library courses and the topics of these readings. The most frequently-assigned readings are identified at multiple units of analysis, as are the topics on which readings are most frequently assigned. While no core set of readings emerged, there was significant consensus on the authors to be included in digital library course reading assignments, as well as the topics to be covered. Implications for the range of assigned readings and topics for digital library education in library science education are discussed.

Pomerantz, J., Wildemuth, B. M., Oh, S., Yang, S., & Fox, E. A. (2006). Briefing: Digital libraries curriculum development. D-Lib Magazine 12(7/8). http://www.dlib.org/dlib/july06/07inbrief.html#POMERANTZ.

A very brief presentation of the module framework developed by the joint Virginia Tech/University of North Carolina project.

Carey, J., Galletta, D., Kim, J., Te’eni, D., Wildemuth, B., & Zhang, P. (2004). The role of human computer interaction in management information systems curricula: a call to action. Communications of the AIS,13, article 23.

This article presents a multifaceted case for the inclusion of the important Human-Computer Interaction (HCI) materials at both the graduate and undergraduate levels of the Association of Information Systems (AIS) model curricula. Based on the presentations and discussions during a panel at the 2003 Americas Conference on Information Systems (AMCIS), we first present the rationale for incorporating HCI materials in MIS curricula. Second, we present a list of relevant and important HCI knowledge and skills that our students need for effective careers. Third, given the limited number of credit hours available to each MIS program, we explore several strategies for the integration of HCI into current and future MIS courses. Finally, drawing upon the rich set of teaching experiences of the authors, we outline a sample syllabus for a separate HCI course and present HCI content that may be addressed in current MIS courses.

Friedman, C. P., Wildemuth, B. M., de Bliek, R., Twarog, R. G., & File, D. D. (1994). Database searching proficiency and problem solving proficiency in a biomedical domain. Teaching and Learning in Medicine, 6, 168-174.

In this article, we examine how proficient medical students are in retrieving information from a database in bacteriology and whether students who were more proficient obtained higher scores on a subsequent posttest. In particular, we examined the computer logs of 26 students who used the INQUIRER database to obtain information relevant to 11 problems assigned during their bacteriology course. The analysis addressed recall and precision as measures of search success, as well as three indices of the more general construct of search proficiency. Student searches were found to be usually successful, as students recalled, on average, about 90% of problem-relevant information available in the database. Of the three indices of search proficiency, only the "efficiency" of these students as databse searchers was related to their subsequent posttest performance.

Wildemuth, B. M. (September/October 1985). How to use information resources on testing and evaluation. Curriculum Review, 39.

Three primary information resources on testing and evaluation are described: the ERIC database, the ETS Test Collection, and the resources of the Buros Institute of Mental Measurements.

Refereed Short Papers, Abstracts, Posters, and Presentations

Pomerantz, J., Wildemuth, B.M., Oh, S., Yang, S., & Fox, E.A. (2008). Evaluation of a curriculum for digital libraries. Poster presented at the Joint Conference on Digital Libraries, Pittsburgh, June 16-20, 2008. Recipient of 3rd place prize in poster competition. http://curric.dlib.vt.edu/DLcurric/JCDLposter.2008-06-05.pdf.

Pomerantz, J., Oh, S., Wildemuth, B.M., Yang, S., & Fox, E.A. (2007). Digital library education in computer science programs.  Short paper presented at the Joint Conference on Digital Libraries, Vancouver, June 20-23, 2007.  http://eprints.cs.vt.edu/archive/00000945/01/Pomerantz_CSsyllabi.pdf.

In an effort to identify the “state of the art” in digital library education in computer science (CS) programs, we analyzed CS courses on digital libraries and digital library-related topics. Fifteen courses that mention digital libraries in the title or short description were identified; of these, five are concerned with digital libraries as the primary topic of the course. The readings from these five courses were analyzed further, in terms of their authors and the journals in which they were published.

Schaeffer, P., Thibodeau, P.L., James, R., Wildemuth, B.M., & Gollop, C. (2007). The informationist and the information specialist revolution: Career pathways for physicians? Poster presented at the Medical Library Association Annual Meeting, Philadelphia, May 18-23, 2007. Poster.

Thibodeau, P.L., James, R., Wildemuth, B.M., Gollop, C., & Schaeffer, P. (2006). New dual-degree pathways: Duke’s medical information specialist training (MIST) program. Poster presented at the Association of American Medical Colleges (AAMC) Annual Meeting, Seattle, October 27-November 1, 2006. Poster.

Thibodeau, P.L., James, R., Schaeffer, P., Wildemuth, B.M., & Gollop, C. (2006). A dual-degree pathway to developing medical information specialists: Transforming physicians into informationists. Poster presented at the meeting of the Mid-Atlantic and Southern Chapters of the Medical Library Association, Atlanta, October 2006. Poster.

Wildemuth, B. M., Blake, C. L., Spurgin, K., Oh, S., & Zhang, Y. (2006). Patients’ perspectives on personal health records: An assessment of needs and concerns. Poster presented at “Critical Issues in eHealth Research 2006,” Bethesda, MD, September 11-12, 2006. Poster.

Yang, M., Wildemuth, B. M., & Marchionini, G. (2004, October). The relative effectiveness of concept-based versus content-based video retrieval.  In Proceedings of the 12th annual ACM international conference on Multimedia, 368-371. Poster.

Three video search systems were compared in the interactive search task at the TRECVID 2003 workshop: a text-only system, which searched video shots through transcripts; a features-only system, which searched video shots through 16 video content features (e.g., airplanes and people); and a combined system, which searched through both transcripts and content features. 36 participants each completed 12 video search tasks. The hypothesis that the combined system would perform better than both the text-only and the features-only systems was not supported, and large topic effects were found. Further analysis showed that concept-based video retrieval worked best for specific topics, whereas the hybrid retrieval techniques which combine both concept- and content-based video retrieval showed some advantage when searching for generic topics. The results have implications for topic/task analysis for video retrieval research, and also for the implementation of hybrid video retrieval systems.

Linnan, L., Wildemuth, B., Harris, J., Rose, J., Collins, L., & Gollop, C. (2003). Bridging the digital divide with library-based interventions.  Paper presented at the Society of Behavioral Medicine (SBM), Salt Lake City, March 2003. Slides.

Boyington, A. R., Wildemuth, B. M., & Dougherty, M. C. (2002). A usability engineering approach to development of a computer-based system for continence health promotion.  Proceedings of the annual meeting of the American Medical Informatics Assocation (AMIA), San Antonio, TX, November 11, 2002, p981. Poster summary.

Lobach, D. F., Arbanas, J. M., Mishra, D. D., Wildemuth, B., & Campbell, M. (2002).  Collection of information directly from patients through an adaptive human-computer interface.  Poster presented at the annual meeting of the American Medical Informatics Assocation (AMIA), San Antonio, TX, November 12, 2002.

Geisler, G., Marchionini, G., Wildemuth, B. M., Hughes, A., Yang, M., Wilkens, T., & Spinks, R. (2002). Video browsing interfaces for the Open Video Project. In CHI 2002: Conference on Human Factors in Computing Systems (Minneapolis, MN, April 20-25, 2002). New York: ACM Press, 514-515.

The Open Video Project is an on-going effort to develop an open source digital video collection that can be used by the information retrieval, digital library, and digital video research communities and ultimately serve an even broader audience. The collection currently contains more than 500 video segments comprising over 20GB of content. Our primary goals for this project are to provide free digital video content to people doing a wide variety of research, to develop a collaborative research environment for people interested in digital video, and to provide a testbed for our own video browsing interface work. This demonstration summarizes the current status of the project through a brief tour of the Open Video web site, describes our current work in developing surrogates to preview video segments, and shows prototypes of an innovative video browsing interface we are developing.

Boyington, A. R., Hall, E. P., & Wildemuth, B. M. (2001). Design and evaluation of a computer-based system for continence health promotion. Poster presented at the Southern Nursing Research Society meeting (Baltimore, MD, February 1-3, 2001).

Purpose: Computer-based systems can be used to deliver information about continence health promotion to older women, but only if those systems are usable by their intended audience. The current study investigates the usability of such a system, focusing on the ease with which the intended audience (older women) can perform information seeking tasks and their overall satisfaction with the system.
Rationale: It is well established that urinary incontinence is a health problem for millions of Americans, particularly older women. Behavioral treatment strategies have been established to be effective, but information about urinary incontinence has not been effectively disseminated to practitioners or to the lay public. Physicians avoid asking older women about bladder control problems, and older women know little about management of them. Women need factual information about continence problems in order to interpret symptoms and make informed decisions. Computer technology offers an additional channel of communication to deliver information on behavioral strategies and advice tailored to individual women's needs and risks. The challenge lies in designing a system for older women that is easy and satisfying to use, and provides the information they want. A cycle of prototype design, testing, analysis, and redesign can be employed to determine whether the prototype is converging toward an appropriate user-interaction design.
Sample: The sample for the usability test sessions are representative of women for whom the computer-based system for continence health promotion will be designed. The nonprobabilistic sample will consist of approximately 30 women (three to five women each session) who are age 55 years or older; African-American or Caucasian ethnicity; self-identified involuntary urine loss and/or symptoms of urinary frequency, urinary urgency, or nocturia; self-reported ability to read and write, to ambulate without difficulty, and to toilet independently; and lack of cognitive impairment as indicated on the Telephone Interview for Cognitive Status - modified.
Methods: A formative evaluation process, followed by redesign, is being applied to enable the researcher to understand how women view prototypes of the system as they are being developed. Early testing will focus on global design decisions and subsequent sessions will test the effectiveness of specific aspects of the system design. The think aloud method (concurrent verbal protocol taking) and interviewing are being employed to collect data. Testing performed in every cycle of iteration will produce quantitative and qualitative data about the prototypes.
Analysis: Analysis will include a description of how structured identification of the observed problems in prototypes of the computer-based system led to solutions that were incorporated into the user-interaction design to improve its usability.

Dempsey, B. J., Wildemuth, B. M., & Geisler, G. Use of an expanding directory interface for WWW legal resources [poster]. Presented at the Digital Libraries '99 Conference, August 11-14, 1999, University of California, Berkeley.

The Directory of Internet-Accessible Legal Resources (DIALeR), http://legalsearch.ils.unc.edu/, was developed to aid in rapid discovery and navigation between high-quality legal sites. It is a hybrid system, incorporating both human selection and indexing of resources and finer-grain searching through search tools available at individual sites. The results of this study provide baseline data concerning the ways that lawyers and law students navigate a directory structure and their selection of resources indexed in that structure. DIALeR transaction logs from February 1, 1999, when the tool was first fielded, through April 12, 1999 were collected and analyzed. There were 342 search sessions conducted during this period, for a total of 2240 user actions. The results of this study indicate that a broad spectrum of Internet users are willing to spend time using well-structured directories of Web resources in particular disciplines.

Invited Presentations and Speeches

St. Amant, R., & Wildemuth, B.M. (2008). eServices and the human factor. Presented at the North Carolina Digital Government Summit, Raleigh, NC, September 3, 2008. Slides.

Wildemuth, B.M. (2007). Introduction to human-computer interaction and interaction design. Presented for the HCI Seminar, Institute of Information Studies and Librarianship, Charles University, October 9, 2007. Slides.

Several key concepts from the field of human-computer interaction are reviewed, including Norman's stages of action model and the idea of affordances. The interaction design lifecycle is then discussed, followed by a discussion (with examples) of the usability of and user experience provided by a particular design.

Wildemuth, B.M. (2007). Bringing your library into view. Presented at Jinonice Information Monday’s, Institute of Information Studies and Librarianship, Charles University, October 1, 2007. Slides.

After the concept of information horizons is introduced, a hypothetical example of an engineer's information horizons is discussed. The role of libraries in this hypothetical case is then reviewed, and two factors that affect a library's position on a potential user's information horizons are examined. It is argued that libraries can improve their positions on those information horizons by supporting information use, as well as information seeking. Four types of services that support information use are suggested: services supporting re-use of existing information, services supporting the creation of new information objects, services supporting learning, and services supporting user collaboration.

Wildemuth, B.M. (2007). Bringing digital libraries into view on an individual’s information horizon. Keynote presentation, “Libraries in the Digital Age,” Dubrovnik and Mljet, Croatia, May 28-June 2, 2007. Slides.

The ideas presented in Prague (above) are also reviewed here, with a focus on the role and position of digital libraries.

Pomerantz, J., & Wildemuth, B.M. (2007). Digital curation issues within the context of a curriculum for digital librarianship. DigCCurr 2007: An International Symposium on Digital Curation, Chapel Hill, NC, April 19, 2007. Slides.

While Tibbo and Lee are developing a curriculum for digital curation, Pomerantz and Wildemuth are involved in an interinstitutional project (with Fox at Virginia Tech) to develop a curriculum for digital librarianship. The process for developing that modular curriculum, the framework for the curriculum, and a template for module development are presented.

Wildemuth, B. M. (2006). CS and ILS as disciplinary homes for digital library education. Workshop on “Developing a Digital Libraries Education Program,” Joint Conference on Digital Libraries (ACM & IEEE-CS), Chapel Hill, NC, June 15, 2006. Slides.

This presentation considered computer science and information and library science as two alternative disciplinary homes for a curriculum in digital librarianship. The differences between the two disciplines are noted, but it is argued that development of a digital library curriculum would benefit from a cooperative venture, rather than two independent efforts.

Wildemuth, B. M. (2006). “Open Video project: How people go about searching video collections.” Invited presentation, SURA/ViDe (Video Development Initiative) Eighth Annual Digital Video Conference, Atlanta, March 29, 2006. Slides.

The research model supporting the studies conducted by the Open Video project is reviewed in detail, with examples from the studies already conducted. The implications of the model for future research are discussed.

Wildemuth, B.M. & Barry, C.L. (2006, February 16). Interview by Sanda Erdelez. “Studying Online Search Behaviors” [radio webcast]. LiS Radio: HIB Spot Series. Columbia, MO: School of Information Science & Learning Technologies, University of Missouri - Columbia. http://lisradio.missouri.edu/archive.php

Wildemuth, B. M. (2004).  Overview of evaluation methods.  Invited presentation, “Developing Archival Use and User Metrics,” sponsored by the Mellon Foundation.  University of North Carolina at Chapel Hill, June 4, 2004. Slides.

A primer on methods that could be used to evaluation user interactions with archives and special collections is presented.

Wildemuth, B. M. (2003). Why conduct user studies? The role of empirical evidence in improving the practice of librarianship. Keynote address. INFORUM 2003, Prague, May 27, 2003. PDF.

To conduct a user study requires a significant amount of time and effort. The understanding of user behaviors that can be gained through a particular user study must be valuable enough, in terms of improving the practice of librarianship, to make the effort worthwhile. A rationale for evidence-based librarianship, based on studies of user behaviors and the information needs that motivate those behaviors, will be presented. This rationale will be illustrated with several example studies, each of which provided empirical evidence that can be directly applied to the improvement of library practices (such as the development and marketing of reference services, the design of training programs for library and database users, and the design of online library systems). The importance of considering the context in which people seek information from libraries will be discussed, as well as the importance of considering the specific characteristics of the library’s patrons.

Technical Reports

Wildemuth, B. M., Russell, T., Ward, T. J., Marchionini, G., & Oh, S. (2005). The influence of context and interactivity on video browsing. TREC VID 2005 Notebook Paper. SILS Technical Report 2006-01. Chapel Hill: University of North Carolina, School of Information and Library Science, Technical Report Series. PDF.

Approaches tested: The goal of this study was to investigate the effects of providing context and interactivity in a retrieval system, supporting the browsing of search result sets. Thus, three systems were developed: (1) a basic system, modeled on the current results list provided by google video searching (runs UNC-BAS-1 and UNC-BAS-2); (2) a similar system, with the context of each shot provided by showing keyframes from the shots appearing just before and after the retrieved shot (runs UNC-CON-1 and UNC-CON-2); and (3) a system that builds on the previous system by offering several mechanisms of interactivity (runs UNC-INT-1 and UNC-INT-2).
Comparative findings: In terms of both performance and user perceptions, the Context+Interactive system was superior. While there were no differences in precision, recall was improved with this system, and users preferred it (based on several measures of user perceptions).
Conclusions: The effects of context on browsing search results were negligible, but should be explored further through re-examination of the definition and operationalization of the concept of context. Interactivity, in combination with context, had positive effects on browsing effectiveness; it was considered easy to use, even though it introduced more complexity into the interface.

Wildemuth, B. M., Yang, M., Geisler, G., Tolleson, T., Elsas, J., Luo, J., & Marchionini, G. (2004). Conceptions of features and semantic clusters as search mechanisms: a pilot study. TREC VID 2004 Notebook Paper. PDF.

Search mechanisms are the interface tools that people can use to search the items in a collection; a common example is a text box for entering search terms. In this pilot study, the Open Video team investigated two mechanisms for providing access to a collection of news videos. The first provided direct access to the subsets of the collection containing particular video features, as identified by other TREC VID participants; the second provided direct access to the subsets of the collection that were identified as semantic clusters, using latent semantic indexing approaches to analyzing the video transcripts. Each access mechanism was depicted in the interface as a series of labeled checkboxes. In the pilot study, four team members completed the TREC VID topic searches and responded to measures of their perceptions of the experience of using each search mechanism. Reasonably high precision was achieved on the user searches across all three systems (0.67-0.74), but none of the systems achieved high recall (0.10-0.11). Mean average precision across three runs (as calculated by NIST) ranged from 0.06-0.09. Searches were completed in approximately 8 minutes across all three systems. User satisfaction with the two experimental systems was mixed. Lessons learned from conducting this pilot study will contribute to the design of a follow-up study investigating the ways in which users of digital video retrieval systems conceptualize search mechanisms that incorporate access to subsets of the collection based on video features or semantic clusters of transcript content.

Wildemuth, B. M., Yang, M., Hughes, A., Gruss, R., Geisler, G., & Marchionini, G. (2003). Access via Features versus Access via Transcripts: User Performance and Satisfaction. TREC VID 2003 Notebook Paper. SILS Technical Report 2003-05. Chapel Hill: University of North Carolina, School of Information and Library Science, Technical Report Series. PDF.

The Open Video Project is specifically concerned with the surrogates that can represent the objects in a digital video collection and the mechanisms through which people can manipulate those surrogates. In TREC VID 2003, we compared the effectiveness of a transcript-only search system, a features-only search system and a search system combining transcript and feature searching. We also presented several different views for users to browse the results pages: a horizontal view, a vertical view, a “before & after” view, and an extra-keyframe view. A within-subjects research design was used, so that each of the 36 participants was exposed to all three search systems. Each participant searched half (12) of the assigned topics. The user satisfaction measures recommended by NIST were augmented by measurements of participants’ perceived usefulness, perceived ease of use, and flow. Results indicated that, with the transcript-only system and the combined system, users were able to achieve higher recall in less time per search. The results from the measures of satisfaction indicate that the users found the transcript-only and combined systems to be more useful and easier to use, and their use resulted in stronger perceptions of enjoyment and concentration than the features-only system. It is concluded that, as users gain experience with features searching, it will be a welcome supplement to transcript searching.

Yang, M., Wildemuth, B. M., Marchionini, G., Wilkens, T., Geisler, G., Hughes, A., Gruss, R., & Webster, C. (2003). Measures of User Performance in Video Retrieval Research. SILS Technical Report 2003-02. Chapel Hill: University of North Carolina, School of Information and Library Science, Technical Report Series. PDF.

Browsing and searching for digital videos online is not as easy as it is with text documents. To address this problem, researchers have begun to create video surrogates to represent video objects. The purpose of this paper is to describe and provide preliminary data regarding six measures that can be used to evaluate the effectiveness of people’s interactions with video surrogates. The six types of performance to be measured are: object recognition (with text stimuli); object recognition (with graphical stimuli); action recognition; gist determination (free text); gist determination (multiple choice); and visual gist determination. While some additional development of the measures is needed, their initial field testing indicates that they are practical and can differentiate multiple levels of performance with video surrogates. These measures will continue to be refined in studies conducted by the Open Video project; we also encourage others to employ them in video retrieval research.

Wildemuth, B. M., & Carter, A. R. (2002). The Perceived Affordances of Web Search Engines: A Comparative Analysis. SILS Technical Report 2002-02. Chapel Hill, NC: University of North Carolina, School of Information and Library Science, Technical Report Series. 25p. PDF.

One way to evaluate the interfaces of search engines is to analyze the perceived affordances offered by each. In this context, the perceived affordances of a search engine are those aspects of the interface that are perceived by its users as allowing particular functions to be invoked. For example, if a search engine provides one text box that is only 10 characters long, users may perceive that it affords the searching of terms that are 10 characters or less. This study analyzed and compared the perceived affordances of nine of the most popular Web search engines (AltaVista, Ask Jeeves, Excite, Google, Hotbot, LookSmart, Lycos/Open Directory, Northern Light, and Yahoo) in September 2001. The criteria for analysis included characteristics of the text box for entering terms, characteristics of the search button, search syntax, the availability and placement of help for entering search terms, methods for limiting the search results, support for modifying a query, features of the directory structure, characteristics of results displays, and methods for setting user preferences. The analysis was conducted by directly examining the interface of each search engine for each feature or characteristic.

In general, some aspects of Web search engine interfaces are becoming more standardized and other aspects vary widely across the search engines. All search engines provide a textbox and some type of accompanying button for entering a query. Almost all of the search engines provide assistance in specifying a query, but only two provide examples of queries on the search page itself. All of the search engines use the same basic syntax for specifying a query, but there is quite a bit of variation in the type and amount of assistance provided via drop-down menus or checkboxes. Direct support for modifying a query was available in only a few of the search engines. The results of searches are reported in fairly standard ways: brief summaries provided, usually 10 per page, in relevance order. The implications of these findings for the design of search engines are discussed.

Wildemuth, B. M., Sonnenwald, D. H., Bollenbacher, W. E., Byrd, G., & Harmon, G. (2001). Mentoring Future Biologists via the Internet: Results from the "Electronic Mentoring for Tomorrow's Scientists" Program. SILS TR-2001-07. Chapel Hill: University of North Carolina, School of Information and Library Science, Technical Report Series. PDF.

The E-Mentoring program provided biology students from two historically minority universities in North Carolina with opportunities to interact and develop relationships with corporate scientists, to expand their learning horizons, and to use technology in a meaningful way. To provide a meaningful context for electronic mentoring for students, the project was integrated with undergraduate and graduate biology courses at rural and urban universities in lower socio-economic areas. To learn from this experience, an intensive evaluation was conducted. Each participant filled out a detailed questionnaire and was interviewed, both before and after their participation in the E-Mentoring program. In addition, messages between students and mentors were archived. These data are analyzed and discussed in this report.

Sonnenwald, D. H., Marchionini, G., Wildemuth, B., Dempsey, B., Viles, C., Tibbo, H., & Smith, J. (2001). Collaboration Services in a Participatory Digital Library: An Emerging Design. SILS TR-2001-03. Chapel Hill: University of North Carolina, School of Information and Library Science, Technical Report Series. 12p. PDF.

Digital libraries need to provide and extend traditional library services in the digital environment. This paper presents a project that will provide and extend library services through the development of a sharium--a workspace with rich content and powerful tools where people can collaborate with others or work independently to explore information resources, learn, and solve their information problems. A sharium is a learning environment that combines the features of a collaboratory, where people collectively engage in research by sharing rich information resources, and a local library, where people come to meet, find information resources, and discuss common interests. To achieve this, collaboarion servies that build on synchronous and asynchronous communication technology should be integrated with other digital library services, including searching, browsing, and information management and authoring services. This paper presents our motivation for providing collaboration services and describes the types of collaboration services that will be included in the digital library.

Webster, L., Brassell, E., Sonnenwald, D. H., Wildemuth, B. M., Harmon, G. L., Byrd, G., & Bollenbacher, W. E. (2000). E-Mentoring Handbook: Lessons Learned from Two Electronic Mentoring Pilot Programs. SILS-TR-2000-3. Chapel Hill: Univeristy of North Carolina, School of Information and Library Science, Technical Report Series. 101p. PDF.

Electronic mentoring was implemented in two pilot studies, the first (in fall 1999) linking students at Elizabeth City State University (NC) with research scientists at Ortho Clinical Diagnostics (in NJ and NY) and the second linking students at North Carolina Central University with research scientists at Glaxo Wellcome (in NC and the UK). A Web-based electronic mentoring system supported the mentor-student pilot studies.

Sonnenwald, D. H., Marchionini, G., Wildemuth, B., Dempsey, B., Viles, C., Tibbo, H., & Smith, J. Collaboration services in a participatory digital library: An emerging design. Paper presented at the Third International Conference on Conceptions of Library and Information Science (CoLIS 3), Dubrovnik, Croatia, May 23-26, 1999. Also available as SILS Technical Report 2001-03. PDF.

Digital libraries need to provide and extend traditional library services in the digital environment. This paper presents a project that will provide and extend library services through the development of a sharium--a workspace with rich content and powerful tools where people can collaborate with others or work independently to explore information resources, learn, and solve their information problems. A sharium is a learning environment that combines the features of a collaboratory, where people collectively engage in research by sharing rich information resources, and a local library, where people come to meet, find information resources, and discuss common interests. To achieve this, collaboration services that build on synchronous and asynchronous communication technology should be integrated with other digital library services, including searching, browsing, and information management and authoring services. This paper presents our motivation for providing collaboration services and describes the types of collaboration services that will be included in the digital library.

Other Publications

Wildemuth, B.M., Pomerantz, J., Oh, S., Yang, S., & Fox, E.A. (2008). Preliminary results from field testing of DL curriculum modules. Paper presented at the workshop, “Education for Digital Stewardship: Libarians, Archivists, or Digital Curators?”, Joint Conference on Digital Libraries, Pittsburgh, PA, June 16-20, 2008. Slides.

Wildemuth, B.M. (2008). Values shape perceptions of value. Paper accepted for the “Values, Value, and Worth: Their Relationship to HCI?” workshop at Association for Computing Machinery, Special Interest Group on Computer-Human Interaction (ACM SIGCHI), Florence, Italy, April 5, 2008. Full text.

It is argued that the values of potential users of information systems shape their judgments about the value of those systems. Thus, it is critical that designers take users’ values into account, just as they take into account users’ physical, cognitive, and affective attributes.

Wildemuth, B. M. (2007). Bringing digital libraries into view on an individual’s information horizon [video recording]. Recorded May 22, 2007.

Wildemuth, B. M. (2007). Subjectivity: Its role in exploratory search processes and evaluation. Paper accepted for “Exploratory Search and HCI” workshop at Association for Computing Machinery, Special Interest Group on Computer-Human Interaction (ACM SIGCHI), San Jose, CA, April 29,2007. Full text.

Subjectivity is inherent in exploratory search. It is evident in the attributes of the searcher before beginning the search (e.g., the searcher’s domain knowledge). In the form of the searcher’s interests, knowledge, cognitive/learning style, and personality, it affects the search process – its overall trajectory. And it comes into play at the end of the search, when the searcher makes judgments about the success of the search. Researchers and designers interested in explora­tory search will need to develop measures of searchers’ beliefs about a search’s success and attitudes toward the search outcomes. These measures can then be leveraged to more directly evaluate the quality of the search process and to design personalized systems that improve a searcher’s performance.

Wildemuth, B. M. (2006). The experience of flow during online searching. Paper presented for discussion at the ASIST SIG USE Research Symposium, 2006: “Information Realities: Exploring Affective and Emotional Aspects in Information Seeking and Use.” Full text.

Wildemuth, B. M. (2006). The illusion of online privacy. Paper presented at the ACM SIG CHI 2006 Workshop, Privacy and HCI: Methodologies for Studying Privacy Issues, April 23, 2006, Montreal. Full text.

After providing a brief discussion of differing conceptions of privacy, studies of Internet users’ privacy-related behaviors and their online privacy concerns are reviewed. Next, the evidence concerning the effects of their concerns on their privacy protection behaviors is discussed. Finally, methods for studying these issues are evaluated, and recommendations for improving these methods are made.

Wildemuth, B. M. (2004). Teaching HCI within the context of information and library science. AIS SIGHCI Newsletter, 3(1), 12. Full text.

Yang, M., Marchionini, G., Wildemuth, B. M., & Mu, X. (2003, October).  The Open Video Project – building an open source digital video archive.  DigiCULT.Info: A Newsletter on Digital Culture, issue 5, 22-26. Full text.

Wildemuth, B. M. (1999). The relationship between a book's genre and the activities it supports. Discussion document prepared for "Designing Electronic Books," a pre-conference workshop sponsored by the Association for Computing Machinery Special Interest Group on Computer Human Interaction, May 16-17, 1999. Full text.

Simple comparisons between printed books and electronic books do not take into consideration the tasks for which books are used. Results from Dillon suggest that reading encompasses two types of tasks: scanning and serial detailed reading. Findings from the field of reading comprehension (Guthrie & Mosenthal) suggest that locating information in documents (i.e., reading-to-do) is a skill that is distinct from prose comprehension. Annotating and highlighting text are tasks that are performed with print media, and are likely to be necessary functions performed with electronic books. In addition, most authors looking forward to an era of electronic books call for the ability to manipulate the text of the book. By delineating the tasks performed when interacting with specific genre, the design space for the "electronic book" becomes more complex. During the workshop, the boundaries of design spaces for particular genre were explored.

Wildemuth, B. M., Friedman, C. P., & Downs, S. M. (1999). Research alerts: Hypertext versus Boolean access to biomedical information: A comparison of effectiveness, efficiency and user preferences. interactions,6(1), 10-11.

See abstract of full article, above.

Wildemuth, B. M. (Ed.) (1998). Collaboration across boundaries: Theories, strategies, and technology. Proceedings of the 1998 ASIS midyear meeting, Orlando, FL, May 16-20, 1998. Medford, NJ: Information Today, for the American Society for Information Science. (Online version available at http://www.asis.org/Conferences/MY98/proceedings.htm).

In response to mature markets, increasing worldwide competition, fast technological development, and decreasing resources, many organizations are adopting collaborative work practices in order to solve complex problems and produce innovative products. To collaborate effectively, we must explore and integrate knowledge and work practices from diverse domains to come to a pragmatic understanding of the work process and expected outcomes. Theories of collaboration have been developed to understand these processes. Strategies and technological support have been developed to overcome some of the major obstacles to collaboration. Outcomes of the application of these strategies and technologies have been evaluated. Such collaborative theories, strategies, technologies, and outcomes were highlighted in the papers, panels and demonstrations presented at the 1998 ASIS midyear meeting.

Wildemuth, B. M. (1995). Defining search success: Evaluation of searcher performance in digital libraries. Discussion document for "How we do user-centered design and evaluation of digital libraries: A methodological forum," 37th Allerton Institute, Graduate School of Library and Information Science, University of Illinois at Urbana-Champaign. Full text.

It is a common belief--and empirical data provides supporting evidence--that some people are more proficient in performing online searches than others. In psychometric terms, "searcher proficiency" is a construct exemplified by a domain of observable behaviors. Prior research has indicated that the construct of searcher proficiency in the context of a traditional database includes such observable behaviors as the precision and recall achieved in the search results; the overlap of the terms selected with a prespecified set of "good" terms; and the time or number of commands used to conduct the search. Finding ways to describe and evaluate the effectiveness of browsing/navigation behavior will be an additional challenge for designers and researchers of digital libraries.

Wildemuth, B. M., & Moore, M. E. (August 1993). End user searching of MEDLINE: Final report. (ERIC Document Reproduction Service No. ED 363 357)

One hundred sixty-one MEDLINE searches conducted by third year medical students were analyzed and evaluated to determine which search behaviors were used, whether those individual moves are effective, and whether there is a relationship between specific search behaviors and the effectiveness of the search strategy as a whole. The typical search took 14 cycles, used 7 terms or concepts, and resulted in the display of 11 citations. The most common moves were selection of a database, entering single-word terms and free-text term phrases, and combining sets of terms. Syntactical errors were also common. Librarians judged the searches to be adequate, and students were quite satisfied with their own searches. Librarians identified many missed opportunities in the search strategies, including the underutilization of the controlled vocabulary, subheadings, and synonyms for search concepts. There were no strong relationships found between the librarians' and students' evaluations of the searches and the measures of searching behaviors. Implications of these findings for system design and user education are discussed. Eleven statistical tables presenting research results are included. The end user questionnaire, lists of categories for coding search strategy moves, a sample of coding of a search, and a search evaluation form are appended. (Contains 15 references.)

Wildemuth, B. M. (Ed.). (1990). External connectivity: Will my office system talk to my online vendor? Transactions of a session of the Mid-Year Meeting of the American Society for Information Science (ASIS), Fort Lauderdale, FL, May 13-17, 1990.

The goal that is generally expressed by the term, connectivity, is to create a seamlessly-interconnected information world. Most corporations address the goal of connectivity only within their own organizational boundaries--interconnecting their own piece of the world. The papers in this volume address the means by which information acquired from external information systems can be most effectively transmitted to a library's clients through internal networks. Robert Stout describes the use of the Communications module of DEC's ALL-IN-1 office system to provide selective dissemination of information (SDI) services to the faculty of the Palmer College of Chiropractic. Jacquelyn Becker next describes the PC-based approach used at Helene Curtis. In the third paper, Alane Love describes a similar appropach used at United Technologies, augmenting it by delivering search results to many different locations using two different office systems.

Wildemuth, B. M. (1989). End-user computing: The adoption of an intellectual technology in corporate settings (Doctoral dissertation, Drexel University, 1989). Dissertation Abstracts International, 50, 2283A.

The goals of the current research were to develop an empirically-grounded model of the adoption process for intellectual technologies, and to identify the variables which have the strongest effects on that process. To accomplish these goals, the adoption of end-user computing applications was studied in five intact departments from three different corporations. Data were collected through a series of five in-depth interviews with each study participant, direct observation of the department's activities, and examination of already-existing documents related to the department's computer use.
Based on the computing applications in use in these five departments, a five-stage model of the adoption process was developed. The five stages are: Resource Acquisition, in which staff, hardware, software and training are acquired; Application Development, in which specific procedures for using a specific hardware/software combination are developed to perform a specific function; Adoption/Renewal, in which the application is adopted by a person other than the developer and/or additional resources are acquired; Routinization/Enhancement, in which the application becomes part of the department's standard operating procedure and/or is enhanced in some way; and External Adoption,in which the application is adopted by someone outside the developer's department.
The results of this study also indicate that the manager's approach to computing, the availability of slack time, the accessibility of hardware/software, and computer expertise within the end-user department have strong effects on the adoption of end-user computing applications. In addition to the implications that these findings have for managers in end-user departments, this study has three implications for researchers: it has clarified the definition of intellectual technologies, proposed a new model of the adoption process that is relevant to intellectual technologies, and developed a method for coding events occurring in an ongoing process.

Gonzalez-Stupp, E. & Wildemuth, B. M. (1985). At home with ERIC: Online searching from your home computer. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. ("After Dark" Edition: ERIC Document Reproduction Service No. ED 267 099; "Knowledge Index" Edition: ERIC Document Reproduction Service No. ED 267 100).

The purpose of this manual is to provide guidance to home computer owners who would like to do their own literature searches of ERIC and other education and social science database. The manual is divided into five chapters. The first two chapters cover the basic concepts of developing a search strategy, and the commands used by a particular system (After Dark, or Knowledge Index). The third chapter discusses various ways to modify your search strategy. Chapter 4 explains how to do some common types of non-subject searches, such as seaches for the works of a specific author. The final chapter provides basic information about a few of the other databses you may wish to search.

Wildemuth, B. M. (Summer 1984). Microcomputer-assisted testing: Resources from ERIC. Educational Measurement: Issues and Practice, 3, 48-49.

A representative sample of citations from the ERIC database, on the topic of microcomputer-assisted testing, are provided.

Wildemuth, B. M. (Ed.). (1981). A bibliography to accompany the Joint Committee's Standards on Educational Evaluation. ERIC/TM report 81. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 222 512).

This annotated bibliography is organized to match the format of the Standards for Evaluations of Educational Programs, Projects, and Materials, including sections on utility, feasibility, propriety, and accuracy of evaluations. It identifies literature that includes in-depth information about the issues covered in the Standards. The Standards are intended to guide evaluations of programs, projects, or materials in elementary, secondary, higher, or adult education. The intended audience includes persons who commission, conduct, or use evaluations, especially teachers, administrators, evaluators, curriculum specialists, school board memebers, legislators, counselors, leaders of educational associations, and parents.

Wildemuth, B. M. (Fall 1979). Procedures for identifying and selecting a minimum competency test. Education Libraries, 5, 8-9.

This brief article outlines the steps that a school district should follow in the first stages of the implementaiton of a minimum competency testing program. They include preparatory activities, defining the project, identifying existing tests, evaluating tests, developing a local test, and sharing the knowledge gained.

Wildemuth, B. M. (1978). Research and evaluation studies from large school systems, 1977. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 166 262).

This bibliography is the result of a search of the ERIC database, and includes documents announced in Resources in Education (RIE) from April 1977 through March 1978. It is limited to documents generated directly or indirectly by large school districts in the United States and Canada. The citations are arranged alphabetically by school district name. A subject index is included.

Wildemuth, B. M. (1977). Test anxiety: An extensive bibliography. TM report 65. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 152 860).

A large amount of research has been done in the past few decades to pinpoint the effects of test anxiety on the performance of those taking tests. Much of that research is listed in this bibliography, which originated with a computer search of the ERIC database, Psychological Abstracts, Comprehensive Dissertation Abstracts, Sociological Abstracts, Exceptional Child Education Abstracts, and the National Technical Information Service database. The bibliography is organized in two sections: the first includes the most recent citations, each with an abstract of the research. The second section lists only the citations for work that was done before 1970. Both sections are indexed. An appendix lists and describes several of the more commonly used measures of test anxiety.

Wildemuth, B. M. (1977). Program evaluation in the arts: An annotated bibliography. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 151 424).

This 56-item annotated bibliography gives teachers and administrators access to information on the evaluation of school fine arts programs. Based upon a computer search of the ERIC database, it cites project reports, journal articles, and dissertations published from 1963 to 1976. The first section includes general discussions and those reports which concern several of the arts; separate sections for the visual arts, dance, drama, and music follow. A subject index is provided.

Wildemuth, B. M. (1977). Minimal competency testing: Issues and procedures. An annotated bibliography. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 150 188).

Due to the current legislative trend toward competency-based eudcation, and the accompanying minimal competency testing, a variety of local school districts and state education agencies are concurrently developing programs to meet the standards set by their state's legislation. This selective bibliogrpahy was compiled in order to aid those involved in such activities. It is based on computer searches of the ERIC database, National Technical Information Service database, Psychological Abstracts, Exceptional Child Education Abstracts, and Dissertation Abstracts. The 28 cited documents which were selected from those searches provide some examples of what has been done in this area, and some discussions of the problems and questions that must be faced.

Wildemuth, B. M. (1977). Research and evaluation studies from large school systems, 1976. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 142 584).

The ERIC database was searched in order to identify documents which reported evaluation research carried by forty large school districts in the United States and Canada. The citations are arranged by the name of the reporting school district. Each of the 149 references is abstracted, and a subject index is provided.

Wildemuth, B. M. (1977). Mastery learning and testing: An annotated ERIC bibliography. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 138 646).

This 136-item annotated bibliography on mastery learning and mastery testing is the result of a computer search of the ERIC database in February 1977. All entries are listed alphabetically by author. A subject index is provided.

Wildemuth, B. M., & Porter, D. E. (1976). State assessment and testing programs: An annotated ERIC bibliography. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 141 389).

There is a growing body of literature in the ERIC database pertaining to state educational assessment and testing programs. Volume I of this bibliograpy includes abstracts of 39 docuemtns and journal articles describing the design and implementation of programs, as well as the technical and political issues which have been addressed by the states. Volume II is a state-by-state annotated listing of 130 descriptive and technical reports issued by states which have had or now have testing or assessment programs. A subject index is provided for each volume.

Wildemuth, B. M. (1976). Cheating: An annotated bibliography. Princeton, NJ: ERIC Clearinghouse on Tests, Measurement, and Evaluation. (ERIC Document Reproduction Service No. ED 132 182).

This 89-item annotated bibliography was compliled to provide access to research and discussions of chating and, specifically, cheating on tests. It is not limited to any educational level, nor is it confined to any specific curriculum area. It is the result of computer searches of the ERIC database and Psychological Abstracts. A subject index is provided.

Return to Barbara Wildemuth's Home Page or Vitae


This page was last modified on September 20, 2008, by Barbara M. Wildemuth.
Address all comments and questions to her at wildem, in the domain ils.unc.edu.
© Barbara M. Wildemuth, 2008. All rights reserved.