Shifts of Focus among
Dimensions of User Information Problems
as Represented during
Interactive Information Retrieval

David Robins

School of Library and Information Science
Louisiana State University

Building on a preliminary study presented at the 1997 ASIS annual meeting (Robins, 1997), the author presents the results of additional analyses of shifts of focus in IR interaction. The results (from the author's dissertation) indicate that users and search intermediaries work toward search goals in nonlinear fashion. Twenty interactions between twenty different users and one of four different search intermediaries were examined. Analysis of discourse between the two parties during interactive information retrieval (IR) shows changes in topic occurs, on average, every seven utterances. These twenty interactions included some 9,858 utterances and 1,439 foci. Utterances are defined as any uninterrupted sound, statement, gesture, etc. made by a participant in the discourse dyad. These utterances are segmented by the researcher according to their inentional focus, i.e., the topic on which the conversation between the user and search intermediary focus until the focus changes (i.e., shifts of focus). In all but two of the 20 interactions, the search intermediary initiated a majority of shifts of focus. Six focus categories were observed. These were foci dealing with: documents; evaluation of search results; search strategies; IR system; topic of the search; and information about the user.

The goal of this study is to increase understanding of information problems as they are revealed in interactions among users and search intermediaries. Specifically, this study seeks to investigate: (a) how interaction between users and search intermediaries reveals aspects of user information problems; and (b) how users and search intermediaries focus on aspects of user information problems during the course of online searches. This paper presents further analysis on the same project reported in Robins (1997).

In this paper, the term "information problem" is used so that a more complete view of the user may be included. "Information need" is too restrictive for the purposes of this study because it implies that there is specific information for which a user has a need, and for which s/he is seeking. Using the term "information need" would then restrict any given situation to that which the information sought is known to exist within a well defined context (Belkin & Vickery, 1985). By "information problem," a broader range of factors affecting search strategy, question negotiation, query formulation, relevance judgments, social factors, project environment, information use and seeking behavior, etc. are implied.

Information retrieval is a dynamically interactive process. Saracevic (1997), states that user modeling (as derived from the observation of IR interaction) is:

If this assertion is true, then it is possible to assume that users, interacting with search intermediaries, will reveal and focus on different dimensions of information problems during the course of any given interaction. Specifically, their interaction will focus on various aspects of the problem at hand at various points in the interaction timeline. In short, participants' interaction may be characterized by shifting among dimensions of information problems. What is not known about such shifts of focus is: (i) whether such shifts are identifiable; (ii) how often such shifts occur; (iii) how long participants focus on any given dimension; or (iv) whether such shifting is manifested in patterns. The dynamic nature of IR interaction is assumed, but there is not a great volume of research to confirm and explicitly describe such assumptions. Specifically, it may be of interest in IR to better understand how participants in IR interaction shift from one focus to another. That is, "What are the foci of attention during IR interaction?" "How long do the participants spend talking about each focus?" "What are the purposes and nature of such foci?"

For the purposes of this study, interaction shifts are any change in focus of the interaction between a user and a search intermediary with respect to the user's information problem. Change of focus is denoted by a change in some topical aspect of the conversation (by shifting to a different aspect of the topic, or by broadening or narrowing the topic, etc.), or by some shift to a non-topical aspect of the information problem (again by narrowing or changing). In other words, a shift may occur between or within topical or nontopical discussions, or, shifts may occur laterally or horizontally within a topical or non-topical discussion. Brown and Yule (1983) characterize "topic-shifts" (p. 94) in the following way.

Given that different aspects (dimensions) of user information problems are discussed during information retrieval interaction, the following research questions guided this study.

1. What aspects/dimensions of users' information problems are discussed by users and search intermediaries during interactive IR?
2. What sequences/patterns of interaction behavior between users and search intermediaries represent attributes of the user's information problem?
3. How much, and on which dimensions of user information problems do participants focus during IR interactions (i.e., what patterns of interaction behavior among users, search intermediaries, and texts are indicative of shifts of approach to the information problem)?

This notion of information problem grows out of a body of literature dating to the mid-1970s that began to break from the traditional study of systems, and begin to focus on users. For example, Dervin (1976; 1977) began looking at social contexts of how people use information, and encouraging information professionals to probe users for such information in order to serve better them. Other researchers began to talk of problematic situations that motivate people to engage in information seeking (Wersig & Windel, 1985).

The hypothesis of information problems that generated the most research during the 1980s and 1990s is Belkin's (1980; Belkin, Oddy & Brooks, 1982) anomalous states of knowledge (ASK). Essentially, Belkin recognized that a user enters into interaction with an IR system due to some acknowledged gap (i.e., anomaly) in her (his) understanding of a topic or situation. Because of the gap of understanding, the user may experience difficulty expressing a need. Therefore, the functions of an IR system/intermediary might include the ability to investigate a user's ASK, rather than an information need.

The ASK hypothesis is the basis on which some researchers approach user modeling. A series of experiments during the 1980s sought to describe the processes by which search intermediaries model user problems (Belkin, Seeger, & Wersig, 1983; Belkin, 1984; Brooks, 1986; Belkin, Brooks & Daniels, 1987). The purpose of these efforts was to provide a basis on which to construct an intelligent intermediary (i.e., automated search intermediary). Therefore, the researchers observed intermediaries during real presearch interview process in order to identify the functions they performed. These functions were then restated in terms of what an intelligent interface should provide for a user.

It was also Belkin, Brooks & Daniels (1987; and Belkin, 1984 among other Belkin efforts) that contained the first notion of "focus shifts" as used in the present study. Their ideas developed partly on the basis of, and partly in parallel with, a study by Grosz and Sidner (1986). Grosz and Sidner put forth the idea that discourse could be segmented according to the intentions of speakers engaged in conversation, or in other texts. Grosz and Sidner consider segmentation in discourse to be a naturally occurring component of interaction. Belkin et al found that such segmentation occurs during information retrieval interaction.

Another facet of Belkin's (1984) and Belkin, Brooks, and Daniels' (1987) research dealt with user modeling. User modeling is the study of the ways in which participants in interactive information retrieval develop understanding of one another (Allen, 1991). Most of the research effort is directed toward understanding how intermediaries model user information problems for the purpose of developing automated intermediaries. However, Belkin identifies 17 types of models that may occur during interactions with information retrieval systems. Examples of these models are: intermediary's model of user; intermediary's of information and information systems; user's model of authors; user's model of intermediaries, information, and the world. This type of modeling is a complex component of IR interaction.

Information retrieval interaction, itself, has been a topic of increased interest in the IR community of late (e.g., Saracevic, 1997; Spink, 1997; Ingwersen, 1992). Models developed in this line of research represent attempts to describe the iterative, and highly dynamic nature of information seeking using interactive computers. Traditional IR models lack a description of such complexity (Saracevic, 1996).

However, most of the studies mentioned above fail to address the full range of cognitive and social activities that occur during interactive IR. For example, sensemaking theories have been suggested by Dervin (1983; in press) and by Weick (1995). Sensemaking is an approach to human behavior that is rooted in the idea of social construction of reality (Weick, 1995). Essentially, it conceives human behavior as action based on environmental cues. These cues are used by individuals in social situations, or in private, to construct plausible explanations and bases for action. Sensemaking is about a continual, ongoing interaction between an individual and his/her environment in an attempt to literally "make sense" of the bewildering complexity inherent in any given life situation. Dervin (1992; 1983b) has used this line of thought to explain information behavior, and to suggest IR system design principles. Dervin's idea of sensemaking relies heavily on the concept of discontinuity, that is, gaps in knowledge at given points in time that cause an individual to seek information. She conceives sensemaking as a bridge that individuals build over the gap between their present knowledge state and whatever resources produce help, and ultimately, information use. One use of the research in this paper is to show that users do not move linearly through the search process. Sensemaking approaches are one way to frame such a notion.

In the following sections, the research design for this study is outlined, and results are presented, followed by a discussion of the findings in light of this review of related literature.

The aim of research in this paper is to describe the nature of interaction shifts between search intermediaries and users. The following research design is used to illuminate such shifts.

Data Corpus

Saracevic and Su (1989) collected the data during a previous study. The study was funded by a grant from the Library Research and Demonstration Program, United States Department of Education (ref. no. R039A80026), with additional funding by DIALOG, and entitled, Nature and Improvement of Librarian-User Interaction and Online Searching for Information Delivery in Libraries. The data consists of transcribed discourse (originally videotaped) between real users and professional search intermediaries during authentic information retrieval interactions. All of the users were either graduate students or faculty in pursuit of some particular research goal. In summary, 40 searches were taped, amounting to over 46 hours of video. Each search averaged nearly 70 minutes (13 minutes presearch; 56 minutes online). Four search intermediaries participated in the study, each averaging over 8 years of search experience. The searches covered a wide variety of topics (46 different databases were used). In addition, a survey was administered to users and search intermediaries both prior to and after each search. Some of the results of these surveys are alluded to later in this report.

Twenty of the 40 transcripts are analyzed in this study. Both user and search intermediary utterances are included in the analysis. The transcripts are drafted as utterances. That is, they are written in the form of:

The above interaction consists of 3 utterances.

Methodology

Twenty (20) of the above mentioned interactions were analyzed in detail for this preliminary study. The goals prescribed by the research questions require that the methodology do the following:

1. Identify shifts within each dialog;
1. Classify each shift according to its type and function;
2. Quantify the number of shifts and utterances in each shift;

Each of these facets of the methodology is discussed in the following sections.

Identification of shifts

Interaction shifts are defined as any change in focus of the conversation between the user and search intermediary with respect to the user's information problem. Interaction shifts are any change in focus of the interaction between a user and a search intermediary with respect to the user's information problem. Change of focus is denoted by a change in some topical aspect of the conversation (by shifting to a different aspect of the topic, or by broadening or narrowing the topic, etc.), or by some shift to a non-topical aspect of the information problem (again by narrowing or changing). In other words, a shift may occur between or within topical or nontopical discussions, or, shifts may occur laterally or horizontally within a topical or non-topical discussion.

Belkin, Brooks and Daniels (1987) identified "focus shifts" (p. 85) in their analysis of user modeling functions of intermediaries in pre-search interviews. They found that intermediaries initiated most coding shifts, a notion consistent with other discourse analysis literature which suggests that shifts in dialog are initiated by participants with higher status (Grosz, 1981). Belkin et al used, in part, certain dialog cues to identify the points at which shifts took place. These cues have been referred to as "frame words" (Sinclair & Coulthard, 1975). For example, such cues might be utterances which contain frame words such as "well," "now," "right," "ok," or "good." Such words in (particularly at the beginning of) an utterance may indicate that the speaker has begun to think about changing the focus of the discussion.

Classification of shifts

Shifts are to be classified according to their type and function. Shifts types refer to a broad class of shifts to be determined as they emerge from the data. Types of shifts concern the source of motivation behind the shift. Shift functions refer to the specific purpose of each shift. The exact nature of types and functions are determined as they emerge from the data consistent with principles of qualitative, grounded theory research (Glaser & Strauss, 1967). However, another facet of the author's dissertation deals with the classification of individual utterances. It is assumed that the categories assigned to the individual utterances within a shift give evidence of the shift's function. Yet, each shift is analyzed as a whole, which may or may not be the sum of its parts.

Quantification of shifts and utterances within shifts

In order to show a clearer picture of the nature of shifts in IR interaction, it is necessary to provide an account of the number of shifts that occur in each interaction, and in aggregate. Accordingly, shifts are to be tabulated by presearch, online and total, for types and functions of shifts.

Identification of focus shifts required the researcher to establish: (a) that interaction participants had changed the intentional focus of their discourse; and (b) at what points during the interaction the focus began and ended (i.e., a starting and ending point for the focus). Robins (1997) demonstrates the manner in which discourse is segmented into individual foci (examples are provided in that paper as well).

A total of 1439 such shifts of focus were identified (presearch = 355, online = 1084) (see Table 1). In all, roughly one fourth of the shifts occurred in the presearch phase; three fourths occurred in the online phase. Within cases, presearch shifts ranged from 8.33% to 40.00% of within shift totals (online ranged from 60.00% to 91.67%). Table 1 also shows how the majority of time in each interaction is spent after the participants go online. Therefore it is not surprising to find in Table 4.18 that in all cases, there are more foci in the online phase of the interaction than in the presearch phase.

Table 1

The figures presented in Table 1 indicate only raw totals. In order to compare shifts of focus between cases, one must determine the rate at which shifts of focus occurred in all interactions, and then test to see whether there is significant variance among the cases. Rate is calculated as the number of utterances spent by the participants during each focus. Utterances per shift were averaged within each case and interaction phase (see Table 2).

Table 2
Utterances Per Shift by Case and Search Phase (i.e., Presearch, Online, Total).

Another pertinent description of the characteristics of focus shifts regards the initiator of shifts. That is, one might ask, "Which of the participants in the interactions were most responsible for initiating changes in focus?" The answer to this question may give clues concerning the active/passive nature of each participant. Table 3 addresses this issue in terms of frequency and percentage within each case. It is interesting to note that in all but two of the interactions, search intermediaries initiated the majority of focus shifts. Overall, search intermediaries initiated shifts from one focus to another 66.71% of the time. In the following section, a classification of foci is presented.

Table 3

Coding Scheme

The next step toward describing the focus of shifts in IR interaction is to develop a coding scheme that characterizes such foci. Since this study's objective is to understand dimensions of user information problems, it is necessary to derive the shift level coding from the utterance level coding described above. The reason for the derivation is that both coding schemes seek to describe the same phenomenon on two different units of analysis (utterances and shifts). After identifying and analyzing each discourse segment, 10 categories of focus were derived for the analysis of focus shifts. Table 4 presents the coding scheme for foci analysis of transcripts.

Table 4.
Shift Level Coding Scheme.

Examples of Coded Foci Taken from Transcripts

In this section, examples of coding application are presented. By doing so, it is possible to demonstrate how the coding was applied.

Examples of Foci by Shift Level Code

Each of the codes described in Table 4 are illuminated with examples below.

Note that in this example, the researcher conducting the videotaping, Louise Su (S) responded to the search intermediary's elicitation for information regarding the number of allowable records to be printed within the limitations imposed by the study itself.

This coding scheme was then applied to the data at large. In the next section, the tabulated results of coding are compiled.

Compilation of Coding Results

This section presents the results of classifying the discourse segments (i.e., shifts of focus). For each category introduced in Table 4, observed occurrences and percentages are compiled. Table 5 shows that participants focused on strategy and evaluation issues in roughly 60% of all foci. The other 40% of their focus was spent among the remaining 8 categories. Interestingly, a loglinear analysis of the shifts among foci showed that participants were no more likely to shift from any one category to any another during interactions. That is, shifts from one type of focus to another appears to be a random event. This and the other findings are discussed in the following sections.

Table 5.
Frequency and Percentage of Codes Related to Shift Level Analysis.

The research to this point has laid a foundation to explore shifts of focus among dimensions of user information problems in IR interaction, and any patterns which may emerge from that study. Few, if any, patterns of behavior were found in this study through traditional methods such as loglinear analysis. Certain patterns, such as the fact that evaluative foci were not found in the presearch phase were not unexpected, and therefore, not discussed. However, the following four points (among many possible points) are suggested by the research in this project:

• Focus shifts occur rapidly
• Participants concentrate on strategy and evaluation
• Interactive IR appears to be chaotic
• There is only moderate evidence of changes in information problem conception

Each of these points is discussed in the following sections.

Focus Shifts Occur Rapidly

As shown in Table 4.19, participants shifted focus, on average, 7.01 utterances (6.67 during the presearch phase, and 7.12 during the online phase). The difference between the online and search phase is not statistically significant. Therefore, patterns that may be attributed to search phase are nonexistent.

However, the fact that participants shift topics of discussion so rapidly is worth noting for two reasons. First, this finding is consistent with notions of sensemaking (Weick, 1995; Dervin, in press) notions of behavior and cognition. In other words, in complex, novel activities such as interactive, mediated information retrieval, people depend on environmental cues and stimuli in order to construct notions of information problems and strategies to work through those problems. Therefore, it is possible that participants in IR interaction do not rely exclusively on problem representations as causal agents in rational decision making. In fact, they may rely on active construction of such representations. If this is true, then these representations must be neither a priori nor ad hoc, because both would lack the dynamic character suggested by rapid shifts among problem dimensions (especially when there appears to be nearly as much problem space as current cognitive state in cognitive space). That is, the former condition suggests a static starting point for representation, and the latter suggests that representation is the culmination of activity. Therefore, since participants do not dwell for long periods on any one topic, and because there is no clear pattern of transition from one topic to the next, the findings suggest a chaotic approach to information problems by participants. At the same time, however, the fact that a majority of utterances were indications of participants' current cognitive state and problem space, also suggests that participants were also moving between uncertainty and certainty with respect to information problems. These findings strongly suggest that participants were in a continuous state of constructing knowledge states based on new information gained through focus on various aspects of user information problems.

Another reason for noting the rapid nature of focus shifts is the implications of this phenomenon for IR system design. Automated intermediary design requires that some type of knowledge base be present to assist end-users with search. In addition, or as an alternative means of intelligence, some way accumulating knowledge from end-users (e.g., neural nets) must be present. These two systems approach intelligent IR from the top-down, and bottom-up, respectively. Systems may also be developed that take advantage of both methods (for example, Mizoguchi, Tijerino & Ikeda, 1995). In any case, however, modeling users from verbal, interactive behavior is problematic. One of the main problems is that conversation in interactive IR is choppy; that is, participants don't talk about any one topic for very long. If, in order to model users' knowledge, machines require patterns that must be identified by way of heuristics, then (i) heuristics are difficult to generate because of lack of patterns, and (ii) assuming heuristics can be developed, machines will find it difficult to match heuristic patterns to real conversation. Therefore, using information problem dimension foci as a means of building knowledge bases would appear to have limited application. However, the fact that a majority of interaction foci concentrated on search strategy and terms, means that evaluation of output may be of some use to systems designers.

Concentration on Strategy and Evaluation

Together, strategic and evaluative foci constitute 60.04% (44.27% and 15.77% respectively) of all foci occurrences. This finding indicates that the majority of foci dealt with input and output from the IR system, that is, the immediately practical aspects of information retrieval. In addition, the notion of terminological determinant in interactive IR as suggested by Saracevic, Mokros, and Su (1990) is supported by these findings. Essentially, terminological determinant holds that the main purpose of any interaction in IR is directed toward generating terms to formulate queries. Strategic foci alone account for 44.27% of all foci (13.06% presearch; 31.20% online), the most frequent of all categories. One of the conditions in a focus that makes it possible for a focus to be coded STRAT is discussion of search terms.

One question raised by the concentration on strategy and evaluation regards user modeling. For example, Belkin (1984) suggests user modeling in IR interaction as a means of developing intelligent intermediaries. Focus categories that might be related to user modeling include TOPIC and USER. TOPIC and USER categories constitute only 14.31% (7.51% presearch; 6.81% online) of all foci in the study. The fact that these foci occur almost equally during the presearch and online phases of interaction indicates that, if user modeling is occurring, it is occurring as much during the online phase as in the presearch. Therefore, user modeling, if it occurs, is an ongoing process. This assertion suggests that user modeling could be discussed in terms of sensemaking, because one of the properties of sensemaking, according to Weick (1995) is that it is an ongoing process. In any case, however, the following questions may be asked about user modeling in interactive IR.

1. To what extent is user modeling a function of terminological determinant? In other words, a search intermediary may have no interest, or even need, to know much about a user in order to perform an effective search. If the search intermediary can (i) generate terms for IR system input, and (ii) receive reasonable output, and (iii) the user is satisfied, then there is no reason to spend resources eliciting detailed information about the user. This notion is consistent with Weick's (1995) assertion that sensemaking requires plausibility above accuracy.
2. To what extent can the quality of user modeling be judged as a function of the amount of visible effort to do so? Since only 14.31% of all foci were attributed to categories that suggest user modeling, does this mean that only a small amount of user modeling occurs during IR interaction? Users filled out a question statement (see Appendix A) before the search that could account for search intermediary knowledge of topic and user background that was not discussed during videotaped interaction. The question statement may have reduced the need for a high quantity of further user modeling. Another possible explanation for the apparently small amount of user modeling is that search intermediaries may not require a great deal of information about a user in order to adequately model the user. Finally, if we assume non-rational behavior on the part of search intermediaries, then the ideal of user modeling itself becomes moot. That is, a search intermediary, under such an assumption, is acting without the time to benefit from representation such as user models. Each new situation presented in IR interaction requires a search intermediary to make a decision without the benefit of reflection (i.e., throwness). There are times, however, in interactions when participants do reflect (i.e., during periods of breakdown). (See, for example, Table 5.3). But for the most part, they appear to be acting without a great deal of reflection.

These two questions, however, need to be researched further.

Process Without Pattern

That participants shifted rapidly among foci (as mentioned earlier) is one indication that shifting of focus is a chaotic process. Another indication of the chaotic nature of focus shifts is the fact that no patterns could be found in the loglinear analysis on focus shifts. Loglinear analysis seeks to find statistically significant shifts from one state to another, and none were found in this analysis. This finding suggests IR interaction to be a highly idiosyncratic process. Each interaction among user, search intermediary, and IR system brings a different form of complex conditions that bring about different behaviors. It would appear that predicting such behaviors is problematic, at best. Therefore, one question to that needs to be addressed by future research regards the efficacy of intelligent intermediaries in situations in which search intermediaries must make subtle judgments. That is, given the chaotic, rapidly shifting nature of human mediated IR interaction, to what degree of subtlety can intelligent intermediaries be expected to interpret user information problems?

Moderate Evidence of Change in User Information Problem

One question that may be generated from this study deals with changes in user information problems that occur during IR interaction. The survey data show that users reported, on average, only moderate changes when they responded to the following questions regarding:

1. their perception of their information problem (2.3 on a scale of 5)
2. changes in their original search question (2.4 on a scale of 5)
3. changes in their personal knowledge (2.56 on a scale of 5).

In addition, the inductive portion of focus shift analysis (i.e., coding scheme development) uncovered no evidence of foci related to changes in user information problems. That is to say, there was not enough evidence to warrant a category. Only very isolated instances were found, such as that shown in Table 5.4. This finding is probably due to the stability and high definition of user information problems in this study. Users in this study ranked their problem definition to be relatively high (an average of 3.95 on a scale of 5). Although, on average, users reported that they were in early stages of their research (2.45 on a scale of 5), most were beginning research on a dissertation. This means that they had finished course work, and therefore, had some domain expertise. In addition, most had well formed question statements such as the one found in Appendix A. Therefore, most seemed to have a good idea of (i) the nature of the literature to be found in a search, and (ii) the quantity of literature to be found on the topic. This finding supports, once again, that Ingwersen's notion that stable, well-defined information problems result in searches with limited uncertainty.

However, there were exceptions to the norm. Seven users responded with higher average rankings to the three questions mentioned above. To be included in this group, users met the following criteria: users must have responded 3 or higher (on a scale of 5) on two of the three questions mentioned above. When those seven users' responses were isolated and averaged, the following rankings were found to the questions mentioned above regarding:

1. their perception of their information problem (4.0 on a scale of 5)
2. changes in their original search question (3.4 on a scale of 5)
3. changes in their personal knowledge (3.1 on a scale of 5).

When this group of seven was compared to the other 13 users studied, a t-test comparison of means showed a significant difference between groups on the first two questions (P<.001, and P<.008, respectively). An identical comparison on the third question revealed no significant difference. This may account, in part, for the level of uncertainty found in earlier phases of this study.

The significance of this finding for focus shifts, however, is that only negligible evidence of focus on changes in problem conception was found by studying foci. Perhaps it would be more fruitful to study changes in information problems during the course of IR interaction in situations in which such problems are not stable, or well formed. For example, Kuhlthau's (1991) subjects were much younger, and perhaps, less skilled at formulating research questions. Such a population may increase the odds of finding changes in information problems. Then, if researchers can see how such changes occur profoundly, they may be able to identify changes when they occur subtly.

The four points regarding shifts of focus mentioned in this section are a beginning in the process of describing the dynamic nature of interactive IR. More research needs to be done in order to find patterns in interaction. The ways in which dimensions of information problems are represented in interactive IR needs more study because of the complex nature of such interaction. While it is possible to observe representations of information problem dimensions, it is more difficult to observe patterns. It is possible that patterns do not exist, but it is also possible that they do exist. It may be necessary to look outside interaction processes in order to find patterns. Identification of behavioral patterns is necessary if our findings are to be incorporated into IR systems.

More research is needed on the identification and understanding of user modeling in the interactive IR process. The present study is only an extension of prior work in a field still in its infancy. The findings here suggest that some modeling does take place, but it is uncertain how much and what kind. Similarly, more research is needed that will identify instances in which users conceptions of information problems changes during the course of IR interactions. All of these efforts will greatly enhance our knowledge of the search process and allow the opportunity for designers to incorporate human factors into IR system design.

Allen, B. L. (1991). Cognitive research in information science: Implications for design. Annual Review of Information Science and Technology, 26, 3-37.

Belkin, N. J. (1980). Anomalous states of knowledge as a basis for information retrieval. Canadian Journal of Information Science, 5, 133-143.

Belkin, N. J. (1984). Cognitive models and information transfer. Social Science Information Studies, 4, 111-129.

Belkin, N. J., Brooks, H. M. & Daniels, P. J. (1987). Knowledge elicitation using discourse analysis. International Journal of Man-Machine Studies, 27, 127-144.

Belkin, N. J., Oddy, R. N., & Brooks, H. M. (1982). ASK for information retrieval: Part I. Background and theory. Journal of Documentation, 38(2), 61-71.

Belkin, N. J., Seeger, T. & Wersig, G. (1983). Distributed expert problem treatment as a model for information system analysis and design. Journal of Information Science, 5, 153-167.

Belkin, N. J. & Vickery, A. (1985). Information in information retrieval systems: A review of research from document retrieval to knowledge-based systems (Library and Information Research Report 35). London: British Library.

Brooks, H. M. (1986). An intelligent interface for document retrieval systems: Developing the problem description and retrieval strategy components. Unpublished doctoral dissertation, City University, London, United Kingdom.

Brown, G. & Yule, G. (1983). Discourse analysis. Cambridge, UK: Cambridge University Pr.

Dervin, B. (1976). The everyday information needs of the average citizen: A taxonomy for analysis. In M. Kochen & J. C. Donohue (Eds.), Information for the community (pp. 19-38). Chicago: American Library Association.

Dervin, B. (1977). Useful for librarianship: Communication not information. Drexel Library Quarterly, 13, 16-32.

Dervin, B. (1983). An overview of sense-making research: Concepts, methods, and results to date. International Communication Association annual meeting. May Dallas

Dervin, B. (1992). From the mind's eye of the user: The sense-making qualitative-quantitative methodology. In J. D. Glazier & R. R. Powell (Eds.), Qualitative research in information management (pp. 61-84). Englewood, CO: Libraries Unlimited.

Dervin, B. (in press). Chaos, order, and sense-making: A proposed theory for information design. In R. Jacobson (Ed.), Information design. Cambridge, MA: MIT Press.

Grosz, B. J. (1981). Focusing and description in natural language dialogs. In A. Joshi, B. Weber, & I. Sag, Elements of discourse understanding (pp. 229-346). Cambridge, UK: Cambridge University Press.

Grosz, B. J. & Sidner, C. L. (1986). Attention, intentions, and the structure of discourse. Computational Linguistics, 12(3), 175-203.

Glaser, B. G. & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Chicago: Aldine.

Ingwersen, P. (1992). Information Retrieval Interaction. London: Taylor Graham.

Kuhlthau, C. C. (1991). Inside the search process: Information seeking from the user's perspective. Journal of the American Society for Information Science, 42(5), 361-371.

Mizoguchi, R., Tijerino, Y. & Ikeda, M. (1995). Task analysis interview based on task ontology. Expert Systems with Applications, 9(1), 15-25.

Robins, D. (1997). Shifts of focus in information retrieval interaction. In C. Schwartz & M. Rorvig (Eds.), ASIS 97: Proceedings of the 60^th ASIS Annual Meeting (pp. 123-134). Medford, NJ: Information Today.

Saracevic, T. (1996). Modeling interaction in information retrieval (IR): A review and proposal. Proceedings of the 59th Annual Meeting of the American Society for Information Science, 33, 3-9.

Saracevic, T. (1997). The stratified model of information retrieval interaction: Extension and application. Proceedings of the 60th Annual Meeting of the American Society for Information Science, 34, 313-327.

Saracevic, T., Mokros, H. & Su, L. (1990). Nature of interaction between users and intermediaries in online searching: A qualitative analysis. Proceedings of the 53rd annual meeting of the American Society for Information Science, 27, 47-54.

Saracevic, T. & Su, L. (1989). Modeling and measuring user-intermediary-computer interaction online searching: Design of a study. Proceedings of the 52nd Annual Meeting of the American Society for Information Science, 26, 75-80.

Sinclair, J. M., & Coulthard, R. M. (1975). Toward an analysis of discourse: The English used by teachers and pupils. Oxford, UK: Oxford University Press.

Spink, A. (1997). Study of interactive feedback during mediated information retrieval. Journal of the American Society for Information Science, 48(5), 382-394.

Weick, K. E. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage.

Wersig, G. & Windel, G. (1985). Information science needs a theory of 'information actions.' Social Science Information Studies, 5, 11-23.

Return to Seminar home page / ASIS home page.