Big Point

Economics and policies drive implementation decisions

T-logs in research and practice

Questions

If people saw web traffic visualizations, would they take different routes? (related to shared views)

Are there studies about customizable portals (ala my library?)?

Are there other task or user taxonomies?

Legal/ethical issues of data collection?

2. Update on Relation Browser projects

Demo Toad interface to Oracle tables, show resulting interface effects (laptop)

3. Readings & theme of user studies

notes from last week (fisheye view concept)

We are moving from concepts and techniques for information seeking interfaces to design and assessment of user studies.

Christel et al paper illustrates two user studies…first suggesting changes in skims and second demonstrating importance of integration of audio and video information. Note the design decisions made:

Research goals (what to emphasize?)

Treatments (various skims…note the choices in each as well as among each)

Tasks (factfinding and gisting in first, gisting in second)—note how these are operationalized…consider the many variations. The actual instruments/stimuli are crucial and there are many decisions to be made that affect results

Instruments (in addition to the stimuli for tasks, questionnaires or interview schedules, protocols for running the subjects, data gathering tools (e.g., website administration?)

Dependent variables (what will actually be measured—accuracy, speed, satisfaction in this case---how will these be captured?)

Subjects (who to include, how to motivate, how/whether to categorize)

Procedures (how to run subjects, e.g, individually vs groups; setting, e.g., workstation, room; assignment and ordering of subjects, assignment and ordering of tasks; what to do under special conditions (NEED TO PILOT!), whether to provide training (and if so, what the effects/biases will be)

Analyses (what statistical or interpretation techniques to use, e.g., if audio/videotaped, will you do verbatim transcripts?)

Interpretation (making sense out of the results, relating to the interfaces, linking to literature/explaining why, etc.)

Writing results (how to organize, what to include, where to send)

Logistics: IRB process, recruiting subjects, running subjects, consent forms, payments, etc.

Koenemann & Belkin paper provides another example where scores of design decisions are made to address the question of user control and relevance feedback in particular

Note the task was not finding documents but constructing a good SDI (routing) query for a topic. Note use of non-LIS subjects (avoiding a common criticism)

Note strong evidence for use of interaction with corresponding performance payoff

Marchionini & Mu paper summarizes four sets of studies using two very different data collection techniques (usability study and eye tracking study).

See IRB procedures: http://research.unc.edu/ors/ethics.html

See Open Video project package same forms http://ils.unc.edu/~march/courses/357_f01/OV_IRB

4. Universal access reading and guidelines (defer until next week)

4. Case #2 Library of Congress Digital Library (American Memory)

user needs assessmentà task/user taxonomy

Design meetings (brainstorming, high-concept, developing trust, prototypes)

Developing tools

Revising and integrating tools

See http://www.cs.umd.edu/hcil/ndl/ for details and examples

5. Interface tours: Since a small group, each person pick a website related to your area of interest, lead the class through the site, pointing out features and techniques that are interesting or troublesome (10-15 minutes).

6. Readings for next meeting:
Learning from eye movements: read Jacob (ACM DL)
Biometrics: read Pankanti, Bolle, & Jain (handout, online for IEEE members)

7. One-minute paper

What was the big point you learned in class today?

What is the main, unanswered question you leave class with today?