In an earlier blogpost, I described a call asking for scholars interested in being part of a focus group in the development of an Oral History (OH) search interface. This call was also sent by email to 113 scholars in our network, after which fifteen people responded to our e-mail (13.2% response-rate, not bad), and one more scholar responded after being tipped by another scholar. In the past two weeks, I’ve interviewed these fifteen scholars via Skype and phone.
The interviews consisted of two parts; first, I wanted to get a better idea of how all these scholars used OH in their own research, and what their perspective on OH was. Second, a basic version of the Oral History Today (OHT) search interface was walked through to get initial feedback and additional user wishes. In this blogpost, I’d like to introduce some interesting preliminary findings from both sections of the interview.
Background of focus group
We have a fairly varied collecion of perspectives in the focus group: four respondents were PhD-students, three senior researchers, two museum-curators, one professor, one assistant professor, one information professional and three respondents with non-research positions.
Regarding their perspectives on OH, we found that:
- Most participants were acquanted with and used OH as a method ( i.e., interviewing as a form of historic research)
- About half had created OH datasets that had been published
- Only five had used other OH collections than their own, of which only 2 had used it as an actual source of information
In short, it appeared that using and publishing OH collections as a dataset for research is not common practice.
With a general idea of the focus group, I performed walk-throughs with each participant in v1 of the OHT search interface. Each participant started on the landing page (see the first image), after which they were asked what their first action would be and how they would then interact with the search interface.
By following participant’s actions, we made several (preliminary) observations:
- Half the respondents employed exploratory search behaviour, while the other half employed directed keyword search behaviour (and the search interface supported either sufficiently).
- Search results were not viewed as a list of possible hits ranked by relevance (of which a portion is probably irrelevant), but as a complete set of which each item had to be viewed.
- The main task after a query is then to manually assess which search results are relevant or important, and which not.
- When opening a search result, the main concern was figuring out what the interview was about, and what the context was in which the interview took place and was published.
After these interviews, we now have 132 user statements. Of these 132, 68 (52%!) were unique, meaning only a single respondent mentioned these, another 29 were mentioned by only two respondents. The main question then is how to manage these user statements and what we can learn from them. The main (preliminary) observations we made were related to how the focus group participants interacted with this search interface, how they tried to assess relevance and what questions they asked using the interface.
How to enable or improve such tasks is the research we will conduct with the Oral History Today project.