Posts filed under ‘Reference’
‘Predatory Reference’ an Interview with Bill Pardue about ‘Slam the Boards.’ Second Slam Coming Up on October 10, 2007!
Bill Pardue is the Virtual Services Librarian at the Arlington Heights (IL) Memorial Library. He worked previously at the Illinois Institute of Technology and received his MSLIS from the University of Illinois in 1992. Bill is also involved with the AskAway Illinois Advisory Committee and manages the website for the statewide VR service.
Bill initiated “Slam the Boards” by inviting librarians “to be bold and invade online answer sites such as Yahoo! Answers, Amazon’s Askville, and the Wikipedia Reference Desk” and to market libraries by “making it clear that this question was answered by a librarian/library professional/etc.”
Here’s my recent interview with Bill.
Marie: Bill, thanks so much for visiting Library Garden today. To get us started, tell me about “Slam the Boards” and especially how the idea occurred to you.
Bill: It was a very social process. I started playing around with Yahoo! Answers on my own and realized that it might be an opportunity for librarians to interact with users who don’t even realize that libraries have reference services. Paula Moore, our Coordinator for Public Services at Arlington Heights, commented that we ought to encourage lots of librarians to do the same. At the Collaborative Virtual Reference Symposium in Denver this past July, I mentioned it to Caleb Tucker-Raymond of the Multnomah County Library. He immediately said that instead of having some vague effort to get librarians more involved, a single day should be picked and promoted, in order to provide a real focal point. It was exactly the thought I needed to take action. Within a week I’d set up a Slam the Boards wiki and started putting the word out on listservs and anywhere else I could leave a comment. Then the viral part took over. In just over a month we had participants listed from the US, Europe, even New Zealand. It just seemed like the right idea at the right time…I just set up the wiki!
Marie: Caleb has such great ideas, I visited him on June 1st at the Oregon Virtual Reference Summit.
What were you hoping to achieve with “Slam the Boards”?
Bill: Mostly awareness on both sides of the question/answer transaction. Awareness among librarians that there’s a large potential patron base that we’re missing and need to promote to, as well as an arena in which we can showcase our excellence. On the asker/patron side, I’d just like a few answer board users to be pleasantly surprised that librarians don’t only provide people with books and videos, but also provide reference service. What I certainly didn’t hope to achieve was a cessation of people using answer boards. It just won’t happen, and people get some very good answers there. But I want librarians to realize that answer boards aren’t “the enemy.”
Marie: I know that one interest you have is in evaluation of the event, and, as a researcher, I’m especially interested in looking at reference quality issues, but would you deem it to have been a success? Why?
Bill: At this point, I’m gaging success in terms of engagement. Of course, it’s great to have a reply chosen as “best answer” now and then, too! The main point, though, is that we were out there, we saw what kinds of questions people ask and we hopefully provided useful, sourced answers. Some folks have started archiving answer board responses in a special QuestionPoint account that will allow for analysis by anyone who’d care to look at them. Currently it’s at about 75 questions (too many of them mine!), but I’m hoping that number increases. Quality’s an interesting issue. In a voting environment like Yahoo! Answers, I ended up feeling extra pressure to give a really good, sourced answer. It even stung a little when someone else’s off-the-cuff reply (which may have said the exact same thing) was voted best. I’d be interested to see how a more thorough study of quality on answer boards is conducted and what kind of results come out of it. You’ve got your work cut out for you!
Marie: So, did Slam the Boards achieve what you had imagined?
Bill: I think it did, partly because it had such a simple goal…get librarians involved, get them to think beyond their library confines and get engaged in some “predatory reference.” We’re still just a drop in the bucket in terms of the total traffic on a site like Yahoo! Answers, so I have no illusions about having a measurable impact on library reference numbers or VR service statistics.
Marie: I’ve heard you talk about “predatory reference” before, and like this radical concept! Would you mind defining it for us at Library Garden?
Bill: I’ll be the first to admit that it’s a somewhat over-dramatic coinage for a fairly straightforward concept. Librarians need to start actively finding reference questions, rather than just waiting for them to come in. Don’t limit your presence to just the reference desk or the library’s IM or VR service. Instead, find out where the where the questions are and start providing answers unsolicited. Being a “virtual services librarian” I tend to think first of online options: looking for points of fact in local discussion forums, blogs, etc. Out of such activity at Arlington Heights, we’ve even worked out partnerships with two local discussion board that take questions from the community. One is the “What’s the Fact” column of the Daily Herald’s Beep Central site. The other is the “Ask an Arlington Heights Librarian” forum.
There are less virtual ways, to do this, too. One local library (and I apologize that I can’t remember which) has been having reference librarians participate as judges for a local bar’s weekly trivia night. The Arlington Heights Memorial Library regularly sends our librarians out to community events (festivals, senior center events, etc.) with a wireless connection that allows us to provide many of the same services that we would at the reference desk. The point is to start being a little…dare I say…pushy about showing off our skills, so that potential users will realize that libraries equal more than just books! I’m sure we could think of other ways to get involved. Show up at village council meetings and if a tough topic comes up, volunteer the library’s reference service to help find some background. When you’re with a group of people, listen for points at which you can mention/promote reference services. If you overhear a local business person talking about doing mailing lists, let them know that the library has tools like ReferenceUSA that can be of use (and that someone on your staff is willing to demonstrate it to them). The opportunities are out there, we just have to be looking for them.
Marie: Do you have any idea about the number of librarians who participated and/or number of questions answered, even if it is a rough guess?
Bill: Ultimately, it’s a tough call. My intuition is in the hundreds of librarians, with maybe a thousand questions…but I have absolutely no way of knowing. Some of the more enthusiastic participants put their names on the wiki.
I counted 98 names there on 10/5/07. If you figure that 2-3 times that many actually participated, and the average “load” was 5 questions (I picked up 25 myself, and I know several others had matched that number), I’d say that 1000 questions isn’t unrealistic.
Marie: This question is from Beth Cackowski of QandANJ “Were the majority of questions answered by librarians, research questions? In other words, were they questions that library customers might expect a librarian to answer, or were they questions that the general public might be surprised to see answered by a librarian, for example: automotive, sports, pop culture, medical, legal?”
Bill: The unfortunate part is that most users don’t have any expectation of what kind of questions a librarian might answer (beyond “do you have a book on…”). To keep things mixed up for myself, I bounced around from category to category, picking up homework help questions, business, arts & humanities, cooking & recipes, geography, etc. I expect others did the same. If you check the list of participants above, you can see that many have added links to their Yahoo! Answer lists, so you can check out how they moved through the categories.
Marie: I definitely agree that many people don’t have a clue as to what types of questions a librarian could answer. Our abilities are usually underestimated.
Here’s a question from Julie Strange of Maryland AskUsNow! “Do you have a sense of how librarians went to find questions? Did they sort through the subjects and go for ones they specialized in? Or did they take new questions as they came in?”
Bill: Cherry-picking is essential on the boards because so many questions aren’t really informational. “What’s your favorite shampoo?” “I really like this girl, but I’m afraid to ask her out. What should I do?” etc. So, after a little digging around, you see that certain categories in any board have a higher ratio of informational vs. social questions and you start to “hang out” there. It’s kind of like “working the room” until you find someone you want to talk to at a party! As far as specializing in a subject, I think that’s very much up to the individual librarian. I consciously tried to be a generalist, but I also picked up a couple of questions in the Science/Astronomy category because that’s my hobby.
Marie: That’s really interesting, I like your “working the room” comparison. Have you gotten any feedback from librarians about their experiences?
Bill: Most of the feedback has been very positive. A lot of librarians were a little taken by the social nature of a lot of the questions, but ultimately were able to find at least a few to answer. Finding out if you received a “best answer” can take several days, so there were numerous messages from librarians when they got word of their “wins.” I got some negative feedback beforehand from some folks who couldn’t see the point of it, but nothing from anyone who actually participated. Of course, there could be all kinds of biases that account for this!
Marie: Did the librarians get much feedback from the users of these services?
Bill: The user feedback is pretty much determined by the mechanisms in place by the answer board. After one question was chosen “best answer,” there was a “nice answer” comment from the user. I’ve received a few like that now. If you give a particularly good answer, you can get “star” ratings, up to five stars. Of course, there’s the voting, too. It can be done by the asker or other readers. It’s nice to see your count of “thumbs up,” but you get a “thumbs down” every now and then. You’ve got to have a thick skin! An interesting anecdote is that I actually had a fairly extensive post-question correspondence with a user who had a tough corporate question. I actually ended up making several phone calls, just as I would have done for my own library’s patron.
Marie: Sounds like you could have parlayed that corporate interplay into some consulting business if you wanted to be more entrepreneurial ;)
Finally, I see that you are encouraging librarians to repeat “Slam the Boards” for October 10th, tell me about your vision to keep it going.
Bill: I’m really hoping this takes on a life of its own. The success of something like this is that it ultimately shouldn’t need a specific set of individuals to keep it going. I’d like to know that there’s a spike in answer board activity each month on the 10th, as well as a baseline through the rest of the month. I’d like to see discussion of this initiative on the existing listservs (it’s a bit too insular to just have its own listserv, I think) and informal discussion groups at conferences. I’d love to hear about a dine-around at Internet Librarian this year! Unfortunately, I can’t make it myself, but that’s all the more reason for others to do this. The best thing is that this is a way to promote library reference service that costs very little money and has the potential over the long run to enhance our image with a user base that almost never thinks about us.
Marie: Nicely put Bill. Thanks so much for your candid answers! Good luck with this month’s “Slam the Boards” on October 10th. I’ll be away at the Library Research Seminar IV in London, Ontario, from Oct. 9-11th, but will see if I can find a wifi hotspot and join in some predatory reference!
Gerry McKiernan of Iowa State University recently put a post on the dig_ref listserv (DIG_REF@LISTSERV.SYR.EDU) that dealt with a topic I have been talking about and thinking about for quite a while now, namely that:
NOW IS THE TIME TO PROMOTE PHONE REFERENCE SERVICES!!!
The ubiquitous nature of phones in everyone’s hands should make it a totally no brainer that we should definitely, absolutely, without question be actively marketing this underused service.
To quote Gerry: “Another Radical (but Conventional) Idea for OnCall, OnDemand, On Site Reference Service. Publicize the Library Hip Reference Help Phone Number… via Library Newsletter/Blog/Liaisons/Campus Groups/Table Tents/Billboards/TV Commercials/Facebook/etc. Why Chat When You Can Really Chat [:-)”
I agree with what most of Gerry is saying, although my research shows that people choose chat over phone reference for reasons of convenience (some love the transcript) and avoidance of awkward silences that occasionally happen on the phone (I’m not kidding about this, just ask any teenager).
His wake-up call to promote phone reference, however, is totally SPOT ON! All his ideas for marketing library phone reference are excellent. Table Tents are an especially good idea since many library users WILL NOT LEAVE or PACKUP their laptops (who can blame them?) in order to approach the reference desk if it is more than 12 feet away from where they are sitting! It would be nice if they saw the phone number on a table tent and could call for help instead of shrugging off their information need. Of course then we have to lift the ill-advised and impossible to enforce ban on cell phones in the library (and replace this policy with one that asks users to be respectful of others when using cell phones in public areas).
September is also the perfect time to do classroom marketing, what with students in Universities and schools undergoing bazillions (ok, thousands surely, maybe even tens of thousands?) library use instruction or orientation sessions!
Here’s my script for anyone doing one of these sessions:
“Greetings students! I am now about to ask you to do something that NONE of your other teachers/librarians have ever asked you to do…” (wait… for it….)
“TAKE OUT YOUR CELL PHONES AND TURN THEM ON” (amid gasps & nervous laughter from startled students, but they will do it eagerly!)
“Now, enter this library reference desk number into your phone BUDDY list…” (give ref desk number…)
“Next, here are the library hours when you can call this number for reference help” (now the students ACTUALLY have a reason to pay attention to the times when the library is open).
Here’s the clincher… “AND during the OTHER hours we are available by…” (chat, IM, e-mail, whatever you have!!)
Thus marketing chat, e-mail, etc. services along with the phone service. Wow, what an exciting old/new idea!
Wouldn’t it be great to see phone reference stats go through the roof? It’s high time to shake our fear of being overwhelmed at the ref desk.
Now is the time! Go for it!
We examined 600 chat transcripts randomly selected from QuestionPoint bank of almost 500,000 transcripts. Here are some of our findings (and an invitation below to the Seeking Synchronicity web site to see the PowerPoint slides and handouts from this presentation).
Do librarians clarify?
75% (in 434 of 581 usable transcripts) librarians did ask clarifying questions.
Did they ask the highly recommended follow-up question? (some version of “Does this completely answer your question?”)
50% (217) of the 434 librarians who clarified did ask the follow up question.
What types of questions were asked?
66% (554 of 838 questions asked by the librarians) were closed questions.
34% (282 of 838) were open.
What did librarians ask about?
Librarians asked users questions about: topic, background, search history, type of resource needed, extent/depth of information needed, if the user wanted a referral and more.
How about the virtual reference users?
Users offered information about: topic, background, extent/depth, and to correct the librarian’s misunderstanding.
Surprising finding! 2 different patterns of clarification!
Librarians clarified more often in the beginning of the interaction
Users clarified in the middle more often.
Most important finding! How to improve accuracy in chat reference?
For the 180 ready reference questions in our sample, we looked at accuracy (see my blog posting of July 10, 2007 for more on ready reference in chat).
Clarifying the query and asking the recommended follow-up question both boosted accuracy.
Always ask clarifying questions, even if you think you understand the question (one user asked for diving instructions, but had made a typo and wanted driving instructions, early clarification would have saved the librarian much searching time!)
Always ask a version of the recommended follow-up question: “Does this completely answer your question?”
Interested in more detail on the above findings? Please click on the above links to see the PowerPoint slides and handouts.
Zuula (http://www.zuula.com/) is a newish metasearch engine that I’ve been enjoying. Unlike most metasearch engines Zuula is not an aggregator. Rather, it displays results in a tabbed format which makes it easy to quickly click along and review results from different search engines. You can also limit your initial searches to these categories: Web, images, news, blog, and jobs. It’s a great interface and, like so many good things in life, Zuula is based right here in the Garden State.
I recently had the opportunity to interview Boris Simkovich, Chief Executive Officer of Zuula LLC.
- Question: Tell us a little bit about yourself. How did you get into the search engine business?
Answer: Well, you probably won’t be too surprised to hear that I didn’t grow up wanting to be in the search engine business!
In college, I studied engineering and economics, and I continued with my study of economics in graduate school, where I got a Ph.D. After that, I spent a couple of years in academia, teaching and doing research in economics, then switched over to the world of management consulting, where I worked for almost ten years.
I finally got involved with the search engine business when the consulting business I was running hired an old graduate school friend of mine, Tim Hunt. Tim also had been working in management consulting, but he had a long-standing hankering to start a software or Internet business. He couldn’t pursue this interest, however, at a traditional consulting firm, so he agreed to come to my firm when I said we would give him the opportunity to pursue a start-up idea while he also was doing consulting work.
So Tim joined the consulting business three years ago, and soon thereafter, we started considering a variety of different new business ideas that might match his interests (and, frankly, my own interests). About two years ago, we settled on the idea that eventually became Zuula.
- Question: Why did you name it Zuula?
Answer: That’s a question that gets asked a lot more than any of us at Zuula expected.
The answer to the question surprises a lot of people. We started off considering name ideas that were very, very different from Zuula. However, as we looked at different options, and also took into account what domain names (Internet addresses) were actually available, the search got more and more difficult. Finally, after playing around with different names that were short, easy to remember, and had available Internet addresses, we settled on the name Zuula.
So, in many ways, Zuula is a made-up name. When we selected it, we didn’t think it had any particular meaning. However, we’ve since found out that the word actually means “to take off” in a central African language, and we hope the meaning is a prediction of sorts for what will happen with Zuula.
- Question: Why did you start Zuula [as opposed to some other Internet-related business]?
Answer: We decided to pursue the ideas that eventually became Zuula for several reasons.
First, we liked the fact that Internet search is such a large business, and almost everyone who uses the Internet uses a search engine from time to time. This meant that it wouldn’t be too hard to find potential users for our new service, and that the service most likely could thrive even if it attracted only a small share of its overall market.
Second, we felt that the features we intended to offer through Zuula would, in fact, appeal to many people who use search engines. Working in consulting, we used search engines all the time. And, based on our own needs, we felt there would be a lot of interest in a search tool like Zuula.
- Question: How long did it take to get Zuula off the ground? What’s involved in launching a new search engine?
Answer: It took a LONG time to get Zuula up and running … a lot longer than we expected.
The basic idea for Zuula came into being in the spring of 2005. It wasn’t until that summer, however, that we decided on the features and functionality you now see in Zuula. From then on, it took almost one and a half years for our programming team to develop and finally launch Zuula.
As you know, Zuula is a meta search engine, which means that it doesn’t have a search index of its own. Instead, Zuula presents search results from a variety of other search engines – Google, Yahoo, MSN, etc. – for a number of different search types. This means that one of the key tasks during the development of Zuula was creating the routines that process the results from other search engines and display them at Zuula.
In addition, we had to design Zuula’s user interface and develop the necessary code for it, and we had to create all the help information and other content necessary for a professional site.
Finally, we had to optimize all of Zuula’s code so that it runs as fast as possible, and we had to establish a server network that was fast, reliable, secure, and cost-effective. In retrospect, I don’t think any of these tasks stand out as having taken particularly longer than we expected. Instead, what we learned is that it doesn’t take very long to complete the core aspects of any given task. It does take long, however, to complete each and every necessary detail of a task … and to get everything to work together successfully.
- Question: Why should I use Zuula instead of Google or other search engines?
Answer: Actually, the beauty of Zuula is that you can leverage its capabilities without having to give up Google (or whatever other major search engine you’re accustomed to using).
Let me explain.
I mentioned earlier that Zuula is a meta search engine, meaning it displays results from a number of other search engines. Unlike other meta search engines, however, Zuula does not aggregate the results from other engines into a single list.
Instead, the results are organized under separate tabs. Results from Google, for example, can be viewed by clicking on the Google tab. Likewise, results from Yahoo are viewable by clicking on the Yahoo tab. There are tabs for essentially all the major search engines, and five different types of search are possible: web, image, news, blog, and jobs.
Moreover, for each type of search, the user can customize the order of the tabs to match his or her preferences. If a user prefers to have Yahoo as her default web search engine, all she has to do is drag the Yahoo tab so it is the first web search tab.
In this way, Zuula’s users can do most of their searching using the search engines they’re already accustomed to. With Zuula, however, users also get quick access to results from other major search engines for those occasions when their default search engines are not enough. There is no need to re-enter search terms, and all search results at Zuula are presented in a consistent, easy-to-read format that clearly distinguishes between organic results and sponsored results (advertising). There’s even a collapsible list of recent searches which can be helpful for difficult searches.
- Question: What developments are planned? What do we have to look forward to?
Answer: We are constantly working on improving Zuula.
For example, we soon will be adding more search engines, and additional search types also should become available in the next several months. Longer term, users should expect to see more customization options and better international support.
And I shouldn’t forget to mention a major upgrade which we’ll be rolling out in the next few weeks. It will involve a unique set of features – based on some innovative new technology – that will make Zuula an even more powerful search tool. Stay tuned!
- Question: What does Zuula offer that might be of particular interest to the readers of Library Garden?
Answer: This may surprise many of your readers, but we’ve always believed that library professionals would be some of Zuula’s first fans. We thought that librarians – as professional researchers — would be particularly attracted to Zuula’s ability to streamline difficult Internet searches.
We also thought that librarians would see Zuula as a way to be more neutral when recommending Internet search tools to patrons. By introducing Zuula in their research guides, handouts, and links on public access computers, librarians can leave it up to their patrons to decide which of the major search engines they want to use for their Internet searches.
Thus, your readers may want to consider adding Zuula to the search boxes in the IE7 or Firefox 2.0 browsers on their public access computers. (There is a link on Zuula’s home page to add Zuula to the browser’s search box.)
Also, we’ll soon be posting html code on the Zuula website which will make it easy to add a Zuula search box to any web page. This, too, may be useful for any public access computers your readers may be responsible for. (There probably will be an announcement at the Zuula blog – http://www.zuulablog.com/ – when the search box code is posted.)
[3/2 Update: The code is ready to go at: http://www.zuula.com/help/ZuulaAddin.html]
- Question: What is your vision for Zuula?
Answer: To be honest, we’re cautious about having a broad, overarching vision for Zuula. When it comes to the Internet, what’s “visionary” today can quickly become ill-informed and misguided tomorrow.
Thus, we try to focus on things closer to the here and now. For Zuula, that means continuing to expand the power and versatility of its search capabilities. As simple minded as it may seem, we want Zuula to be one of the best ways to find information on the Internet.
- Question: I would love to be able to run a search and then subscribe to an RSS feed for that search that brings me updated results in real time (a’la technorati, delicious, flickr, etc.) Any plans to add that kind of RSS component?
Answer: Sorry, but we’re not currently planning any RSS functionality like you’ve described. However, we very much welcome suggestions, comments, and criticisms from our users, and we’ll consider adding this feature in the future.
- Question: So what’s the business model here? It doesn’t look like you’re selling advertising—how do plan to stay in business?
Answer: You’re absolutely correct – we’re not selling advertising. And I suspect it will be quite some time before we start selling advertising directly on our own.
Currently, the advertising we display at Zuula is the same advertising that users would see if they carried out their searches directly at the search engines whose results we display. Zuula receives no money whatsoever from the advertisements. Any revenue created by the advertisements is 100 percent retained by the search engines whose results are displayed alongside the advertisements.
Obviously, this approach is not sustainable in the long run. Instead, we expect sometime later this year to begin implementing arrangements to allow us to retain some or all of the revenue generated by the advertising at Zuula. This won’t happen overnight, but it’s certainly the long-term direction of the site.
- Question: I love the idea of the Zuula blog, but the posting has been pretty sparse. What do you see as the blog’s purpose and do you have plans to beef up the postings?
Answer: It’s unfortunate but true that the Zuula blog has taken a back seat to other work we’ve been doing since Zuula’s launch late last year.
However, you should start to see more regular postings at the site in the coming month or two. For now, our intention is for the blog to be a way to “highlight and announce.” “Highlight,” in the sense of explaining to our users features that Zuula has, but which they may not have noticed. “Announce,” in the sense of publicizing new features and functionality that we’ve developed for Zuula.
That’s not to say that our plans for the blog’s content are set in stone. Indeed, we’d be happy to hear from your readers what sort of content they’d like to see at the blog.
Get thee over to the Wall Street Journal and read this gloves-off (you know, in a genteel way) debate between Jimmy Wales, founder of Wikipedia, and Dale Hoiberg, editor-in-chief of Britannica. Here’s a taste:
Mr. Hoiberg: No, we don’t publish rough drafts. We want our articles to be correct before they are published. We stand behind our process, based on trained editors and fact-checkers, more than 4,000 experts, and sound writing. Our model works well. Wikipedia is very different, but nothing in their model suggests we should change what we do.
Mr. Wales: Fitting words for an epitaph… …We are open and transparent and eager to help people find criticisms of us. Disconcerting and unusual, I know. But, well, welcome to the Internet.
Personally, it took me a while to get to the point where I feel a fair level of trust in the quality of Wikipedia. I think Wales has done an excellent job of creating a system that maximizes the benefits of open source collaboration, while minimizing the drawback and dangers of having too much openness. I’m reminded of the brilliant article Clay Shirkey wrote a few years ago, “A Group is it’s own worst enemy“. Shirkey, building off of the concepts expressed by psychologist W.R. Bion in his seminal work,”Experiences in Groups“, wrote,
Group structure is necessary to defend the group from itself. Group structure exists to keep a group on target, on track, on message, on charter, whatever. To keep a group focused on its own sophisticated goals and to keep a group from sliding into these basic [destructive] patterns. Group structure defends the group from the action of its own members. (emphasis is mine, pjb)
I remember being struck by Bion’s work when I first read him in a college psych class, but Shirkey really brings it home. Although Shirkey is mostly focusing on social software, the concepts expressed in “Own Worst Enemy” are applicable well beyond that topic, and you might find yourself reflecting on the structure and health of your library (or your Bridge club, or your — um, make that OUR — government). Geek confession: I keep a copy of Shirkey’s article in a “Ponderables” binder on my night table and re-read it regularly.
But I digress. Point is, Wales has done a great job of keeping Wikipedia from being it’s own worst enemy, and I’ve seriously warmed up to Wikipedia as a trusted source.