Jennifer A.

Fielding

Rethinking CRAAP
Getting students thinking like fact-checkers in evaluating web sources

F or over two decades, librarians have been

at the forefront of helping their patrons and
students discern what online information is reli-
wider scope now has the potential to facilitate
the acceptance of misinformation as fact. Once
a site is deemed “credible,” all information on
able, and what may be biased or outright false. it is frequently trusted and taken at face value.
Particularly as more formal information literacy This is clearly problematic, since studies show
programs developed at the college and university that once information has been accepted as valid
level (and the attendant inclusion of information and assimilated by the user, it becomes far more
literacy in many general education programs), difficult to counter, even with accurate facts.2,3,4
academic librarians have developed curricula and In addition, considering the disciplinary shift
taught students how to evaluate web sources for away from static Information Competency Stan-
credibility. In many institutions, this has frequently dards for Higher Education to the more organic
been achieved via a “one-shot” session with a and contextual approach of the Framework for
checklist of sorts, often some variation of the Information Literacy for Higher Education, it
CRAAP Method (Currency, Reliability, Authority, would seem apparent that the pedagogy sur-
Accuracy, and Purpose) developed nearly 15 rounding the critical assessment of web sources
years ago at California State University-Chico.1 must also evolve.
The CRAAP method encourages the user to
perform an in-depth analysis of the website to The Stanford Study
determine its credibility, often by finding and A 2017 Stanford working paper by Sam Wine-
analyzing the “About Us” section, and thoroughly burg and Sara McGrew highlights this evolution
exploring the site to determine if there are named in stark relief, assessing the critical evaluation
authors, what their credentials may be, and the skills for web content between students, facul-
stated purpose or mission of the publishing orga- ty, and professional fact-checkers. They found
nization. Additional CRAAP assessments include that faculty (arguably information-savvy critical
how recently the site has been updated, whether thinkers) performed barely better than under-
page links are working and lead to other reliable graduates in assessing the credibility of web
information, and whether the site is a commercial, content, primarily because they used the deep-
nonprofit, or educational site. Students focus dive type of assessment endorsed by methods
on the site itself, performing a “deep dive” into like CRAAP—thoroughly examining the site
what they find at a particular URL. As currently itself.
employed, the CRAAP method does not explicitly Fact-checkers, on the other hand, almost
encourage leaving the site to place any content immediately began an independent verification
found there in a wider context.
However, in recent years, the dissemination
of mis- and disinformation online has become Jennifer A. Fielding is coordinator of library services at
increasingly sophisticated and prolific, so re- Northern Essex Community College-Lawrence Campus,
email: jfielding@necc.mass.edu
stricting analysis to a single website’s content
without understanding how that site relates to a © 2019 Jennifer A. Fielding

C&RL News December 2019 620

process, a strategy the researchers dubbed “lateral that GSK is the pharmaceutical company Glaxo-
reading”—opening multiple tabs, and searching SmithKline, while the publisher of Medline is the
for independent information on the publishing National Library of Medicine.
organization, funding sources, and other factors Students then skimmed information on both
that might indicate the reliability and perspective of these organizations on sites like Wikipedia and
of the site and its authors or sponsors. news sites, and were asked for their assessment
This lateral reading approach produced on which would deliver more trustworthy infor-
significantly better results for the fact-checkers— mation regarding asthma diagnosis, prognosis,
both in critical assessment and in the speed of and treatments. Frequently at this point, students
their conclusions—than the “vertical reading” would spontaneously start discussing the informa-
deep dive did for both the student group and tion they found regarding GSK’s legal battles, and
the faculty group.5 the ethical implications of drug companies giving
health advice, which then often led to animated
Informal trials with lateral searching discussions regarding the responsibilities of infor-
In an effort to trial and potentially incorporate mation creators.
this new research finding in teaching methods, The simple shift to a lateral reading method not
a group of community college librarians revised only visibly engaged students more thoroughly in
their basic one-shot content to include a “lateral the process, but also directly applied several of
reading” assessment with their first-year stu- the ACRL Framework for Information Literacy for
dents. Higher Education principles: Searching as Strategic
Northern Essex Community College (NECC) Exploration, Information Has Value, and Authority
is a two-year associates degree- granting institu- is Constructed and Contextual.
tion 30 miles north of Boston, Massachusetts. In-class discussions around these concepts
Almost all students in every program take an were often robust and student-driven, and fre-
English 101 and/or English 102 course, during quently led to additional explorations of various
which research and information literacy skills are websites. One student remarked they saw the
frequently addressed in both the content and/ activity like “detective work” and enjoyed that
or with a “one-shot” library visit. In addition, as aspect of it. While this feedback is certainly an-
part of the college’s Core Academic Skills require- ecdotal, it demonstrates the difference in framing
ments, all students graduating with an associate’s that helped engage students more actively in the
degree must also complete a course designated evaluation process.
as “intensive” in information literacy within their Future classroom exercises are anticipated to
discipline. focus on specific outcomes and behaviors articu-
In the Fall 2018 semester, several NECC lated in the Framework, particularly exhibiting
librarians adapted the portion of some one-shot “mental flexibility and creativity” and developing
sessions where the CRAAP method would have an “awareness of the importance of assessing
been used to assess the credibility of a website. content with a skeptical stance and with a self-
In one example of this alternative “lateral read- awareness of their own biases and worldview.”6
ing” activity, students analyzed two results from Since these early, unstructured efforts have
a Google search on “asthma” and compared sites been so promising, discussions by the librarians
for two of the top ten results: asthma.com versus around next steps in applying these pedagogical
medlineplus.gov/asthma.html. changes have been focused on adapting the ap-
While the instructor did note the difference proach to differing disciplinary perspectives across
in the domain names as a likely indicator of the the curriculum (i.e., incorporating student activities
purpose of each site (part of the CRAAP method), evaluating sites specific to history, psychology,
students were then encouraged to open new tabs criminal justice, etc.), and development of an
and search for information on both GSK (which assessment method to verify lateral searching’s
students had identified as the publisher of asthma. efficacy versus traditional methods for NECC’s
com) and Medline. Students quickly discovered students. Should the shift be incorporated into

December 2019 621 C&RL News

ongoing teaching, communication with faculty will curate their information to suit their interests and
also be important, as (ironically due to the diligent propagate information nearly instantaneously,
efforts of librarians) many faculty use CRAAP as evolving information literacy instruction across
the website assessment “standard” in their courses the curriculum has broad implications not just
(librarians frequently receive requests from faculty for the research process, but for issues such as
to “teach the session on CRAAP”). citizenship, democracy, and social responsibility.
It is important to note here, however, that
the Stanford study attributed the success of the Notes
fact-checkers to both the lateral reading strategy 1. Sarah Blakeslee, “The CRAAP Test,” LOEX
described above, but also their “robust knowledge Quarterly 31, no. 3 (2004).
of sources to inform their decisions,”7 for instance, 2. John Cook and Stephan Lewandowsky,
understanding that a nonprofit site does not neces- The Debunking Handbook (St. Lucia, Australia:
sarily connote altruism, and that purported news University of Queensland, 2011), http://sks.to/
sites can lean heavily left or right in their reporting. debunk.
“Fact checkers also possessed knowledge of on- 3. Elizabeth Kolbert, “Why Facts Don’t
line structures, particularly how search results are Change Our Minds,” The New Yorker (February
organized and presented. They knew that the first 27, 2017).
result was not necessarily the most authoritative.”8 4. Alice Marwick and Rebecca Lewis, “Me-
These findings indicate that any strategies dia Manipulation and Disinformation On-
taught regarding information evaluation must also line,” Data & Society (May 15, 2017), https://
be paired with content on search engine ranking, datasociety.net/output/media-manipulation
personalization, and Eli Pariser’s now well-known -and-disinfo-online/.
filter bubble effect.9 5. Sam Wineburg and Sarah McGrew, “Lateral
Reading: Reading Less and Learning More When
Conclusions Evaluating Digital Information,” Stanford History
It is widely acknowledged that the current in- Education Group-Working Paper (September
formation landscape places an increasing bur- 2017), https://papers.ssrn.com/sol3/papers.
den on the information consumer. The lack of cfm?abstract_id=3048994.
editorial control in a web environment, coupled 6. ACRL “Framework for Information Literacy
with personalized search engine results and filter for Higher Education,” 2015, www.ala.org/acrl/
bubbles of mis/disformation on social media and standards/ilframework.
other platforms10,11,12 makes obvious the need for 7. Wineburg and McGrew, “Lateral Reading.”
librarians to evolve our pedagogy to teach and 8. Ibid.
encourage lateral, fact-checking behaviors and 9. Eli Pariser, The Filter Bubble: What the In-
dispositions. ternet is Hiding From You (New York: Penguin
As such, while very useful for many early mani- Press, 2011).
festations of web content, I would argue that the 10. Ibid.
CRAAP “deep-dive” examination of a specific web 11. Amy Mitchell, Elisa Shearer, Jeffrey
source is no longer wholly adequate in light of the Gottfried, and Michael Barthel, “Pathways to
increasing sophistication of the web, nonexistent News,” Pew Research Center, July 7, 2016,
barriers to content creation, and the muddling ef- www.journalism.org/2016/07/07/pathways
fect of social media on information consumption -to-news/.
and sharing. While it is clear that each of CRAAP’s 12. Chris Meserole, “How Misinforma-
individual assessments have ongoing value, it tion Spreads on Social Media—And What
has become vitally important to place informa- To Do About It,” The Brookings Institution
tion into a wider context to adequately evaluate (May 9, 2018), https://www.brookings.edu
its credibility, as well as teach how information /blog/order-from-chaos/2018/05/09/how
is ranked and presented on search engines and -misinformation-spreads-on-social-media
social media. As consumers are able to both -and-what-to-do-about-it/.

C&RL News December 2019 622