EDUCATIONAL

RESOURCE
Volume 12 Issue 1 2020

DOI: 10.21315/eimj2020.12.1.8 Seven Steps to Construct an Assessment

ARTICLE INFO Blueprint: A Practical Guide
Submitted: 26-12-2019
Accepted: 31-01-2020 Muhd Al-Aarifin Ismail, Mohamad Najib Mat Pa, Jamilah Al-
Online: 10-04-2020 Muhammady Mohammad, Muhamad Saiful Bahri Yusoff

Department of Medical Education, School of Medical Sciences,

Universiti Sains Malaysia, Kelantan, MALAYSIA

To cite this article: Ismail MA-A, Mat Pa MN, Mohammad JAM, Yusoff MSB. Seven steps to
construct an assessment blueprint: a practical guide. Education in Medicine Journal. 2020;12(1): 71–
80. https://doi.org/10.21315/eimj2020.12.1.8

To link to this article: https://doi.org/10.21315/eimj2020.12.1.8

ABSTRACT
Blueprint is a map or specification of assessment items based on educational outcomes and its primary
function is to support the validity of assessment with regard to its content – content validity. It helps
to align assessment items with the intended learning outcomes and students learning experience. The
primary focus of this study is to describe seven practical steps in the construction of an assessment
blueprint. The steps are somehow flexible, as we try to elaborate the possibilities. This study covers
several issues that might arise during blueprinting construction and suggestions on how to tackle the
issues. A well-constructed blueprint is essential and important to ensure the validity of any assessment
content is aligned with the intended learning outcomes and learning experience.

Keywords: Blueprinting, Assessment blueprint, Student assessment, Validity, Constructive alignment

Corresponding author Muhd Al-Aarifin Ismail, Department of Medical Education, School of Medical Sciences,
Universiti Sains Malaysia, 16150 Kubang Kerian, Kelantan, Malaysia | Email: alaarifin@
usm.my

INTRODUCTION Validity of Assessment

Blueprint is defined as a complete plan that Blueprinting acts as a valid tool to align
explains how to do or develop something objectives with assessment, helps in
(1). In assessment, the term refers to a map distribution of appropriate weightage and
or specification of assessment to ensure that questions across the topics. It should be
all aspects of the curriculum and educational an integral part of assessment (5) and
domains are covered by the assessment the process of blueprinting will act as a
programmes over a specified period of source of evidence to support its content
time (2). It helps curriculum developers validity (6). As far as threat of validity
to match various competencies with the is concerned, educators must seriously
course content and the appropriate modality consider any element that can reduce the
of assessment (3). Assessment blueprint validity of assessments. As described by
is an essential step for enhancing validity Messick (7), there are two major sources of
of assessment and constructive alignment validity threats which are construct under
particularly for high stakes examinations (4). representation and construct-irrelevant
variance (8). Construct under representation

 Malaysian Association of Education in Medicine and Health Sciences and 71

Penerbit Universiti Sains Malaysia. 2020
This work is licensed under the terms of the Creative Commons Attribution (CC BY)
(http://creativecommons.org/licenses/by/4.0/).
 Education in Medicine Journal 2020; 12(1): 71–80

refers to the under sampling or biased functions such as guiding the selection of
sampling of the curriculum or course content learning experiences (9).
for assessment. It occurs when there are too
few items covering a curricular content. Constructive Alignment
Construct-irrelevant variance on the other
hand occurs because of flawed item formats, The three pillars of education consist of
items that are too easy or too hard, or the intended learning outcomes, teaching and
choice of inappropriate test modalities. In learning activities, and assessment tasks
general, the aim of the blueprinting is to (Figure 1) – it is referred as the constructive
reduce these two major threats to validity. It alignment (10). The congruence between
helps institutions to identify test instruments these three pillars of education can be
appropriate to the constructs and contents of facilitated by a blueprint (11, 12). It will
the assessment. Although its primary function ensure the assessment tasks are aligned with
is to validate evaluation content, a well- the intended learning outcomes and teaching
constructed blueprint can also serve other and learning activities.

Figure 1: Constructive alignment.

BENEFITS OF ASSESSMENT thinking skills and assessment of in-depth

knowledge.
BLUEPRINT
f. Sample content, competencies and tools
Blueprint helps to match various for the assessment in a rational and
competencies with the course content and balanced manner.
the appropriate modality of assessment (3).
Recent study by Patil et al. (5) found that
blueprinting helps to: BLUEPRINTING PROCESS AT SCHOOL
a. Ensure questions being asked in OF MEDICAL SCIENCES, UNIVERSITI
examination are aligned to objectives. SAINS MALAYSIA (usm)
b. Ensure that there are no questions that
are out of syllabus. The school adopts SPICES curriculum (13)
that has two phases – Phase I (pre-clinical)
c. Make assessment fair to the students. consists of Year 1 and Year 2, and Phase II
d. Design the instructional strategies. (clinical) consists of Year 3 until Year 5.
This study described seven practical steps
e. Ensure that the selected test items
for creating a blueprint based on School of
provide appropriate emphasis on
Medical Sciences practice. Department of

72 https://eduimed.usm.my
 EDUCATIONAL RESOURCE | Practical Guide to Construct an Assessment Blueprint

Medical Education facilitates the blueprinting blueprinting construction. In our example,

process through series of workshops and prior the blueprint for this course is prepared for
to the workshops, a template of blueprint is GMT 104 (Microbiology, Immunology
prepared, and it consists of a series of rows and Pathology course). It is prepared
and columns using Microsoft Excel software for Semester 1, Professional I and
(Table 1). The flow of the blueprinting Supplementary examinations. It is prepared
workshops is described in Figure 2. for 2017/2018 academic session. The
assessment tools involved are multiple true
Blueprinting Construction false (MTF), short essay question (SEQ)
and objective structured clinical examination
Step 1: Define the blueprinting purpose and scope (OSCE). The total numbers of questions
needed are 14 for MTF, four questions
The first step in any blueprinting for SEQ and three questions for OSCE.
construction is to identify its purpose and For Professional I and Supplementary
scope. The following questions should be Professional I Examination, six questions of
addressed during this initial step: MTF and one SEQ question for each set. No
OSCE question is required for this course
a. Which semester or phase of study the (Table 1).
blueprint is prepared for?
b. How many academic session the The other important question that needs to
blueprint is prepared for? be addressed is about the institution standard
statements for the students in that phase
c. Which courses are involved? to be achieved. The standard statement is:
d. What is the assessment tools involved? “The graduate of this medical programme
e. How many questions for each assessment should demonstrate adequate knowledge
tool? for safe clinical decision and management,
be able to work with supervision, equipped
with standard clinical skills, and conduct
These are some of the crucial questions that themselves professionally”.
should be answered during the first stage of

Figure 2: The flow of the blueprinting workshop at School of Medical Sciences, USM.

https://eduimed.usm.my 73
74
Table 1: Blueprint for GMT 104 (Microbiology, Immunology and Pathology) course

Column: 1 2 3 4 5 6 7 8 9

SEMESTER I PRO. I SUPP. PRO. I Name of

Course Mode Impact Frequency I×F
MTF SEQ OSCE MTF SEQ MTF SEQ lecturer

https://eduimed.usm.my
MICROBIOLOGY
Classification of medically important L 3 3 9 – – – – – – – A
bacteria
Classification and development of L 2 2 4 1 – – – – – – C
Education in Medicine Journal 2020; 12(1): 71–80

medically important fungi

Classification and replication of L 3 2 6 1 – – – – – – F
viruses
Microbial genetics and principles of L 2 2 4 – 1 – – – 1 – C
antimicrobial resistance mechanisms
Transport media, growth and P 2 2 4 1 1 – – – – B
microbial culture
Classification of medically important L 3 2 6 – 1 – 1 – – – K
parasites
Microscopic examination of bacteria L 2 3 6 – – – 1 – 1 – Z
and fungi
Host-pathogen interactions PBL 2 2 4 – – – – – – – A
Basic medical microbiology P 3 3 9 1 – – – – – – C
IMMUNOLOGY
Cells and organs of the immune L 3 2 6 1 – – – – 1 – D
system
T & B cell development L 1 2 2 – – – – – – – D
Complement and cytokines L 3 2 6 1 – – – – 1 – F
Immune response and regulation L 2 2 4 1 – – – – – – G
Principle of antigen and antibody PBL 3 3 9 – – 1 1 – – – E
interactions
(continue on next page)
 Table 1: (continued)

Column: 1 2 3 4 5 6 7 8 9

SEMESTER I PRO. I SUPP. PRO. I Name of

Course Mode Impact Frequency I×F
MTF SEQ OSCE MTF SEQ MTF SEQ lecturer
Principles of immunisation L 3 3 9 1 – – – 1 – – W
Principles of tolerance and L 1 2 2 – 1 – – – – – H
autoimmunity
Introduction to clinical immunology L 3 3 9 1 – – 1 – – – J
Immunological tests P 3 3 9 – – – – – – – F
PATHOLOGY
Acute inflammation PBL 3 3 9 1 – – 1 – – – D
Chronic inflammation L 2 3 6 – – – – – – F
Tissue healing and repair L 2 3 6 1 – – – – – – Q
Molecular basis of neoplasia: L 2 1 2 – 1 – – – 1 – A
oncogene and tumour suppressor
genes
Clinical aspects of tumours and host L 1 3 3 1 – – – – – – C
defense against tumours
Oedema and congestion L 3 3 9 – – – – – – 1 P
Thromboembolic disorder L 1 3 3 1 1 1 F
Neoplasia GSL 3 3 9 1 1 A
Total I x F 155
Total questions 14 4 3 6 1 6 1
Notes: I × F : Impact × Frequency; L : Lecture; PBL : Problem-based learning; GSL : Guided self-learning; PRO. : Professional examination; SUPP. PRO. : Supplementary professional
examination; MTF : Multiple true false; SEQ : Short essay question; OSCE : Objective structured clinical examination; P : Practical
Due to confidentiality issue, the data shown above has been edited and it does not represent the real blueprint for GMT 104.

https://eduimed.usm.my
EDUCATIONAL RESOURCE | Practical Guide to Construct an Assessment Blueprint

75
 Education in Medicine Journal 2020; 12(1): 71–80

Step 2: Tabulate curricular content (Column 1) (14). However, the content importance level
is not easy to define (14).
The curricular content can be listed in
many ways according to curricular setting. At School of Medical Sciences, the impact
It can be listed according to course learning and frequency of each curricular content are
outcomes, clinical presentations, topics based on the criteria shown in Table 2 and
(lectures, practical session) and many more. Table 3. For Phase 1, the criteria for impact
At School of Medical Sciences, teaching of and frequency are different with Phase 2 due
the undergraduate curriculum is organised to different training level. For Phase 1, the
according to course topics, therefore the focus is more on preparing the students for
curricular content listed in our blueprints Phase 2, whereas the focus of Phase 2 is to
contain the list of topics. Column 1 in produce novice competent house officers.
Table 1 shows the 26 topics for this course. The weightage for each curricular content
is produced by multiplying the impact and
Step 3: Identify impact and frequency (Column 3 frequency (I × F). This produces I × F
and 4) product (Column 5) for curricular content
which ranges from 1 to 9.
Each assessment has a predetermined
number of items. The number of items University of Calgary blueprinting process
cannot be simply distributed equally based has their own approach to define impact and
on the list of curricular content. This is frequency (14). Potential life-threatening
because each curricular content has different conditions or the potential for significant
level of importance. Some measure of disease prevention and how frequently
relative weighting of content areas must be a presentation is encountered in clinical
decided beforehand so that priority can be practice is considered (see Table 4).
given to major areas when creating items

Table 2: Weighting criteria for impact and frequency of the curricular contents for undergraduate Phase 1 at
the School of Medical Sciences

Impact Frequency
1 Less important for Phase 2 1 Rarely applied in Phase 2
2 Important for Phase 2 2 Commonly applied in Phase 2
3 Very important for Phase 2 3 Frequently applied in Phase 2

Table 3: Weighting criteria for impact and frequency of the curricular contents for undergraduate Phase 2 at
the School of Medical Sciences

Impact Frequency
1 Less important for house officers 1 Rarely applied in clinical practices
2 Important for house officers 2 Commonly applied in clinical practices
3 Very important for house officers 3 Frequently applied in clinical practices

76 https://eduimed.usm.my
 EDUCATIONAL RESOURCE | Practical Guide to Construct an Assessment Blueprint

Table 4: Weighting criteria for impact and frequency of the clinical presentations for undergraduate renal
course at the University of Calgary

Impact Frequency
1 Non-urgent, little prevention potential 1 Rarely seen
2 Serious, but not immediately life threatening 2 Relatively common
3 Life threatening emergency and/or high potential for 3 Very common
prevention impact

Step 4: Categorise curricular content based on It is worth to note that, the first technique
relative weightage is more suitable for any assessment that
has less questions number compared to the
At School of Medical Sciences, the number of curricular content items, whereas
curricular contents are classified into “must the second technique is more suitable for
know”, “should know” and “nice to know” any assessment that has more number of
(3) based on their I × F. For any curricular curricular content items as compared to
content that has I × F of 6 or 9, they are total number of questions.
classified under “must know” knowledge,
for curricular content that has I × F of 3 Step 5: Decide on percentage of questions for
or 4, they are considered as “should know” each category
knowledge, whereas for curricular content
that has I × F of 1 or 2, they are put under As discussed earlier, the questions cannot be
“nice to know” knowledge. All curricular distributed equally throughout the curricular
contents will then be arranged and put into content. Logically, more questions should
their categories in different sheet in the be asked from “must know” category and
Microsoft Excel file. less questions from “should now” and “nice
to know”. The percentage of questions for
Alternative technique in determining each category may vary from one blueprint
weightage is by dividing the I × F for to the others. Therefore, we need to decide
each curricular content (column 5 of on percentage of each category in every
Table 1) with the total for the I × F column blueprinting process. How many percent
(155 in our example) to provide a relative questions from “must know”, “should
weighting for each curricular content, which know” and “nice to know” knowledge? At
corresponds to the proportion of evaluation School of Medical Sciences, particularly for
items for this presentation. The higher the Phase 1 curriculum, we have decided to take
number produced, the more important the 60% for “must know”, 30% for “should
curricular content, thus more questions know” and 10% for “nice to know” content.
should be constructed from these curricular
contents.

Table 5: Classification of curricular content based on their weightage

SEMESTER I PRO. I SUPP. PRO. I

IxF Category Percentage
MTF SEQ OSCE MTF SEQ MTF SEQ
6–9 Must know 60 % 13 3 2 6 1 6 1
3–4 Should know 30 % 6 2 1 1 0 1 0
1–2 Nice to know 10 % 1 0 0 0 0 0 0
Total 100 % 20 5 3 7 1 7 1

https://eduimed.usm.my 77
 Education in Medicine Journal 2020; 12(1): 71–80

For the second technique, to get the number curriculum has their own assessment task,
of questions for each curricular, multiply the be it for theory or clinical assessments and
total number of items on the evaluation with have been decided earlier. These tasks must
the relative weighting for each curricular be consistent with the learning outcomes
content, and then round it up or down to and teaching activities for that course. For
the nearest whole number. example, to assess theories or cognitive
skills, we can use multiple choice questions
Step 6: Decide on number of item for each and OSCE to assess clinical skills.
assessment task (Column 6, 7, 8)
Step 7: Assign questions to lecturers for items
As discussed earlier, each assessment task preparation (Column 9)
should have its predetermined number
of items. Based on the information from The final step in blueprint construction
Step 5, we can now decide on how many is to identify question makers for items
questions should be constructed from each preparation. Practically, the one who
category. For example, the number of items teaches the topic should prepare the
cannot be equally distributed based on the questions. However, there are circumstances
list of curricular content. whereby other lecturers need to prepare the
questions. It can be discussed during this
There are variety of tasks that can be stage (Table 6).
evaluated for any curricular content. Each

Table 6: The summary of seven steps in constructing a blueprint

Steps Description
1. Define the blueprinting Identify its purpose and scope.
purpose and scope For which semester or phase of study?
Which academic session?
Which courses?
What assessment tools?
How many questions?
2. Tabulate curricular content Curricular contents – course learning outcomes, clinical presentations
or topics – are listed based on curricular setting.
3. Identify impact and The impact and frequency for each curricular content are identified
frequency based on the selected criteria.
4. Categorise curricular content The curricular contents are classified as “must know”, “should know”
based on relative weightage and “nice to know” knowledge.
5. Decide on percentage of Determine how many percent questions should be constructed from
questions for each category “must know”, “should know” and “nice to know” knowledge.
6. Decide on number of item Decide on how many questions should be constructed for each
for each assessment task category of curricular content.
7. Assign questions to lecturers Identify question makers for items preparation. Practically, the one
for items preparation who teaches the curricular content should prepare the questions.

78 https://eduimed.usm.my
 EDUCATIONAL RESOURCE | Practical Guide to Construct an Assessment Blueprint

ISSUES RELATED TO BLUEPRINT distribute the blueprint to students since

they will study all the curricular content
USM Department of Medical Education listed.
has been conducting many workshops
on blueprint construction. We have
come across many issues related to the CONCLUSION
blueprinting process. There are a couple of
commonly asked questions that should be This study described seven practical
addressed in this study. First, if we use the steps to construct a blueprint. Based on
weighting criteria for impact as mentioned the experience handling blueprinting
in Table 4, what would happen to curricular process, solutions to several issues related
content which are very important i.e., to weightage of curricular content and
life threatening (impact, I = 3) but not dissemination of the blueprint to students
frequent (frequency, F = 1). These types were discussed. Despite it is a resource
of curricular content would fall under intensive process, it will provide the utmost
“nice to know” category. Thus, possibility benefit to both teachers and learners. This
of these curricular content to be selected is because a well-constructed blueprint is a
will be lesser compared to other curricular valuable educational tool that can improve
content. For this issue, we suggest that any the quality of assessment in medical
life-threatening condition will automatically education, and thus will ensure the highest
be classified under “must know” category quality of graduates produced.
irrespective of their frequency. Therefore,
these types of cases will have higher chance
to be selected. ACKNOWLEDGEMENTS
Second issue is about disclosure of the This collaborative work was funded
blueprint to students. Can it be distributed by the USM Short Term Grant (304/
to the students? Some institutions do PPSP/61313161). We would like to thank
distribute their blueprint to students USM for allowing us to conduct the series
and some do not. It was reported that of blueprinting workshop at its institutions
distributing assessment blueprint to students and our greatest appreciation to our dean
do not improve student performance, but and deputy dean (academic & alumni) who
significantly increased the perception of are very supportive. Special thanks also to
fairness of the evaluation process (15). Our our entire USM School of Medical Sciences
suggestion for this issue is that, it depends faculty members.
on the institutional policies. Different
institution will have their own policies. If we
disclose the blueprint to students, will it lead REFERENCES
to strategic learning? In other word, if we
distribute the blueprint to students, will they 1. Cambridge University Press. Blueprint. In:
only study on the curricular content which Cambrigde Dictionary. 2018. Available at:
will come out in examination? Theoretically https://dictionary.cambridge.org/dictionary/
yes, but based on study, the theory is english/blueprint
unsupported (14). Our suggestion is simple,
2. Sood R, Paul V, Mittal S, Adkoli B, Sahni
if the total number of curricular content
P, Kharbanda O. Assessment in medical
is more than the number of questions; we
education: trends and tools. New Delhi: KL
suggest not disclosing it to students since
Wig CMET, AIIMS; 1995.
disclosure of these blueprints might lead to
strategic learning. However, if the number
of curricular content is less than the number
of questions, then it is not a problem to

https://eduimed.usm.my 79
 Education in Medicine Journal 2020; 12(1): 71–80

3. Adkoli B, Deepak K. Blueprinting in 10. Biggs J. Constructive alignment and why it is

assessment: principles of assessment in important to the learning process. UK: The
medical education, 1st ed. New Delhi, India: Higher Education Academy; 1999.
Jaypee Brothers Medical Publishers Ltd;
2012. p. 205–13. https://doi.org/10.5005/jp/ 11. Bordage G, Brailovsky C, Carretier H, Page
books/11647_19 G. Content validation of key features on a
national examination of clinical decision-
4. Boland J, Finn Y, Geoghegan R. making skills. Acad Med. 1995;70(4):276–
Blueprinting assessment to enhance 81. https://doi.org/10.1097/00001888-199
constructive alignment. 2014. In: 16th 504000-00010
Ottawa Conference, Transforming
Healthcare through Excellence in 12. Bridge PD, Musial J, Frank R, Roe T,
Assessment and Evaluation. 25–29 April. Sawilowsky S. Measurement practices:
Ontario, Canada. methods for developing content-valid
student examinations. Med Teach.
5. Patil SY, Gosavi M, Bannur HB, Ratnakar 2003;25(4):414–21. https://doi.org/10.1080/
A. Blueprinting in assessment: a tool to 0142159031000100337
increase the validity of undergraduate
written examinations in pathology. Int 13. Harden RM, Sowden S, Dunn WR.
J Appl Basic Med Res. 2015;5(Suppl Educational strategies in curriculum
1):S76–9. https://doi.org/10.4103/2229- development: the SPICES model. Med
516X.162286 Educ. 1984;18(4):284–97. https://doi.org/10
.1111/j.1365-2923.1984.tb01024.x
6. Sireci SG. The construct of content validity.
Social Indicators Research. 1998;45(1– 14. Coderre S, Woloschuk W, McLaughlin K.
3):83–117. https://doi.org/10.1023/A:10069 Twelve tips for blueprinting. Med Teach.
85528729 2009;31(4):322–4. https://doi.org/10.1080/
01421590802225770
7. Messick S. Validity. In: Linn R, editor.
Educational measurement. New York: 15. McLaughlin K, Lemaire J, Coderre S.
Macmillan Publishing; 1989. p. 13–103. Creating a reliable and valid blueprint for
the internal medicine clerkship evaluation.
8. Downing SM, Yudkowsky R. Assessment Med Teach. 2005;27(6):544–7. https://doi
in health professions education. UK: .org/10.1080/01421590500136113
Routledge; 2009. https://doi.org/10.4324/
9780203880135

9. Hamdy H. Blueprinting for the assessment

of health care professionals. The Clinical
Teacher. 2006;3(3):175–9. https://doi.org/10
.1111/j.1743-498X.2006.00101.x

80 https://eduimed.usm.my