Englisia: Journal of Language, Education, and Humanities

November 2023. Vol.11, No.1, 306-323

The development of an online-based TPACK

questionnaire for ELT teachers

ST Ayu Surayya*, Maman Asrobi, Siti Maysuroh, Zukhruf Farizi

Universitas Hamzanwadi, Indonesia

Manuscript received February 24, 2023, revised July 10, 2023, accepted July 22, 2023,
and published online November 7, 2023.

Recommended APA Citation

Surayya, ST. A., Asrobi, M., Maysuroh, S., & Farizi, Z. (2023). The development of an
online-based TPACK questionnaire for ELT teachers. Englisia: Journal of Language,
Education, and Humanities, 11(1), 306-323. https://doi.org/10.22373/ej.v11i1.17364

ABSTRACT

The concept of Technological Pedagogical Content Knowledge (TPACK) has gained

significant attention in the field of education due to the widespread adoption of innovative
technologies, particularly accelerated by the COVID-19 pandemic and the surge in online
teaching. To effectively measure TPACK, the utilization of a well-designed and tailored
questionnaire has emerged as a sophisticated approach. The aim of this research is to
develop a questionnaire that is valid and reliable specifically for the context of online
learning. In order to ensure the validity of the questionnaire, expert validation was
conducted with three professionals from diverse backgrounds, including an Indonesian
professor, a Ph.D. student, and a local education supervisor. By incorporating their
recommendations, the questionnaire underwent revisions and refinements. Subsequently,
the validity of the questionnaire was assessed by administering it to 43 participants across
four Indonesian provinces: West Java, East Java, East Boneo, and West Nusa Tenggara.
The findings indicated that the questionnaire demonstrated strong validity and reliability,
making it a valuable tool for evaluating teachers' TPACK in online teaching
environments. Statistical analysis using the SPSS 23.0 software confirmed the validity
and reliability of all 24 questionnaire items, further endorsing its readiness for future
research endeavors. Additionally, this questionnaire can be tailored to meet the unique
requirements of teachers across different education programs, thereby enhancing its
applicability and versatility. In conclusion, this study contributes to the field of online
teaching by providing a robust instrument for assessing teachers' TPACK, facilitating
targeted interventions and improvements in instructional practices.

*Corresponding Author:

ST Ayu Surayya
Universitas Hamzanwadi
Jl. Cut Nyak Dien No.85, Pancor, Kec. Selong, Kabupaten Lombok Timur, Nusa Tenggara Barat 83611, Indonesia
Email: ssurayya@hamzanwadi.ac.id
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

Keywords: EFL teacher; Online learning in Indonesia; TPACK survey; TPACK

questionnaire; Education in pandemic

1. Introduction
The COVID-19 pandemic has resulted in a rapid and simultaneous transformation
of the educational systems worldwide, emphasizing the need for technology in education.
This can be observed through research and government regulations that highlight the
digital sphere's role in reshaping education (Kang, 2021; Mahzan, 2021; Schleicher,
2020). Various topics such as assessment during the pandemic, the voices of students and
teachers, teaching methods, and media have become readily accessible and important
areas of study. This dynamism underscores the adaptable nature of education, which
evolves based on real-world needs.
In line with this notion, technology is crucial in pandemic education. Maria et al.
(2020) argue that teachers' competence, previously referred to as "pedagogical content
knowledge," has evolved over the past 30 years. Specifically, the term "content" in this
context signifies that teachers should possess advanced knowledge in their specific
teaching fields. The "pedagogical" aspect pertains to their understanding of teaching
methods, assessment, teaching tools, and student interactions. In 2006, Koehler and
Mishra extended these competencies to include technology, resulting in the concept of
Technological Pedagogical Content Knowledge (TPACK), which encompasses the three
essential knowledge components for teachers in the digital era.
TPACK is a framework comprising three fundamental components: technological
knowledge (TK), pedagogical knowledge (PK), and content knowledge (CK). It also
includes four hybrid components: pedagogical content knowledge (PCK), technological
pedagogical knowledge (TPK), technological content knowledge (TCK), and
technological pedagogical content knowledge (TPCK). The incorporation of technology
within this framework reflects its significant influence on education, impacting over 50%
of teaching processes and activities (Fransson & Holmberg, 2012), particularly during the
pandemic (Ammade et al., 2020; Maria et al., 2020). Qualitative analysis and
questionnaires are commonly employed to study and assess TPACK in both qualitative
and quantitative research approaches.
Questionnaires play a pivotal role in TPACK survey research, providing a means
to gather abstract ideas, beliefs, concepts, and attitudes (Jr et al., 2014; Roopa, 2012)
related to TPACK. They serve as suitable instruments for identifying teachers'
professional needs (Mahony, 2015). Today, questionnaires have evolved beyond
traditional printed formats and can be administered online, facilitating survey research in
the digital age.
TPACK has become a popular topic of study in survey research and is widely
searched on the internet. For instance, Akman and Güven (2015) developed a
questionnaire to analyze TPACK and self-efficacy perceptions among social science
Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 307
 The development of an online-based TPACK questionnaire for ELT teachers

teachers and teacher candidates. Their questionnaire comprised 50 items, employing a

five-point scale ranging from "I do not know" (1) to "I know in a very good level" (5),
yielding a Cronbach's Alpha reliability coefficient of 0.977. Fuad et al. (2020)
investigated Indonesian Language Teachers' TPACK for online learning by adapting the
questionnaire developed by Schmid et al. (2020). Their questionnaire consisted of 30
items divided into TK, CK, PK, PCK, TCK, PCK, and TPCK groups. Participants
indicated their responses on a five-point scale, ranging from "strongly agree" (SA) to
"strongly disagree" (SD). However, no validity test was conducted for the questionnaire.
Additionally, Maria et al. (2020) developed a TPACK questionnaire comprising 28 items
and utilizing a five-point scale.
However, previous TPACK questionnaires have primarily focused on general
teaching aspects, without explicitly addressing the specific teaching setting. Yet, the true
value of a framework lies in its ability to explain and make sense of the real world
(Imenda, 2014; Jabareen, 2009; Nilsen, 2015). Unfortunately, the process of developing
questionnaires often varies in quality and lacks consistent, rigorous standards (Maria et
al., 2020; Jr et al., 2014). Therefore, as educational supervisors, the researchers are
motivated to provide comprehensive and valid tools to support education, particularly in
the context of online pandemic education. Developing a TPACK questionnaire
specifically designed for online teaching is crucial to minimize biased interpretations,
considering the emergence of online learning as a new educational culture. In this study,
the researchers aim to develop a concise TPACK questionnaire based on valid and reliable
measurements, particularly in the English as a Foreign Language (EFL) context. Thus,
the research question is focused on developing an online-teaching-based TPACK
questionnaire for ELT teachers.

2. Literature review
The primary objective of this study is to develop a TPACK questionnaire
specifically tailored for online teaching. To achieve this goal, the researchers will first
provide a comprehensive review of the TPACK framework as the theoretical foundation
for this study. This review will focus on discussing the key components of TPACK,
namely Content Knowledge, Pedagogical Knowledge, Technological Knowledge,
Technological Content Knowledge, and Technological Pedagogical Knowledge.

2.1. TPACK
The concept of Technological Pedagogical Content Knowledge (TPACK) revolves
around the integration of knowledge required in teaching and technology, and how
teachers can develop that knowledge within themselves. TPACK specifically refers to the
knowledge teachers need to effectively incorporate technology into their curriculum and
instructional practices. It encompasses the interplay between teachers' content
knowledge, pedagogical knowledge, and technology knowledge, highlighting their
interconnectedness (Knolton, 2014).
308 | Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

Figure 1. The technological pedagogical content knowledge framework

From the figure above, it shows that these three basic components in the TPACK
framework indicate how these three bodies of knowledge extract, prod, and influence
each other and create three new concepts, which are Pedagogical Content Knowledge,
Technological Content Knowledge, and Technological Pedagogical Knowledge. For
more details, the explanation as follow:

2.1.1. Content Knowledge (CK)

Content Knowledge (CK) refers to the comprehensive understanding and expertise
that English teachers possess in their specific subject matter. This knowledge
encompasses various aspects such as concepts, theories, evidence, and organizational
frameworks within the field. It also includes familiarity with the best practices and
established approaches for effectively conveying this information to students. For
teachers to be effective, they must have a deep knowledge and comprehension of the
subjects they teach, including a grasp of essential facts, concepts, theories, and procedures
within the discipline. Additionally, they should be familiar with explanatory frameworks
that help organize and connect ideas, as well as possess knowledge of the rules of
evidence and proof that apply to their field (Shulman, 1986).

2.1.2. Pedagogical Knowledge (PK)

Pedagogical Knowledge (PK) refers to the practical knowledge and understanding
that English teachers possess about the processes, methods, and approaches to teaching
and learning. This component encompasses various aspects, including teaching strategies
that cater to individual learning needs and effective methods for presenting subject matter
to students (Kanuka, 2006). In essence, it encompasses the practicalities, procedures, and
methods necessary for successful teaching and learning (Koehler et al., 2007). For
example, this knowledge encompasses understanding the purposes, value, and aims of

Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 309
 The development of an online-based TPACK questionnaire for ELT teachers

education, as well as more specific areas such as general classroom management

strategies, awareness of students' learning styles, course planning, and learning
assessment. Pedagogical Knowledge equips teachers with the skills and techniques
needed to create engaging and effective learning environments for their students.

2.1.3. Technological Knowledge (TK)

Technological Knowledge (TK) refers to teachers' knowledge of various
technologies, technological tools, and resources, as well as their ability to effectively
utilize them. TK encompasses understanding educational technologies and considering
their potential applications within a specific subject area or classroom context. It involves
recognizing when and how technology can facilitate or hinder learning and adapting to
new technological advancements. As Koehler et al. (2007) suggest, TK encompasses all
instructional materials, ranging from traditional tools like a blackboard to more advanced
technologies. In essence, TK encompasses a wide range of technologies utilized in
learning environments, reflecting the teacher's familiarity and proficiency with these tools
(Margerum-Leys & Marx, 2002). Teachers with strong TK possess the skills and
knowledge to effectively incorporate technology into their instructional practices.

2.1.4. Pedagogical Content Knowledge (PCK)

Pedagogical Content Knowledge (PCK) refers to teachers' knowledge of how to
effectively teach specific subject areas within the pedagogical context. This knowledge
encompasses various aspects such as curricula development, student assessment, and
reporting of results. PCK involves understanding the relationship between pedagogy and
the practicalities that support teaching practices. Its aim is to enhance teaching practices
by establishing stronger connections between the content being taught and the
pedagogical approaches used to convey it. PCK is specific to a particular subject area and
involves transforming content into instruction. This may involve presenting the subject
matter in various ways, adapting instructional materials, and catering to the individual
needs and alternative ideas of students (Harris et al., 2009; Sahin, 2011). PCK equips
teachers with the knowledge and skills to effectively bridge the gap between content
knowledge and instructional strategies in order to facilitate meaningful learning
experiences for their students.

2.1.5. Technological Content Knowledge (TCK)

Technological Content Knowledge (TCK) is a theoretical framework that is
primarily defined and influenced by the capabilities and characteristics of technologies
themselves. According to Slough and Connell (2006), TCK emphasizes the
comprehensive intersection of technology and content. They illustrate this concept using
the analogy of lenses, where technology and content lenses are used to view teaching and
learning. In this framework, technology and content become intertwined, forming a

310 | Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

unified perspective. The lenses also serve to enhance teaching and learning by providing
a more focused approach and facilitating collaborative professional development.
It is important to note that TCK differs from the broader Technological Pedagogical
Content Knowledge (TPACK) framework in its conceptualization of technology. TPACK
views technology as a distinct domain of knowledge separate from content and pedagogy,
focusing on the areas where these three realms of knowledge overlap. In contrast, TCK
places greater emphasis on the integration of technology and content, recognizing the
interconnectedness and inseparable nature of these two components. By understanding
and applying TCK, educators can harness the potential of technology to enhance content
delivery and facilitate meaningful learning experiences. This framework provides a lens
through which teaching and learning can be magnified, enabling a more focused and
collaborative approach in professional development.

2.1.6. Technological Pedagogical Knowledge

Technological Pedagogical Knowledge (TPK) refers to teachers' understanding of
how technology can shape and influence specific pedagogical practices. Margerum-Leys
and Marx (2002) suggest that TPK involves grasping general pedagogical strategies and
applying them to the use of particular technologies. It encompasses integrating
technological tools and equipment with appropriate instructional designs and strategies
while being aware of their strengths and limitations.
Sahin (2011) highlights the fact that many popular computer software applications
are not specifically designed for educational purposes but rather for business,
entertainment, communication, and social interaction. Therefore, it is the responsibility
of teachers to go beyond the general uses of these technologies and effectively integrate
them into instruction. TPK enables teachers to utilize technology in ways that align with
pedagogical goals and enhance the learning experience for students. By understanding
the affordances and limitations of technology, teachers can select and incorporate
appropriate tools and methods to support their instructional practices.

2.2. Questionnaire Development

Questionnaires are valuable tools for collecting data through a series of structured
questions that respondents fill out or answer. They are commonly used for measuring
factual variables, obtaining relevant information for the study's objectives, and ensuring
high validity and reliability of the collected data.
In the process of developing a questionnaire, the first step involves determining the
constructs of the research variables. These constructs are synthesized from theories that
have been analyzed and discussed, as presented in theoretical studies or literature reviews.
The constructs are described through conceptual definitions of the variables, including
their dimensions and indicators that will be measured (Oyebanji & Jumoke, 2017). Based
on these constructs, specific indicators are identified for measurement.

Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 311
 The development of an online-based TPACK questionnaire for ELT teachers

Subsequently, questionnaire items are created to measure the predetermined

indicators, following the process outlined in steps 4 and 5 of instrument preparation and
development. The design of the questionnaire items must align with the chosen
questionnaire type that is appropriate for measuring the identified indicators of the
variables under investigation (Oyebanji & Jumoke, 2017).

3. Method
This study follows the Research and Development (R&D) method as outlined by
Borg and Gall (1983), which is a systematic approach used to create and validate
educational products, including the development of a questionnaire. The development
process in this study consists of seven steps, as depicted in Figure 2.
Research &
Litrature Studies

Designing Questionnare Experts

Analysis

Revising
Questionnaire Questionnaire distribution/data collection

Data Analysis

Valid and Reliable

Questionnaire

Figure 2. Seven steps of the development of TPACK online-teaching based

questionnaire

The first step involved conducting research and reviewing relevant literature. Two
samples of the questionnaire were used during this step, chosen for their clarity and
comprehensibility. In the second step, the questionnaire was designed using a four-level
scale, including "Very Agree," "Agree," "Disagree," and "Very Disagree." This choice of
scale, as suggested by Chyung et al. (2017), helps avoid uncertain midpoints and increases
participants' sensitivity to the scale.
Furthermore, the questionnaire was developed in two languages, English and
Bahasa Indonesia, to accommodate the diverse participants. Three experts with different
qualifications, based on their education, profession, and knowledge of TPACK, were
involved in the revision process. After incorporating the experts' suggestions, the
questionnaire was distributed to EFL teachers for data collection. The questionnaire link
was shared globally through various social media platforms such as Facebook, Indonesian
teachers' groups on Telegram, and Twitter.
A total of 65 responses were collected from EFL teachers in Java, Borneo, and West
Nusa Tenggara. Data analysis was conducted using SPSS version 23.0, which eliminated
43 participants' responses based on the validity assessment of their answers. The
population of this study consisted of EFL teachers from both private and state junior and
senior high schools in Indonesia. However, the sample consisted of 43 EFL teachers from

312 | Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

Java, Borneo, and West Nusa Tenggara, who had varying teaching experience ranging
from 1 to more than 20 years. The sampling method used in this study was non-probability
convenience sampling, where participants were selected based on their willingness to
voluntarily respond to the questionnaire and the validity of their responses as measured
by SPSS version 23.0.

Table 1
Demographics variable.
Demographics Variable Accepted Responds Percentage
(N)
Gender Male 20 47%
Female 23 53%
Location Borneo 10 23%
Java 15 35%
West Nusa Tenggara 18 42%
Education Bachelor 27 63%
Magister 16 37%
Teaching < 5 years 9 21%
Experience 5-10 years 13 30%
10-15 12 28%
15-20 9 21%
Level Private Junior High 35%
15
School
Public Junior High 23%
10
School
Private Senior High 14%
6
School
Public Senior High 28%
12
School
Total 43

Table 1 provides a summary of the demographics of the participants, including

gender, location, education, teaching experience, and school level. The table shows the
number and percentage of eliminated and accepted responses in each category.
Given the quantitative design of this research, there were no specific limitations or
criteria for participant selection. The study employed convenience sampling, which
involves choosing members who are easily accessible and willing to participate
(Bhardwaj, 2019). Therefore, all EFL teachers in junior and senior high schools who were
willing to answer the questionnaire were included as part of the sample.

4. Findings and discussion

The construction of the questionnaire in this study was modified and adapted from
various sources, namely Fuad et al. (2020), Loi (2021), and Schmid et al. (2020). These
sources were selected because they are recently published research journals. However,
the selection of specific statements for inclusion in this questionnaire was based on their

Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 313
 The development of an online-based TPACK questionnaire for ELT teachers

relevance to the understanding and application of TPACK in the context of online

learning.
For instance, item PK 1 or item 16 in the questionnaire developed by Schmid et al.
(2020) was not included in this study. The statement "I know how to assess student
performance in a classroom" was deemed unrelated to the online learning-based context
of this questionnaire. Therefore, careful consideration was given to selecting statements
that align with the specific focus of this study and the online teaching environment.

Table 2
Item questionnaire construction.
Item construction Sources
Schmid, et.al, Fuad, et. al. Loi
(2020) (2020) (2021)
Technological Knowledge
TK1 TK1
TK2 TK1 TK3, TK4
TK3 TK1, TK2, TK3, TK2
TK4
TK4 TK1 TK3, TK4
TK5 TK7
Content Knowledge
CK1 CK3
CK2 CK2 CK2
CK3 CK6
Pedagogical Knowledge
PK1 PK2, Pk3 PK1, PK2, PK3,
PK5
PK2 PK4 PK4
Pedagogical and Content
Knowledge
PCK1 PCK1
PCK2 PCK2
Technological Content
Knowledge
TCK1 TCK3 TCK1
TCK2 TCK4
Technological Pedagogical
Knowledge
TPK1 TPK1 TPK1
TPK2 TPK2 TPK2
TPK3 TPK2

314 | Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

TPK4 TPK4
TPK5 TPK3, TPK4 TPK5
Technological Pedagogical Content Knowledge
TPACK1 TPCK2
TPACK2 TPCK2 TPCK5
TPACK3 TPCK3 TPCK1
TPACK4 TPCK1 TPCK3
TPACK5 TPCK4

Based on the experts' comments, the questionnaire underwent minimal revisions.

One of the experts emphasized the need for consistency between the English and Bahasa
Indonesia versions of the questionnaire. As a result, eight revisions were made to ensure
that the translations accurately conveyed the same meaning and context as the original
questionnaire. Adjustments were made to ensure consistency between the two versions,
such as modifying sentences and ensuring accurate translations. Another expert
recommended adding numbers to each item in the questionnaire to improve organization
and ease of reference. Taking this suggestion into account, numbers were incorporated
into the questionnaire to facilitate clarity and help respondents navigate through the items
more effectively.
Regarding the length of the questionnaire, a third expert expressed the opinion that
it was quite long. While the term "short" was debated, it was decided to retain the 24-item
length. This decision was based on the consideration that the questionnaire was indeed
shorter compared to previous questionnaires developed by Maria et al. (2020), Fuad et al.
(2020), and Loi (2021), which consisted of 54 items.
Overall, the feedback from the experts played a valuable role in refining the
questionnaire. The revisions focused on ensuring consistency between language versions,
organizing the items with numbered references, and acknowledging the questionnaire's
relative brevity in comparison to previous studies. These adjustments aimed to improve
the questionnaire's coherence and alignment with the research objectives, which involve
investigating Technological Pedagogical Content Knowledge (TPACK) in the context of
online teaching.
Simultaneously, the table provides an overview of the questionnaire items and their
sources of construction. The items were derived from multiple sources, incorporating
relevant statements from Schmid et al. (2020), Fuad et al. (2020), and Loi (2021).
Technological Knowledge items, such as TK1, TK4, and TK5, were based on Schmid et
al. (2020). CK2 drew upon insights from all three sources, while CK1 and CK3 were
influenced by Loi's (2021) research. Similarly, Pedagogical Knowledge items
incorporated elements from Schmid et al. (2020) and Fuad et al. (2020), and Pedagogical
and Content Knowledge items were constructed based on Loi's (2021) work. The
questionnaire also included items related to Technological Content Knowledge,
Technological Pedagogical Knowledge, and Technological Pedagogical Content
Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 315
 The development of an online-based TPACK questionnaire for ELT teachers

Knowledge, with their sources specified accordingly. In summary, the questionnaire

items were carefully selected and adapted from various sources, considering the feedback
from experts and aligning with the research objectives.

Table 3
SPSS measurement.
Statements of Willingness to Understand and Implement The p<
α
TPACK 0.05
Technological Knowledge
TK1 I am willing to learn how to solve technical problems in .70
teaching
TK2 I am willing to try new technologies. .61
TK1 I am willing to use technology .67
TK2 I am willing to keep up with new technological developments .67
that are important for me
TK3 I am willing to spend my time working with different .63
technologies
Initial TK Subscale .84
Content Knowledge
CK1 I am willing to add sufficient knowledge about English .74
language content.
CK2 I am willing to use various ways and strategies to develop my .73
understanding of English language content
CK3 I am willing to use English in every activity .63
Initial CK Subscale .89
Pedagogical Knowledge
PK1 I am willing to add my knowledge about teaching techniques .68
in VLE
PK2 I am willing to assess student performance in my VLE class .67
Initial PK Subscale .87
Pedagogical and Content Knowledge
PCK1 I am willing to adjust my teaching to what is currently .
understood or not understood by the students
PCK2 I am willing to teach English content which needed by the .
students
Initial PCK Subscale .76
Technological Content Knowledge
TCK 1 I am willing to learn about the technology that I can use to .78
understand and practice English language material
TCK2 I am willing to use technology as the media for me to learn .67
my content knowledge

316 | Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

Initial TCK Subscale .75

Technological Pedagogical Knowledge
TPK1 I am willing to find out the technology for an effective .76
teaching approach
TPK2 I am willing to find out technology that enhances student .67
learning
TPK3 Online teaching and learning have made me think more .78
deeply about how technology can affect the teaching
approach I use in class
TPK4 I am willing to spend my time thinking critically about how .76
to use technology in the classroom
TPK1 I am willing to adjust the use of technology that I learn to .74
various teaching activities
Initial TPK Subscale .87
Technological Pedagogical Content Knowledge
TPACK1 I am willing to find out a technology to use in the classroom .78
that enhances what I teach, how I teach, and what students
learn.
TPACK2 I am willing to find out the technology that enhances content .77
for an English subject.
TPACK3 I am willing to teach the appropriate subject by combining .76
English language material, technology, and teaching
approaches in my online class
TPACK4 I am willing to use strategies that combine content, .70
technology, and teaching approach in my online class
TPACK5 I am willing to help other colleagues with using the content, .73
technology, and teaching approaches in my school and/or
district.
Initial TPACK Subscale .92

In terms of validity, item-total correlations were computed to assess the relationship

between individual items and their respective subscales. Higher correlations indicate
stronger associations. Among the technological knowledge subscale, TK1 displayed a
correlation of 0.70, indicating a positive relationship with the overall subscale. Similarly,
CK1 and CK2 within the content knowledge subscale showed correlations of 0.74 and
0.73, respectively, indicating a strong relationship with the overall subscale. TCK1 in the
technological content knowledge subscale exhibited a correlation of 0.78, suggesting a
positive association with the subscale. Additionally, TPACK1 and TPACK2 in the
technological pedagogical content knowledge subscale displayed correlations of 0.78 and
0.77, respectively, signifying a strong relationship with the overall subscale.

Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 317
 The development of an online-based TPACK questionnaire for ELT teachers

Reliability was assessed using Cronbach's alpha, which measures internal

consistency reliability. A higher Cronbach's alpha indicates greater reliability among the
items within each subscale. The initial TK subscale demonstrated a Cronbach's alpha of
0.84, indicating good internal consistency. Similarly, the initial CK subscale displayed a
Cronbach's alpha of 0.89, suggesting a high level of reliability. The initial PK subscale
exhibited a Cronbach's alpha of 0.87, indicating good internal consistency among the
items. The initial PCK subscale demonstrated a Cronbach's alpha of 0.76, suggesting
acceptable reliability. The initial TCK subscale displayed a Cronbach's alpha of 0.75,
indicating reasonable internal consistency. Lastly, the initial TPACK subscale showed a
high Cronbach's alpha of 0.92, signifying excellent internal consistency reliability.
These findings provide support for the validity and reliability of the questionnaire
items within their respective subscales. The item-total correlations suggest that the items
are significantly associated with their respective subscales, indicating their relevance in
measuring the constructs of interest. The high Cronbach's alpha values indicate that the
items within each subscale are internally consistent, suggesting that they are measuring a
similar underlying construct. Overall, these results provide confidence in the
questionnaire's ability to effectively assess willingness to understand and implement the
TPACK framework.
The TPACK study was initiated in 2006, and since then, numerous comprehensive
studies have been conducted in this field, including the development of a short
questionnaire. The short questionnaire was designed with concise statements and a
reduced number of items to allow respondents to complete it quickly. This consideration
is important because teachers often have multiple responsibilities and limited time to
participate in research. By using a simple and easily understandable questionnaire,
respondents can grasp its meaning and purpose without much difficulty. Ensuring that
respondents understand the questions in the desired manner is crucial, as highlighted by
Sreejesh et al. (2014). Therefore, creating a short questionnaire provides a convenient and
comfortable experience for the respondents.
While a short questionnaire offers time efficiency, it is essential to address the issue
of item validity. Some items in the research did not demonstrate high validity. This may
be due to excessive focus on the characteristics of a short questionnaire, which led to
fewer items being included. To achieve high validity, each indicator should ideally consist
of at least three items. It is important to have multiple items for each indicator to ensure
the representation of the construct in case any item is found invalid during the validation
process (Bashooir & Supahar, 2018). However, in the present questionnaire, the CK and
PK domains only had two items, which is fewer than the other domains like TK, PCK,
TCK, PCK, and TPACK, which had three or more items. This suggests that maintaining
the label of a "short questionnaire" for TPACK is challenging since each domain of
TPACK should ideally have three or more items. Therefore, while the goal of building a
short questionnaire is to make it convenient for respondents, it is crucial to ensure its

318 | Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

validity and reliability, not only through item validity measurement but also from the
perspective of experts.
The involvement of experts in the questionnaire development process is important.
Translating a questionnaire into another language is a complex task (Tsang et al., 2017).
The validation process involves ensuring the accuracy of language translation, the
appropriate use of punctuation, and the overall consistency of the questionnaire to align
with the intended concept. This process is known as linguistic validation, where the aim
is to create an adapted questionnaire that maintains comparability to the original version
in terms of meaning and concept (Nipa et al., 2020). Professionals carefully examine each
component and the overall structure, ensuring that the adapted questionnaire is sourced
from qualified and reputable references (Elangovan & Sundaravel, 2021).
Selecting a qualified questionnaire source is an important consideration in
questionnaire development. Researchers need to choose appropriate indicators for each
item from well-established and respected previous studies. This process is part of the
qualitative phase in questionnaire development, which involves an extensive literature
review, identification of constructs and measures, development of questionnaire items,
and seeking expert advice on the inclusion of constructs and pre-selected items in the
research instrument (Nikitina et al., 2016).
Furthermore, several factors influence questionnaire development, such as the
gender of participants, the level of workplace in which English teachers operate, and their
teaching experience. Gender plays a significant role in education and research
participation. Female EFL teachers tend to be more active in survey research, displaying
better engagement in answering questions, providing clarifications, and responding
promptly. Women often exhibit greater competency in areas such as empathy, intuition,
teamwork, self-control, and care (Xin et al., 2019). In the education field, females have
shown greater pedagogical knowledge than males, while males tend to possess stronger
technological knowledge (Öz, 2015). This finding is reflected in the TPACK scores,
where female teachers tend to have higher scores in PK compared to CK and TK in the
writing context (Surayya & Asrobi, 2020).
The level of the school where English teachers work also impacts their participation
in research. Junior school teachers are more actively involved in teacher development,
resulting in a higher number of participants from junior high schools compared to senior
high schools (Tindowen et al., 2020). The reasons behind this phenomenon require further
investigation, as limited studies have explored differences between teachers in senior and
junior high schools based on professional demands, loyalty, and other factors.
Teaching experience also influences the willingness of English teachers to
participate in voluntary research. In this study, a large majority of the respondents
(75.4%) had relatively short teaching experience of 0-9 years, while a smaller portion
(24.6%) had longer experience of 10-20 years. This finding contrasts with the statement
by Buenvinida et al. (2020) that teachers with greater experience are better prepared.
However, novice teachers may be more open to technology, while experienced teachers
Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 319
 The development of an online-based TPACK questionnaire for ELT teachers

may have more commitments and responsibilities, leaving less time for voluntary
research (Thien et al., 2014).
While the majority of participants in this questionnaire had bachelor's degrees, it is
important to note that the level of education does not solely determine the willingness of
English teachers to participate in voluntary research. The high number of teachers with
bachelor's degrees in the research sample aligns with the overall educational landscape in
Indonesia, where a majority of teachers hold bachelor's degrees (Kemenristekdikti, 2018).
Teachers' participation in research also connects to the professionalism principle outlined
in the legal law of the Republic of Indonesia in 2003, emphasizing the need for teachers
and lecturers to enhance the quality of education (Dewan Perwakilan Rakyat Indonesia,
2005).
English teachers in Indonesia are active in online platforms, particularly on
Facebook and Telegram. Therefore, conducting an online survey research proves to be a
viable choice in Indonesia. Web surveying in academic research is a relatively new
phenomenon in a rapidly evolving technological context (Allen & Roberts, 2010). This
questionnaire development, based on an online survey format, highlights the importance
of teachers not only as educators but also as active contributors to research, contributing
to significant transformations in the education process.

5. Conclusion
Ensuring the validity and reliability of research findings requires the use of a valid
and reliable instrument, such as a well-constructed questionnaire. The process of building
a questionnaire involves essential steps like item construction, expert validation, and
measuring with statistical software like SPSS. These steps are interdependent and crucial
in establishing a questionnaire that is ready to be used for assessing the TPACK of EFL
teachers in online learning.
Furthermore, it is worth mentioning that this questionnaire can serve as a valuable
resource for other researchers and teachers in different fields. By adapting and modifying
the questionnaire to suit their respective areas of expertise, such as economics, science,
art courses, and so on, researchers and teachers can effectively measure the TPACK or
related constructs within their specific domains.
In summary, establishing a valid and reliable instrument, such as a well-constructed
questionnaire, is essential for obtaining trustworthy research findings. The steps involved
in building a questionnaire, including item construction, expert validation, and statistical
measurement, are integral to this process. Future researchers should ensure an adequate
number of items in each domain when measuring TPACK using Cronbach's alpha.
Moreover, other researchers and teachers can adapt this questionnaire to suit their specific
areas of expertise and assess similar constructs within their respective fields.

320 | Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

Acknowledgement
Special gratitude is extended to the participants of this study who generously
dedicated their valuable time and actively participated. Their willingness to contribute
has been instrumental in the progress of this article. Additionally, heartfelt appreciation
is expressed to Prof. Bambang Yudi, the esteemed professor who graciously served as the
expert validator. Furthermore, thanks are extended to P3MP Universitas Hamzanwadi for
their unwavering belief and support, granting the opportunity to conduct this research.
Their continued trust and encouragement have been pivotal in facilitating the successful
execution of this study for the third time.

References
Akman, Ö., & Güven, C. (2015). TPACK survey development study for social sciences
teachers and teacher candidates. International Journal of Research in Education and
Science, 1(1), 1–10.
Allen, P. J., & Roberts, L. D. (2010). The ethics of outsourcing online survey research.
International Journal of Technoethics, 1(3), 35–48.
https://doi.org/10.4018/jte.2010070104
Ammade, S., Mahmud, M., Jabu, B., & Tahmir, S. (2020). TPACK model based
instruction in teaching writing: An analysis on TPACK literacy. International
Journal of Language Education, 4(1), 129–140.
https://doi.org/10.26858/ijole.v4i2.12441
Bashooir, K., & Supahar, S. (2018). Validitas dan reliabilitas instrumen asesmen kinerja
literasi sains pelajaran Fisika berbasis STEM. Jurnal Penelitian Dan Evaluasi
Pendidikan, 22(2), 219–230. https://doi.org/10.21831/pep.v22i2.19590
Bhardwaj, P. (2019). Types of sampling in research. Journal of the Practice of
Cardiovascular Sciences, 5(3), 157–172. https://doi.org/10.4103/jpcs.jpcs_62_19
Buenvinida, L. P., Rodriguez, M. T. M., Sapin, S. B., Alforja, N. S., & Panopio, F. P.
(2020). Senior high school teachers’ readiness in implementing 21st century
learning: An input to school improvement plan. Advances in Social Sciences
Research Journal, 7(9), 42–51. https://doi.org/10.14738/assrj.79.8967
Chyung, S. Y. Y., Roberts, K., Swanson, I., & Hankinson, A. (2017). Evidence-based
survey design: The use of a midpoint on the likert scale. Performance Improvement,
56(10), 15–23. https://doi.org/10.1002/pfi.21727
Dewan Perwakilan Rakyat Indonesia. (2005). Undang-Undang (UU) tentang guru dan
dosen nomor 14. Dewan Perwakilan Rakyat Indonesia, 2.
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rj
a&uact=8&ved=2ahUKEwjWxrKeif7eAhVYfysKHcHWAOwQFjAAegQICRAC
&url=https%3A%2F%2Fwww.ojk.go.id%2Fid%2Fkanal%2Fpasar-
modal%2Fregulasi%2Fundang-undang%2FDocuments%2FPages%2Fundang-
undang-nomo
Elangovan, N., & Sundaravel, E. (2021). Method of preparing a document for survey
instrument validation by experts. MethodsX, 8(March), 101326.
https://doi.org/10.1016/j.mex.2021.101326
Fransson, G., & Holmberg, J. (2012). Understanding the theoretical framework of
Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 321
 The development of an online-based TPACK questionnaire for ELT teachers

technological pedagogical content knowledge: A collaborative self-study to

understand teaching practice and aspects of knowledge. Studying Teacher
Education, 8(2), 193–204. https://doi.org/10.1080/17425964.2012.692994
Friebel, G., Lalanne, M., Richter, B., Schwardmann, P., & Seabright, P. (2021). Gender
differences in social interactions. Journal of Economic Behavior and Organization,
186, 33–45. https://doi.org/10.1016/j.jebo.2021.03.016
Fuad, M., Ariyani, F., Suyanto, E., & Shidiq, A. S. (2020). Exploring teachers ’ TPCK :
Are Indonesian language teachers ready for online learning during the COVID-19
outbreak ? 8(4), 6091–6102. https://doi.org/10.13189/ujer.2020.082245
Imenda, S. (2014). Is there a conceptual difference between theoretical and conceptual
frameworks? Journal of Social Sciences, 38(2), 185–195.
https://doi.org/10.1080/09718923.2014.11893249
Jabareen, Y. (2009). Building a conceptual framework: Philosophy, definitions, and
procedure. International Journal of Qualitative Methods, 8(4), 49–62.
https://doi.org/10.1177/160940690900800406
Jr, A. R. A., Rochelle, J. S. La, Dezee, K. J., Gehlbach, H., Artino, A. R., Rochelle, J. S.
La, Dezee, K. J., Artino, A. R., Rochelle, J. S. L. A., Dezee, K. J., & Gehlbach, H.
(2014). Developing questionnaires for educational research : AMEE Guide No . 87.
Medical Teacher, 36(6), 463–474. https://doi.org/10.3109/0142159X.2014.889814
Kalra, R. (2018). Minding the gap in vocabulary knowledge: Incidental focus on
collocation through reading. Arab World English Journal, 2(July), 125–131.
Kang, B. (2021). How the COVID-19 pandemic is reshaping the education service (Vol.
1). Springer Singapore. https://doi.org/10.1007/978-981-33-4126-5_2
Kemenristekdikti. (2018). Indonesia higher education statistical year book 2018. Pusdatin
Kemenristekdikti, 15(21), 1–7.
https://pddikti.ristekdikti.go.id/asset/data/publikasi/Statistik Pendidikan Tinggi
Indonesia 2018.pdf
Koehler, M. J., & Mishra, P. (2006). Technological pedagogical content knowledge: A
Framework for teacher knowledge. Teachers College Record, 108(6), 1017–1054.
http://one2oneheights.pbworks.com/f/MISHRA_PUNYA.pdf
Loi, N. Van. (2021). Vietnamese high-school teachers’ perceptions of TPACK in
teaching English as a foreign language. European Journal of Education Studies,
8(4), 183–198. https://doi.org/10.46827/ejes.v8i4.3693
Mahony, L. (2015). Developing an online questionnaire for determining teachers’
perceptions of their preparedness for working with children experiencing parental.
AARE Conference, 1–17.
Mahzan, S. K. (2021). Teachers competency in Malaysia during Covid19 pandemic.
Albukhay Social Business Journal, 2(2), 22–33.
Maria, E., Schmid, M., Brianza, E., & Petko, D. (2020). Developing a short assessment
instrument for Technological Pedagogical Content Knowledge (TPACK) and
comparing the factor structure of an integrative and a transformative model
computers & education developing a short assessment instrument for tech. Elsevier,
157(103967), 1–12. https://doi.org/10.1016/j.compedu.2020.103967
Nikitina, L., Don, Z. M., & Cheong, L. S. (2016). Construction and validation of a
questionnaire on language learning motivation. Zbornik Instituta Za Pedagoska
322 | Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023
 ST Ayu Surayya, Maman Asrobi, Siti Maysuroh, & Zukhruf Farizi

Istrazivanja, 48(2), 284–300. https://doi.org/10.2298/ZIPI1602284N

Nilsen, P. (2015). Making sense of implementation theories, models and frameworks.
Implementation Science, 10(1), 53. https://doi.org/10.1186/s13012-015-0242-0
Nipa, S., Phongnarisom, C. M., & Sriboonreung, T. (2020). The impact of students’
proficiency in English on science courses in a foundation year program.
International Journal of Linguistics, Literature and Translation (IJLLT), 3(4), 39–
45. https://doi.org/10.32996/ijllt.2020.3.4.5
Okas, A., van der Schaaf, M., & Krull, E. (2014). Novice and experienced teachers’ views
on professionalism. Trames, 18(4), 327–344. https://doi.org/10.3176/tr.2014.4.02
Oyebanji, A., & Jumoke, O. (2017). Research variables: Types, uses and definition of
terms. Research in Education, July 2017, 43–55.
https://www.researchgate.net/publication/342897909
Öz, H. (2015). Assessing pre-service english as a foreign language teachers’
technological pedagogical content knowledge. International Education Studies,
8(5), 119–130. https://doi.org/10.5539/ies.v8n5p119
Roopa, S. (2012). Questionnaire designing for a survey. The Journal of Indian
Orthodontic Scoiety, 46(4), 273–277. https://doi.org/10.5005/jp-journals-10021-
1104
Schleicher, A. (2020). The impac of COVID-19 on education insights from education at
a glance 2020. OECD, 1–31.
Schmid, M., Brianza, E., & Petko, D. (2020). Developing a short assessment instrument
for Technological Pedagogical Content Knowledge (TPACK.xs) and comparing the
factor structure of an integrative and a transformative model. Computers and
Education, 157(May), 1–12. https://doi.org/10.1016/j.compedu.2020.103967
Sreejesh, S., Mohapatra, S., & Anusree, M. R. (2014). Business research methods (Issue
July). Springer. https://doi.org/10.1007/978-3-319-00539-3
Surayya, S. A., & Asrobi, M. (2020). Tracing technological pedagogical content
knowledge (TPACK) on practical EFL teachers in writing context. Voices of English
Language Education Society, 4(2), 177–190.
https://doi.org/10.29408/veles.v4i2.2417
Thien, L. M., Abd Razak, N., & Ramayah, T. (2014). Validating teacher commitment
scale using a Malaysian sample. SAGE Open, 4(2).
https://doi.org/10.1177/2158244014536744
Tindowen, D. J., Bautista, J. A., Mira, S., & Echalar, H. J. (2020). Senior high school
teachers ’ professional and organizational commitment and their job satisfaction.
International Journal Of Arts Humanities and Social Sciences Studies, 5(9), 142–
150. www.ijahss.com
Tsang, S., Royse, C. F., & Terkawi, A. S. (2017). Guidelines for developing, translating,
and validating a questionnaire in perioperative and pain medicine. Saudi Journal of
Anaesthesia, 11(5), S80–S89. https://doi.org/10.4103/sja.SJA_203_17
Xin, J., Zhang, Y., Tang, Y., & Yang, Y. (2019). Brain differences between men and
women: Evidence from deep learning. Frontiers in Neuroscience, 13(March), 1--10.
https://doi.org/10.3389/fnins.2019.00185

Englisia: Journal of Language, Education, and Humanities | Vol.11, No.1, November 2023 | 323