clinical performance

Considering the interdependence of clinical

performance: implications for assessment and
entrustment
Stefanie S Sebok-Syer, Saad Chahine, Christopher J Watling, Mark Goldszmidt,
Sayra Cristancho & Lorelei Lingard

INTRODUCTION Our ability to assess RESULTS Although some independent

independent trainee performance is a key performances were described, participants
element of competency-based medical spoke mostly about the exceptions to and
education (CBME). In workplace-based clinical disclaimers about these, elaborating their
settings, however, the performance of a trainee sense of the interdependence of trainee
can be deeply entangled with others on the performances. Our analysis of these
team. This presents a fundamental challenge, interdependence patterns identified multiple
given the need to assess and entrust trainees configurations of coupling, with the
based on the evolution of their independent dominant being coupling of trainee and
clinical performance. The purpose of this supervisor performance. We consider how
study, therefore, was to understand what faculty the concept of coupling could advance
members and senior postgraduate trainees workplace-based assessment efforts by
believe constitutes independent performance supporting models that account for the
in a variety of clinical specialty contexts. collective dimensions of clinical
performance.
METHODS Following constructivist grounded
theory, and using both purposive and CONCLUSION These findings call into
theoretical sampling, we conducted individual question the assumption of independent
interviews with 11 clinical teaching faculty performance, and offer an important step
members and 10 senior trainees (postgraduate toward measuring coupled performance. An
year 4/5) across 12 postgraduate specialties. understanding of coupling can help both to
Constant comparative inductive analysis was better distinguish independent and
conducted. Return of findings was also carried interdependent performances, and to
out using one-to-one sessions with key consider revising workplace-based assessment
informants and public presentations. approaches for CBME.

Medical Education 2018: 52: 970–980 This is an open access article under the terms of the Creative Commons Attrib
doi: 10.1111/medu.13588 ution License, which permits use, distribution and reproduction in any
medium, provided the original work is properly cited.

Centre for Education Research and Innovation, Schulich School of Correspondence: Stefanie S Sebok-Syer, Centre for Education
Medicine and Dentistry, Western University, London, Ontario, Research & Innovation, Schulich School of Medicine & Dentistry,
Canada Western University, Suite H110, Health Sciences Addition,
London, Ontario N6A 5C1, Canada.
Tel: 519 661 2111 (ext. 89113);
E-mail: stefanie.sebok-syer@schulich.uwo.ca

970 ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd;
MEDICAL EDUCATION 2018 52: 970–980
 The question of coupling

Assessment research has identified sources of

INTRODUCTION variance such as content specificity,12 rater
variance,13 or the influence of context,14 which
Despite an increasing understanding that most impact the accuracy of assessments in complex,
clinical practice is not truly ‘independent’, the workplace-based performance environments. A
ability to assess independent postgraduate trainee recent review of rater-based assessments has also
performance is a key premise of competency-based noted inconsistencies that exist between assessment
medical education (CBME). To achieve this goal, approaches, psychometric assumptions and human
programmes have been required to develop novel capabilities when it comes to assessing trainees in
competency-based assessment tools focused on the workplace-based clinical contexts.15 Other scholars
observation and entrustment of trainees as they of workplace-based assessment have also recognised
complete various tasks.1 Characterised as entrustable the challenges of assessing in these environments,
professional activities (EPAs), such tasks are with some advocating for more reliance on
observable and measurable behaviours that qualitative data regarding performance,16 and
supervisors can trust trainees to carry out others arguing for better training of raters to
(independently) with success once they have achieved minimise variance17 or more nuanced
a particular level of competence.2 These understanding of the sociocultural influences on
entrustment decisions are particularly important practices such as direct observation.18 Recognising
when it comes to senior trainees preparing for that many learning activities take place in team
independent practice because they are used to settings in the clinical workplace, a variety of
judge their readiness to provide unsupervised approaches have emerged for assessing students’
patient care.3 However, in workplace-based clinical collaboration competencies.19
settings, performance outcomes can be difficult to
link to one particular trainee because trainees’ Although this current scholarship has usefully
performance is often inseparably tied to the faculty elaborated the complexity of assessment in clinical
members who supervise them.4 Although senior and workplace settings, it has tended to maintain
trainees may routinely provide patient care that is the perspective that the independent trainee is the
neither directly observed nor prospectively approved focus of assessment attention. This assumption is
by faculty supervisors,5 even routines such as regular pervasive but largely tacit and therefore
case review interweave faculty members’ and underexplored. The current study makes explicit,
trainees’ clinical decisions and actions.6 and questions, this fundamental assumption. In
Furthermore, studies of clinical supervision suggest doing so we situate ourselves within a rich tradition
that faculty members enact various practices that of sociocultural understandings of clinical learning
control and even counteract trainee independence and practice. Many medical education scholars have
to safeguard patient safety, based on factors such as argued for the need to move beyond the field’s
clinical context, patient acuity and trainee dominant individualist, cognitivist approaches to
experience.7 Given that these supervisory practices learning, to engage sociocultural orientations from
are sometimes invisible, trainees may perceive more education, social science and humanities-based
independence than they actually have.8 knowledges.20–23 Social learning theories,
sociomaterial theories and complexity theory have
Within medical education, our language has all been embraced for their ability to foreground
changed from preparing trainees for independent entangled dimensions of collaborative work in
practice to preparing them for unsupervised or dynamic clinical systems. Arguments for the value of
indirectly supervised practice, acknowledging that these orientations highlight their ability: to grapple
physicians rarely practise in isolation from others. productively with learning relationships in situated
However, this shift has not yet translated into apprenticeships;20 to account for both human and
assessment discourse, which retains a strong focus material factors involved in workplace practice and
on, and assumption of, independence.9–11 Given learning;24 and to trace non-linear relationships
that assessment models are conventionally designed among complex system processes,21 including those
to measure independent performance, they struggle that may be maladaptive for learning.22 Empirical
when faced with the collective nature of clinical research with these orientations is powerfully
performance in workplace-based training advancing our appreciation of issues such as the
environments. In these environments, identifying influences of supervisor interruption during case
independent trainee performance for assessment review,25 the entanglement of collaborative practices
purposes presents a profound challenge. in clinical teamwork,26 the inherent contradictions

ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd; 971
MEDICAL EDUCATION 2018 52: 970–980
S S Sebok-Syer et al

of simulation-based education,27 and the absence of ceased data collection when we reached theoretical
attention to power in interprofessional education.28 sufficiency, which did not mean that no new ideas
would have been identified with more data
Although there has been robust consideration of collection, but rather that we had achieved
the sociocultural dimensions of practice and sufficient data collection to enable an
learning, a sociocultural orientation has rarely been understanding of the dimensions of
applied to issues of assessment in medical education. interdependence.30,31 During the interview,
The current study addresses this gap by bridging participants were asked to identify instances of
these two domains and questioning the assumption trainees’ independent clinical performance and to
of independent performance. Such questioning is describe how such performances are currently
more than a theoretical exercise. Given the documented or captured within the clinical training
expectation within CBME that trainees will be setting. Participants were probed regarding how the
assessed and judged based on the evolution of their performances they mentioned were influenced by
independent clinical performance, we need to supervisory relationship or team context, two
better understand under what circumstances their sensitising concepts drawn from the relevant
performance is, or is not, independent. The literature. Interviews were recorded and transcribed
purpose of this study, therefore, was to understand verbatim and de-identified prior to data analysis.
what faculty members and senior postgraduate
trainees believe constitutes independent trainee We conducted constant comparative inductive
performance in a variety of clinical specialty analysis using an iterative process in which data
contexts. Our research question was: Which trainee collection and data analysis were concurrent, each
actions and decisions in the clinical workplace are informing and influencing the other. Following
likely, in the context of CBME, to be considered a constructivist grounded theory,29 the researchers
reflection of trainees’ independent clinical actions engaged in three analytical stages of coding: initial,
and decisions? focused and theoretical. Initial and focused coding
took place iteratively as transcripts became available.
Initial coding consisted of reading the interview
METHODS transcripts line-by-line to identify ideas. Building
upon our initial codes, focused coding was then
This study was approved by the institutional Health used to highlight concepts or themes within the
Sciences Research Ethics Board. We used a transcripts; an early theme identified was termed
constructivist grounded theory approach to explore ‘the question of decoupling’, which referred to
the nature of ‘independent’ trainee clinical participants’ persistent references to factors
performances in workplace-based settings.29 Data influencing the independent performances they
collection and analysis occurred in an iterative were trying to explain. Each new incident,
fashion. Using a purposive sampling technique, we experience or perspective described by a participant
conducted individual interviews with 11 clinical was compared with previous incidents, experiences
teaching faculty members and 10 senior trainees and perspectives to define and refine the theme –
(postgraduate year [PGY]4/5) across 12 the question of decoupling. Our interview guide was
postgraduate specialties: anaesthesia, emergency also periodically revised in light of this developing
medicine, otolaryngology/head and neck surgery, analytical process. The iterative nature of collection
general surgery, critical care medicine, internal and analysis also allowed us to use theoretical
medicine, neurology, obstetrics and gynaecology, sampling as the study proceeded, seeking
orthopaedic surgery, pathology, paediatrics, and participants from training contexts that might
psychiatry. We sought a diverse sample because we elaborate or challenge our early understanding of
anticipated that independence would be this issue of decoupling. Particular attention was
characterised differently in various specialty paid to these discrepant examples so that our
programmes, and our aim was to produce a rich analysis could reflect their occurrence. In regular
description of the features of independent meetings of the analysis group (SSS, LL and SC),
performance that could inform assessment strategies three decoupling subcategories (supervisor, team
across a variety of programmes making the shift to and system) were discussed and definitions refined,
CBME. E-mails were sent to faculty members and following which the entire dataset was recoded, with
trainees at a single, midsized Canadian medical careful attention to discrepant instances that
school, inviting them to participate in a 30–45- challenged the integrity of these thematic
minute semi-structured individual interview. We categories. At this point in the analysis, we

972 ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd;
MEDICAL EDUCATION 2018 52: 970–980
 The question of coupling

determined that the ‘system’ subcategory was too performance. In analysing these exceptions and
sparsely populated to be richly described. disclaimers, we were able to describe patterns of
interdependence, which we eventually
Once all thematic categories were finalised and the conceptualised as ‘coupling’. In this results section,
data organised accordingly, theoretical coding we first describe the pattern of responses regarding
explored the relationships among them, and independent performances, and then we describe
analytical memos were created to reflect theoretical two dominant configurations of interdependent or
insights and questions about these relationships. At coupled performances that we identified in the data.
this point in the process, we shifted our attention Faculty participants are identified by Participant F#
from decoupling to coupling, reflecting our and senior resident trainee participants by
emerging understanding that, while participants Participant R#.
were explaining why they felt a particular clinical
performance was not independent, they were also Independent trainee performance
providing insight into why it was interdependent.
Insights from theoretical coding were presented to Participants were able to identify ‘independent’
other members of the research team, both trainee performances, but there were diverging
individually and in group meetings, for discussion, perspectives in our data about the opportunities to
elaboration and refinement, with special attention demonstrate independence. Independence was
to our discrepant examples. Investigator described by our trainee participants (PGY4/5) as
triangulation strengthened the analysis, as our team occurring when they perform a clinical task, such as
includes experts in measurement and assessment, gross examination of specimens, without direct
teamwork, clinical supervision, postgraduate supervision (Participant R2), when they provide
training and qualitative research.32 Our approach patient care without prior approval and only consult
also included strategies for returning findings for before disposition or discharge (Participants R5 and
refinement and elaboration, which serves as a R7), or when they are on-call and make clinical
measure of rigour and strengthens our ability to decisions without having to consult with faculty
explore our findings’ resonance and evaluate members (Participants R6 and R8). On the one
potential transferability to other contexts. There hand, a number of trainee participants
were three venues for this: (i) discussions with four characterised ‘everything’ they do as independent.
local key informants from specialties outside of As a senior trainee from emergency medicine
those interviewed for this study (i.e. cardiology, explained:
reconstructive surgery, radiology and urology); (ii)
two local medicine grand-rounds sessions at Usually the staff don’t see my patients. I’ll have
different hospitals, where findings were formally done everything, the history and physical, order
presented and participants could ask questions and investigations and even disposition the patient,
engage in discussions; and (iii) a public whether they go home or be admitted to a
presentation in another province, to an audience of service. I feel comfortable ordering advanced
clinical teachers from a variety of specialty contexts. imaging and CTs, MRIs and my staff trust me to
Our discussion of findings with participants and do that as well. (Participant R7)
other audiences suggested that the notion of
coupling resonated strongly and that it was Most trainees also acknowledged that independent
experienced differently in different specialty performance was easier to observe in the later
cultures and organisational settings. stages of postgraduate training (i.e. PGY4/5) and in
clinical situations where trainees are working
without direct supervision:
RESULTS
So, somebody at my stage. I just finished my time
The interviews began by asking participants to as the chief of the service, so I’m probably as
describe instances of independent trainee senior as you get. And I would say that at this
performance. Most participants asserted that some point I’m very independent. For administrative
clearly independent performances existed in their purposes we’ll say, there will always be a staff
clinical training context. However, consistent across person in the room or available, but the majority
all interviews, participants spent most of their time of surgeries I either do on my own or I’m doing
detailing an array of disclaimers for, and exceptions with a staff person available to assist as I need
to, their examples of independent trainee them. (Participant R4)

ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd; 973
MEDICAL EDUCATION 2018 52: 970–980
S S Sebok-Syer et al

Ordering practices were among the most common trainee and supervisor. Also evident, but less well
examples cited to reflect independent trainee elaborated, in the data were configurations of
performance. For instance, when asked what sort of coupling between trainees and other members of
independent actions and decisions a trainee might the interprofessional team.
make throughout a shift, one faculty member
responded with ‘order lab tests, order medications, Coupling of trainees and supervisors
and [order] chest x-rays’ (Participant F1).
The predominant configuration of coupling evident
However, trainee participants also challenged the in the data was that between a trainee and his or
very notion of independence within postgraduate her supervisor. The nature and degree of
training, as a result of their experience that ‘the interdependence of trainee and supervisor
resident and faculty work as a team’ (Participant depended on the clinical context and the type of
R3). Similarly, a faculty participant from critical care supervision provided. In some clinical contexts,
commented: such as the operating room, coupling was described
in such strong terms that participants questioned
Yeah, the structure of our care delivery is that the whether trainees could ever be considered
residents rarely have the final say on anything, independent. As one faculty member from
unlike elsewhere in the hospital. We have obstetrics and gynaecology explained:
residents, we have senior residents that we call
fellows, and then there is the consultant. So, the I don’t believe our residents are given true total
residents rarely get to do anything on their own. independence prior to graduation. Because
(Participant F8) medicolegally, for example, at the time of a birth
if something were to occur, the culture in at least
As these comments suggest, trainees work as part of my department in our institution is for the staff
a collaborative team, where performance is person to be available in person at all times. So,
interdependent. you may function independently in the sense of
you may put the forceps on and deliver the baby,
Interdependent trainee performance but not without me actually watching what you’re
doing still. (Participant F6)
As faculty members and trainees described what
constituted independent trainee performance, all of Another surgical faculty member explained that,
them spoke at length about how a performance that because of a profound sense of responsibility for
seems to be independent may not be. They offered the surgical outcome, his level of supervision never
detailed exceptions, disclaimers and complicating abated to allow a trainee full independence:
factors, such as ‘it’s tricky’ (Participant R5),
‘unfortunately’ (Participant R5) and ‘but it is My patients, their complication rates should be
difficult, if not impossible, to separate them by my complication rate, not my residents’
individual practitioner’ (Participant F8), and they complication rate. I’m watching them like a hawk
appeared to struggle to draw a line around clearly and if I think they’re going to make a mistake we
independent performance. For instance, one faculty take over. The outcomes of my patients I
member from emergency medicine described a consider them to be my outcomes, not my
complicating factor of trainees being steered toward residents’ outcomes. (Participant F10)
the sicker patients:
The clinical context in surgery, where the faculty
Yes, that [ordering of x-rays, ultrasound and CTs] member is omnipresent in the operating room,
is definitely something. Now theirs [trainees] supported such careful watching. But this was not
would be skewed, I don’t know how you interpret unique to surgery. Participants from non-surgical
it. (Participant F2) programmes also reported a sense of being carefully
watched. An internal medicine trainee told a story
In analysing these parts of the interviews, we about:
identified recurring patterns of interdependence
performance; we have conceptualised these patterns . . . the busiest night I think I’ve ever had to work
as ‘coupling’ of trainee performance with that of and partly it was because we had a couple of very
other team members. The main configuration of sick patients and at one point, at 11:00 or so, the
coupling described by our participants was between staff actually called me and said look it, I was just

974 ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd;
MEDICAL EDUCATION 2018 52: 970–980
 The question of coupling

cruising [the EHR system] and I noticed that you their awareness of the work that occurs behind the
guys have had a lot of consults. Do you have any scenes. One faculty participant from neurology
questions or are you worried about anybody? It pointed out that some trainees sign-off on dictations
was like the most wonderful moment because it without an awareness that faculty members will
was just so comforting to know that somebody review and edit them:
was looking over my shoulder someplace.
(Participant R6) They must come to me. As a consultant, they’ll
dictate for me. Even if they sign-off on them,
Although this trainee expressed relief at knowing he which for the most part we ask them not to so
or she was not performing alone but were being that I can review and edit. But occasionally
watched, others expressed frustration at what they some residents, if they’re off-service residents,
perceived to be a constraint on their independence. they’re used to just signing off on their own
The following example comes from pathology and dictations. It still comes to me, I have to sign-off
laboratory medicine: on it, and I can put an addendum. Once it’s
been finalized by the resident, I can’t edit that,
It could be the day before I graduate, and I but I can put an addendum at the bottom.
cannot sign out the simplest specimen. So (Participant F4)
everything I do still needs to be looked over by a
pathologist. It’s an odd thing where I graduate It was not only behind-the-scenes actions that
the next day and I can sign out everything. So coupled trainee performance with supervisors, it was
something magical happens that night, and I can also actions taking place after the trainee’s role in
suddenly have the ability. It’s a challenge in our patient care had ended. Most trainees admitted
specialty . . . Right now, we don’t have a lot of knowing very little about the outcomes of their
independence from that standpoint. Everything patients, or how the associated clinical
we do is, kind of, always looked over. (Participant documentation for such patients was edited as a
R2) result of a faculty member’s supervision, because
the trainee often did not continue to provide the
Although the ordering of medications, tests or patient’s care:
imaging was often mentioned as an independent
trainee performance, many faculty participants Sometimes they go straight to the next place.
acknowledged that such orders are often placed So, that longitudinal course of how that
only after consultation with a supervisor: ‘for the patient did and what happened and how did
most part, our residents wait to have the discussion your decision maybe affect that patient’s
before they actually would order the medication’ outcome, we don’t often get. We get the
(Participant F4). Trainees also commented that immediate, like how they did the next day or
affirming clinical orders with one’s supervisor within the next week or couple of weeks.
before they are placed was a common convention: (Participant R4)

This whole very institution, I think you’ll find at Faculty members commented that the transient
[our medical school] everything that we do is nature of postgraduate training, in which they
reviewed with the staff first so that is another rotate in and out of clinical workplace settings,
layer of I would say complexity that any kind of makes it difficult for trainees to appreciate how
orders that we put in is very likely to have already strongly their performance is coupled with their
been discussed with our staff first in an on call supervisors’, who may have altered clinical
scenario. (Participant R1) decisions:

These discussions are rarely evident in I change the note, but if . . . next time I see him,
documentation, such that ordering practices may if I see him again, I might mention it, but half of
appear independent (tagged to the trainee in the them, I never see them again. (Participant F9)
electronic health record, for instance) but actually
reflect performance that is coupled with the Other configurations of coupling
supervisor.
Although coupling between supervisors and trainees
Participants’ responses to the question of was dominant in the data, some participants
independent performance depended in part on provided examples of coupling between trainees

ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd; 975
MEDICAL EDUCATION 2018 52: 970–980
S S Sebok-Syer et al

and other members of the health care team. Across

most of the programmes our participants DISCUSSION
represented, trainees participated as one of many
learners on a clinical teaching team. As one trainee In asking faculty members and trainees to
explained, the established hierarchy on these describe instances of independent trainee
teaching teams meant that their performance was performance, we gained insight into the
intimately connected to the performance of other interdependence of trainee performance. We have
learners also engaged in patient care activities: characterised this phenomenon as ‘coupling’ to
capture the interdependence of trainee
There are two residents actually so usually one performance with both supervisors as well as with
senior and one junior. It doesn’t matter whether other trainees and health care professionals. In
it’s senior or junior. The medical student finds this section, we situate this conceptual
whichever is available, and then reviews the case. understanding of coupling within the scholarship
Dependent on how much work has already been on clinical supervision and teamwork, and
done by the medical student, there may or may elaborate its implications for workplace-based
not be more work for the resident to follow-up assessment. Our aim is to offer the notion of
on so time to disposition is a challenge when you coupling as a conceptual bridge between the
add in the fact that there’s a medical student also traditional assessment focus on independent
involved. (Participant R1) performance and the emerging assessment
challenge of accounting for collective performance
Many participants pointed out that the teaching in complex clinical environments. We acknowledge
team structure presents challenges to the attribution that this will be an uneasy marriage of
of clinical performances, as each trainee’s actions or epistemological orientations, but we would
decisions will be coupled, to varying degrees, with a contend that such a marriage is necessary to
more senior team member with whom they grapple productively with the challenge of
reviewed their plans. And it is not entirely assessing coupled performances in clinical training
predictable which team members are coupled at any settings.
given time: for instance, one trainee reported that
‘anything that I feel needs urgent attention, such as First, though, a note about the term coupling. We
an airway emergency. I would contact both the have used it to represent interdependence between
senior resident as well as the attending’ (Participant two team members (e.g. trainee and supervisor) and
R8). the term had resonance with participants in our
return of findings interviews and presentations. We
Participants also offered examples of how trainee recognise, however, that coupling is also a term
independence is modulated by the roles and used in organisational science to describe the
behaviours of team members, such as nurses, social nature and degree of interdependence of
workers and laboratory technicians. One example components in complex systems. According to
related to the task of obtaining informed consent: coupling theory, system elements are conceptualised
according to their degree of responsiveness (i.e.
In an interprofessional practice often it’s a social capacity and ability to respond to changes) and
worker who does that [obtain informed consent]. distinctiveness (i.e. preservation of an independent
In medicine, it is of necessity needing to be role within a system).33 Our use of the term
much more interprofessional in its practice. I ‘coupling’ in this paper reflects only the most basic
would hope that our senior residents know how of coupling arrangements: between two human
to work with other members of the team. elements in a system. However, coupling theory
(Participant F11) allows for more elaborate relations of
interdependence, including among multiple factors,
As this example suggests, in some institutional both human and material.34 Although the data
contexts, a trainee’s practice of obtaining informed from this study contained only a few elaborate
consent is likely to be coupled with the social descriptions of coupling (e.g. coupling between
worker’s practice. Given that this task is an trainee, faculty member and system factors such as
entrustable professional activity for trainees as they rotation schedule or patient census), we expect that
transition to residency, entrustment decisions would as research continues in this domain, we can draw
need to account for this coupling. productively on organisational coupling theory to

976 ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd;
MEDICAL EDUCATION 2018 52: 970–980
 The question of coupling

deepen our understanding of the multiple patterns task without direct supervision (e.g. a lumbar
of interdependence shaping trainees’ clinical puncture); (ii) gather clinical data without prior
performance. approval (e.g. ordering bloodwork); or (iii) manage
clinical decision making (e.g. discharge from the
It is not surprising that trainee performance is emergency room). An appreciation of coupling
coupled with that of a supervisor. Many studies have allows us to clearly articulate such characteristics of
highlighted the power and importance of the independent performance. Once articulated,
supervisory relationship in medicine’s workplace- educators can identify which performances in their
based training model. Hauer and colleagues’4 own programmes have these characteristics and
review of clinical supervision identified five make strategies for the best way to assess them.
important factors that impact trainee independence:
supervisor, trainee, supervisor–trainee relationship, Characterising interdependence
task and context. Kennedy’s observational and
interview study of clinical supervision found that The concept of coupling we have put forth provides
both supervisors and trainees enact strategies to a language for articulating the degree to which a
balance the goals of trainee independence and trainee’s performance is interdependent with
patient safety.8,35 Goldszmidt et al.36 suggested that another individual(s). With this language, we can
supervisors may enable or inhibit trainee bridge the gap between the assumption of
independence based on their response to independent performance and the reality of
institutional factors such as patient census and interdependent performance. This gap may explain
discharge pressures. Additionally, Goldszmidt and some of the current challenges in trying to assess
colleagues argued that the ideal performance of a trainee performance in authentic, clinical workplace
teaching team is one in which the group comes to settings. Given that assessment is largely influenced
an increasingly refined and shared understanding by measurement and psychometrics, obstacles such
of the patient’s needs over the trajectory of the as the large amount of unexplained variance in
hospital stay, even as individuals rotate on and off workplace-based assessments or the inability to
the team.6 In this way, Goldszmidt’s work has account for more than one object of measurement
shifted the focus from clinical supervision to the in a single observation15,38 have received much
intersecting practices of the individuals within the attention in the literature.
collective team.
Our work attempts to marry assessment and
Although there is a robust literature on clinical measurement approaches with an appreciation of
supervision, these studies do not explicitly consider sociocultural understandings. We have no
the question of how we conceptualise and assess expectation that this marriage will be an easy one,
trainee performance within the context of as it is likely to challenge sacred assumptions from
supervision. Our results extend our understanding each of these scholarly communities. However, we
of the intersecting practices of supervisors and would argue that this marriage between assessment
trainees, and have implications for assessing trainee and sociocultural perspectives is necessary in order
performance. We have described various to authentically capture the interdependence of
configurations of coupled performance between clinical performance in workplace-based settings. To
trainees and other members of the clinical teaching assess the various configurations of coupled
team, particularly supervisors. We contend that performances, we need approaches that can assess
viewing trainee performance through the lens of multiple individuals, numerous task dimensions and
coupling can help us in two ways: (i) to identify various outcome measures. We also need to
moments where performance may be truly consider how coupled performances could be
independent, and (ii) to appreciate when and how further characterised by distinguishing between
performances are interdependent. contribution and attribution39 in outcomes
assessment. Emerging approaches in educational
Defining independence measurement might prove useful to this end. For
example, Andrews et al.40 recently used the
Independence, by definition, means being free Andersen/Rasch (A/R) multivariate item response
from control, influence or support of others, with theory (IRT) model to assess interaction patterns in
the ability to think and act for oneself.37 As we seek dyads and explore how interaction patterns relate to
independent performances to assess, we should look performance outcomes. Using a simulation-based
for instances in which trainees (i) perform a clinical collaborative problem-solving task, they found that

ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd; 977
MEDICAL EDUCATION 2018 52: 970–980
S S Sebok-Syer et al

interaction patterns from two individuals who were they were working, in order to track not only the
previously unacquainted, working to solve a science ordering decision itself, but the supervisory
problem, correlated with performance outcomes. influence and sociocultural conditions surrounding
They also characterised the four interaction it. Trainees’ ordering practices may well change not
patterns: cooperative, collaborative, fake only as trainees gain expertise, but also as their
collaboration and dominant/dominant that were supervisors change (and as other salient aspects of
displayed between the dyads. Wilson and the clinical context change from rotation to
colleagues41 have also advocated for the use of rotation); an assessment approach designed
models, such as the multi-level Rasch model, which specifically for coupled performances would allow
consider both unidimensional and observation and interpretation of such changes and
multidimensional analyses when conducting adjust this information as the configurations of
assessments within collaborative environments. coupling change.
Wilson, Gochyyev and Scalise41 used data from
partners collaborating in an online learning Limitations
environment to show that roughly 90% of total
variance was explained by groups. These studies As with any constructivist grounded theory, the
from educational measurement provide early findings are a product of the context within which
approaches for assessing skills such as collaboration the study was conducted. In our context, the
in ways that capture aspects of both independent findings are representative of a single medical
and interdependent dimensions of performance school; others will need to explore the resonance of
and characterise performance along a spectrum coupling in their own settings. Our sampling
rather than creating a false dichotomy. Within strategy of interviewing a few supervisors and
medical education, a variation of the Rasch trainees from a wide range of programmes has
measurement model proposed by Wilson and allowed us to describe the phenomenon of
colleagues has already been used to capture aspects coupling. However, we cannot yet describe
of rater’s collaborative performance.42 All of these meaningful variations in particular clinical contexts
aforementioned approaches require us to consider and postgraduate programmes, and purposeful
the collective in order to meaningfully assess sampling will be required in future studies to
authentic clinical performance. pursue such insights. Additionally, our sample
included a subset of the many postgraduate
Our study advocates for a shift towards assessment programmes in which trainees perform clinical
that (i) is more precise about when trainee work; as systematic inquiry into coupling continues,
performance is independent and (ii) can account we expect the transferability of the concept to be
for coupling within trainee performance. To strengthened by examples and exceptions from a
illustrate the implications of coupling for wider variety of trainee performance contexts. Given
assessment, consider the example of trainee the nature of our data, this paper presents the
ordering practices as an outcome variable. An simplest configuration of interdependence:
approach that conceptualises this outcome as coupling between dyads. Future research will need
coupled would take into account the influence of to elaborate an increasingly sophisticated
particular faculty members’ supervisory practices on description of the configurations of coupling that
the extent to which a trainee order is an shape trainee performance in clinical workplaces.
independent decision or a coupled one. It would
also help to determine whether the clinical action
could be fairly attributed to a trainee or whether CONCLUSION
the contribution of the trainee to the final outcome
represents a more accurate reflection of the The concept of collective competence entered
conditions under which the trainee performs. medical education discourse in 2008; however, our
Furthermore, these conditions are not limited to field has yet to find a meaningful way to translate
supervisor and trainee; other team members, this into assessment practices. The concept of
clinical resources and medical protocols could also coupling provides a way forward, as it helps us
contribute to the interdependence of clinical begin to map the landscape ‘in between’
practice. Therefore, assessment approaches would independent performance and collective
need to incorporate not only data about the performance and to think more purposefully about
trainee, but also data about which faculty member the constructs we intend to measure. To date, the
was working with the trainee and where and when assessment of independent clinical performance has

978 ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd;
MEDICAL EDUCATION 2018 52: 970–980
 The question of coupling

been fraught with confounding factors such as nonparticipant observational study. Acad Med
unexplained variance; we suggest that this ‘noise’ 2017;92:792–9.
might, in some cases, be a signal of coupling. With 6 Goldszmidt M, Dornan T, Lingard L. Progressive
this premise, we propose that educators use the collaborative refinement on teams: implications
concept of coupling to better distinguish for communication practices. Med Educ 2014;48 (3):
301–14.
independent from interdependent trainee
7 Kennedy TJT, Regehr G, Baker GR, Lingard L.
performances so that we begin to develop ways to
Point-of-care assessment of medical trainee
assess interdependent performances as
competence for independent clinical work. Acad Med
interdependent. If we can develop and employ 2008;83:S89–92.
assessment approaches that more accurately 8 Kennedy TJT, Lingard L, Baker GR, Kitchen L,
measure trainees’ coupled performance in Regehr G. Clinical oversight: conceptualizing the
authentic clinical environments, it will have relationship between supervision and safety. J Gen
profound implications for competence judgements Intern Med 2007;22:1080–5.
in postgraduate training. 9 Dijksterhuis MGK, Voorhuis M, Teunissen PW,
Schuwirth LWT, ten Cate OTJ, Braat DDM, Scheele F.
Assessment of competence and progressive
Contributors: SSS contributed to the conceptual design, independence in postgraduate clinical training. Med
acquisition, analysis and interpretation, and drafted the Educ 2009;43 (12):1156–65.
work; SC contributed to conceptualisation and analysis, 10 Gofton WT, Dudek NL, Wood TJ, Balaa F, Hamstra
and revised the work; CJW contributed to the SJ. The Ottawa surgical competency operating room
conceptualisation and interpretation, and revised the evaluation (O-SCORE): a tool to assess surgical
work; MG and SC contributed to the interpretation and competence. Acad Med 2012;87:1401–7.
revised the work; LL contributed to the conceptual 11 Sebok-Syer SS, Klinger DA, Sherbino J, Chan TM.
design, analysis and interpretation, and drafted the work. Mixed messages or miscommunication? Investigating
All authors gave final approval of the submitted paper the relationship between assessors’ workplace-based
and agree to be accountable for all aspects of the work. assessment scores and written comments. Acad Med
Acknowledgements: The authors wish to acknowledge the 2017;92:1774–9.
faculty members and residents who participated in this 12 Eva KW. On the generality of specificity. Med Educ
study. 2003;37 (7):587–8.
Funding: (i) Schulich School of Medicine & Dentistry 13 Gingerich A, Ramlo SE, van der Vleuten CPM, Eva
Dean’s Research Innovation Grant; (ii) Academic Medical KW, Regehr G. Inter-rater variability as mutual
Organization of Southwestern Ontario (AMOSO) disagreement: identifying raters’ divergent points of
Innovation Fund. view. Adv Health Sci Educ Theory Pract 2017;22:819–
Conflicts of interest: Nothing to disclose. 38.
Ethical approval: This study was approved by the 14 Sebok SS, Roy M, Klinger DA, De Champlain AF.
institutional Health Sciences Research Ethics Board (File Examiners and content and site: oh My! A national
Number: 108391). organization’s investigation of score variation in large-
scale performance assessments. Adv Health Sci Educ
Theory Pract 2015;20:581–94.
REFERENCES 15 Gingerich A, Regehr G, Eva KW. Rater-based
assessments as social judgments: rethinking the
1 Gingerich A. What if the ‘trust’ in entrustable were a etiology of rater errors. Acad Med 2011;86:S1–7.
social judgement? Med Educ 2015;49 (8):750–2. 16 Govaerts M, van der Vleuten CPM. Validity in work-
2 ten Cate O. Entrustability of professional activities based assessment: expanding our horizons. Med Educ
and competency-based training. Med Educ 2005;39 2013;47 (12):1164–74.
(12):1176–7. 17 Kogan JR, Conforti LN, Iobst W, Holmboe ES.
3 Touchie C, ten Cate O. The promise, perils, problems Reconceptualizing variable rater assessments as both
and progress of competency-based medical education. an educational and clinical care problem. Acad Med
Med Educ 2016;50 (1):93–100. 2014;89:721–7.
4 Hauer KE, ten Cate O, Boscardin C, Irby DM, Iobst 18 Watling C, LaDonna KA, Lingard L, Voyer S, Hatala
W, O’Sullivan PS. Understanding trust as an essential R. ‘Sometimes the work just needs to be done’: socio-
element of trainee supervision and learning in the cultural influences on direct observation in medical
workplace. Adv Health Sci Educ Theory Pract training. Med Educ 2016;50 (10):1054–64.
2014;19:435–56. 19 Olupeliyawa AM, O’Sullivan AJ, Hughes C,
5 Sagasser MH, Fluit CRMG, van Weel C, van der Balasooriva CD. The teamwork mini-clinical
Vleuten CPM, Kramer AWM. How entrustment is evaluation exercise (T-MEX): a workplace-based
informed by holistic judgments across time in a family assessment focusing on collaborative competencies in
medicine residency program: an ethnographic health care. Acad Med 2014;89:359–65.

ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd; 979
MEDICAL EDUCATION 2018 52: 970–980
S S Sebok-Syer et al

20 Bleakley A. Broadening conceptions of learning in 33 Orton JD, Weick KE. Loosely coupled systems: a
medical education: the message from teamworking. reconcepualization. Acad Manage Rev 1990;15:203–23.
Med Educ 2006;40 (2):150–7. 34 Lingard L, McDougall A, Levstik M, Chandok N,
21 Bleakley A. Blunting Occam’s razor: aligning medical Spafford MM, Schryer C. Using loose coupling theory
education with studies of complexity. J Eval Clin Pract to understand interprofessional collaborative practice
2010;16:849–55. on a transplantation team. J Res Interprof Pract Educ
22 Hodges BD, Kuper A. Theory and practice in the 2014;3:1–17.
design and conduct of graduate medical education. 35 Kennedy TJT, Lingard LA. Questioning competence:
Acad Med 2012;87:25–33. a discourse analysis of attending physicians’ use of
23 Mann KV. Theoretical perspectives in medical questions to assess trainee competence. Acad Med
education: past experience and future possibilities. 2007;82:S12–5.
Med Educ 2011;45:60–80. 36 Goldszmidt M, Faden L, Doran T, van Merrienboer J,
24 Fenwick T. Sociomateriality in medical practice and Bordage G, Lingard L. Attending physician variability:
learning: attuning to what matters. Med Educ 2014;48 a model of four supervisory styles. Acad Med
(1):44–52. 2015;90:1541–6.
25 Goldszmidt M, Aziz N, Lingard L. Taking a detour: 37 Oxford English Dictionary. Oxford English Dictionary
positive and negative effects of supervisors’ Online; nd. Independence. http://www.oed.com/vie
interruptions during admission case review w/Entry/94323?redirectedFrom=independence#eid.
discussions. Acad Med 2012;87:1382–8. [Accessed 14 Augest 2017.]
26 McDougall A, Goldszmidt M, Kinsella EA, Smith S, 38 Hodges B. Assessment in the post-psychometric era:
Lingard L. Collaboration and entanglement: an actor- learning to love the subjective and collective. Med
network theory analysis of team-based Teach 2013;35:564–8.
intraprofessional care for patients with advanced 39 van Melle E, Gruppen L, Holmboe ES, Flynn L,
heart failure. Soc Sci Med 2016;164:108–17. Oandasan I, Frank JR, International Competency-
27 Fenwick T, Abrandt Dahlgren M. Towards socio- Based Medical Education Collaborators. Using
material approaches in simulation-based education: contribution analysis to evaluate competency-based
lessons from complexity theory. Med Educ 2015;49 medical education programs: it’s all about rigor in
(4):359–67. thinking. Acad Med 2017;92:752–8.
28 Paradis E, Whitehead CR. Louder than words: power 40 Andrews JJ, Kerr D, Mislevy RJ, von Davier A, Hao J,
and conflict in interprofessional education articles, Liu L. Modeling collaborative interaction patterns in
1954-2013. Med Educ 2015;49 (4):399–407. simulation-based task. J Educ Meas 2017;54:54–69.
29 Charmaz K. Constructing Grounded Theory: A Practical 41 Wilson M, Gochyyev P, Scalise K. Modeling data from
Guide Through Qualitative Analysis. London, UK: Sage collaborative assessments: learning in digital interactive
Publications 2006. social networks. J Educ Meas 2017;54:85–102.
30 Dey I. Grounding Grounded Theory: Guidelines for 42 Yeates P, Sebok-Syer SS. Hawks, doves, and Rasch
Qualitative Inquiry. London, UK: Academic Press 1999. decisions: understanding the influence of different
31 Varpio L, Ajjawi R, Monrouuxe LV, O’Brien BC, Rees cycles of an OSCE on students’ scores using many
CE. Shedding the cobra effect: problematising facet Rasch modeling. Med Teach 2017;39:92–9.
thematic emergence, triangulation, saturation, and
member checking. Med Educ 2017;51:40–50. Received 4 October 2017; editorial comments to author 14
32 Thurmond VA. The point of triangulation. J Nurs December 2017, 8 February 2018; accepted for publication 20
Scholarsh 2001;33:253–8. February 2018

980 ª 2018 The Authors. Medical Education published by Association for the Study of Medical Education and John Wiley & Sons Ltd;
MEDICAL EDUCATION 2018 52: 970–980