0% found this document useful (0 votes)
138 views8 pages

Evolution of Audio Recording in Field Surveys: Abstract

1. Computer audio-recorded interviewing (CARI) uses digital audio recordings to monitor field survey interviews for quality assurance. Advances in technology now allow inexpensive and routine audio recording of full interviews on laptop computers. 2. CARI provides a record of interviews that can be used to detect falsification of data and evaluate interviewer and questionnaire performance. It also acts as a deterrent to curbstoning by signaling to interviewers that their work may be monitored. 3. Reviewing audio recordings allows monitors to check for indicators of questionable authenticity like unnatural response patterns or silence, aiding quality assurance efforts. CARI provides a less costly alternative to traditional quality control methods like re-interviews.

Uploaded by

Nilson Evilásio
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
138 views8 pages

Evolution of Audio Recording in Field Surveys: Abstract

1. Computer audio-recorded interviewing (CARI) uses digital audio recordings to monitor field survey interviews for quality assurance. Advances in technology now allow inexpensive and routine audio recording of full interviews on laptop computers. 2. CARI provides a record of interviews that can be used to detect falsification of data and evaluate interviewer and questionnaire performance. It also acts as a deterrent to curbstoning by signaling to interviewers that their work may be monitored. 3. Reviewing audio recordings allows monitors to check for indicators of questionable authenticity like unnatural response patterns or silence, aiding quality assurance efforts. CARI provides a less costly alternative to traditional quality control methods like re-interviews.

Uploaded by

Nilson Evilásio
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Evolution of Audio Recording in Field Surveys

M. Rita Thissen, Sridevi Sattaluri, Emily McFarlane, and Paul P. Biemer


RTI International, Research Triangle Park, NC 27709

Abstract: earliest in-person surveys. Traditionally, field staff


have worked largely unobserved, with occasional
The tools of field survey administration change quickly. shadowing by supervisory personnel or re-contacting
By taking advantage of new technology and adapting it the respondent to confirm the interview’s authenticity
for time-honored needs, survey managers can boost the and inquire about the professionalism of the interviewer.
effectiveness, efficiency and quality of data collection.
One method which has evolved rapidly is computer It can also be difficult to evaluate or confirm the
audio-recorded interviewing (CARI), an approach to effectiveness of questionnaire items in field surveys,
ensuring the quality of data through unobtrusive whether from a usability perspective, such as the ability
recording by the computer of the audio portion of in- of interviewers to read the questions in a fluent and
person interviews, much as silent monitoring has been understandable manner, or from the perspective of
used to ensure quality at call centers. clarity for the respondent, such that the response
provides the desired information without the need for
Several developments in the past few years have explanation or probing. While focus groups or
improved the technical feasibility of CARI for routine cognitive interviews in advance of data collection may
and inexpensive use in field studies. Advances in file offer insight into presentation and response patterns, the
compression and available bandwidth enable collection practice does not fully anticipate field conditions.
of longer recordings with little strain on transmission
capacity and no burden to the interviewer. Use of a Now the situation has changed. Many computers now
simple external file for specifying items to be recorded have audio recording capabilities, and some have built-
in a Blaise instrument offers great flexibility in selecting in microphones. With this technology, surveys can be
portions of the interview for auditing, even permitting set up to collect digital audio recordings in an
modification of the recorded-item list while an unobtrusive manner while the interview is taking place.
instrument is in production. A web-based monitoring With computer audio-recorded interviewing (CARI),
application, for use by trained reviewers in evaluating sound files can be created electronically without the
the audio files, can now provide access to centrally need for external equipment and can be transmitted
located audio files by geographically distributed staff. along with response data files and tracking information.
Because the recording process is “invisible,” once
Progress has also been made from an operational consent has been given, it can provide a faithful
viewpoint. Work has been done to determine the representation of the reality of in-person data collection.
minimum amount of recording needed to achieve
agreement among reviewers as to the authenticity of the The technology provides a potent tool for deterring and
recorded session, and cost modeling shows that CARI detecting falsification, providing performance feedback
can provide quality assurance at equal or reduced costs and enabling study of questionnaire item effectiveness.
compared to more traditional approaches of re-interview
or telephone verification. 2. Audio Recording Technology, Past and Present

Use of CARI on several national surveys has provided From the marketing of the Dictaphone in 1907 (Nuance
production experience to bolster laboratory tests. This Communications 2005) to the availability of miniature
article reviews the progress of CARI technology in the recorders embedded in portable electronic devices today
years since it was introduced, with an emphasis on (Dwyer et al,1998), people have been discovering ways
feasibility for routine use with field surveys. to take advantage of audio recording tools to capture
voices for later review. While the early acoustic
Key Words: survey technology; audio recording; recorders proved helpful for journalistic interviews, they
computer audio-recorded interviewing (CARI); sound were not usable for large-scale research surveys; the
file; quality assurance; performance management; field introduction of cassette tapes improved convenience for
interview; in-person interview interviewing (Stockdale, 2002).

1. Introduction With the advent of digital recording, and as computers


Monitoring the performance of field staff and the began offering built-in sound cards, the task of
quality of data collection has been challenging since the capturing audio records became easier and sound
quality improved. Sound files now can be recorded • Placement of the microphone with respect to noise-
electronically through sound cards and software on producing hardware (keyboard, fan and disk drive)
laptops, handhelds and other portable devices, making • Placement of the microphone with respect to the
this technology handy for use in field surveys. interviewer and respondent.
• Microphone control settings.
Figure 1. Milestones of audio recording history Some internal microphones are adequate to capture
Audio Storage Invention Widespread voices within 8 feet or so of the laptop when configured
Use properly, at a quality level that allows a listener to
Wax cylinder 1885 Early 1900’s to distinguish among multiple voices and discern the
1940’s spoken content.
Magnetized wire 1898 1940’s
Magnetic coatings 1928 1930’s to 3. Audio Recording for Quality Assurance
on plastic tape present
Compact cassette 1963 1960’s to 1990’s Although there are several advantages offered by
Pulse code 1937 1990’s implementing CARI, perhaps the most compelling
modulation reason is to confirm the authenticity of data for a
Digital audio 1971 1990’s reduced cost compared to traditional verification
microprocessor methods. CARI can act as a deterrent to curbstoning and
SoundBlaster audio 1989 1990’s as a tool for detecting questionable interviews.
card Interviewers who are aware that monitors may listen to
First use of CARI 1999 Present parts of each interview may be less likely to falsify data,
Portable digital 2003 Present because the audio file acts as a “witness” to their
voice recorders actions. In this way, the simple presence of CARI can
reduce cheating.
In 1999, use of digital audio recording was first
developed and deployed on a national field survey, as Speech patterns heard in audio files provide information
the result of innovative work by RTI developers R. to the monitors about the veracity of the interview, as
Suresh, A. Bethke and P. Cooley. Use of CARI has indicated by the timing and phrasing of questions and
grown since then, as the feasibility and utility of the responses. In a normal interchange, people pause
approach have been confirmed. Electronic recording between words, phrases or sentences, as they consider
requires little attention during the interview, as there are their answers or express their views (Kowal et al, 1975;
no tapes to change, no additional equipment to set up O’Connell and Kowal, 1983).
and no distraction during the interview. Feedback from
respondents and interviewers indicates that most people Figure 2. Indicators of questionable authenticity
forget about digital recording when the microphone is Silence No voices can be heard, although room
internal, once the interview gets underway. noises and key clicks are audible
Mumbling The interviewer can be heard, but appears
Many laptops now have built-in microphones, sound to be speaking to him or herself
cards and adequate disk space for conducting audio Unnatural The respondent answers too quickly or
recording. Handheld digital recorders from some patterns laughs in inappropriate places
companies offer audio recording capabilities but Comments The respondent or interviewer makes
function much like analog tape recorders, requiring the comments suggesting the interview is
user to switch them on and off manually. A few being falsified
handhelds offer programmed recording capabilities plus Same The same respondent’s voice is heard in
an internal microphone. Most laptop and many handheld voice multiple interviews or does not match the
computers allow use of an external microphone instead stated sex or age of the respondent.
of the internal one, for improved audio fidelity.
However, the visible hardware calls attention to the For example, when an interviewer acts alone and
recording process and may be more likely to affect the falsifies data, there may be no voice at all in the
respondent’s and interviewer’s behavior. recording or only one voice without the expected
pausing, inflection or clarity of voice which would be
Audio fidelity from any device depends on a number of expected in a two-way exchange. If the interviewer
factors. When using a laptop with internal microphone, enlists someone to pose as the respondent, the
these include accomplice may display inappropriate attitudes or
emotions, make unexpected remarks, respond without
pausing to understand the question or pause at unnatural interviewer’s presentation of an item and the subject’s
places while inventing an answer. CARI monitors listen response provides a clear indication of whether the item
to the recordings, and quickly become adept at succeeds in several ways:
distinguishing between recordings of normal • Readability – based on the interviewer’s fluency in
interviewing circumstances and suspicious ones, by presenting the item
listening for characteristics such as those in Figure 2. • Clarity of content – based on the respondent’s ease
(Thissen and Rodriguez, IBUC 2004) of understanding
Survey items which evoke negative reactions or require
Using CARI, a survey organization may reduce its re- frequent explanations are detrimental to the response
contact efforts and costs. CARI monitoring may replace rate and increase the level of burden on both interviewer
most telephone verification calls or field re-interview. and subject. Using CARI, especially during field testing
However, it remains important to have a second means of an instrument, allows the survey specialist to evaluate
for following up a small sample of the cases since some the success of the questionnaire items in eliciting the
respondents may refuse to allow audio recording, and desired information.
interviewers may attempt to use that option to prevent
detection of poor interviewing habits or curbstoning. 5. Privacy, Security, Consent and Legalities
The benefits of CARI plus optional re-contact are
twofold: it tells the interviewing staff that they cannot For CARI to be used during an interview, participants
avoid monitoring even if they discourage their must give express consent for the interview to be
respondents from allowing CARI, and it allows recorded. Respondents are told that their participation is
comparison of the two approaches to confirm the voluntary and that their information and responses are
validity of the results. confidential and will only be used for statistical
purposes. In two national field studies using CARI,
4. Data Collection Methodology approximately 83% of respondents in one survey on a
highly sensitive topic agreed to allow the interview to
Another benefit of CARI is that it provides a method for be recorded, and 93% of respondents on another less
identifying questionnaire problems and data collection sensitive survey agreed. (Wrenn-Yorker and Thissen,
difficulties in interviewer-respondent interactions. Field FedCASIC 2005). For those who do not allow
staff do not always conduct interviews in an optimal recording, traditional verification methods such as
manner, and it can be difficult to obtain reliable telephone verification interviews are used.
information about their performance. While personal
observation can provide a wealth of information, the All survey data, including CARI recordings, must be
presence of an observer may bias the evaluation. CARI safeguarded. In addition to design considerations based
offers a unique opportunity to listen to the interview on user needs, careful attention must be paid to security
exactly as it took place, without observation effects. and privacy issues when dealing with human data. In
the United States, laws and regulations direct the
During the first few weeks in the field, feedback can be management of personal identification information,
an important tool for reinforcing lessons learned during health records and other specific types of data.
training. A CARI monitor may be able to provide Computing professionals must be aware of federal, state
feedback, either praise or constructive criticism, about and local requirements for confidentiality in storage,
the way in which the interview was conducted. transmission and release of personal information.
Improper question administration which can be detected Institutional review boards oversee all research data on
through CARI includes human subjects, to ensure that the studies contribute to
• Paraphrasing the greater good without harming individuals. To
• Improper probing comply with guidelines and regulations, information
• Suggested responses systems may need to include authentication
• Poor enunciation mechanisms, audit histories and user records. These are
• Improper commentary regulatory rather than usability requirements but are
CARI can be used to identify positive behavior such as essential components of survey information systems.
• Precise adherence to protocol
• Adept handling of difficult situations Given heightened consciousness of confidentiality and
• Consistency, honesty, and professional behavior security concerns, care is required in handling audio
recordings. Even though the survey may not deliberately
CARI can also be used to evaluate the usability of record personally identifying information, it cannot be
questionnaire items. The audio recording of an guaranteed to avoid it. For this reason, audio files are
best treated as sensitive data, much the way response
data is handled. Encryption may be desirable for digital sampling rate and a single channel. Recording two
recordings, and special care may need to be taken in channels (stereo) would require twice the storage space
handling tapes, if the recording is by analog device. and provides no extra quality since a single microphone
is generally used. Audio quality also is affected by
6. Audio File Formats sampling rate, compression and audio file format, and
the settings given above are minimal for useful files.
Digital audio recording can take place with various
levels of sound quality, and the resulting files may be To reduce the space for audio files, a compression
stored in various electronic formats. The sound technique may be used. Coder-decoder algorithms
recording algorithm affects the following: (CODECS) offer ways to store recordings in less space.
• Audio file size and storage requirements They eliminate silence and mathematically map the
• Required software for recording and playback sampled analog sound frequencies instead of preserving
• Quality of sound on playback the actual data points. CODECS may be employed as a
• Platform requirements and CPU demands post-processing step after creation of the sound file or as
• Cost and licensing issues. a real-time action at the time of recording.

Many audio file formats have been developed over the CODECS (compression–decompression techniques)
years, and their sheer variety may seem baffling to the were developed for use with audio recording, to reduce
new observer. Recent attention has been given to mp3 the size of sound files. It is possible for audio recording
(Motion Picture Group Audio Layer 3) format, but to combine the digitizing process and compression at
many other file formats exist as well. A few of the once. For use in surveys, the system designer can
common formats are listed in Figure 3. choose among simple recording with no compression,
simultaneous recording and compression or recording
Microsoft Windows operating systems include Sound followed by compression. Section 11 discusses these
Recorder software which writes to the wave file format, approaches in a comparison of post-recording
and the Windows Media Player which can play back compression with simultaneous recording and
wave files and a number of other non-proprietary compression
formats. The PCM (pulse code modulation) digital
recording algorithm is used in various encoders 7. Integrating Audio Recording with Survey
including Sound Recorder, and records uncompressed Software
sound with no required licensing.
A variety of technologies have been in use to implement
Figure 3. Common audio file formats survey instruments, such as Blaise (Statistics
Name File Use Netherlands), CASES (University of California,
Extension Berkeley) and web-based technologies like ASP.NET
Wave .wav Windows (Microsoft). Audio recording components have been
uncompressed successfully incorporated in all these environments.
One of the challenges of incorporating audio recording
MP3 .mp3 Compressed audio
in a survey instrument is to make the process
RealMedia .rm Compressed audio
unnoticeable to the interviewer. The recording process
RealAudio .ra Compressed, for
must not slow the system or provide any visual or
streaming audio
audible clue as to when it starts and stops.
AIFF .aiff Macintosh default
uncompressed Audio recording can be added to Blaise instruments by
CD Audio .cda Music CD tracks using either of two programming approaches. One
Active Streaming .asf Streaming audio approach uses a Blaise procedure which in turn invokes
Format an external application to start and stop the recorder.
Using this approach requires complex programming
Wave files are not especially efficient at storage, but the within Blaise in every place the recording application
recording process places little demand on the computer. needs to be invoked, to keep track of whether recording
The size of a particular wave file depends on the is already in progress or needs to be started or stopped
recording parameters selected in its creation. For each (Thissen and Rodriguez, IBUC 2004).
available audio file format, there is a choice of sampling
rate, bandwidth, number of channels and other The second approach uses the Blaise alien router.
parameters. For RTI’s current CARI system, the Starting with version 4.6, Blaise introduced the alien
standard configuration is 16 bit bandwidth, 11.25 KHz router as part of the Blaise component pack. The alien
router technology allows the invocation of an external dialog. (See Section 11 below for a comparison of
component before and after every survey item. Use of recording parameters and file sizes.) Assuming an
the alien router externalizes the complexities of tracking instrument were programmed to collect three one-
the recorder state. It also opens up the possibility of minute recordings which were compressed to 100KB
maintaining a text list of items to be recorded, external each, the case management system would have 300KB
to the instrument. This reduces the complexity of to transmit for every case. If the interviewer transmits
instrument programming and allows easy modification one case each day, these files can be sent using a dialup
of the list of items to be recorded, without any need to connection. The use of broadband allows transmitting a
modify the data model or recompile the instrument larger number of files or larger size files at a faster rate.
(Thissen and Sattaluri, 2006b).
The third option, using removable external media and
For CASES instruments the recording can be integrated shipment, can be used when entire interviews or lengthy
by spawning a separate application to start and stop an sections are recorded. However, security concerns, the
external recorder (Wrenn-Yorker and Thissen, 2005). effort of handling external media and the possibility of
loss make this approach less desirable than automatic
When a survey is offered in multiple modes by using a transmission via dialup or broadband. Still, it may
web-based instrument, field interviewing may take prove useful when recording interviews in their entirety
place through a website running on the laptop without or when other forms of file transfer are not available.
continuous connection to the internet. In that case, the
audio recording component can be achieved by Audio recordings may contain personal identifying
installing a client side Java applet and Java scripting, information, whether by intention or by accident, and so
similar to the way in which CARI can be implemented it is important to protect these files by using encryption
for internet-based surveys (Suresh, 2005) tools while they reside in any location accessible to
unauthorized individuals. In addition, if files are
Once a survey instrument has been enabled with CARI transferred over the internet, secure socket layer (SSL)
technology, survey information systems (Thissen, 2004) certification can be used, which provides a way to
must also be expanded to handle the audio data files. encrypt the data stream during transmission.
From a case management and data security perspective,
CARI files are no more than response data stored in a 10. CARI Monitoring
different format. Issues and concerns are the same for
files containing audio response data as they are for files After audio files are received at a central location, the
of textual responses. File protection on the laptop, monitoring process may be as simple as opening up the
transmission to a central site, central storage, access by files using a free player tool like Windows Media Player
authorized researchers and eventual deletion must all be or Real Player. However, since manual case
planned with the same security and confidentiality used management is impractical for all but the smallest of
for traditional response files. surveys, it is best to build a system that provides an
interface for reviewing the files and a database for
9. Transmission recording evaluations.

There are several options for transferring audio files The monitoring system might be a client-server
from the field laptop to a central management system. application or a browser-based application located on an
The files can be sent using dialup transmission, internal or external network. Client-server applications
broadband, or removable media like flash drives restrict access to an organization’s internal network and
shipped by secure delivery methods. For small surveys, locally-located users, due to poor performance of
it may be practical to leave audio files on the laptops database connections over long distance. A web-based
until the end of data collection. With the pervasiveness approach has advantage of being available from any
of broadband access at homes through cable modem or workstation which has access to the network, supporting
DSL (digital subscriber line telephone service), the organizations with review staff distributed nationally or
capacity for transmitting large files has greatly even internationally (Thissen and Sattaluri, 2006a).
increased. Still, researchers must plan for transmission
when using CARI, since audio files can be large. Regardless of the implementation, it should provide
role-based access to protect the security of the
The choice of transmission option may depend on the information stored in the audio files. For example,
size of files being transmitted. It is found that three levels of access might be designed into the system:
uncompressed audio recording consumes about one • CARI monitoring staff, who listen to and
megabyte of disk space for each minute of recorded evaluate audio files
• Supervisory staff, who designate monitors, At RTI, files are recorded with the Windows native
manage caseloads and track review-completion Sound Recorder software called from Blaise or CASES,
status resulting in file sizes of about one MB/minute
• System administrators who configure new uncompressed. Use of the LAME (The LAME Project)
surveys and create new logins and passwords. open source compression algorithm and appropriate
parameters yields an average compression ratio of
For large surveys, the system may also include an approximately 11:1 without loss of audio quality,
algorithm to select a specified percentage of files to be resulting in about 100KB files for one minute of audio.
reviewed per interviewer. Ideally, it would offer the
flexibility to adjust review rates for any field Figure 5. File sizes obtained by concurrent recording
interviewer for any active survey, so that quality and compression
assurance personnel can increase monitoring of any Input Number Average
interviewer who has been suspected of improper data CODEC Sound Of Files MB/Min
collection practices. (Hartman et al, 2006) Quality Tested
MPEGRec Low 4 0.98
11. Audio and Operational Results MPEGRec Mod 3 1.68
MPEGRec V.High, 2 0.96
In this section, we present some results of RTI’s Mono
experience with CARI technology. The data given MPEGRec V.High, 1 1.80
below were obtained by lab test, field test and Stereo
production survey use of CARI processes. RealMedia Low 24 0.34
RealMedia Mod 3 0.51
A comparison of recording alternatives is shown in RealMedia V.High, 2 0.34
Figure 4, with an indication of the resulting playback Mono
sound quality. The column labeled “MB Per Min” lists RealMedia V.High, 1 0.47
the number of megabytes of storage required for one Stereo
minute of sound when using the uncompressed wave
file format. Similar patterns of relative file size can be In another experiment, we recorded sound directly to a
found for other file formats. compressed format, without intervening storage as a
wave file. In a CARI system, this requires the
Figure 4. Recording parameters instrument to call a specific recording application and
Band- Sampling Chan- Sound MB Per CODEC, such as MPEGRec (mp3), producing a
width nels Quality Min compact file that is ready to encrypt and transmit. The
8 bit 11.25 KHz 1 Low 0.66 simplicity of this approach was attractive because
16 bit 11.25 KHz 1 Medium 1.31 compression was immediate and effective, as shown in
8 bit 22.5 KHz 1 Medium 1.79 Figure 5. On the down side, simultaneous compression
16 bit 22.5 KHz 1 High 1.19 and recording tax the computer’s processing power.
16 bit 44.1 KHz 1 Very 5.25 This reduces system performance, produces lag and
High visible indication of recording processes, and limits its
16 bit 44.1 KHz 2 Very 12.3 usefulness.
High
Figure 6. Loudness Effect on File Size
We have looked at alternative processes for File Sound Averaged MB Per
compressing existing audio files. A wave file was Format Level Over # of Minute
compressed as a separate step after recording, using a Files
specific CODEC and selected recording parameters. In Wave Silent 6 1.30
terms of a CARI system, this process might be Wave Quiet voice 9 1.31
performed by the case management system after the Wave Voice 6 1.32
interview was completed but prior to transmission. MP3 Silent 6 0.97
Using this approach, compression ratios ranged from a MP3 Quiet voice 8 0.96
factor of 2 to 75. In general, if the recording was of MP3 Voice 6 0.97
very high fidelity stereo, the original file would be very RM Silent 6 0.34
large and compress greatly. Lowering the recording RM Quiet voice 8 0.34
quality produces a smaller file originally but RM Voice 6 0.34
proportionally less compression.
We tested whether loudness had any effect on the size 8 shows the distribution of field performance problems
of the recorded output file by looking at the level of found in one study after review of approximately 5600
sound in audio files compared to file size, for CARI interviews. A single case might be assigned multiple
files which where all recorded under identical problem codes, and so the problem count total is greater
configuration settings on the same laptop. Figure 6 than the number of affected cases (Wrenn-Yorker and
shows the results of the comparison, demonstrating that Thissen, FedCASIC, 2005).
there was no apparent effect of loudness on audio file
size. In general, field interviewers and respondents have been
accepting of the technology. In a feedback study, 82%
The quality of the sound files from the field is of of interviewers felt neutral or positive about use of
interest, as an indicator of the feasibility of gathering CARI and a post-interview survey of 283 respondents
information for large numbers of interviews. Figure 7 found that 70% of the respondents reported they had no
shows results from reviewing a sample of 11% of the reaction one way or the other, 15% reported liking the
first 1500 completed interviews from a survey. The idea, while 13% disliked the idea (Herget et al, 2005).
asterisk (*) indicates that the default rating was chosen, As noted above, assent to CARI by respondents ranged
as opposed to an explicitly-defined score. Rating the from around 83% in one survey to 93% in another. This
file quality rating was optional through the monitoring assent was independent of consent to conduct the
interface if the quality was acceptable for review interview (Wrenn-Yorker and Thissen, 2005).
(Hartman et al, 2006).
A small experiment was conducted to determine the
Figure 7. CARI sound file quality distribution minimum number of CARI audio files required for
Sound Quality Number of Interviews making consistent monitoring evaluations, that is, how
1 – Poor 4 many audio files were required before reaching a point
2 – Passable 5 where listening to additional audio files for an interview
3 – Adequate 21 had no effect on the determinations. This work
* – Acceptable 48 suggested that three audio files each of 30-second
4 – Good 49 duration may be adequate for verification purposes.
5 – Excellent 37 After review of three files, CARI monitors reached 97%
agreement with the ratings found by review of five files,
Problems noted with audio files included background indicating that three files provide sufficient information
noise, static, faintness of voices, key tapping, hum and for evaluation purposes.
other recording problems which interfered with
detection of vocal content. Audio files were considered It is difficult to compare costs precisely between CARI
adequate if voices could be plainly heard and operations and more traditional re-interview or
understood, regardless of other noises. This definition verification processes, because the traditional systems
of quality differs from any commonly used to rate the tend to be well established while CARI systems are still
quality of audio recording for other purposes, such as evolving. A theoretical cost-analysis model was created
musical entertainment, but it is appropriate for survey to compare the expected costs of operating both systems
evaluation purposes. at the same “steady state” in which all systems had been
implemented. Analysis of that model suggests that the
Figure 8. Field performance problems detected steady-state cost of verification is less with CARI than
through CARI for the traditional approach, but actual data were not
Count % of Problem Definition available for that comparison.
Cases
13 0.2 Authenticity Questionable 12. Visions of the Future
217 3.9 Reading - Minor Deviation
Looking forward, we see expanded use of CARI in field
72 1.3 Reading - Major Deviation
surveys, for monitoring survey quality and also as an
73 1.3 Recording Errors
integral part of data collection. Advances in digital
44 0.8 Unprofessional Behavior signal processing may support automation of activities
86 1.5 Inappropriate Probing now being done by CARI monitors or coders.
79 1.4 Feedback not Neutral
1 0.01 Incorrect Incentive Provided With regard to data quality monitoring, it may be
possible one day to screen a large portion of the audio
We have also gathered operational information on field files automatically for evidence of falsification. For
staff performance from production use of CARI. Figure example, software may be able to distinguish between
audio files with and without voices and to identify the Herget, D., Biemer, P.P., Morton, J. and Sand, K.
number of differing voices within a single recording. (2005), “Computer Audio Recorded Interviewing
This technology could be employed for a population (CARI): Additional Feasibility Efforts of
census or large survey that requires many interviews to Monitoring Field Interview Performance”,
be screened very quickly for falsification. Audio Presented at Federal Conference on Statistical
processing software may be able to determine Methods.
respondent qualities such as whether a voice is male or Kowal, S., O'Connell, D.C. and Sabin, E.J. (1975)
female, or to match spoken interviewer words with the “Development of Temporal Patterning and Vocal
predefined question text, for evaluation of how well the Hesitation in Spontaneous Narratives”, Journal of
interviewer followed protocol. Psycholinguistic Research, Vol. 4, p. 195-207.
The LAME Project, LAME Compression Software,
CARI can also be used as a data collection tool. A http://lame.sourceforge.net/index.php
number of surveys tape record respondent responses Ming, J., Hazen, T.J. and Glass, J.R. (2006), "Speaker
that are subsequently coded, and CARI offers a Verification Over Handheld Devices with Realistic
convenient, unobtrusive alternative for collecting these Noisy Speech Data," Proceedings of the
recordings. Matching audio responses to a dictionary of International Conference on Acoustics, Speech, and
expected words might allow automated coding of open- Signal Processing, pp I-637 to I-640.
ended items or of an “other-specify” option of multiple- Nuance Communications, Inc. (2005), “About
choice items. Dictaphone”,
http://www.dictaphone.com/aboutus/history.asp
Farther in the future, recordings may be transcribed O’Connell, D.C. and Kowal, S. (1983), “Pausology”. In
automatically to text with can be parsed and analyzed. Computers in Language Research, Sedelow, W. A.
Current commercial software often requires “training” Jr. and Sedelow, S.Y. (eds), Berlin-New York:
the package to recognize the user’s voice, which limits Walter de Gruyter & Co., pp. 221-301.
usefulness in the field. However, research is underway Statistics Netherlands, Statistical Informatics
on speech-to-text conversion tools in uncontrolled or Department, P.O. Box 4000, 2270 JM Voorburg,
“noisy” surroundings (Ming, et al, 2006), which may The Netherlands.
broaden its applicability to include home environments. Stockdale, A. (2002), “Tools for digital audio recording
in qualitative research”, Social Research Update,
Acknowledgements pp 1-4
Suresh, R. (2005). “Web-Based Computer Audio
The authors would like to acknowledge the work of Recorded Interview (Web-CARI).” Presented at the
Albert Bethke, Phil Cooley and R. Suresh in the International Field Directors and Technology
invention of CARI, the contributions of Frank Mierzwa Conference 2005, Atlanta, GA
in cost modeling and of Pauline Robinson in file Thissen, M. R., and Rodriguez, G. (2004), “Recording
compression studies. Finally we would like to Interview Sound Bites Through Blaise
recognize the contributions of the U.S. Census Bureau Instruments”, Proceedings of the International
to the field of audio-recorded interviewing. Blaise Users’ Conference, pp. 411-423.
Thissen, M.R., and Sattaluri, S. (2006a) “Computer
References Audio-Recorded Interviewing (CARI)”, Presented
at The International Field Directors and
Biemer, P.P., Hergert, D., Morton, J. and Willis, W. Technologies Conference, Montreal
(2000), “The Feasibility of Monitoring Field Thissen, M.R, and Sattaluri, S. (2006b), “Research and
Interview Performance Using Computer Audio Development in Audio-Recorded Interviewing, Part
Recorded Interviewing (CARI)”, Proceedings of the II”, Presented at The International Field Directors
American Statistical Association’s Section on Survey and Technologies Conference, Montreal, Canada
Research Methods, pp. 1068-1073 University of California, Berkeley, Software Support
Dwyer; J.J, Godin, D.K., Colon, R.S., Sr., Rothschild, S. Services, “Computer-Assisted Survey Execution
Pawlowski, J.J., and Vaughan, J.C. (1998), “Voice System (CASES),” CSM Program, 358 Barrows
file management in portable digital audio recorder”, Hall #3820, Berkeley, CA 94720.
United States Patent 6671567 Wrenn-Yorker, C. and Thissen, M.R.(2005), “Computer
Hartman, P., Wrenn-Yorker, C., Sattaluri, S. and Audio Recorded Interviewing (CARI)
Thissen, M.R. (2006), “Research and Development Technology”, Presented at the Federal Computer-
in Audio-Recorded Interviewing”, Presented at Assisted Survey Information Collection
Federal Computer Assisted Survey Information (FedCASIC) Conference.
Collection (FedCASIC) Conference

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy