Pronunciation Assessment Annotation Guidelines .............................................................................

Objective ......................................................................................................................................... 2

Steps ................................................................................................................................................ 2

Pronunciation Tags ......................................................................................................................... 2

Transcription Requirements................................................................................................................ 3

US Spelling ...................................................................................................................................... 3

Casing .............................................................................................................................................. 3

Initialisms/Acronyms/Spelled-out Letters .................................................................................... 4

Titles/Honorifics ............................................................................................................................. 4

Punctuation .................................................................................................................................... 4

Numbers.......................................................................................................................................... 5

Times ............................................................................................................................................... 5

Contractions.................................................................................................................................... 5

Symbols/Special Characters ........................................................................................................... 5

Informal Words ............................................................................................................................... 6

Filler Words ..................................................................................................................................... 6

False Starts ...................................................................................................................................... 6

Truncations ..................................................................................................................................... 6

OK vs. okay...................................................................................................................................... 6

Spacing ............................................................................................................................................ 7

Side Searching ................................................................................................................................. 7

Transcription Tagging ..................................................................................................................... 7

Spelling Resources............................................................................................................................... 8

Dictionaries ..................................................................................................................................... 8
 Pronunciation Assessment Annotation Guidelines

Objective

Correct errors in the reference text and annotate words to highlight mispronunciations.

Steps

Reference text will be presented for each hit in the ‘Transcription’ box. Your job is to:

1. Listen to the audio and correct any word errors in the reference text

2. Go through the modified text to check for mispronunciations

3. Tag each mispronounced word with an <M> tag. You should place the tag after the
mispronounced word.

Pronunciation Tags

The M tag that you select (M1 through M5) will depend on the severity of the mispronunciation.
The higher the degree is, the more serious the mispronunciation is.

Mispronunciation Ranks Chat

M1 Is easy to understand throughout, has minimal effect on intelligibility.
M2 Can generally be understood throughout, though clarity reduced.
M3 Has caused some difficulty for the listener.
M4 Huge effort to understand.
M5 Unintelligible.
Examples:

If there is a very minor mispronunciation of “birthday”, the following would apply:

• it is my birthday <M1> today

If “birthday” requires a huge effort to understand, the following would apply:

• it is my birthday <M4> today

If the spoken word is unintelligible and it is therefore impossible to accurately transcribe the word,
you should retain the word in the reference text (even if clearly unsuitable) and follow it with the
<M5> tag. In this example, the reference has wrongly identified “many” as the unintelligible word:

• Reference text - today is going to be many fun

• Transcribe as - today is going to be many <M5> fun

If the audio is unintelligible but the reference text does not contain a word or words that can be
used, please make a best guess and tag the word(s) with <M5>.

Do not overuse the <M5> tag. It should only be used if a word is unintelligible or has been cut off
(see False Starts/Truncation guidance below).

Mispronounced words should be transcribed using their standard spelling. Do not try to capture
the way a word was spoken by modifying its normal spelling.

If there is no mispronunciation in the utterance, do not use the <M> tag.

You can select <M> tags by either right clicking on your mouse to bring up the dropdown menu or
by using the F shortcut keys to select the desired tag - F1 = <M1>, F2 = <M2>, etc.

Transcription Requirements

US Spelling

En-US spelling must be used.

Casing

Capital letters should be used to begin proper nouns. This includes names, places,
products, days of the week, months, etc. Capitals are also used for the personal pronoun
“I”, and for initialisms/acronyms/spelled-out letters (see guidance below).

Examples:

• I spoke to Gina yesterday

 • did you know that David is leaving town on Monday

A capital letter is only needed at the start of a transcription if any of the above applies.

Initialisms/Acronyms/Spelled-out Letters

Common initialisms such as CD, DVD, and PC should be transcribed with upper case
letters and no spacing – CD, DVD and PC. Plurals should be transcribed with the upper
case ‘S’ and no space - CDS, DVDS and PCS.

Uncommon initialisms, such as MDD, should be transcribed with letters separated by a

space - M D D. Plurals of uncommon acronyms should be transcribed as M D DS.

Acronyms that are pronounced as words should be transcribed with upper case letters
and no space - eg NASA and NATO.

Spelled out words should be formatted with upper case letters and a space – the correct
spelling for her name is G R A C I E.

Titles/Honorifics

Titles should not be abbreviated. “Mr” should be “mister”, “Dr” should be “doctor”, “Mrs” should
be “missus” etc. When used before a name as an honorific, the first letter should be capitalized.
When used without the proper noun, it should not.

Examples:

• I went to see the doctor today

• I went to see Doctor Adams today
• I don’t know about that mister
• I don’t know about that Mister Adams
• Sir Elton John is very talented
• Can I borrow a book sir
• It was nice to meet you ma’am

Punctuation

Punctuation is not required, other than apostrophes and hyphens.

Spoken punctuation should be transcribed as heard, rather than with symbols.

Examples:

• google dot com

• david at gmail dot com
Numbers

All numbers should be spelled out in full, as they are said. For instance, if a speaker says "104" as
"one oh four", this is what should be transcribed. Numbers should not be hyphenated.

Examples:

• five laptops arrived yesterday

• he did twenty two push ups
• she arrived at three oh five in the afternoon

Times

Time should be spelled out in full. If verbalized, “a.m.” and “p.m” should be spelled as such.

Examples:

• see you at eight a.m. tomorrow

• she is arriving at four thirty p.m.

Contractions

Contractions that are common should be written as one word. The following is a list of common
contraction endings. If it is unclear whether the second word is contracted, transcribe each word
fully. For example, if it is unclear whether the speech is "he's" or "he is", transcribe it as the latter.

'll for "will" or "shall"

've for "have"
n't for "not"
're for "are"
's for "is" or "has" or "us" (in "let's")
'd for "would"
'm for "am"

Symbols/Special Characters

Symbols/special characters should not be used.

Examples:

• I picked it up for $150 –> I picked it up for one hundred and fifty dollars
• we made a saving of 50% –> we made a saving of fifty percent
• 2 + 2 = 4 –> two plus two equals four
Informal Words

Informal pronunciations are fine if that is what is said.

Examples:

• do you wanna go to the park

• the party is gonna be fun

Filler Words

Filler words such as “um” and “uh” should be transcribed if they are obvious and
prominent. Ignore filler words if they are not obvious.

False Starts

If you encounter a false start or multiple false starts, transcribe the false start as a complete word
and use the <M5> tag to highlight that the word was not heard in full.

Example:

• Audio - I went to the bea bea bea beach

• Transcribe as - I went to the beach <M5> beach <M5> beach <M5> beach

Ignore minor noises/vocal stumbles that are not identifiable as a false start.

Truncations

If a word is cut off at the end of the audio but the word that the speaker intended to say is obvious,
transcribe the word using standard spelling and use the <M5> tag to highlight that the word was
not heard in full.

Example:

• Audio – he was so funny that I couldn’t stop laughi

• Transcribe as – he was so funny that I couldn’t stop laughing <M5>

OK vs. okay

In order that the team’s transcriptions are consistent, please use “OK”.
Spacing

There should be a single space between words and before and after <M> tags.

Side Searching

Make use of side searching to help with understanding context or identifying names of places,
people, trademarks, etc.

Transcription Tagging

No transcription tags are used for this project. The <M> tag is the only tag that you will use.
 Spelling Resources

Dictionaries

Dictionary Name Dictionary URL

Lexico https://www.lexico.com

American Heritage https://www.ahdictionary.com/

Dictionary

Merriam https://www.merriam-webster.com/

Cambridge Dictionary https://dictionary.cambridge.org/dictionary/essential-

(American English) american-english/

Dictionary.com https://www.dictionary.com/