Workshop 1

FETP-Intermediate

Data Planning, Management,

and Quality Control

FETPI2.0_d20_Data_Planning_Management_2020-06.pptx Version 2.0

 Learning Objectives
At the end of this lesson, you will be able to:
▪ Describe the purpose and components of a data
analysis plan
▪ Describe the process for becoming familiar with a
data set, particularly the variables and values in
the data set
▪ Create tables shells
▪ Conduct quality-control data checks, including
duplication and missing data

2 Epidemiologic Bias
 Introduction
▪ You have been given access to a data set and
you have been told to analyze it.

▪ How do you begin?

3 Epidemiologic Bias
 Steps in Approach to Data Analysis
▪ Familiarize yourself with the structure of the
data set
▪ Draft an analysis plan
▪ Create a working copy of the data set
▪ Look at the actual data in the data set
▪ Clean the data in the working file
▪ Revise and add detail to the analysis plan
▪ Create an analysis file
▪ Follow the analysis plan to analyze the data

4 Epidemiologic Bias
 Revealing the Structure of a Data Set
▪ How is the data set stored, e.g., ASCII, CSV,
Excel file, SAS file, MS Access / Epi Info file, etc.
▪ What are the variables in the data set?
▪ For each variable,
– What is its format?
– How is it coded? What do the codes mean?
Data dictionary would help
▪ How many records?

5 Epidemiologic Bias
Original VCT File (from Log Book to Excel)

– Date of visit – Type

– Client code – No. of contacts identified
– Sex – No. clients followed up
– Age – Variance
– Occupation – Comment
– Address – Total number of results
– Pretest counsel (Y/N), date for period
– Tested (Y/N), date – Number of results
– Test result returned within 21 days
– Posttest counsel (Y/N), date
– Date results returned to client
 Data Dictionary
▪ Critical to each dataset
▪ Enables anyone to become familiar with the
dataset
▪ List each variable with Example
– Variable name – DT-Visit
– Description – Date of clinic visit
– Variable type – Date (dd-mm-yyyy)
– Response options / – 01-Jan-2017 to present
Allowable values
– Instruction / Comment – Cannot be left blank
7 Epidemiologic Bias
 Example: Data Dictionary
Data source: Patient interviews
Comments: Data collected 16/03/2019 - 30/11/2019

8 Epidemiologic Bias
 What if no data dictionary available?
Create one!
▪ Variable name
▪ Variable description
▪ Variable type
▪ Permissible values
▪ Comments

9 Epidemiologic Bias
 Group A Streptococcus Outbreak
Questionnaire
In the last two weeks have you had any of the following symptoms?
Sore throat Yes No
Headache Yes No
Swollen lymph glands
around your neck Yes No
Fever Yes No
Muscle aches Yes No
Stomach ache Yes No
Vomiting Yes No
What day did your symptoms begin? _______________
(MM/DD)
Did you see a doctor regarding your illness? Yes No
Did the doctor take a culture? Yes No
If yes, what were the results? Positive Negative Unknown
Did you attend the School
Banquet held on 11 May? Yes No

10 Epidemiologic Bias
 Group A Streptococcus Outbreak
Questionnaire (Annotated)
In the last two weeks have you had any of the following symptoms? (Throat)
Sore throat Yes No (Headache)
Headache Yes No (Glands)
Swollen lymph glands
around your neck Yes No (Fever)
Fever Yes No (Aches)
Muscle aches Yes No (Stomach)
Stomach ache Yes No (Vomit)
Vomiting Yes No
What day did your symptoms begin? _______________
(MM/DD) (DT-Onset)

Did you see a doctor regarding your illness? Yes No

(MD)
Did the doctor take a culture? Yes No
(Culture)
If yes, what were the results? Positive Negative Unknown
(Results)
Did you attend the School
Banquet held on 11 May? Yes No (Banquet)

11 Epidemiologic Bias
 Steps in Approach to Data Analysis
▪ Familiarize yourself with the structure of the data
set
▪ Draft an analysis plan
▪ Create a working copy of the data set
▪ Look at the actual data in the data set
▪ Clean the data in the working file
▪ Revise and add detail to the analysis plan
▪ Create an analysis file
▪ Follow the analysis plan to analyze the data

12 Epidemiologic Bias
 Why Plan the Analysis?
▪ For existing dataset:
– What are the goals of the analysis?
– What can you analyze?
– Use analysis time efficiently
▪ For planned study, outbreak investigation:
– Assures data collection and database structure
are useful for your analysis
– Use analysis time efficiently

13 Epidemiologic Bias
 Maxims for Planned Studies

Do not design a study without knowing

how you plan to analyze the data!

Do not design a questionnaire without

knowing how you plan to analyze the
data!

14 Epidemiologic Bias
 Factors that Influence Analysis
▪ Goals of analysis / research questions
▪ Available variables
▪ Study design
▪ Sampling

15 Epidemiologic Bias
 What are the goals of the analysis?
▪ What are the questions you want to answer?
▪ For example,
– Time, place, and person (TPP) characteristics
of persons tested at VCT clinics
– Percent HIV positive overall, and by TPP
– Describe how quickly results were returned
(overall, by TPP and positivity, and % not
returned)
– Determine the proportion of results returned
within 21 days overall and by TPP

16 Epidemiologic Bias
 Basic Analysis Plan
▪ How to get from electronic data to final report
▪ Content
– Source(s) of data
– Available variables, created variables (data
dictionary)
– Key variables, e.g., exposure, outcome
– Sequence of analyses, tables, figures, with
• Measures of occurrence, e.g., incidence
• Measures of association, e.g., odds ratio
• Measures of precision, e.g., 95% CI
• Statistical tests, e.g., Chi-square

17 Epidemiologic Bias
 Steps in Approach to Data Analysis
▪ Familiarize yourself with the structure of the data
set
▪ Draft an analysis plan
▪ Create a working copy of the data set
▪ Look at the actual data in the data set
▪ Clean the data in the working file
▪ Revise and add detail to the analysis plan
▪ Create an analysis file
▪ Follow the analysis plan to analyze the data

18 Epidemiologic Bias
 Steps in Approach to Data Analysis
▪ Familiarize yourself with the structure of the data
set
▪ Draft an analysis plan
▪ Create a working copy of the data set
▪ Look at the actual data in the data set
▪ Clean the data in the working file
▪ Revise and add detail to the analysis plan
▪ Create an analysis file
▪ Follow the analysis plan to analyze the data

19 Epidemiologic Bias
 Before Analyzing Data…Look at the Data
▪ Look at each variable
– Range
– Frequency distribution (individual values)
– Percent missing

20 Epidemiologic Bias
DT-Visit — Frequency Distribution
DT-Visit Frequency Range:
01-Mar-14 1 01-Mar-14 to 7-Apr-17
02-Feb-15 1 Percent Missing:
11-Jan-16 1 0%
02-Jan-17 4 Frequency Distribution:
03-Jan-17 5 pre-2017 3
04-Jan-17 4 Jan 2017 118
… … Feb 2017 155
5-Apr-17 1 Mar 2017 309
Apr 2017 10
7-Apr-17 1
Total 595
Total 595
Partial Line listing — Sorted by DT-Visit
Rec. No. DT-Visit Clinic Type DT-pretest DT-test TestResult

298 01-Mar-04 PMTCT 01-Mar-07 01-Mar-07 Neg

141 02-Feb-05 VCT 02-Feb-07 02-Feb-07 Neg
27 11-Jan-06 PMTCT 11-Jan-07 11-Jan-07 Neg
1 02-Jan-07 VCT 02-Jan-07 02-Jan-07 Neg
2 02-Jan-07 VCT 02-Jan-07 02-Jan-07 Neg
296 02-Jan-07 PMTCT 02-Jan-07 02-Jan-07 Neg
297 02-Jan-07 PMTCT 02-Jan-07 02-Jan-07 Missing
3 03-Jan-07 VCT 03-Jan-07 03-Jan-07 Neg
4 03-Jan-07 VCT 03-Jan-07 03-Jan-07 Neg
5 03-Jan-07 VCT 03-Jan-07 03-Jan-07 Neg
 Steps in Approach to Data Analysis
▪ Familiarize yourself with the structure of the data
set
▪ Draft an analysis plan
▪ Create a working copy of the data set
▪ Look at the actual data in the data set
▪ Clean the data in the working file
▪ Revise and add detail to the analysis plan
▪ Create an analysis file
▪ Follow the analysis plan to analyze the data

23 Epidemiologic Bias
 Data Cleaning

▪ Screen

▪ Diagnose

▪ Resolve

24
 Data Cleaning
– Duplicate records
▪ Screen – Invalid and out-of-range codes
– Missing data
– Outliers (unlikely values)
– Lack of variability
▪ Diagnose
– Unlikely patterns, including
reverse polarity
– Skip pattern checks
▪ Resolve – Logic checks

25
 Look for Duplicate Records
Rec. Client Test Result Test Date Client Time to
No. Code Sex Age Date Date Result Informed Informed
1 ABC64F F 27 2 Jan 21 Jan Neg 21 Jan 19
2 DEF21F F 20 2 Jan 18 Jan Neg 18 Jan 16
3 GH78M M 19 3 Jan 3 Feb Neg 3 Feb 31
4 IJK32F F 35 3 Jan 28 Jan Neg 28 Jan 25
5 LMN88F F 22 3 Jan 2 Jan Neg 2 Jan -1
6 OPQ37M M 42 3 Jan Neg -39816
7 RST44F F 71 4 Jan 15 Jan Neg 15 Jan 11
8 UVW55F M 37 4 Jan -39817
9 XYZ17F F 24 4 Jan 24 Jan Neg 24 Jan 20
Records 10 – 196
197 OPQ37M M 42 3 Jan 3 Mar Neg 3 Mar 59

26 Epidemiologic Bias
 Look for Invalid or Out-of-Range Values
▪ Frequency distribution of each variable
▪ Look for invalid codes, e.g.,
– Legitimate codes: 1=female, 2=male
• What to do with 3 or 9?
• What to do with F or M?
▪ Look for out-of-range or outlier values
– Visit date from before study began?
– 82-year-old at PMTCT?

27 Epidemiologic Bias
 Look for Inconsistencies, Skip Problems,
Logic Errors (1 of 2)
▪ Look for implausible response to one variable
based on response to another variable
▪ Look for skip inconsistencies
– If contraception = none, then contraception
type cannot be IUD
▪ Compare dates
– Date of discharge before date of admission?

28 Epidemiologic Bias
 Look for Inconsistencies, Skip Problems,
Logic Errors (2 of 2)
▪ Look for logic errors
– Surgical code = hysterectomy, sex = male

▪ Confirm calculations
– If both DOB and age collected, does age =
int(DT_visit − DOB)/365.25?

29 Epidemiologic Bias
 Review Open-Response Variables
Grande Anse Beach, xxx Parish, Grenada

30
Write
Epidemiologic Biasdown the name of the parish.
 How many ways can one spell
(or misspell) St. George Parish?
▪ St George ▪ St Geoeges
▪ St. George ▪ St Georegs
▪ St Georges ▪ St Georgres
▪ St. Georges ▪ St Goerges
▪ St George’s ▪ Saint George
▪ St. George’s ▪ etc.
▪ St Georges’
▪ St. Georges’

31
 Data Cleaning

▪ Screen
Go back to the original data
source, if possible
– Error
▪ Diagnose
– Missing data
– True extreme
– Cannot determine
▪ Resolve

32
 Data Cleaning

▪ Screen

▪ Diagnose
– Change to correct, valid value
– Leave unchanged
▪ Resolve – Flag as permanently questionable
edit failures – Flag as permanently missing
– Delete record

33
 Resolving Edit Failures
▪ Webster’s Law: Never let an error age!
▪ DOCUMENT, DOCUMENT, DOCUMENT
▪ Be consistent in decision-making
▪ Avoid “second guessing”
▪ Update the working file

34 Epidemiologic Bias
 Steps in Approach to Data Analysis
▪ Familiarize yourself with the structure of the data
set
▪ Draft an analysis plan
▪ Create a working copy of the data set
▪ Look at the actual data in the data set
▪ Clean the data in the working file
▪ Revise and add detail to the analysis plan
▪ Create an analysis file
▪ Follow the analysis plan to analyze the data

35 Epidemiologic Bias
 Analysis Plan — Details
▪ Specify:
– Categories for continuous variables
• Age
• Incubation period (if prolonged)
• Weight, height, BP
• Duration of hospitalization, treatment, etc.
– Summary measures
▪ Sequence: from simple to complex
– Descriptive to analytic to subgroup analyses
▪ Create table shells
36 Epidemiologic Bias
 Which 10-year Age Groups Would You
Choose?
Choice 1 Choice 2 Choice 3
0 – 10 0 – 10 0–9
10 – 20 11 – 20 10 – 19
20 – 30 21 – 30 20 – 29
30 – 40 31 – 40 30 – 39
40 – 50 41 – 50 40 – 49
50 – 60 51 – 60 50 – 59
60 – 70 61 – 70 60 – 69
≥ 70 ≥ 71 ≥ 70
Unknown

37 Epidemiologic Bias
 Creating Categories
▪ Mutually exclusive, all inclusive
▪ Choices
– Standard categories for the disease
– Equal intervals
– Equal numbers within each group
▪ Include category for unknown or missing values
▪ If calculating rates, numerator categories must
match denominator (population, census)
categories

38 Epidemiologic Bias
 Choices, e.g., Age
Age Group (yrs) #Cases
Mean age = ___ 0–9 ___
Median age = ___ 10 – 19 ___
Range = ___ – ___ 20 – 29 ___
30 – 39 ___
40 – 49 ___
50 – 59 ___
60 – 69 ___
Unknown ___
Total ___

39 Epidemiologic Bias
 Line Listing of Cases of Leptospirosis,
Country K, August 20xx
Date of
onset of Date of Red
ID Symptoms Sex House Age (yr) symptoms hospitalization Fever Chills Headache Myalgia Jaundice eyes
1 Y M 23 54 08/09 08/09 Y Y Moderate n Y n
2 Y M 57 69 08/05 08/07 Y Y none n n n
23 Y M 65 61 08/01 08/06 Y Y Severe Y n n
24 Y F 58 63 08/07 08/10 Y Y Severe n n n
25 Y M 13 9 08/06 08/12 Y Y Severe n n n
26 Y F 33 37 08/12 08/14 Y Y Moderate Y Y n
27 Y F 5 44 08/08 08/10 Y Y Severe Y Y n
28 Y M 5 17 08/08 08/11 Y Y Severe Y Y n
29 Y F 45 28 08/10 08/10 Y Y Moderate Y n n
71 Y M 38 28
102 Y M 17 21 08/05 08/09 Y Y Severe Y n n
103 Y M 8 30 08/05 08/06 Y Y Severe Y n n
109 Y M 9 22 08/10 08/12 Y Y Severe Y n n
110 Y M 35 67 08/04 08/06 Y Y Severe Y Y Y
113 Y F 47 9 08/17 08/17 Y Y Severe Y n n
114 Y M 39 38 08/15 08/15 Y Y none n n n
115 Y M 14 22 08/03 08/06 Y Y Severe n n n
116 Y F 3 19 08/10 08/11 Y Y Severe Y n n
118 Y F 3 22 08/07 08/12 Y Y Moderate Y n n
119 Y M 30 19 08/05 08/08 Y Y Mild Y n n
120 Y F 12 35 08/04 08/09 Y Y Severe Y Y n
122 Y F 2 64 08/06 08/11 Y Y Moderate Y n n
123 Y M 16 27 08/10 08/12 Y Y Moderate Y n n
126 Y F 25 61 08/10 08/11 Y Y Severe Y Y n

40 Epidemiologic Bias
 Summary of Age of Cases of
Leptospirosis, Country K, August 20xx
Mean age = ___
36 Age Group (yrs) #Cases
29
Median age = ___ 0–9 2
___
9 – ___
Range = ___ 69 10 – 19 3
___
20 – 29 7
___
Number of Cases by Age
8
30 – 39 4
___
7 40 – 49 1
___
6 50 – 59 1
___
5
60 – 69 6
___
4
3 Unknown 0
___
2 Total 24
___
1
0
0-9 10-19 20-29 30-39 40-49 50-59 60-69
41 Epidemiologic Bias
 Analysis Plan — Details
▪ Specify:
– Categories for continuous variables
• Age
• Incubation period (if prolonged)
• Weight, height, BP
• Duration of hospitalization, treatment, etc.
– Summary measures
▪ Sequence: logical, systematic
▪ Create table shells

42 Epidemiologic Bias
 Table Sequence in a Typical
Surveillance Report
numbers graph
Time and/or rates
Place numbers or map or table
rates
numbers table
Person
and/or rates
Agent, numbers, frequency
e.g., subtypes percentages distribution
numbers, frequency
Clinical
percentages distribution
Determinants, numbers, frequency
e.g., risk factors percentages distribution
43 Epidemiologic Bias
 Table Sequence in a Typical
Analytic Report
▪ Descriptive to analytic to subgroup analyses

Table 1 Descriptive epi (who is in the

and/or figures study, when, where)

Table 2
Clinical features
and/or figures

Table 3
Key association(s) or finding(s)
and/or figures

Table 4+
Subgroup or other analyses
and/or figures
44 Epidemiologic Bias
 Measures of Association
for Different Types of Studies

Measure of Association

Prospective Cohort Study Risk ratio or rate ratio

Retrospective Cohort Study Risk ratio or rate ratio

Case-control Study Odds ratio

Cross-sectional Study Prevalence ratio

45 Epidemiologic Bias
 Analysis Plan — Details
▪ Specify:
– Categories for continuous variables
• Age
• Incubation period (if prolonged)
• Weight, height, BP
• Duration of hospitalization, treatment, etc.
– Summary measures
▪ Sequence: from simple to complex
– Descriptive to analytic to subgroup analyses
▪ Create table shells
46 Epidemiologic Bias
 What is a Table Shell?
▪ Also called “dummy table”
▪ Table shell = table with title and category labels,
but with no data
▪ Can (should) include statistics
▪ Created as part of analysis plan, filled in during
actual data analysis

47 Epidemiologic Bias
 Table Shell — Example
Table. COVID-19 Case fatality proportion (CFP) by age and sex,
Country S, 2020
Males Females Total
No. Rate / No. Rate / No. Rate /
Age Cases 100,000 CFP Cases 100,000 CFP Cases 100,000 CFP
0–4 25 4.2 0.0% 23 3.9 8.7% 48 4.0 4.2%
5 – 14 41 4.3 0.0% 53 5.5 0.0% 94 4.9 0.0%
15 – 24 91 12.2 1.1% 107 13.6 1.9% 198 12.9 1.5%
25 – 34 157 29.5 0.6% 198 33.4 2.0% 355 31.5 1.4%
35 – 44 160 44.1 3.1% 128 32.9 3.1% 288 38.3 3.1%
45 – 54 112 35.2 13.4% 84 28.3 11.9% 196 31.9 12.8%
60+ 78 42.7 38.5% 54 25.8 29.6% 132 33.7 34.8%
Unknown 7 na na 9 na na 20 na na
Total 671 18.2 7.7% 656 17.2 5.8% 1,331 17.7 6.8%

48 Epidemiologic Bias
 Table Shell — Example
Table. COVID-19 Case fatality proportion (CFP) by age and sex,
Country S, 2020
Males Females Total
No. Rate / No. Rate / No. Rate /
Age Cases 100,000 CFP Cases 100,000 CFP Cases 100,000 CFP
0–4
5 – 14
15 – 24
25 – 34
35 – 44
45 – 54
60+
Unknown
Total

49 Epidemiologic Bias
 Table Shell — Purpose
▪ Used to help visualize how data can be organized
and summarized

▪ Used to evaluate how the relationships between

variables will be assessed

▪ Provides a template to guide you through the

steps of your analysis

50 Epidemiologic Bias
 Create Table Shells
Questions from cross-sectional study (age 25+):
1. What is your gender? 4. How many hours per
 Female week do you exercise?
 Male  Less than 1 hour
 1 – 4 hours
2. What is your age?
______ (in years)  5 – 10 hours
 11 – 20 hours
3. How many alcoholic
 More than 20 hours
beverage drinks do you
consume per week, on
average?
______ drinks / week
51 Epidemiologic Bias
 Table Shells (Q1, Q2)
Gender Number Pct Age group
Female (years) Number Pct
Male 25 – 34
Other 35 – 44
Missing 45 – 54
Total 55 – 64
≥ 65
Unknown
Total

Mean = ____
Median = ____
Range = ___ – ___

52 Epidemiologic Bias
 Table Shells (Q3, Q4)
Drinks / week Number Pct Exercise
0 (hours / week) Number Pct
1 <1
2 1–4
3 5 – 10
4 11 – 20
5 ≥ 21
6 Unknown
etc. Total

Then create groups based

on the data
Mean = ____
Median = ____
53 Epidemiologic
Range = ___Bias
– ___
 Study of Circumcision and HIV, Uganda
HIV+ HIV− Total Risk
Circum+ _____ _____

Circum− _____ _____

Total _____ _____ _____

RR = _____, 95% CI = (___ , ___)

Pearson Chi-square = _____,2-tail P-value = ____

Ref:Bias
Epidemiologic Gray RH et al. AIDS 2000; 14:2371–2381.
54
 Study of Circumcision and HIV, Uganda
HIV+ HIV− Total Risk
Circum+ 18 890 908 1.98%
Circum− 154 4,454 4,608 3.34%
Total 172 5,344 5,516 3.12%

RR = 0.59, 95% CI = ( 0.37, 0.96)

Pearson Chi-square = 4.642, 2-tail P-value = 0.03

Ref:Bias
Epidemiologic Gray RH et al. AIDS 2000; 14:2371–2381.
55
 Steps in Approach to Data Analysis
▪ Familiarize yourself with the structure of the data
set
▪ Draft an analysis plan
▪ Create a working copy of the data set
▪ Look at the actual data in the data set
▪ Clean the data in the working file
▪ Revise and add detail to the analysis plan
▪ Create an analysis file
▪ Follow the analysis plan to analyze the data

56 Epidemiologic Bias
 Analysis File
▪ Purpose of analysis file is to facilitate the analysis
▪ Use standard naming convention for file (can
include creation date in name)
▪ Create and add variables
– needed for specified tables
– that will be used repeatedly during analyses
▪ Use easily identifiable variable names
▪ Keep unique identifiers for merging
▪ Document differences when updating files

57 Epidemiologic Bias
 Steps in Approach to Data Analysis
▪ Familiarize yourself with the structure of the data
set
▪ Draft an analysis plan
▪ Create a working copy of the data set
▪ Look at the actual data in the data set
▪ Clean the data in the working file
▪ Revise and add detail to the analysis plan
▪ Create an analysis file
▪ Follow the analysis plan to analyze the data

58 Epidemiologic Bias
 Analyzing the Data
▪ In general, follow the analysis plan
▪ But remain flexible
– Different groupings of categories
– Pursue interesting, unanticipated findings
▪ Analysis often uncovers additional errors
– Fix error, document decision
– Create updated analysis file

59 Epidemiologic Bias
 Summary
▪ Planning the analysis is an essential step in the process
of study design, and should be completed prior to
collecting data

▪ table shells are an organizational tool for planning the

data analyses

▪ All datasets must be “cleaned” before starting the analysis

▪ There are many possible sources of data errors

▪ The most common types of data errors are missing data,

duplications, and outliers or improbable values

60 Epidemiologic Bias
 Conclusion
▪ Analysis planning is an essential step
▪ Analysis plan is based on what questions you need to
answer, what data you have, what information you want to
communicate
▪ Create table shells to guide analysis and presentation
▪ Do not assume that data are clean
▪ Leave original data alone; created cleaned file, analysis
file
▪ Review and clean the data, document all decisions
▪ There is no substitute for looking at the data
▪ Note: You might find additional errors during the analysis!
Fix and document.
61 Epidemiologic Bias
 Learning Objectives
At the end of this lesson, you will be able to:
▪ Describe the purpose and components of a data
analysis plan
▪ Describe the process for becoming familiar with a
data set, particularly the variables and values in
the data set
▪ Create tables shells
▪ Conduct quality-control data checks, including
duplication and missing data

Questions?
62 Epidemiologic Bias