Scribd
Upload
93 views2 pages

10 Question Checklist For Understanding The Behavior Your Data

This document outlines a 10-point checklist for understanding the behavior of data in analyses. It includes questions about whether results align with expectations, how the data is distributed, making fair comparisons over time, using appropriate baselines, accounting for chance, sample sizes, significance versus importance, appropriate comparisons, considering multiple explanations, and proving causality. The checklist aims to help ensure data analyses are rigorous, comprehensive and provide meaningful insights.

Uploaded by

dreamer4077
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
93 views2 pages

10 Question Checklist For Understanding The Behavior Your Data

This document outlines a 10-point checklist for understanding the behavior of data in analyses. It includes questions about whether results align with expectations, how the data is distributed, making fair comparisons over time, using appropriate baselines, accounting for chance, sample sizes, significance versus importance, appropriate comparisons, considering multiple explanations, and proving causality. The checklist aims to help ensure data analyses are rigorous, comprehensive and provide meaningful insights.

Uploaded by

dreamer4077
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

10 QUESTION CHECKLIST FOR UNDERSTANDING

THE BEHAVIOR OF YOUR DATA

1. What did you expect?


 Do the results you see align with what you and/or the customer expected to see?
 If not, what are the possible causes?

2. How is it shaped?
 Does a distribution of the data reveal any of the following:
o High or low outliers
o Two-tailed distribution
o Clusters of distinct groups that share characteristics within the population?
 What are the possible causes

3. How big is the difference?


 When comparing data from two or more time periods, does your comparison statistic fairly and
accurately represent the difference? (i.e. relative comparison, absolute comparison, percent
increase)

4. Bigger than what?


 Did you provide baseline or comparable data to the data in your analysis?
o Data from prior time periods
o Data from an external source (i.e. benchmarks)

5. Did it happen by chance?


 Do your summary statistics include the margin of error? (i.e. standard deviation)
 Did you apply your logic to another time period to validate that findings are similar?
o If findings are different, what is the root cause?
 Seasonality
 Product changes
 Business rule changes

6. Size matters
 For summary statistics, how does the population size affect the findings?
o Did you share the N when presenting the average, percentage, ratio, etc.?
o Is the population or sample large enough to achieve statistical significance?
 Did you adequately explore and explain the possibility of results being due to random chance?

7. Significant isn't necessarily important


 If your analysis demonstrates statistical significant findings, are they meaningful to the business?
o Does a correlation between variables present an opportunity for the business to take
action? Or is it merely an interesting observation?
8. Apples are different from oranges
 Did you make an effort to ensure that comparison data is representative of the data used in your
analysis? (i.e. if benchmark data pertains to commercial business only, do not compare it to
Medicare)

9. One explanation is never enough


 Did you investigate and address multiple explanations for your findings? This could include:
o Changes in practice patterns
o Changes in diagnostic or procedural coding
o Changes in products / benefits
o Seasonality
o Increased utilization
o Implementation of a cost / utilization control program

10. Did this cause that?


Did you scientifically prove causality in your documentation? Causality is possible if:
 the cause and effect variables show statistical correlation (i.e. P < or =0.05)
 the cause happened before the effect (i.e. UM program implemented and then utilization
decreased)
 the results are reproducible over different time (i.e. 12 months of data show an increasing cost
associated with hospital admits)
 the cause-and-effect relationship makes sense (i.e. increased utilization leads to increased
aggregate cost NOT increased utilization leads to increased unit cost)

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy