|
Things I try to remember when editing medical articles
edit- This overview and this list of cognitive biases. If I want extra credit, I review this glossary: [1]
- The h-index ([2]) and impact factor of journals. In general, I try to stick to an IF > 2 though this is not canon. These can usually be found with a simple Google search.
- The evidence hierarchy — to help me see the forest from the trees.
- Systematic reviews > meta analyses > double blinded randomized control trials > animal studies > in vitro. Randomized controlled trials allow for more confidence in conclusions while meta analyses and reviews are more of a consensus. Neither is perfect. It's a long way from bench to bedside.
- In the absence of evidence we must rely on a priori reasoning and analogy. These are part of the scientific method, but generally inappropriate for encyclopedic content. I am very careful with case reports, cross sectional epidemiology and cohort studies as they are prone to spurious relationships.
- The plural of anecdote is not evidence and correlation does not equal causation.
- Publication bias — by searching for unpublished trials, displaying funnel plots, and using statistics like fail-safe N to investigate the possibility of suppressed research. A small fail-safe N or asymmetric funnel plot suggest bias.
- Conflicts of interest, not just in declarations at the end of the article but by looking up the authors' online resumés, research histories and paid lectures. NIH funded studies are preferred but can still have serious issues. Money, ego and prestige are insidious. It is difficult to convince a man of something if his salary depends on him believing the opposite.
- Retraction Watch publishes a list of scientists with the most retracted papers, either due to p-hacking, poor statistical methods or even actively fabricating data. You can access this list here: [3]
- Journal lists:
- The Abridged Index Medicus — a list of 114 journals that are generally gold standard. Another is the 2003 Brandon/Hill list which includes 141 journals, though it is no longer maintained.
- Beall's list — a compilation of problematic journals, discussed comprehensively here: [4] It has not been updated in some time and there are limitations but still a phenomenal open-source candle in the dark. Be cautious of hijacked and vanity "journals". MDPI, Frontiers and Hindawi are some of the more frequent offenders.
- CiteWatch — Wikipedia's homage to Beall; an excellent resource that is updated twice monthly.
- Cabells' Predatory Reports — the successor to Beall's; a comprehensive multidisciplinary update. Unfortunately provided by a paid subscription service only available to institutions, not individual researchers - [5]
- Headbomb's plug-in.
- Lies, damned lies, and statistics — the methods and results sections are crucial.
- I usually start out by looking at diagrams/tables and carefully reading the captions because pictures are easier for my reptile brain to digest. Looking at p intervals and sample sizes give me some sense of an idea's sincerity. This alone lands me light years ahead of where I would have been just reading the abstract. It can be overwhelming at first, but gets easier with practice.
- Bayesian analyses > frequentist inferences. The former is a deductive probability, the latter inductive and binary. Combined Bayesian + frequentist analyses are better than either individually, with the truth often living where they meet.
- Overadjustment bias for conclusions that emerge or disappear only after correction for confounding variables. There could be a causal path. Cox proportional hazards models, in particular, are susceptible.
- As an example: incorrect adjustment for blood pressure while studying the relationship between obesity and kidney failure. Obesity causes high blood pressure, which is its mechanism for destroying your kidneys. Correcting for hypertension obscures the mechanism and causes a Type II error. This method can also be inverted to cause Type I errors. Such mistakes induce bias instead of preventing it.
- Cox models also try to force data into linearity and falter with J- or U-shaped correlations.
- Distribution of p-values in meta-analyses to distinguish Monte Carlo type approaches from p-hacking.
- For controversial or mainstream topics, I take a peak at the talk page and edit history to see if the issue I'm about to revise has been explored, especially if it's in a subject I don't usually edit.
- If an article I want to read is behind a paywall, sometimes I try e-mailing the author a kind note to ask for a copy. This usually works, especially if I pack in a compliment or two. Researchers are like plants; they flourish with attention.
- Sci hub is a search engine to read older articles behind paywalls, though more recent publications are not available.
- Images need to be CC BY or CC BY SA. For NC and ND licensed images you can upload them to NC Commons.
Some quotes
editFor the left brain
edit
All heuristics are equal, but availability is more equal than others.
The One begets the Two. The Two begets the Three, and the Three begets the 10,000 things.
In a series of studies in 2005 and 2006, researchers at the University of Michigan found that when misinformed people, particularly political partisans, were exposed to corrected facts in news stories, they rarely changed their minds. In fact, they often became even more strongly set in their beliefs. Facts, they found, were not curing misinformation. Like an underpowered antibiotic, facts could actually make misinformation even stronger.
Arguing with an idiot is like playing chess with a pigeon. It's just going to knock the pieces over, shit on the board, and then strut around like it won.
It is difficult to get a man to understand something when his salary depends on his not understanding it.
People would rather believe a simple lie than the complex truth.
The popularity of a scale rarely equates to its validity.
For the right brain
edit
True humility is not thinking less of yourself. It is thinking of your self less.
I never gave away anything without wishing I had kept it; nor kept it without wishing I had given it away.
When once a man is launched on an adventure as this, he must bid farewell to hopes and fears, otherwise death or deliverance will both come too late to save his honour and his reason!
In this world Ellwood, you must be oh so smart, or oh so pleasant. Well for years I was smart; I recommend pleasant. And you may quote me.
Frank Sinatra saved my life once. He said, "Okay, boys. That's enough."
If you want to go fast, go alone. If you want to go far, go together.
Always look on the bright side of life.