Start with the question, not the conclusion
A useful paper answers a specific question: in which people, with which treatment, compared with what, and for which outcome? ‘This lens gives better vision’ is too broad. Better at reading a chart in daylight, using a computer without glasses, or driving with fewer disturbing halos? These are different questions.
Write down what matters to you before reading. Then look for that outcome in the paper. A result about something else may be interesting without answering your question.
What kind of study are you reading?
In a randomised controlled trial, chance determines which study group a participant joins. A suitable comparison helps separate the treatment’s effect from differences between the people receiving it. Randomisation is valuable, but poor follow-up, biased measurements or selective reporting can still weaken the result.
In an observational study, researchers follow or analyse people whose treatment was not assigned by randomisation. The groups may differ for other reasons. That makes a claim of cause and effect harder to establish, not the study worthless.
Prospective means the study follows participants forward according to a plan. Retrospective means it looks back at existing records. These describe timing; ‘prospective’ does not automatically mean randomised or interventional.
A case series describes patients without a comparison group. A systematic review searches for and appraises studies using explicit methods; a meta-analysis statistically combines compatible results. Pooling weak or very different studies does not automatically create strong evidence.
Evidence levels I–V: a starting point, not a verdict
The earlier TMX Academy teaching article used a five-level diagram, I–V. A simplified reading of that scheme is:
- Level I: randomised trials and systematic reviews of such trials.
- Level II: prospective comparative studies without randomisation.
- Level III: retrospective comparative studies.
- Level IV: case series and other studies without a comparison group.
- Level V: expert opinion and preclinical work, such as laboratory or animal experiments.
This is a teaching hierarchy, not one universal grading system. Exact categories depend on the system and the question. A laboratory experiment can be excellent evidence about optical behaviour under its test conditions without establishing how a patient will feel after surgery.
Oxford CEBM explicitly warns that a hierarchy is not a definitive judgement of evidence quality or a treatment recommendation. Read the design and its limitations, not just a Roman numeral. Oxford CEBM: how to use evidence levels.
Four GRADE levels: how confident can we be in the result?
GRADE asks a different question: how reliable are the combined findings for a particular outcome? Its four categories describe confidence, not four types of study.
| Confidence | Plain-language meaning |
|---|---|
| High | Strong confidence in the estimated effect. |
| Moderate | Reasonable confidence, but a meaningfully different effect remains possible. |
| Low | Limited confidence; the actual effect may differ substantially. |
| Very low | Great uncertainty about the estimate. |
The same treatment can have stronger evidence for one benefit and weaker evidence for another benefit or harm. A GRADE rating is not a quality badge for an entire brand. This article explains GRADE; it does not assign formal GRADE ratings to lenses. GRADE Working Group.
Five concerns can lower confidence: biased study methods, inconsistent results, evidence that does not directly fit the question, imprecise estimates, and missing or selectively published results. Randomisation does not automatically make the final answer highly reliable. Cochrane Handbook, chapter 14.
Look at who was included—and who was not
Before using an average result to imagine your own future vision, check the participants’ age, eye conditions, previous surgery and eligibility criteria. Did the study exclude the very problem you have? How long were participants followed? A short follow-up cannot answer every long-term question.
Also check what was counted: people, eyes, or measurements? In an explicitly hypothetical example, 100 eyes from 50 people are not 100 independent patients. Results from both eyes can be related. Check whether the analysis accounts for that relationship rather than treating every eye as an unrelated person.
A prestigious journal, famous author or large sample is a reason to read carefully—not permission to stop checking. Size can improve precision but cannot undo a biased comparison.
Percentages: always ask ‘out of how many?’
Here is invented arithmetic, not a treatment result: an unwanted outcome occurs in 20 of 100 people in one group and 10 of 100 in another. That is a 50% relative reduction, but an absolute difference of 10 percentage points. Both statements describe the same numbers; one sounds more dramatic.
Ask for the starting risk, actual group sizes, follow-up time and absolute difference. For ‘90% satisfied’, ask who answered, what satisfaction meant, and whether people with missing responses were included. Cochrane: presenting absolute and relative effects.
‘Statistically significant’ is not the same as useful
A small p-value does not tell you how large a benefit is, whether it matters in everyday life, or the probability that the claim is true. Read the size of the difference and its confidence interval, not only ‘p < 0.05’. The interval helps show the statistical precision of the estimate under the study’s assumptions; it is not a promised range for your own result.
Likewise, ‘no statistically significant difference’ does not prove that two treatments are equivalent. The study may simply be too imprecise to distinguish them. American Statistical Association: principles for interpreting p-values.
‘Not worse’ does not mean ‘better’
A non-inferiority study asks whether a new option is not unacceptably worse than the treatment it is compared with for a specified outcome. It needs a justified margin defining what ‘unacceptably worse’ means. Meeting that test does not, by itself, establish superiority, identical results, or fewer side effects.
Always ask what the study was designed to show. A claim about ‘better overall vision’ needs evidence for that claim, not a successful test of non-inferiority for one measurement. FDA guidance on non-inferiority explains this principle for drug trials; it is not a rating of any IOL.
Marketing highlights strengths. Read what it leaves out
A practical reading habit—in medicine as elsewhere—is to expect advertising to emphasise the most attractive feature. That is an editorial rule of thumb, not proof that every omission hides a failure. An absent number may not have been measured, may not be in the short summary, or may simply not have been highlighted.
Do not turn ‘not reported’ into either ‘no problem’ or ‘the product is bad’. Ask for the full paper and the missing outcome. If a claim is supported only by laboratory testing, a selected subgroup or one clinic’s experience, the wording should reflect that limit.
For an IOL, seeing small letters on a bright chart does not answer every question about reading comfort or vision in difficult lighting. Look separately for glasses use, contrast, bothersome halos and patient-reported daily function. Compare like with like—not a favourable number from one paper with a differently measured number from another. See why contrast matters and what laboratory lens tests can and cannot show.
Funding and transparency: questions, not accusations
Industry funding does not automatically invalidate a study, and a lack of industry funding does not guarantee good methods. Look for the sponsor’s role, conflicts of interest, a registered trial and an accessible protocol. Were the main outcomes chosen in advance? Are missing participants, harms and changes to the analysis explained?
CONSORT 2025 helps identify what a randomised trial report should disclose. It is a reporting checklist—not a certificate that the treatment works or a substitute for judging bias. Read the CONSORT 2025 statement.
Three questions to take to a consultation
- What did the study compare, and how well does it answer my own question?
- How large is the benefit, how reliable is the estimate, and what problems might matter to me?
- What remains unknown—and would it change my choice?
Weak evidence does not automatically mean a treatment fails. It means the confidence of the claim should be modest. Strong evidence about an average benefit still cannot decide your personal suitability. Use the paper to improve your questions, then discuss benefits, trade-offs and your eye findings with the clinician.
Where this explanation began
This new IOL Adviser editorial article develops the patient-facing theme of TMX Academy’s ‘Scientific publications—how to understand what deserves trust?’ (15 October 2024), updated against the methodological sources linked above. It is not a reproduction of a course or an endorsement by a manufacturer.
Educational information—not a diagnosis or medical prescription. An eye-care professional must assess your eyes and medical suitability before treatment or surgery.