TEACHING POINTS:: How to "read' a Journal/Study article

Ace844 · January 5, 2006

Hi all,

Since there are a number of us here who regularly "cite," and post journal articles. I thought it may be helpful to those who don't know how how, or have never been taught to have a refrence to see how, or if it is applicable to them.. Also, I wanted to post a way for you to "evaluate" what you are reading...

Please select the type of article that you are appraising below:

Therapy Question- A question concerning the effectiveness of a treatment or preventative measure

Diagnosis Question - A question concerning the ability of a test to predict the likelihood of a disease

Prognosis Question - A question concerning outcome of a patient with a particular condition

Harm Question - A question concerning the likelihood of a therapeutic intervention to cause harm

________________________________________________________________________________

_____________________

Appraising Relevant Overview or Meta-Analysis

To appraise a systematic overview, there is a series of questions that you must answer.

First of all, was the study valid?

Did the overview address a focused clinical question?

Were the criteria used to select articles for inclusion clearly stated and appropriate?

Is it likely that important relevant studies were missed by the search strategies employed?

Was the validity of the included studies appraised?

Were the validity criteria used appropriate to the nature of the clinical question?

Were the assessments of the studies with respect to inclusion criteria reproducible?

Were the results similar from study to study? If heterogeneity was present, was it clinically significant?

After thinking about the above questions, Is there a FATAL flaw in the study?

If there is, then toss this study and go onto the next one. If not, continue the appraisal.

--------------------------------------------------------------------------------

What were the results?

Were the results reported in the form that you need to apply them to your practice?

i.e. relative risk or odds ratio for therapy, likelihood ratio for diagnosis, etc.

If not, is data given which allows you to calculate the needed parameters from the reported results?

Were the results pooled, and were the confidence intervals sufficiently narrow to eliminate ambiguity?

Appraising Relevant Therapy Studies

To appraise a study, there is a series of questions you must answer.

First of all, was the study valid?

Was the study randomized?

Was follow-up complete and adequate and were all patients kept in the groups to which they were assigned?

Was the study blinded?

Aside from the intervention(s) being studied, were the patients treated identically in the different groups?

Were the groups similar at the start of the study?

After thinking about the above questions, Is there a FATAL flaw in the study?

If there is, then toss this study and go onto the next one. If not, continue the appraisal.

--------------------------------------------------------------------------------

What were the results?

Did the study CLEARLY show harm or benefit?

Were confidence intervals reported and was the entire interval on one side

or the other of the HARM-BENEFIT line?

If no confidence intervals, were P values reported and were they less then .05

--------------------------------------------------------------------------------

Are the results applicable to my patients?

Were all clinically important outcomes reported?

Were absolute risk data reported for dichotomous outcomes ?

What was the number needed to treat for the important dichotomous outcomes?

Number needed to treat calculator

Are the overall results clinically significant?

Would these results apply to my patients?

Did the treatment cause harm which might outweigh the benefit for my patient(s)?

--------------------------------------------------------------------------------

Overall: Does this study answer the original clinical question and is the answer meaningful?

--------------------------------------------------------------------------------

If this study is does not answer the original question or if the answer is not meaningful after the above appraisal, then toss out the study and proceed to the next one

Appraising Relevant Diagnosis Studies

To appraise a study, there is a series of questions you must answer.

First of all, was the study valid?

Was there an independent comparison with an acceptable " Gold Standard "

Did the study population involve an appropriate spectrum of disease?

Were subjects without disease clinically symptomatic?

Was the gold standard test performed on all patients who underwent the diagnostic test in question?

Were the methods described in sufficient detail to permit duplication?

After thinking about the above questions, Is there a FATAL flaw in the study?

If there is, then toss this study and go onto the next one. If not, continue the appraisal.

--------------------------------------------------------------------------------

What were the results?

What are the likelihood ratios(s)?

likelihood ratio calculator

--------------------------------------------------------------------------------

Are the results applicable to my patients?

Did the disease spectrum in the study population reflect that of the patients for whom I might need this test?

Are there other reasons that this test might perform differently in my patient population?

Can the accuracy reported for this test be achieved in my practice setting?

Will the results effect my management of my patients?

Bayes Theorem nomogram (you must have entered figures into calculator above)

Do the benefits of this test justify the costs, risks, and inconvenience to my patients?

--------------------------------------------------------------------------------

Overall: Does this study answer the original clinical question and is the answer meaningful?

--------------------------------------------------------------------------------

If this study is does not answer the original question or if the answer is not meaningful after the above appraisal, then toss out the study and proceed to the next one.

Appraising Relevant Prognosis Studies

To appraise a study, there is a series of questions you must answer.

First of all, was the study valid?

Was there a representative sample of patients as a well-defined point in the course of the disease?

Was the follow-up sufficiently long and complete?

Were the outcomes well-defined and unambiguously measurable?

Were important prognostic factors identified and were their effects on relevant outcomes reported?

After thinking about the above questions, Is there a FATAL flaw in the study?

If there is, then toss this study and go onto the next one. If not, continue the appraisal.

--------------------------------------------------------------------------------

What were the results?

How large is the likelihood of the outcomes in a specified period of time?

Are confidence intervals reported?

--------------------------------------------------------------------------------

Are the results applicable to my patients?

Was the study population sufficiently similar to my own?

Do the results help to identify which of my patients will most benefit from specific therapies?

Do the results allow me to tell my patients what they want to know?

--------------------------------------------------------------------------------

Overall: Does this study answer the original clinical question and is the answer meaningful?

--------------------------------------------------------------------------------

If this study is does not answer the original question or if the answer is not meaningful after the above appraisal, then toss out the study and proceed to the next one.

Appraising Relevant Studies on Harm

To appraise a study, there is a series of questions you must answer.

First of all, was the study valid?

Were the comparison groups similar with respect to factors which could forseeably influence outcome?

Were outcomes and exposures measured in the same way in all groups?

Was follow-up sufficiently long and complete?

Did exposure precede the appearance of the symptoms in question?

For medicines, is there a dose-response relationship with respect to the symptoms in question?

After thinking about the above questions, Is there a FATAL flaw in the study?

If there is, then toss this study and go onto the next one. If not, continue the appraisal.

--------------------------------------------------------------------------------

What were the results?

How large is the Relative Risk (RR) or Odds Ratio (OR) of the measured outcomes in the exposed group?

Are confidence intervals given for the Relative risk or Odds ratio?

--------------------------------------------------------------------------------

Are the results applicable to my patients?

Is the magnitude of the risk sufficiently large to obviate the need for work-up of other causes of symptoms?

Is my patient at particular risk for a bad outcome from this exposure?

Do I need to stop the exposure?

--------------------------------------------------------------------------------

Overall: Does this study answer the original clinical question and is the answer meaningful?

--------------------------------------------------------------------------------

If this study is does not answer the original question or if the answer is not meaningful after the above appraisal, then toss out the study and proceed to the next one.

Definitions::

- A -

Absolute risk and its reduction

This is the percentage of subjects in any group or sub-group that experiences a discrete bad outcome such as death or admission to the hospital. An efficacious therapy serves to reduce that risk. For example, if 15% of the placebo group died and 10% of the treatment group died, the absolute reduction in the risk of death is 5%.

Accuracy

The proportion of all test results (positives and negatives) which agreed with the gold standard.

Applicability (also called external validity, generalizability, relevance)

This is the degree to which the results of an observation, study, or review are likely to hold true in your practice setting.

For a detailed discussion, click HERE.

--------------------------------------------------------------------------------

- B -

Bayes' Theorem

This is a simple formula that says that if a particular test result is twice as likely to occur in patients with a disease, condition, or injury than in patients without, then, it is twice as likely that the patient with the result being tested for actually has the disease as compared to any randomly selected similar patient who has not been tested. If you don't like thinking about things like this, just use the nomogram in the users guides or the calculator on the diagnosis appraisal page.

Bias

This is any factor which might change the results of a study from what they would have been if that factor were NOT present. The direction of bias may be unpredictable. For example, giving a team a ten point advantage might seem to give that side an advantage but some teams actually play much better when they have to come from behind! The validity of a study is integrally related to the likelihood that the results have been biased by factors extraneous to the study design.

Blinding

The "masking" or concealment from study subjects, caregivers, or others involved in the study of any detail(s) of the study which could introduce Bias. For example, not telling patients or doctors which patient gets placebo or actual drug; or not telling radiologists the clinical assessment of patients whose films they are reading.

--------------------------------------------------------------------------------

- C -

Case-control study

This might be considered a randomized controlled trial played backwards. People who get sick or have a bad outcome are identified and "matched" with people who did better. Then, the effects of the therapy or harmful exposure which might have been administered at the start of the trial are evaluated. In other words, you first find the people who did poorly and then look at the therapy or exposure and compare it to people who didn't get the therapy. Needless to say, this is a crude way of doing a study. When the effect of interest is HARM, this may actually be the only ethical way of doing the study.

Case report

This includes single case reports and published case series'. These are searchable as a separate category in the MEDLINE database

C.A.T.

see critically appraised topic

Clinical significance

Results are clinically significant when they make enough difference to you and your patient to justify changing your way of doing things. For example, a drug which is found in a megatrial of 50,000 adults with acute asthma to increase FEV1 by only 0.5% (P value<.0001) has failed this test of significance.

Cochrane Collaboration

An international organized effort to organize all existing clinical studies into systematic reviews easily accessible to practicing clinicians and to otherwise facilitate the process of bringing clinical evidence to bear on decision making in patient care.

Cohort study

Also called a "prospective observational study", this design follows a group of patients, called a "cohort", over time to determine general outcome as well as the outcomes of different subgroups.

Cointervention

A therapy or other ancillary treatment which is NOT under investigation which is given to study patients.

Confidence intervals

An interval around an observed parameter such as relative risk which is guaranteed to include the true value to some level of confidence (usually 95%). That level of confidence is only justified to the extent that bias is absent from the study. A well known election poll advertises itself "this poll is accurate to within 2 percentage points 99% of the time." This is a way of saying, in language aimed at voters (perhaps a skewed sample from the standpoint of IQ) that the 99% CI around the reported percentages is + 2.

Controlled clinical trial

Any study which compares two groups by virtue of different therapies or exposures fulfills this definition.

Critical appraisal

The process of assessing and interpreting evidence systematically considering its validity, results, and relevance. For more information, consult the User's Guides.

Critically appraised topic (C.A.T.)

A 1 or 2 page summary of a search and critical appraisal of the literature related to a focused clinical question. This summary should be kept in an easily accessible place so that it can be used to help make clinical decisions.

--------------------------------------------------------------------------------

- D -

Dichotomous outcome

Any outcome measure in which there are only two possibilities, like dead/alive, admitted/discharged, graduated/sent to glue factory. Beware of potentially fake dichotomous outcome reports such as "improved/ not improved", particularly when derived from continuous outcome measures. For example, if I define a 10 point or greater increase in peak expiratory flow in a study of acute asthma as "improved", I may show what looks like a tremendous benefit when the results were clinically insignificant. This is lesson 2a in "How To Lie With Statistics."

Double blind

A single blind study means that someone (patient or physician) does not know what is going on. Double blind means that at least two people (patient and physician) don't know what's going on. Triple blind might mean that the paper is written before the results are tabulated. The whole point is to prevent bias.

--------------------------------------------------------------------------------

- E -

Effect size

The difference in measured outcomes attributed to a therapeutic intervention. This term is encountered in meta-analyses when different studies have measured different things. For example, results of an asthma study which measured FEV1 could be combined with those of another study which measured return visits to the ED using a statistically derived generic effect size. Do you prefer skiing or red wine?

Effectiveness

I buy a BMW which test drives miraculously on the dealerâ€™s special runway. I then find that the roads in the area where I live have all been closed. This is a breakdown of effectiveness. See efficacy.

Efficacy

The BMW I have selected for a test drive blows all four tires, stalls out and crashes on the dealerâ€™s special runway. I spend two days in the hospital. This is a breakdown in efficacy. See effectiveness.

Event rate

This is a term for absolute risk.

Exposure

Anything you can be exposed to: a drug, a surgical procedure, time, sexual harassment, rounds, even a diagnostic test. Most commonly encountered in therapy, prognosis or harm studies where the EFFECT of an "exposure" is the subject of the study.

External validity

See applicability.

--------------------------------------------------------------------------------

- F -(empty)

--------------------------------------------------------------------------------

- G -

Generalizability

See applicability.

Gold standard

No longer relevant in the realm of high finance from whence it originated, this term gained new life when it was decided that it should refer to a reference standard for evaluation of a diagnostic test. For the purposes of a study, the "gold standard" test is assumed to have 100% sensitivity and specificity. This may well constitute an exaggerated estimate of the reference test. Choice of the "gold standard" must therefore be evaluated in appraising a diagnosis study.

--------------------------------------------------------------------------------

- H -

Harm-Benefit Line

On a graph of outcomes, this line divides results favoring therapy from results favoring the control.

Heterogeneity

Also called "homogeneity" but having nothing to do with sexual preference, this term is used to designate a statistical test used to determine whether results from a set of independently performed studies on a particular question are similar enough to make statistical pooling valid. Are the apples sufficiently red and the oranges sufficiently green to be able to add them up and report the total number of "orpples"? As in other matters, statistical tests do not guarantee clinical relevance.

Homogeneity

See heterogeneity.

--------------------------------------------------------------------------------

- I -

Incidence

The rate at which an event occurs in a defined population over time. To be distinguished from prevalence.

Intention-to-treat

Intentions... that with which the path to hell is lined. Patients assigned to a particular treatment group by the study protocol should be retained in that group for the purpose of analysis of the study results no matter what happens. Patients redefined or dropped from a study early on as a result of protocol violations unlikely to create bias may validly be considered exceptions to this rule.

Internal validity

See validity.

--------------------------------------------------------------------------------

- J -

(empty)

--------------------------------------------------------------------------------

- K -

(empty)

--------------------------------------------------------------------------------

- L -

Likelihood Ratio

An operator defined as the percentage of patients positive by gold standard for a particular disease, condition or injury who have a particular test result divided by the percentage of patients without the problem who have that same test result. A likelihood ratio of two means that the test result in question is twice as likely to come a patient with the problem as it is from a patient without the problem. The LR may be derived from reported sensitivity and specificity or from a clear understanding of the above definition. To see how the LR is used, see Bayesâ€˜ Theorem; to actually use it, see the nomogram.

To see how the Likelihood Ratio is generated, use the calculator

--------------------------------------------------------------------------------

- M -

Meta-analysis

A review of a focused clinical question following rigorous methodological criteria and employing statistical techniques to combine data from independently performed studies on that question. To learn more, see the Userâ€™s Guide.

--------------------------------------------------------------------------------

- N -

Nomogram for Likelihood Ratio

Null hypothesis

What do you do when you want others to be maximally impressed with what you do? You DECREASE EXPECTATIONS, then what you do accomplish looks even better! The null hypothesis is the assumption that there is no difference between the groups and that the treatment you are studying has no effect. Any difference in outcome actually observed between the groups is then evaluated in relationship to the "zero expectation" hypothesis.

Number needed to treat (NNT)

The number of patients who must receive a particular therapy for one to benefit. You might tell a patient that an NNT of 10 means that the chance that he/she will benefit in this way from the treatment is 1 in 10. To calculate NNT use the calculator.

--------------------------------------------------------------------------------

- O -

Observational study

Any study of therapy, prevention or harm in which the exposure is not assigned to the individual subject by the investigator(s). A synonym is "non-experimental"; examples are case-control and cohort studies.

Odds ratio

The odds of an event, understood best by those who enjoy wagers, is the number of times it occurred (a) divided by the number of times it didnâ€™t (, or a/b. This contrasts with the probability of an event which is the number of times it occurred divided by the number of times it could have occurred, or a/a+b. The odds ratio is the ratio of the odds of an event in one group divided by the odds in another group. When the event rate or absolute risk in the control group is small (less than 20% or so), then the odds ratio is very close to the relative risk.

--------------------------------------------------------------------------------

- P -

Placebo

The thing you give a study subject who has been assigned to the control group to make them think they are getting the treatment you are studying.

Point estimate

The exact result that has been observed in a study. The confidence interval tells you the range within which the result is likely to lie.

Post-test probability

The likelihood that your patient has the disease, condition or injury you are testing for at the moment the result of the test you (or someone) ordered is delivered to you. To calculate it you need the pretest probability or prevalence and also the likelihood ratio for the test in question. To do this, you could use Bayes theorem or, if you are lazy (and practical), use the nomogram.

Pre-test probability

At the point you order a diagnostic test, you already have some idea of how likely your patient is to have the disease, condition or injury in question. You think of this as small, medium or large. "Pretest probability" means putting a number on the estimate you have already made. A difference of 10% in either direction will not change the effect of the diagnostic test. Putting the number on your clinical estimate will, however, allow you to determine what the test result means, should you want to know. This is also called prevalence.

Prevalence

The proportion of people in a defined group who have a disease, condition or injury. In the context of diagnosis, this is also called "pre-test probability." To be distinguished from incidence.

Prospective study

Any study done forwards in time. This is particularly important in studies on therapy, prognosis or harm, where retrospective studies make hidden biases very likely.

Publication bias

A possible bias which can effect systematic overviews to the extent that studies on the question at hand with conflicting results may not have been published.

P value

The probability that the difference(s) observed between two or more groups in a study would occurred if there were no differences between the groups other than those created by random selection. The assumption underlying the p-value is the null hypothesis.

Power

The chance that an experimental study will correctly observe a statistically significant difference between the study groups. This may be considered the "sensitivity" of the study trial itself for detecting a difference when it is there.

--------------------------------------------------------------------------------

- Q -

(empty)

--------------------------------------------------------------------------------

- R -

Randomization

A technique which gives every patient an equal chance of winding up in any particular arm of a controlled clinical trial.

Randomized Controlled Trial

A controlled clinical trial in which the study groups are created through randomization.

Relative risk and its reduction

The probability of an event in one group divided by the probability of the same event in another group. Generally the event is a bad one and the rate in the therapy group (when it is a therapy study) is in the numerator. When a benefit has been observed, this ratio is less than one. Subtracting the ratio from one gives the relative risk reduction, which is the percentage by which the risk in the control group has been reduced by the therapy.

Reliability

Sometimes used loosely, this actually refers to the reproducibility of a measurement procedure. It is NOT the same as validity or applicability of a study.

Retrospective study

Any study in which the outcomes have already occurred before the study has begun.

Risk factor

Any aspect of an individualâ€™s life, behavior or inheritance which increases the likelihood of a disease, condition or injury.

--------------------------------------------------------------------------------

- S -

Sensitivity

The probability that a patient with a disease, condition or injury will test positive by a particular test for the problem.

Sensitivity analysis

An analytical procedure to determine how the results of a study would change if the facts were different or different studies included. This is chiefly important in meta-analysis or complex techniques such as decision analysis and cost-effectiveness analysis.

Specificity

The probability that patients without a particular disease, condition or injury will test negative for the problem by a particular test.

Statistical power

see Power

Statistical significance

A measure of how confidently an observed difference between two or more groups can be attributed to the study interventions. The p value is the most commonly encountered way of reporting statistical significance. The methods assume that the study is free of bias. Clinical significance is entirely independent from statistical significance.

Stratified randomization

A way of ensuring that the different groups in an experimental trial are balanced with respect to important factors which could effect outcome.

Spectrum

In a diagnosis study, the range of clinical presentations and of relevant disease advancement exhibited by the subjects included in the study.

Systematic overview

A formal review of a focused clinical question based on a comprehensive search strategy and structured critical appraisal.

--------------------------------------------------------------------------------

- T -

Threshold Probabilities

The level of suspicion at which your clinical decision changes

--------------------------------------------------------------------------------

- U -

User's Guide

A guide to using literature that was developed by the evidence based medicine group from McMaster University in Canada. For further information, please click on McMaster link on main menu.

Utility

Particularly for a diagnostic test, this is a measure of whether the patient is truly better off as a result of the test. A test could have high sensitivity, specificity and good likelihood ratios and still have low utility if it is very invasive or poses other risks or inconvenience to the patient. It belongs under the section of a diagnostic review.

--------------------------------------------------------------------------------

- V -

Validity

The degree to which the results of a study are likely to be true, believable and free of bias. This is entirely independent of the precision of the results (p value) and does not predict the of the results to your patients. For a detailed discussion of validity, Literature Class Validity Importance Applicability

All

Therapy

Diagnosis

Harm

Prognosis

Overview

Decision Analysis

Practice Guideline

Utilization Review

Outcome Analysis

--------------------------------------------------------------------------------

- W -

(empty)

--------------------------------------------------------------------------------

- X -

(empty)

--------------------------------------------------------------------------------

- Y -

(empty)

--------------------------------------------------------------------------------

- Z -

(empty) --------------------------------------------------------------------------------

Here are some articles to also read about this!!::

1) Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How to get started. JAMA 1993; 270: 2093-2095.

2) Guyatt GH, Sackett DL and Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA 1993; 270:2598-2601.

3) Guyatt GH, Sackett DL and Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? JAMA 1994; 271:59- 63.

4) Jaeschke R, Guyatt G and Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. Are the results of the study valid? JAMA 1994; 271 (5):389-391.

5) Jaeschke R, Gordon H, Guyatt G & Sackett DL. Users' guides to the medical literature. III. How to use an article about a diagnostic test. B. what are the results and will they help me in caring for my patients? JAMA 1994; 271:703-707.

6) Levine M, Walter S, Lee H, Haines T, Holbrook A & Moyer V. Users'guides to the medical literature. IV. How to use an article about harm. JAMA 1994; 271 (20):1615-1619.

7) Laupacis A, Wells G, Richardson S , Tugwell P. Users' guides to the medical literature. V. How to use an article about prognosis. JAMA 1994; 272:234-237.

Richardson WS , Detsky AS. Users' guides to the medical literature. VII. How to use a Clinical Decision Analysis. A. Are the results of the study valid? JAMA 1995; 273 (16):1292-1295.

9) Richardson WS, Detsky AS. Users' guides to the medical literature. VII. How to use a Clinical Decision Analysis. B. What are the results and will they help me in caring for my patients? JAMA 1995; 273 (20):1610-1613.

10) Guyatt GH, Naylor CD, Juniper E et al. Users' guides to the medical literature. XII. How to use articles about health-related quality of life. Evidence-Based Medicine Working Group. JAMA 1997; 277 (15):1232-1237.

11) Oxman AD, Cook DJ, Guyatt GH. Users guide to medical literature. VI. How to use an overview. JAMA 1994; 272:1367-71.

12) Wilson MC, Hayward RS, Tunis SR, Bass EB, Guyatt GH. Users guide to medical literature VIII How to use clinical practice guidelines (A). Are the recommendations valid. JAMA 1995; 274:570-4.

13) Wilson MC, Hayward RS, Tunis SR, Bass EB, Guyatt GH. Users guide to medical literature VIII. How to use clinical practice guidelines (. What are the recommendations and will they help you in caring for your patient? JAMA 1995; 274:1630-2

14) Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ. Users guide to medical literature IX. A method for grading health care recomendations. JAMA 1995; 274:1800-4

15) Naylor CD , Guyatt GH. Users guide to medical literature X. How to use an article reporting variations in the outcomes of health services. JAMA 1996; 275:554-8.

16) Naylor CD , Guyatt GH. Users guide to medical literature XI. How to use an article about a clinical utilization review. JAMA 1996; 275:1435-9

17) Drummond MF, Richardson WS, OBrien BJ, Levine M and Heyland D. Users guide to medical literature XIII. How to use an article on economic analysis of clinical practice: A. Are the results of the study valid? JAMA 1997; 277:1552-1557.

1 OBrien BJ, Heyland D, Richardson WS, Levine M , Drummond MF . Users guide to medical literature XIII How to use an article on economic analysis of clinical practice: B. What are the results and will they help me in caring for my patients? JAMA 1997; 277:1802-1806

19) Dans AL, Dans LF, Guyatt GH, Richarson S. Users guide to medical literature XIV. How to decide on the applicability of clinical trial results to your patient. JAMA 1998; 27; 279:545-9

20) Richardson S, Wilson M, Guyatt GH, Cook DJ, Nishikawa J. Users guide to medical literature XV. How to use an article about disease probability for differential diagnosis. JAMA 1999; 281:1214-9.

21) Guyatt GH, Sinclair J, Cook DJ, Glasziou P. Users guide to medical literature XVI. How to use a treatment recommendation. JAMA 1999; 281:1836-43.

22) Barratt A, Irwig L, Glasziou P, et.al. Users guide to medical literature XVII How to use guidelines and recommendations about screening. JAMA 1999; 281:2029

23) Randolph AG, Haynes RB, Wyatt JC, Cook DJ, Guyatt GH. Users guide to medical literature XVIII How to use an article evaluating the clinical impact of a computer-based clinical decision support system. JAMA 1999; 282: 67- 74.

Hope this helps,

ACE844

Ace844 · June 22, 2006

Hello Everyone,

Here's another 'take' on appraising the literature 'critically'...

Hope this Helps,

ACE844

(Reading Smart: Discovering What the Data Do and Don’t Say

By Elizabeth A. Criss @ RN, MEd)

When a commercial says "four out of five" people agree, what does that mean? The advertiser is hoping you think it means 80 percent of all people support that particular product or idea. But couldn’t it also mean something else? For instance, what if they had only asked five people for their opinions or mailed out only 10 surveys and received five responses – four people for, one against. There are many other possible combinations that could produce these numbers and still not represent 80 percent of the population. Is this wrong? It’s hard to say. The best response is probably that results, like beauty, are in the eyes of the beholder.

That’s all well and good for TV commercials, but this same "data torturing" can occur in medical research. Raw data generated by a project really doesn’t mean anything until it’s analyzed, and the tools used to analyze this information and the way data is compared determine what conclusions can be drawn. That can leave a lot of room for interpretation.

Let’s say you’re interested in finding the latest research on the pneumatic anti-shock garment (PASG). Flipping through the journals, you find a study evaluating the effect of PASGs on nontrauma patients. The abstract states this is a prospective study done on 300 patients during a 12-month period. The findings of the study indicate that PASGs are of little value in the treatment of nontrauma patients in the prehospital environment.

Intrigued by these findings, you read the article. The results section describes the 300 patients. You note that the study divided the patients into two groups: blood pressure (BP) > 60 mmHg and BP < 60 mmHg. To assist in understanding the results, the authors include Tables 1 through 3.

Moving on to the discussion, you note the authors’ conclusion: "For the majority of nontraumatic patients, the PASG is not beneficial and possibly increases mortality." To support their conclusion, there is a more lengthy and detailed explanation than you found in the abstract. Looking back over the information in Tables 1 and 2, you believe this to be a reasonable conclusion.

But what about Table 3? Didn’t it demonstrate that PASG use in these patients reduced mortality? It did, but the authors’ conclusions are still valid. It’s important to note that the authors said "in the majority of "patients," not that the results applied to all patients. So, why didn’t the authors make more reference to the group in Table 3?

Table 3 highlights a subgroup, patients, with a BP of < 60 mmHg that was positively affected by PASG use. Sometimes groups like this are left out due to the small number of patients in the subgroup; a small sample size does not allow the authors to calculate meaningful statistics or draw any significant conclusions. Without statistics, the most the authors can do is discuss the result as a possible trend. Nevertheless, the authors should at least mention this group as a potential area for future research. Another possibility for leaving subgroups out of a discussion is that they did not support the author’s original hypothesis. Although not entirely ethical, this has been done.

The point of all this is that it is important to understand that data can be manipulated. Researchers will sometimes drop patients who don’t fit the desired hypothesis or support a certain position. It is important for you, the reader, to scrutinize the literature and account for all the patients. If the authors say "majority," instead of "all," find out where the rest of the population went. Be suspicious. Ask yourself if these patients were deliberately left out, or if the sample was just too small to be meaningful.

Most of the research published today is well-controlled and scrutinized by professional review panels. However, it doesn’t hurt to become critical reader and ask questions.

Elizabeth Criss, RN, MEd, serves on the Prehospital Care Research Forum Board of Advisers, is a senior research associate at the University of Arizona in Tucson and a base hospital coordinator at University medical Center in Tucson.

This article was reprinted from JEMS, March 1994.

Table 1

All Study Participants

Number of Patients Number of Deaths Percent Mortality

PASG 165 50 30.3

No PASG 135 30 22.3

Table 2

Patients with BP>60 mmHg

Number of Patients Number of Deaths Percent Mortality

PASG 115 44 38.3

No PASG 102 22 21.6

Table 3

Patients with BP<60 mmHg

Number of Patients Number of Deaths Percent Mortality

PASG 50 6 12

No PASG 33 8 24.2

Ace844 · June 22, 2006

(4/17/2002 Reading Emergency Medical Services Literature p. 1 / 1

It is important to have a basic understanding of research study prior to interpreting research articles and abstracts.

Dr. Jim Upchurch and Jeri D. Pullum illustrate the significance of scrutinizing EMS research articles. Special thanks to Marc Colbeck and his stafffor sharing this great article to our readers.Be sure to visit www.paramedicine.com By Jim Upchurch @ M.D., NREMTPandJeri Pullum, MS)

Introduction:Emergency Medical Services (EMS) literature is the printed word concerning EMS. This includes articles, books, computer software and WebPages. The literature can be divided into two broad categories: research literature and non-research literature. Both serve an important purpose. Research literature helps to direct the practice of EMS by showing us what works and what doesn’t. The non-research literature may also change our EMS practice through the spread of information concerning various topics in EMS. These topics can be educational, personal opinion, survey data, etc. The article you are currently reading is non-research literature with the intent to describe two categories of EMS literature and offer an approach to effective reading.EMS is a broad field dealing with the medical care provided for patients who become suddenly ill or injured. The field includes both out-of-hospital and in-hospital emergency healthcare providers.

There are publications devoted entirely to EMS, such as JEMS and Prehospital Emergency Care, and there are general medical journals that regularly print articles about EMS. Until the past decade, virtually everything that was considered literature was in print form: books, magazines, journals and newspapers. A growing source for literature is the Internet. Readers must apply the same critical reading skills to Internet sources as they do to print sources.Those who know the literature gain a better understanding of the field and can use that information to continually improve their out-of-hospital medical practice. Research LiteratureAccording to Webster, research is the careful, systematic study and investigation in some field of knowledge, undertaken to discover or establish facts or principles. Our interest is in the field of medicine.

Articles published on medical research can be thought of as a snapshot of the medical topic being investigated. This snapshot gives us one view of a problem. Sometimes that single view is so powerful as to change our patient approach through just one study. One example is the investigation of penetrating chest trauma and treatment with the Military Anti-Shock Trousers (MAST), now often referred to as the Pneumatic Anti-Shock Garment (PASG). This study was done in Houston, Texas, several years ago, and clearly demonstrated that patients who were shot or stabbed in the chest died more often when MAST were applied than when they were not. This one study has changed the way we treat penetrating chest trauma in the field. However, the benefit of MAST or PASG in other situations remains unanswered due to a lack of research. It often takes several research projects or snapshots to give us a good picture of the subject under investigation.

What is EMS research literature? Research articles are generally published in medical journals that are issued several times a year. This helps to distribute the results of medical research faster than waiting for textbooks to be released or updated. Prehospital Emergency Care is currently the only medical journal dedicated to out-of-hospital medicine. EMS research articles are also found in the Annals of Emergency Medicine and occasionally in other medical journals. EMS research occupies a very small space in the overall field of medical research. Other research not specifically EMS also can impact out-of-hospital practice. The ongoing research in the treatment of heart disease is one major example.So, what is a ‘good’ research article? It is one that gives us a true snapshot of the subject studied. The first step toward finding these articles is to look in a refereed medical journal. The author(s) published in these journals must work their way into the journal by passing one or more in-depth reviews of their article. The reviews are performed by one or more experts on the research subject contained in the article. It can take a year or longer before the article is accepted for publication.

Journals that require a review process will identify this fact inside their journal. Prehospital Emergency Care and Annals of Emergency Medicine are two examples of refereed medical journals. After obtaining an article to read (or perhaps, one provided by your medical director), how do you get the most from your reading? First, you will need access to a dictionary (English and medical). It is important to look up words that are unfamiliar; otherwise, an important message from the article may be lost. A terminology gap can exist between what the author writes and the reader’s word memory bank. Dictionaries help close the gap. Other useful tools may include medical texts, drug references, etc. Next, it is helpful to understand how a research article is put together.

In general, research articles are divided in to the Study Objective or Introduction, Methods, Results, Discussion and Conclusion. All but the Discussion is briefly covered at the beginning of an article. This is called the Abstract. The remainder of the research article discusses all divisions in detail.The Study Objective or Introduction contains the research question. This part of the article should clearly describe what is being studied. Background information and a discussion of related studies (literature review) may also be provided here.Methods is a description of how the study was conducted. Retrospective and prospective methods are frequently used. Retrospective studies use previous patient contacts to look for answers to the study objective. This is accomplished by reviewing medical records from a hospital or ambulance or from a database like the trauma registry. A prospective study is set up before patient contact. An example is the MAST study mentioned earlier. MAST was applied on one day but not the next. Patient response to treatment with and without MAST was documented in a prospective manner. The Results section describes the data collected by the researchers and whether it meant anything.

By time you get to this point in the article, you should have a fair understanding of what the researchers are looking for. Statistics help determine if the results of the study were real or due to outside influences. This is called statistical significance. There are different statistical tests used to calculate statistical significance. The selected test method should be identified along with the results. Statistical significance is important but does not guarantee clinical significance. Clinical significance is whether the study shows an impact on patients. The authors should identify the clinical significance they found. The Discussion should be straightforward and review the study up to this point. Look for a description of the limitations of the study. This is a clue that the author(s) are trying to present a true picture of their findings.Finally, the Conclusion should contain the author(s) response to the Study Objective based on the research. This may be included in the Discussion.Non-research LiteratureNon-research literature can take many forms: reporting on practices in the field, product reviews, announcements of new procedures, or feature stories on unusual or noteworthy people, agencies or events.

A critical reader can gain much valuable information from these articles. Product reviews help readers make informed decisions about purchases. Reporting on practices in the field helps readers stay up-to-date and helps determine whether practices in their own areas need updating. Feature stories can provide motivational material or can spawn ideas of how to adapt innovations to improve local EMS services.As with the research article, critical reading is important to gain the most from the non-research article. Although there is nothing wrong with reading for entertainment, the critical reader can improve comprehension through guided reading. You can perform a self-review of the information you read by answering the following questions.Who wrote the article? The critical reader should check the writer’s credentials. The credentials should support the notion that he or she has some knowledge of the topic under discussion. For example, an article on surgical airways written by a social worker probably should not carry much weight (unless the social worker is also a paramedic, R.N. or M.D.). However, that same social worker writing an article on critical incident stress is more likely to be useful. Also, the critical reader will pay attention to the credentials of other sources the writer is using for information. All this helps the reader shape an opinion about the article.

Many feature stories are project reports and are often submitted by someone involved in the project. There is nothing inherently wrong with this practice, but without the strict scientific standards that are required of research articles, some writers involved in the project may paint a biased “rosy” picture. The critical reader should be suspicious if the writer fails to acknowledge any adverse circumstances, results or features. What is the writer’s perspective? A physician may have a different view of the subject than an emergency medical technician, nurse or non-medical author. Medical professionals are not the only people who can write about EMS; people with many different backgrounds can offer great insight into various EMS topics. But their perspective is bound to be different.What is the article’s perspective? Does the writer presents various points of view? Obviously, an article about a controversy in the field should include all viewpoints. This consideration applies to other topics as well.

A product review should note poor features of the product, as well as the good features, indicating an unbiased evaluation. Critical readers can gain valuable insight from weighing opposing viewpoints for themselves or from understanding what does or does not work. What are the credentials of the source: the journal, magazine, newspaper or Web site? The critical reader should find out how long the source has been operating and who publishes it. A critical reader can get a good sense of the tone of the source by evaluating its contents over time. Web sites, in particular, can come and go in the blink of an eye. That does not mean that a new magazine or web site is automatically suspect, but the critical reader realizes that such sources have not had time to establish a reputation. Publishers can range from professional organizations, industry groups with specific special interests, educational institutions, or communications organizations. The publisher’s philosophy influences the types of information published and exerts at least some editorial control.

Does the author describe reading objectives and resources used in their work? You should be informed early on what it is you will gain by spending the time to read the author’s creation. Resources utilized by the author should be recorded in the offering.Does the information apply to you? The reader should consider both how well the information applies to his or her particular situation and the age of the information. The reader should look carefully at how the reported situation is both similar to and different from his or her own experience. Further, the reader must evaluate whether the information is dated or if it still applies.Does the author use correct grammar and spelling (yes grammar!)? If the text does not flow well it may be the result of poor sentence structure and/or spelling. A writer who does not take the time to use correct grammar and spelling may not have taken the time to develop good content. And speaking of content, the ideas presented should flow in a logical manner. Now, where do we find these articles? JEMS and EMS Magazine are major contributors to the written word on EMS. The articles are generally well written by experienced writers and provide useful information. Other sources include various web sites, Fire based journals, Search and Rescue journals, and occasionally medical journals other than Prehospital Emergency Care or Annals of Emergency Medicine.

SummaryWith the information overload provided by the Internet and the many publications that address EMS, it is important to develop a sense of what is worth your valuable reading time and to get the most from what you do read. The reader who can keep an open mind but remain skeptical has the most to gain from regular and studied reading of the EMS literature. Hopefully this introduction will point you in the right direction. So read! ReferencesRowntree D. Statistics Without Tears: A Primer for Non-Mathematicians. New York: Charles Scribner’s Sons; 1981.Cuddy PG, Elenbaas RM, Elenbaas JK: Evaluating the medical literature. Part I: abstract, introduction, methods. Ann Emerg Med. 1983; 12:549-555. Elenbaas JK Cuddy PG, Elenbaas RM: Evaluating the medical literature. Part III: results and discussion. Ann Emerg Med. 1983; 12:679-686. Huck SW. Reading Statistics and Research, 2nd ed. New York: Addison Wesley Longman; 2000.Merriam-Webster’s Collegiate Dictionary. Springfield, MA: Merriam-Webster, Inc. 1999; 995.

About the authors:Jim Upchurch. Dr. Upchurch, is a family physician by training, emergency physician by geography, and practicing EMT. He has been course coordinator and medical director for a series of paramedic courses since 1988 that have combined face-to-face instruction with distance learning techniques. He can be reached at upchurch@mcn.net.Jeri D. Pullum. Jeri has spent the past 10 years working with rural EMS providers, primarily on training projects for the Critical Illness and Trauma Foundation. She has designed, produced and authored numerous interactive computer-based training programs and contributes substantially to both the development of grant projects and in overseeing their completion. Using her background in print journalism and bachelor’s degree in journalism from the University of Montana, Jeri writes and edits scripts, training materials, proposals, reports and newsletters. She was an editor for a daily newspaper and worked for several years for a video production company. Jeri has a master’s degree in distance education and instructional technology. You can contact her at jpullum@attbi.org.

Ace844 · July 23, 2006

(Michael G. Stewart @ MD, MPH

The Bobby R. Alford Department of Otorhinolaryngology and Communicative Sciences

Baylor College of Medicine

Houston, Texas USA

Ronald B. Kuppersmith, MD, MBA

Section of Otolaryngology-Head and Neck Surgery

Virginia Mason Medical Center

Seattle, Washington USA

Aletta S. Moore, BA, Dip Lib

The Bobby R. Alford Department of Otorhinolaryngology and Communicative Sciences

Baylor College of Medicine

Houston, Texas USA)

Although we have seen an unprecedented increase in the quantity of information available, the quality of our understanding hasn't improved much at all. Indeed, the increase in information has only made the haystack larger, while the needle hasn't grown a millimeter."

-- D. Weinberger[1]

The use of computers has revolutionized the act of searching the medical literature, and the number of physicians performing their own searches continues to increase significantly.[2][3] This is the result of several converging trends:

The compendium of published medical literature is enormous and thus is not immediately accessible to most clinicians.

The volume of new information added to the literature is expanding quickly.

Practicing physicians are becoming more computer literate, and medical schools and residency-training programs are teaching students and residents how to integrate computers into their everyday practice.

There are now few, if any, medical offices that do not have computers.

The older, cumbersome access to searching medical databases has evolved into easier "point-and-click" or menu-driven interfaces.

Evidence-based medicine continues to grow in importance.[4]

In the past, literature searches were used primarily for research and manuscript preparation. The continuing importance of this type of literature search was emphasized during the recent controversy over the death of a research subject, in which an inadequate literature search was cited as a contributing factor.[5] Today, however, computer-assisted literature searches are being used to assist in patient care and clinical problem solving,[6][7][8] and many computer-assisted systematic literature reviews have been published in the contemporary literature.[9][10] Literature searches are reported to be extremely

Literature searches are reported to be extremely useful in several aspects of patient care, including choosing the most appropriate diagnostic tests, properly diagnosing medical conditions, and developing and implementing suitable treatment plans.

useful in several aspects of patient care, including choosing the most appropriate diagnostic tests, properly diagnosing medical conditions, and developing and implementing suitable treatment plans.[7][8] Users also have reported that findings from literature searches have resulted in improved patient outcomes overall and, in some cases, have contributed to the implementation of life- or organ-saving treatments.[8] In fact, in one study, the use of online literature searches early in the course of hospitalization resulted in significantly lower hospital costs and charges and in shorter lengths of stay than when literature searches were not performed in matched control patients with the same diagnoses.[6] Even more impressive, the patients in the literature-search group had a higher average level of illness severity than did the patients in the control group, which normally results in higher costs and longer length of stay.

Quality Of Literature Searches

The quality of literature searches is traditionally assessed with two variables: recall and precision.[11] Recall refers to the ratio of relevant citations retrieved by a search to the actual number of relevant citations present in the literature. Precision refers to the ratio of relevant citations retrieved by a search to the number of total citations retrieved by a search. In the past, searches often were mediated by medical librarians, who were knowledgeable and trained in search techniques; these were considered the "gold standard" of literature searches. Studies have shown, however, that clinician searches can approach the quality of librarian searches after a relatively short period of training and practice.[3]

Available Sources Of Literature Searches

Databases maintained through the National Library of Medicine

MEDLINE is a database of journal citations that covers general medicine, nursing, dentistry, veterinary medicine, health care systems, and preclinical sciences. As of early 2002, MEDLINE contained citations to over 11 million articles from over 4600 biomedical journals from the United States and 70 other countries dating back as far as 1966. Approximately 460,000 references were added in 2001.

In 1997, the National Library of Medicine (NLM) began providing free, unlimited access to MEDLINE on the Internet through a search interface called PubMed, which was developed in association with the National Center for Biotechnology Information at the NLM. Its search engine, called Entrez, is flexible and powerful, allowing users to perform both simple and complex searches. PubMed can be accessed through links at the NLM Web site (www.nlm.nih.gov ) or directly (www.ncbi.nlm.nih.gov/entrez/query.fcgi ).

Changes and improvements continue to be made to PubMed. Most aim to synthesize and "add value" to the information. For example, links from PubMed articles refer users to gene sequences in public databases such as Genbank or to the National Institutes of Health (NIH)-maintained Online Mendelian Inheritance in Man database. Plans are underway to hyperlink phrases in PubMed abstracts to relevant sections of major textbooks.[12] The LinkOut feature is designed to point to additional Web-accessible resources (e.g., databases, consumer health information, research tools) and to offer direct access to the citation's full-text article. The NLM's goal -- to provide free access to full-text articles -- has met with some resistance, mainly from the publishers of journals; however, there is increasing support for providing the full text of at least the older articles free of charge. Many newer articles probably will remain embargoed for some time.

Because indexing articles often takes time, MEDLINE had published a separate PREMEDLINE database, which contained information supplied during the prepublication period by publishers of medical and scientific journals. This information now has been incorporated into MEDLINE, so delay is no longer an issue. Notably, this prepublication material does not include medical subject headings (MeSH) and must be searched with free text. Additionally, the NLM is gradually adding citations for material published before the mid-1960s through a database called OLDMEDLINE (available through the NLM gateway at gateway.nlm.nih.gov/gw/Cmd ), which currently contains citations from 1958 to 1965. Approximately 2 additional years of citations are being added during each calendar year.

Although there are other search engines available and some libraries and medical centers may have their own search interfaces, the free access and expanded features of PubMed have made it the most frequently used medical search engine. Thus, this article focuses primarily on techniques that help the user search efficiently using PubMed. These techniques, however, can be applied and used with other databases.

MEDLINE interfaces other than PubMed

In addition to PubMed, there are several commercially available interfaces to MEDLINE; however, because they are fairly expensive for an individual user, most are accessed through medical or hospital libraries or educational institutions. The ones most commonly used are maintained by Ovid Technologies (http://www.ovid.com/sales/medical.cfm ) and Silver Platter (www.silverplatter.com/catalog/meds.htm ).

Other important medical databases

In addition to MEDLINE, conscientious searchers should check other information sources. The Cochrane Library (www.update-software.com/cochrane ) is an excellent collection of research reviews on the effect of health care interventions. Because it has been indexed by MEDLINE since only 2000, physicians should make it a practice to search the Cochrane Library directly. It also includes the Database of Abstracts of Reviews of Effectiveness (DARE), and the Cochrane CENTRAL register of controlled trials, which indexes them back to the 1948 streptomycin-for-pulmonary-tuberculosis trial, which is generally regarded as the first modern controlled clinical trial.[13] A current individual subscription to the Cochrane Library is $285 per year, but it is often available through local medical or hospital libraries.

For clinical trials, the NIH maintains its own Web site (www.clinicaltrials.gov ), and the National Cancer Institute' site (www.cancer.gov/cancer_information ) includes information on clinical trials, research initiatives, and clinical practice guidelines. The British equivalent of the NIH's clinical trials registry is the metaRegister of controlled trials (www.controlled-trials.com ), which can be a fruitful source of information.

Exerpta Medica's EMBASE (www.embase.com ), which is available free through many medical and hospital libraries or through individual subscriptions, is a bibliographic database of 3500 journals and concentrates on drug information; therefore, it is especially useful for identifying articles on controlled clinical trials. Recent research has shown the importance of searching both MEDLINE and EMBASE to ensure comprehensive coverage.[14]

Citation indexing, such as with the Institute for Scientific Information's Science Citation Index (www.isinet.com/isi/products/citation/sci/ ) also can add another dimension to medical literature searching. By following chains of citations, the benefits of old-fashioned browsing can be achieved.

For the clinician who is interested in evidence-based medicine, the American College of Physicians-American Society of Internal Medicine sponsors ACP Journal Club (www.acpjc.org ) and Best Evidence (www.acponline.org/catalog/electronic/best_evidence.htm ), which are useful sites for finding summaries of high-quality studies and are available to nonmembers for $85 per year.

For guidelines and consensus statements, the Agency for Healthcare Research and Quality in partnership with the American Medical Association and the American Association of Health Plans sponsors the National Guideline Clearinghouse (www.guidelines.gov ), which is a good start. Other useful sites include InfoPOEMS Database from the Journal of Family Practitioners (www.infopoems.com/index.cfm ), the CDC Recommends Prevention Guidelines System (www.phppo.cdc.gov/cdcRecommends/AdvSearchV.asp ), and NIH Consensus Statements (text.nlm.nih.gov/nih/upload-v3/CDC_Statements/cdc.html#list ).

MDConsult from Elsevier Science provides full-text access to approximately 40 medical textbooks, 50 medical journals, comprehensive drug information, and more than 600 clinical practice guidelines.

MDConsult (www.mdconsult.com ) from Elsevier Science provides full-text access to approximately 40 medical textbooks, 50 medical journals, comprehensive drug information, and more than 600 clinical practice guidelines. The service is subscription based, but many medical schools and health care systems license the service and make it available to physicians in their systems.

Additionally, the following otolaryngology journals all provide free full-text access to current subscribers and have many value-added features over the actual journals: Archives of Otolaryngology-Head and Neck Surgery (www.archotol.ama-assn.org ), Annals of Otology, Rhinology, and Laryngology (www.annals.com ), Laryngoscope (www.laryngoscope.com ), and Otolaryngology-Head and Neck Surgery (www.elsevierhealth.com/oto ).

Finally, Lexis-Nexis (www.lexisnexis.com ) is potentially an extremely valuable research resource. Although not focused on health care, it is a comprehensive database of legal, public record, business, and news information. Because Lexis-Nexis frequently is licensed to academic institutions, it is usually available to affiliated personnel. Additionally, access to specific documents can be purchased on a pay-per-view basis.

Search Techniques

Whatever database or search interface is used, the searcher should take advantage of all provided training materials to obtain the best possible search results. Most systems provide either a user manual or online help features. For achieving maximal effectiveness, the importance of reviewing the help information cannot be overemphasized and applies to any database interface (including Internet search engines). Fortunately, PubMed provides a helpful online tutorial, a list of frequently asked questions, and a comprehensive help file.

Regardless of the search interface used, the key factors for the success of any computer-based search are the search strategy and the search technique. The two most important methods of achieving these are (1) planning a strategy for searching before going on-line, and (2) monitoring the results of the search as it progresses. Similarly, understanding the techniques of efficient searching leads to higher-quality search results. In the following subsections, the authors summarize the basic concepts needed for successful literature searches.

Formulating the question

To search efficiently, one must formulate an answerable question, which can then be transformed into a set of searchable concepts. One commonly used technique is to visualize the title of the article that would answer your question exactly. Then analyze that title, break it down into its component concepts, and perform a search on those concepts, first individually and then combined. For example, "Are tubes beneficial for otitis media?" is not a focused clinical question that lends itself to a literature search, because it does not follow the mnemonic PICO (patient, intervention, comparison, outcome).[15] [Table 1] shows how the PICO categories can focus a clinical question on otitis media. Depending on the intent of the question, however, the PICO categories may change. For example, if the question concerns a prognosis, then the "Comparison" category might remain blank.

Table 1. An example of formulating a searchable clinical question using the PICO system.

Heading Example

Patient or population Children with recurrent otitis media

Intervention Antibiotics

Comparison Watchful waiting

Outcome Reduced recurrences

MeSH terms

The entire search process relies on an accurate formulation of the search question, which then is translated into a search query by breaking it down into concepts. MEDLINE uses concepts to index articles. To standardize this indexing, the NLM created a special list of terms called MeSH terms, which are used in MEDLINE and other MeSH-based databases. MeSH terms constitute a thesaurus that contains all the concepts that appear in the literature. New MeSH terms are created and added to the thesaurus as significant new or modified concepts appear in the literature. Today, there are approximately 19,000 main MeSH terms. All scientific articles are indexed accurately, hierarchically, and uniformly into MEDLINE using an average of 10 to 12 MeSH terms that describe each article's content. Using MeSH terms, therefore, contributes to the precision and completeness of any search.

Users should be aware of the MeSH vocabulary and know what terms are not included. This requires reviewing the MeSH vocabulary itself, because MeSH terms are not always intuitive. By default, PubMed attempts to "map" the search queries to the closest MeSH term, but if this is not possible, the system performs only a free-text search. Furthermore, users should not rely on the expectation that commonly used terms will map to the correct MeSH term. For example, although nosebleed and its alternate spelling nose bleed both map to the MeSH term epistaxis, the term bloody nose does not. This is more often a problem for the nonprofessional searcher, but it illustrates the importance of using precise terminology.

Another example of the importance of using MeSH terms is trying to search using laryngeal cancer, which is not a MeSH term, instead of laryngeal neoplasms, which is. Using the latter ensures a more comprehensive search result.

There is a MeSH browser available from the main PubMed search page that should help users find the appropriate MeSH terms. Additionally, check the subheadings that can be applied to the MeSH term for which you plan to search by clicking on "Display Details" for the term. These MeSH subheadings allow users to focus the search more precisely to a diagnosis, drug therapy, surgery and so forth.

Choosing a MeSH term as a "major heading" also can focus a search. Although articles usually are indexed using 10 to 12 MeSH terms, only a few are defined as the major focus of an article. Searching a MeSH term as a major heading can be useful for narrowing a large search result into a more manageable number of citations.

Using the MeSH vocabulary requires some learning; however, the effort one puts into becoming familiar with the MeSH vocabulary pays off in improved search quality.

Searches also may be limited by language, geographic area, age group, gender, and a variety of other terms called check tags. Limiting searches in PubMed is simply done by using the "Limits" link immediately under the box in which the search terms are entered. There are also MeSH terms for publication type, which indicate the type of article rather than its concept (e.g., review, clinical trial, meta-analysis). The publication types also can be added through the "Limits" link or by entering the publication type followed by "[pt]" as a separate search term (e.g., "review[pt]").

Using the MeSH vocabulary requires some learning; however, the effort one puts into becoming familiar with the MeSH vocabulary pays off in improved search quality. Authors have compared literature searches performed using both free text and MeSH terms, and MeSH-based searches have higher recall and precision than do free-text searches.[11]

Free-text searches

In a free-text search, the computer looks for those particular words in either the title or the abstract of a citation. The obvious weakness of a free-text search is that all possible synonyms and variants of a term (e.g., caustic ingestion, corrosive burns, caustic stenosis, esophageal burn, and so forth) must be entered to ensure a comprehensive result (i.e., one with high recall). Additionally, PubMed and most systems also require the user to enter both singular and plural forms and variant spellings of each word. Another problem is that all of the articles that contain those words anywhere in the abstract or title are retrieved -- even articles that are not pertinent to the search topic.

Free-text searching can be useful when there are no appropriate MeSH terms for the search in question; however, even under those circumstances, free-text searching is used mostly in conjunction with the MeSH vocabulary.

Advanced free-text searching techniques that use "word stemming" (i.e., the expansion of synonyms) exist but are beyond the scope of this article.

Boolean search techniques

After you have identified your search concepts or terms, MEDLINE and most other databases allow the use of Boolean logic to improve search specificity. To use Boolean logic in searches, first dissect the search query into concepts (usually MeSH terms) and search each of these separately. Then combine them using the following Boolean operators: AND, OR, or NOT. These operators must be entered in the search window using capital letters. The operator AND between two terms retrieves only those articles in which both concepts appear. The operator OR retrieves any article in which either of the two terms appear. By using NOT before a term, one eliminates any article that contains the concept. For example, if a clinician is searching for clinical trials in sinusitis, an appropriate starting point for the search would be "sinusitis AND clinical trial" (both of which are MeSH terms); however, if the clinician is searching for articles on "sarcoidosis AND sinusitis," perhaps not all potentially relevant articles for sarcoidosis would be retrieved; some might be indexed under granulomatous disease. In this case, the following strategy can be used:

Search 1: "sinusitis"

Search 2: "sarcoidosis OR granulomatous disease"

Search 3: "sinusitis AND sarcoidosis OR granulomatous disease"

This sample search illustrates an important point: one should always check the MeSH vocabulary. It turns out that granulomatous disease is not a MeSH term and that its nearest match -- granulomatous disease, chronic -- refers to a rare immunodeficiency syndrome. On further searching of the MeSH browser, one discovers that sarcoidosis is indexed under the heading lymphoproliferative disorders. A review of other diagnoses under that heading demonstrates that it is a more appropriate MeSH term for this search. Thus, a more appropriate search would be as follows:

Search 1: "sinusitis"

Search 2: "sarcoidosis OR lymphoproliferative disorders"

Search 3: "sinusitis AND sarcoidosis OR lymphoproliferative disorders"

Although this search might result in the retrieval of many nonrelevant articles (e.g., sinusitis and lymphoma) -- a low search precision -- it may yield some articles of interest that would not have been identified using only "sarcoidosis AND sinusitis." If the clinician did not want those articles on lymphoma, he or she could do a fourth search:

Search 4: "sinusitis AND sarcoidosis OR lymphoproliferative disorders NOT lymphoma"

The searcher, however, should be cautious about using the Boolean operator NOT, especially in free-text searches. In the last search, using "NOT lymphoma" would rule out every article that contains the term lymphoma in its title or abstract, even if the article itself focused on sarcoidosis.

Other PubMed features

Sign In

TEACHING POINTS:: How to "read' a Journal/Study article

Recommended Posts

Ace844

Ace844

Ace844

Ace844

Join the conversation

Browse

Activity

My Activity Streams

Store

My Details

Support