Communication of Results on Risk Factors Studies: Confidence Intervals

 

Sorana-Daniela BOLBOACĂ1, Lorentz JÄNTSCHI2

 

1”Iuliu Haţieganu” University of Medicine and Pharmacy, 13 Emil Isac, 400023 Cluj-Napoca, Romania

2Technical University of Cluj-Napoca, 15 Constantin Daicoviciu, 400020 Cluj-Napoca, Romania

sbolboaca@umfcluj.ro, lori@academicdirect.org

 

 

           Abstract

Purpose: Starting from the hypothesis that confidence intervals are used in medical research as a criterion of trustworthiness and robustness of findings, the aim of the research was to determine whether the medical parameters communicated as results in abstracts of risk factors studies published in PubMed database and in Romania online journals are accompanied by confidence intervals.

Method: The search strategy included four keywords and some limitations (publication data, the language of publication, and studies on human). Four inclusion criterions were imposed. The obtained results were summarized and analyzed with Statistica.

Results: A number of 3191 were identified after applying the search strategy on PubMed database. Almost three and a half percent (n = 110) remained after inclusion the confidence intervals as keyword. Sixty articles out of one hundred and one accomplished the inclusion criteria. A number of sixty-five articles from nine Romanian online journals were identified. After applying the inclusion criterions, five articles out of sixty-five were included into the analysis. Testing the null hypothesis that there are not significant differences between the number of articles indexed in PubMed database and the number of articles published in Romanian online journals which refer as outcome the relative risk or odds ratio with associated confidence intervals, a p-value less than 0.0001 was obtained.

Conclusion: It can be conclude that the publication standards in Romanian medical journals must by arise in order to become aligned with the international trends and standards.

           Keywords

Risk Factors, Cohort Study, Relative Risk, Odds Ratio, Confidence Intervals (CIs)

 

 

Introduction

 

            The inclusion of the best available evidence into clinical decision is the main goal of evidence-based practice [1]. In spirit of evidence-based practice, the physicians must to be able to translate the knowledge resulted from medical research by including them into individual decisions [2], [3]. In the assessment of an article, the physicians must deal with fundamental statistics such as the standard deviation, the standard error and the confidence intervals (CIs) [4].

            Confidence intervals defines as an estimated range of values that is likely to include an unknown population parameter, the estimated range being calculates from a given set of sample data [5] are used nowadays as a criterion of assessment of the trustworthiness or robustness of the finding [6]. If independent sample are take repeatedly from same population, and the confidence interval is calculated for each sample, then a certain percentage (called confidence level) of the interval will include the unknown population parameter. Confidence intervals are usually computed for the 95%. The CIs offer to the physicians the possibilities to be more certain about the clinical value of a statistical parameter and to decide on what degree he/she can rely on the results [4].

            Starting from the hypothesis that confidence intervals and their widths are used in medical research as a criterion for results trustworthiness and robustness, the objective of the research was to determine whether the medical parameters communicated in abstracts as results of risk factors studies published in PubMed database and in Romania online journals are accompanied by associated CIs.

Material and Method

 

            Screening for risk factors studies were search using PubMed electronic database for articles in English and Romanian online journals for articles in Romanian.

            The following keywords were used in searching strategy: screening AND relative risk OR odds ration OR cohort OR/AND confidence intervals. In searching strategy the following limits were imposed:

·        Dates: 2003, February - 2006, February;

·        Human or animals: Humans;

·        Languages: English;

·        Type of article: Clinical Trial, Editorial, Meta-Analysis, Review, Guideline.

            For inclusion into the study the abstract of the reported research had to be a screening and had to have:

·        The abstract available;

·        Data concerning risk factors;

·        The results express as relative risk or odds ration;

·        A cohort design of the research.

            The confidence intervals keyword it was choused to be optional (OR/AND) because in the Romanian journals the confidence intervals it is not as frequently used as into the PubMed indexed journal. Also, the type of article was not imposed in searching the Romanian online journals.

            Studies identified through the above search strategy that met the inclusion criteria were included into the analysis. There were collected the following data:

(1) The name of statistical parameter;

(2) The confidence intervals associated with the parameter (as dichotomial variable: Yes/No);

(3) The abbreviation of the journal;

(4) Publication type.

            Data were summarized and analyzed with Statistica 6.0 software. The 95% CIs for proportions were calculated based on the binomial distribution hypothesis [7].

 

 

Results

 

            There were identified a number of 3191 after applying the search strategy on PubMed database. Almost three and a half percent (n = 110) remained after inclusion of confidence intervals keyword. Sixty articles out of one hundred and one accomplished the inclusion criteria. Five articles refer as results both statistical parameters (relative risk and odds ratio). The distributions of the type of statistical parameter and associated confidence intervals, and the type of article according with the journal abbreviation are presented in Table 1.

            The first step of searching the Romanian journal databases was represented by identification of online journals. There were identified a number of ninety-five Romanian medical journals. Sixty-three of them (66.31%, 95%CIs [55.80-75.78]) did not had an online version, eleven (11.22%, 95%CIs [6.33-19.99]) were accessible online after payment of a fees, and five (5.26%, 95%CIs [2.12-11.57]) had un-functional addresses.

            The second step was represented by application of the search strategy. There were identified sixty-five articles from nine journals (Stetoscop, Cercetări experimentale şi medico-chirurgicale, Timişoara Medical Journal, Romanian Journal of Internal Medicine, Observatorul Medical, Infomedica, Clujul Medical, Brain Aging International Journal, Acta Medica Transilvanica, and Revista Română de Psihiatrie).

            Thirty-five percent did not refer any statistical parameter. Fifty-eight percent (95% CIs [46.18-70.75]) refer as results the following statistical parameters:

·        Ratio: thirteen articles;

·        Risk of progression: one article;

·        Individual risk: one article;

·        Prevalence: five articles;

·        P-value: one article;

·        Mortality: one article;

·        Mean: one article;

·        Frequency: three articles;

·        Likelihood ration: one article.

            After applying of the inclusion criterions, five articles out of sixty-five were included into the analysis (95% CIs [3.10- 16.90]) (see Table 2).

 

Table 1. Type of statistical parameter according with type of article: PubMed database

Journal abbreviation

Statistical parameter

Type of article

RR

OR

CIs

Acad Emerg Med

1

 

1

Controlled Clinical Trial

Am Heart J

1

 

1

Multicenter Study

Am J Cardiol

1

 

1

Multicenter Study

Am J Obstet Gynecol

 

1

1

Clinical Trial

Am J Prev Med

 

1

1

Clinical Trial

Arch Gen Psychiatry

 

1

1

Meta-Analysis

Arch Neurol

1

 

1

Multicenter Study

Atherosclerosis

1

 

1

Clinical Trial

Birth Defects Res A Clin Mol Teratol

1

 

1

Multicenter Study

Breast

1

1

n.a.

Review

Can J Surg

 

1

1

Meta-Analysis

Cancer Causes Control

 

1

1

Multicenter Study

Cancer Epidemiol Biomarkers Prev

1

1

2

Clinical Trial

Cancer Sci

1

 

1

Clinical Trial

Cardiol Young

1

 

1

Multicenter Study

Chest

 

1

1

Meta-Analysis

CMAJ

1

1

1

Clinical Trial

Cochrane Database Syst Rev

 

12

10

5 Meta-Analysis + 5Review

Crit Care Med

1

1

2

Clinical Trial, Multicenter Study

Drugs Aging

1

 

1

Clinical Trial

Eur J Cancer

 

1

1

Clinical Trial

Exp Mol Med

1

 

1

Multicenter Study

Fertil Steril

1

 

1

Multicenter Study

Hernia

 

1

1

Multicenter Study

Int J Cancer

1

1

2

Clinical Trial + Multicenter Study

J Am Soc Nephrol

 

2

1

Clinical Trial + Review

J Card Surg

 

1

1

Meta-Analysis

J Clin Oncol

 

2

1

Multicenter Study + Meta Analysis

J Epidemiol Community Health

1

 

1

Clinical Trial

J Fam Pract

 

1

1

Clinical Trial

J Hypertens

 

1

1

Multicenter Study

J Natl Cancer Inst

2

2

4

2 Meta-Analysis + Multicenter Study

+ Randomized Controlled Trial

J Viral Hepat

 

1

n.a.

Review

J Womens Health (Larchmt)

1

1

1

Multicenter Study

Neurology

 

1

1

Guideline

Obstet Gynecol

 

3

3

2 Clinical Trial + Meta-Analysis

Pediatr Infect Dis J

1

 

1

Multicenter Study

Saudi Med J

 

1

1

Multicenter Study

Scand Cardiovasc J

1

 

1

Multicenter Study

Sex Transm Dis

1

 

1

Multicenter Study

Thromb Haemost

 

1

1

Meta-Analysis

Total

23

42

56

1 Controlled Clinical Trial; 8 Review;

19 Multicenter Study; 1 Guideline

15 Clinical Trial; 13 Meta-Analysis

RR = Relative Risk; OR = Odds Ratio; CIs = Confidence Intervals; n.a. = not available

 

Table 2. Type of statistical parameter according with type of article: Romanian online journals

Journal title

Statistical parameter

Type of article

RR

OR

CIs

Stetoscop

2

 

1

Medical News

Clinical Guideline

Experimental & Medical-Surgical Researches

2

1

 n.a

2 Original Article

Total

4

1

1

1 Medical News

1 Clinical Guideline

2 Original Articles

RR = Relative Risk; OR = Odds Ratio; CIs = Confidence Intervals; n.a. = not available

 

            Fifty-six out of sixty included articles from PubMed database refer as outcome relative risk or odds ration and associated confidence intervals (95%CIs [83.36-98.31]). One out of five articles from Romanian online journals refers as outcome relative risk with associated confidence intervals (95%CIs [4.0-76.0]).

            Testing the null hypothesis that there are not significant differences between the number of articles indexed in PubMed database and the number of articles published in Romanian online journals which refer as outcome the relative risk or odds ratio with associated confidence intervals, a p-value less than 0.0001 was obtained.

 

 

Discussion

 

            In specialty literature there is a trend which impose specification of the confidence intervals associated with statistical parameters, being considered as useful tools in study results interpretations and in generalizing of results [8]. Thus, confidence intervals are found in study of assessment of a new therapeutic attitude [9], [10], the effectiveness and usefulness of a diagnostic test [11], [12], in epidemiological studies [13], as well as in research which present the evaluation of risk factors [14], [15]. The main question at which the confidence intervals respond in medical studies is “What is the range of real effects that are compatible with used data?”. When 95% confidence intervals are used, these confidence intervals will contain in 95% of the time the true value of the outcome effect (eg. relative risk, odds ratio, etc). In interpretation of confidence intervals for relative risk or odds ratio, the findings are non-significant if the confidence intervals embraces the value of no effect, which in these cases is equal with one. The most important reasons of using confidence intervals associated with relative risk and odds ration in risk factors studies are as follows: (1) emphasized precision of estimation; (2) its meaning is reasonable clear for any physicians (there is no confusion between significance and magnitude as in p-value interpretation); (3) nowadays journals require them.

            Analyzing the results of study two remarks can be made. First remark refers the use of confidence intervals associated with relative risk or odds ration in risk factor assessments. Ninety-three percent from articles indexed in PubMed and published between 2003, February and 2006, February proved to take into consideration the necessity of confidence intervals. Opposite, twenty percent from the articles published in Romanian online journals provide the confidence intervals associated with relative risk, even if the necessity of using the confidence intervals was first published in specialty literature in 1986 [8]. In reality, the differences are bigger, because the sample size of articles published in PubMed is twenty-two time bigger comparing with the sample size of articles published in Romanian online journals. Second remark is indirect related with the objective of the study and refers the type of articles in which the statistical parameters of interest were used. Looking at the results of articles indexed in PubMed, it can be observed that the majority of the articles were multicenter studies (eighteen articles), followed by the clinical trials (fifteen articles) and meta-analysis (thirteen articles). Regarding the type of articles, there could not be made any comparisons because the types of articles published in Romanian online journals are not the same as that indexed in PubMed, and on the other hand the number of articles is to less (two original articles, one clinical guideline, and one medical news, see Table 2).

            The absence of the confidence intervals associated with relative risk or odds ratio in the articles published into Romanian online journals could be due to unfamiliarity of Romanian researcher with confidence intervals. The best solution for dismiss the unfamiliarity with confidence intervals could be short training on basic statistics for physicians interested on the subject or a didactic articles on confidence intervals. Another dimension of the problem refers the absence of the instruments which to compute the confidence intervals for relative risk and odds ratio. Even if for example SPSS and Statistica software did not provide these types of calculations, the physicians could use EpiInfo free software, which is able to compute based on a normality distribution the confidence intervals for relative risk and odds ratio.

            Based on this study, it can be conclude that the publication standards in Romanian medical journals must by arise in order to become aligned with the international trends and standards. The request of confidence intervals associated with statistical parameters had to be imposed by the medical journals editors and Romanian authors must to comply and to include them in original articles.

 

 

Acknowledgement

            Research was partly supported through project ET/46/2006 by UEFISCSU Romania.

 

 

References

 


[1] Sackett D. L., Rosenberg W. M., Gray J. A., Haynes R. B., Richardson W. S., Evidence-based medicine: what it is and what it isn’t, Br Med J, 1996, 312(7023), p. 71-72.

[2] Kitson A., Harvey G., McCormack B., Enabling the implementation of evidence based practice: a conceptual framework, Qual Health Care, 1998, 7(3), p. 149-158.

[3] Sussman S., Valente T. W., Rohrbach L. A., Skara S., Pentz M. A., Translation in the health professions: Converting science into action, Eval Health Prof, 2006, 29(1), p. 7-32.

[4] Medina S. L., Zurakowschi D., Measurement Variability and Confidence Intervals in Medicine: Why Should Radiologist Care?, Radiology, 2003, 226, p. 297-301.

[5] Huw D., What are confidence intervals? What is…., Hayward Group Publication, 2003, 3, p. 1-9.

[6] Bender R., Berg G., Zeeb H., Tutorial: Using confidence curves in medical research, Biom J, 2005, 47(2), p. 237-247.

[7] BinomialDistribution, ©2005, Virtual Library of Free Software, AcademicDirect [cited March 2006]. Available from: URL: http://vl.academicdirect.org/applied_statistics/binomial_distribution/

[8] Gardner M. J., Altman D. G., Confidence intervals rather than P values: estimation rather than hypothesis testing, Br Med J (Clin Res Ed), 1986, 292(6522), p. 746-750.

[9] Guyatt G., Jaeschke R., Heddle N., Cook D., Shannon H., Walter S., Basic statistics for clinicians: 2. Interpreting study results: confidence intervals, CMAJ, 1995, 152(2), p. 169-173.

[10] Scheen A. J., Assessment of therapeutic efficacy, an essential step in evidence-based medicine, Rev Med Liege, 2000, 55(4), p. 206-210.

[11] Newcombe R. G., Two-sided confidence intervals for the single proportion: Comparison of seven methods, Stat Med, 1998, 17, p. 857-872.

[12] Simel D. L., Samsa G. P., Matchar D. B., Likelihood ratios with confidence: sample size estimation for diagnostic test studies, J Clin Epidemiol, 1991, 44, p. 763-770.

[13] Bewick V., Cheek L., Ball J., Statistics review 11: assessing risk, Crit Care, 2004, 8(4), p. 287-291.

[14] Zhu K., Hunter S., Payne-Wilks K., Sutcliffe C., Bentley C., Roland C. L., Williams S. M., Potential differences in breast cancer risk factors based on CYP1A1 MspI and African-American-specific genotypes, Ethn Dis, 2006, 16(1), p. 207-215.

[15] Bosetti C., Gallus S., La Vecchia C., Aspirin and cancer risk: an updated quantitative review to 2005, Cancer Causes Control, 2006, 17(7), p. 871-888.