Communication of Results on Risk Factors Studies: Confidence Intervals
SoranaDaniela BOLBOACĂ^{1}, Lorentz JÄNTSCHI^{2}
^{1}”Iuliu Haţieganu” University of Medicine and Pharmacy, 13 Emil Isac, 400023 ClujNapoca, Romania
^{2}Technical University of ClujNapoca, 15 Constantin Daicoviciu, 400020 ClujNapoca, Romania
sbolboaca@umfcluj.ro, lori@academicdirect.org
Abstract
Purpose: Starting from the hypothesis that confidence intervals are used in medical research as a criterion of trustworthiness and robustness of findings, the aim of the research was to determine whether the medical parameters communicated as results in abstracts of risk factors studies published in PubMed database and in Romania online journals are accompanied by confidence intervals.
Method: The search strategy included four keywords and some limitations (publication data, the language of publication, and studies on human). Four inclusion criterions were imposed. The obtained results were summarized and analyzed with Statistica.
Results: A number of 3191 were identified after applying the search strategy on PubMed database. Almost three and a half percent (n = 110) remained after inclusion the confidence intervals as keyword. Sixty articles out of one hundred and one accomplished the inclusion criteria. A number of sixtyfive articles from nine Romanian online journals were identified. After applying the inclusion criterions, five articles out of sixtyfive were included into the analysis. Testing the null hypothesis that there are not significant differences between the number of articles indexed in PubMed database and the number of articles published in Romanian online journals which refer as outcome the relative risk or odds ratio with associated confidence intervals, a pvalue less than 0.0001 was obtained.
Conclusion: It can be conclude that the publication standards in Romanian medical journals must by arise in order to become aligned with the international trends and standards.
Keywords
Risk Factors, Cohort Study, Relative Risk, Odds Ratio, Confidence Intervals (CIs)
Introduction
The inclusion of the best available evidence into clinical decision is the main goal of evidencebased practice [1]. In spirit of evidencebased practice, the physicians must to be able to translate the knowledge resulted from medical research by including them into individual decisions [2], [3]. In the assessment of an article, the physicians must deal with fundamental statistics such as the standard deviation, the standard error and the confidence intervals (CIs) [4].
Confidence intervals defines as an estimated range of values that is likely to include an unknown population parameter, the estimated range being calculates from a given set of sample data [5] are used nowadays as a criterion of assessment of the trustworthiness or robustness of the finding [6]. If independent sample are take repeatedly from same population, and the confidence interval is calculated for each sample, then a certain percentage (called confidence level) of the interval will include the unknown population parameter. Confidence intervals are usually computed for the 95%. The CIs offer to the physicians the possibilities to be more certain about the clinical value of a statistical parameter and to decide on what degree he/she can rely on the results [4].
Starting from the hypothesis that confidence intervals and their widths are used in medical research as a criterion for results trustworthiness and robustness, the objective of the research was to determine whether the medical parameters communicated in abstracts as results of risk factors studies published in PubMed database and in Romania online journals are accompanied by associated CIs.
Material and Method
Screening for risk factors studies were search using PubMed electronic database for articles in English and Romanian online journals for articles in Romanian.
The following keywords were used in searching strategy: screening AND relative risk OR odds ration OR cohort OR/AND confidence intervals. In searching strategy the following limits were imposed:
· Dates: 2003, February  2006, February;
· Human or animals: Humans;
· Languages: English;
· Type of article: Clinical Trial, Editorial, MetaAnalysis, Review, Guideline.
For inclusion into the study the abstract of the reported research had to be a screening and had to have:
· The abstract available;
· Data concerning risk factors;
· The results express as relative risk or odds ration;
· A cohort design of the research.
The confidence intervals keyword it was choused to be optional (OR/AND) because in the Romanian journals the confidence intervals it is not as frequently used as into the PubMed indexed journal. Also, the type of article was not imposed in searching the Romanian online journals.
Studies identified through the above search strategy that met the inclusion criteria were included into the analysis. There were collected the following data:
(1) The name of statistical parameter;
(2) The confidence intervals associated with the parameter (as dichotomial variable: Yes/No);
(3) The abbreviation of the journal;
(4) Publication type.
Data were summarized and analyzed with Statistica 6.0 software. The 95% CIs for proportions were calculated based on the binomial distribution hypothesis [7].
Results
There were identified a number of 3191 after applying the search strategy on PubMed database. Almost three and a half percent (n = 110) remained after inclusion of confidence intervals keyword. Sixty articles out of one hundred and one accomplished the inclusion criteria. Five articles refer as results both statistical parameters (relative risk and odds ratio). The distributions of the type of statistical parameter and associated confidence intervals, and the type of article according with the journal abbreviation are presented in Table 1.
The first step of searching the Romanian journal databases was represented by identification of online journals. There were identified a number of ninetyfive Romanian medical journals. Sixtythree of them (66.31%, 95%CIs [55.8075.78]) did not had an online version, eleven (11.22%, 95%CIs [6.3319.99]) were accessible online after payment of a fees, and five (5.26%, 95%CIs [2.1211.57]) had unfunctional addresses.
The second step was represented by application of the search strategy. There were identified sixtyfive articles from nine journals (Stetoscop, Cercetări experimentale şi medicochirurgicale, Timişoara Medical Journal, Romanian Journal of Internal Medicine, Observatorul Medical, Infomedica, Clujul Medical, Brain Aging International Journal, Acta Medica Transilvanica, and Revista Română de Psihiatrie).
Thirtyfive percent did not refer any statistical parameter. Fiftyeight percent (95% CIs [46.1870.75]) refer as results the following statistical parameters:
· Ratio: thirteen articles;
· Risk of progression: one article;
· Individual risk: one article;
· Prevalence: five articles;
· Pvalue: one article;
· Mortality: one article;
· Mean: one article;
· Frequency: three articles;
· Likelihood ration: one article.
After applying of the inclusion criterions, five articles out of sixtyfive were included into the analysis (95% CIs [3.10 16.90]) (see Table 2).
Table 1. Type of statistical parameter according with type of article: PubMed database
Journal abbreviation 
Statistical parameter 
Type of article 

RR 
OR 
CIs 

Acad Emerg Med 
1 

1 
Controlled Clinical Trial 
Am Heart J 
1 

1 
Multicenter Study 
Am J Cardiol 
1 

1 
Multicenter Study 
Am J Obstet Gynecol 

1 
1 
Clinical Trial 
Am J Prev Med 

1 
1 
Clinical Trial 
Arch Gen Psychiatry 

1 
1 
MetaAnalysis 
Arch Neurol 
1 

1 
Multicenter Study 
Atherosclerosis 
1 

1 
Clinical Trial 
Birth Defects Res A Clin Mol Teratol 
1 

1 
Multicenter Study 
Breast 
1 
1 
n.a. 
Review 
Can J Surg 

1 
1 
MetaAnalysis 
Cancer Causes Control 

1 
1 
Multicenter Study 
Cancer Epidemiol Biomarkers Prev 
1 
1 
2 
Clinical Trial 
Cancer Sci 
1 

1 
Clinical Trial 
Cardiol Young 
1 

1 
Multicenter Study 
Chest 

1 
1 
MetaAnalysis 
CMAJ 
1 
1 
1 
Clinical Trial 
Cochrane Database Syst Rev 

12 
10 
5 MetaAnalysis + 5Review 
Crit Care Med 
1 
1 
2 
Clinical Trial, Multicenter Study 
Drugs Aging 
1 

1 
Clinical Trial 
Eur J Cancer 

1 
1 
Clinical Trial 
Exp Mol Med 
1 

1 
Multicenter Study 
Fertil Steril 
1 

1 
Multicenter Study 
Hernia 

1 
1 
Multicenter Study 
Int J Cancer 
1 
1 
2 
Clinical Trial + Multicenter Study 
J Am Soc Nephrol 

2 
1 
Clinical Trial + Review 
J Card Surg 

1 
1 
MetaAnalysis 
J Clin Oncol 

2 
1 
Multicenter Study + Meta Analysis 
J Epidemiol Community Health 
1 

1 
Clinical Trial 
J Fam Pract 

1 
1 
Clinical Trial 
J Hypertens 

1 
1 
Multicenter Study 
J Natl Cancer Inst 
2 
2 
4 
2 MetaAnalysis + Multicenter Study + Randomized Controlled Trial 
J Viral Hepat 

1 
n.a. 
Review 
J Womens Health (Larchmt) 
1 
1 
1 
Multicenter Study 
Neurology 

1 
1 
Guideline 
Obstet Gynecol 

3 
3 
2 Clinical Trial + MetaAnalysis 
Pediatr Infect Dis J 
1 

1 
Multicenter Study 
Saudi Med J 

1 
1 
Multicenter Study 
Scand Cardiovasc J 
1 

1 
Multicenter Study 
Sex Transm Dis 
1 

1 
Multicenter Study 
Thromb Haemost 

1 
1 
MetaAnalysis 
Total 
23 
42 
56 
1 Controlled Clinical Trial; 8 Review; 19 Multicenter Study; 1 Guideline 15 Clinical Trial; 13 MetaAnalysis 
RR = Relative Risk; OR = Odds Ratio; CIs = Confidence Intervals; n.a. = not available 
Table 2. Type of statistical parameter according with type of article: Romanian online journals
Journal title 
Statistical parameter 
Type of article 

RR 
OR 
CIs 

Stetoscop 
2 

1 
Medical News Clinical Guideline 
Experimental & MedicalSurgical Researches 
2 
1 
n.a 
2 Original Article 
Total 
4 
1 
1 
1 Medical News 1 Clinical Guideline 2 Original Articles 
RR = Relative Risk; OR = Odds Ratio; CIs = Confidence Intervals; n.a. = not available 
Fiftysix out of sixty included articles from PubMed database refer as outcome relative risk or odds ration and associated confidence intervals (95%CIs [83.3698.31]). One out of five articles from Romanian online journals refers as outcome relative risk with associated confidence intervals (95%CIs [4.076.0]).
Testing the null hypothesis that there are not significant differences between the number of articles indexed in PubMed database and the number of articles published in Romanian online journals which refer as outcome the relative risk or odds ratio with associated confidence intervals, a pvalue less than 0.0001 was obtained.
Discussion
In specialty literature there is a trend which impose specification of the confidence intervals associated with statistical parameters, being considered as useful tools in study results interpretations and in generalizing of results [8]. Thus, confidence intervals are found in study of assessment of a new therapeutic attitude [9], [10], the effectiveness and usefulness of a diagnostic test [11], [12], in epidemiological studies [13], as well as in research which present the evaluation of risk factors [14], [15]. The main question at which the confidence intervals respond in medical studies is “What is the range of real effects that are compatible with used data?”. When 95% confidence intervals are used, these confidence intervals will contain in 95% of the time the true value of the outcome effect (eg. relative risk, odds ratio, etc). In interpretation of confidence intervals for relative risk or odds ratio, the findings are nonsignificant if the confidence intervals embraces the value of no effect, which in these cases is equal with one. The most important reasons of using confidence intervals associated with relative risk and odds ration in risk factors studies are as follows: (1) emphasized precision of estimation; (2) its meaning is reasonable clear for any physicians (there is no confusion between significance and magnitude as in pvalue interpretation); (3) nowadays journals require them.
Analyzing the results of study two remarks can be made. First remark refers the use of confidence intervals associated with relative risk or odds ration in risk factor assessments. Ninetythree percent from articles indexed in PubMed and published between 2003, February and 2006, February proved to take into consideration the necessity of confidence intervals. Opposite, twenty percent from the articles published in Romanian online journals provide the confidence intervals associated with relative risk, even if the necessity of using the confidence intervals was first published in specialty literature in 1986 [8]. In reality, the differences are bigger, because the sample size of articles published in PubMed is twentytwo time bigger comparing with the sample size of articles published in Romanian online journals. Second remark is indirect related with the objective of the study and refers the type of articles in which the statistical parameters of interest were used. Looking at the results of articles indexed in PubMed, it can be observed that the majority of the articles were multicenter studies (eighteen articles), followed by the clinical trials (fifteen articles) and metaanalysis (thirteen articles). Regarding the type of articles, there could not be made any comparisons because the types of articles published in Romanian online journals are not the same as that indexed in PubMed, and on the other hand the number of articles is to less (two original articles, one clinical guideline, and one medical news, see Table 2).
The absence of the confidence intervals associated with relative risk or odds ratio in the articles published into Romanian online journals could be due to unfamiliarity of Romanian researcher with confidence intervals. The best solution for dismiss the unfamiliarity with confidence intervals could be short training on basic statistics for physicians interested on the subject or a didactic articles on confidence intervals. Another dimension of the problem refers the absence of the instruments which to compute the confidence intervals for relative risk and odds ratio. Even if for example SPSS and Statistica software did not provide these types of calculations, the physicians could use EpiInfo free software, which is able to compute based on a normality distribution the confidence intervals for relative risk and odds ratio.
Based on this study, it can be conclude that the publication standards in Romanian medical journals must by arise in order to become aligned with the international trends and standards. The request of confidence intervals associated with statistical parameters had to be imposed by the medical journals editors and Romanian authors must to comply and to include them in original articles.
Acknowledgement
Research was partly supported through project ET/46/2006 by UEFISCSU Romania.
References
[1] Sackett D. L., Rosenberg W. M., Gray J. A., Haynes R. B., Richardson W. S., Evidencebased medicine: what it is and what it isn’t, Br Med J, 1996, 312(7023), p. 7172.
[2] Kitson A., Harvey G., McCormack B., Enabling the implementation of evidence based practice: a conceptual framework, Qual Health Care, 1998, 7(3), p. 149158.
[3] Sussman S., Valente T. W., Rohrbach L. A., Skara S., Pentz M. A., Translation in the health professions: Converting science into action, Eval Health Prof, 2006, 29(1), p. 732.
[4] Medina S. L., Zurakowschi D., Measurement Variability and Confidence Intervals in Medicine: Why Should Radiologist Care?, Radiology, 2003, 226, p. 297301.
[5] Huw D., What are confidence intervals? What is…., Hayward Group Publication, 2003, 3, p. 19.
[6] Bender R., Berg G., Zeeb H., Tutorial: Using confidence curves in medical research, Biom J, 2005, 47(2), p. 237247.
[7] BinomialDistribution, ©2005, Virtual Library of Free Software, AcademicDirect [cited March 2006]. Available from: URL: http://vl.academicdirect.org/applied_statistics/binomial_distribution/
[8] Gardner M. J., Altman D. G., Confidence intervals rather than P values: estimation rather than hypothesis testing, Br Med J (Clin Res Ed), 1986, 292(6522), p. 746750.
[9] Guyatt G., Jaeschke R., Heddle N., Cook D., Shannon H., Walter S., Basic statistics for clinicians: 2. Interpreting study results: confidence intervals, CMAJ, 1995, 152(2), p. 169173.
[10] Scheen A. J., Assessment of therapeutic efficacy, an essential step in evidencebased medicine, Rev Med Liege, 2000, 55(4), p. 206210.
[11] Newcombe R. G., Twosided confidence intervals for the single proportion: Comparison of seven methods, Stat Med, 1998, 17, p. 857872.
[12] Simel D. L., Samsa G. P., Matchar D. B., Likelihood ratios with confidence: sample size estimation for diagnostic test studies, J Clin Epidemiol, 1991, 44, p. 763770.
[13] Bewick V., Cheek L., Ball J., Statistics review 11: assessing risk, Crit Care, 2004, 8(4), p. 287291.
[14] Zhu K., Hunter S., PayneWilks K., Sutcliffe C., Bentley C., Roland C. L., Williams S. M., Potential differences in breast cancer risk factors based on CYP1A1 MspI and AfricanAmericanspecific genotypes, Ethn Dis, 2006, 16(1), p. 207215.
[15] Bosetti C., Gallus S., La Vecchia C., Aspirin and cancer risk: an updated quantitative review to 2005, Cancer Causes Control, 2006, 17(7), p. 871888.