Quantitative and Qualitative Analysis of PubMed-Indexed Biomedical Publications in Oman from years 2005-2009

Dileep K. Rohra*, Syed I. Azam

How to cite this Article

Rohra DK, Azam SI. Quantitative and Qualitative Analysis of PubMed-Indexed Biomedical Publications in Oman from years 2005-2009. Oman Med J 2011 March; 26(3):160-165.

How to cite this URL

Rohra DK, Azam SI. Quantitative and Qualitative Analysis of PubMed-Indexed Biomedical Publications in Oman from years 2005-2009. Oman Med J 2011 March; 26(3):160-165.. Available from http://www.omjournal.org/fultext_PDF.aspx?DetailsID=94&type=fultext

Abstract

Objectives: Research activities are promoted at the government and the institutional levels in Oman. However, the quantity and quality of research conducted in various institutes of Oman has not been measured. Therefore, this study was conducted to analyze the quantity and quality of biomedical publications emanating from Oman over the last five years (2005-2009).

Methods: Data regarding the biomedical publications of Omani origin published from January 2005 to December 2009 were retrieved from Pubmed. All abstracts were manually checked to exclude false retrievals. The journal impact factor (IF) for 2008 was applied for every publication. Full-length original articles, review articles and publications mentioning new research methods/new technique were assigned full IF of the journal. However, half of the journal IF were assigned to correspondences/commentaries, case reports/series and short communications.

Results: Of biomedical publications, 752 were retrieved from Pubmed and 519 publications were included for final analysis after excluding false positives. The number of biomedical publications steadily increased over the last five years, but it was not statistically significant. Original research articles constituted more than half (54.3%) of all the publications, followed by case reports/series (30.1%). Sultan Qaboos University (SQU) contributed more than half of the biomedical publications (51.5%), followed by the Ministry of Health (MoH), which contributed 38% of the publications. Interestingly, the mean IF of the publications was not significantly different throughout the years.

Conclusions: The number of publications emanating from Oman increased over the last five years; however, the quality of research has not improved. Furthermore, the research in Oman is mainly conducted by the government-run institutions and there is hardly any contribution from the private institutions.

Keywords: Biomedical research; Journal impact factor; Publications; Pubmed; Oman.

Introduction

The Sultanate of Oman is an Arab country in the Middle East and a member state of the Gulf Cooperation Council countries. It borders with the United Arab Emirates, Saudi Arabia and Yemen. This country has a very recent history of education and research. This can be judged from the fact that the only public sector university of Oman; Sultan Qaboos University (SQU) was started in 1986. Recently, private institutes of higher education have also started their operations such as Sohar University (2000), Oman Medical College (2001), University of Nizwa (2004) and Dhofar University (2004).

The research activities are promoted at the government and the institutional levels. Even the guidelines for academic promotions of the faculty working in these institutes emphasize heavily on the research activities. Adequate funding is also available for research from the government as well as other sources. The presence of Oman in biomedical research can be observed in the international publications. However, the quantity of research conducted in various institutes of Oman has not been measured. In order to assess the number of publications published in indexed medical journals, Pubmed is a valuable tool.¹ This search engine can be used as a means to obtain vital data for analysis. Research productivity of various institutes, countries and regions can be determined using Pubmed.²

In addition to quantity, the quality is increasingly recognized as a critical aspect while evaluating the research. Evaluating the quality of scientific research is not an easy task. In the 1970s, some objective parameters were suggested as a means of evaluating the quality of research.^3,4 Among these parameters, impact factor (IF) of the journal is widely employed as a tool to judge the quality of scientific research.^5-7 The idea of IF was first coined by Garfield in 1955.⁸ It is an instrument for the assessment of quality of the journal monitored periodically by Thomson Scientific (formerly International Scientific Institute-ISI), Philadelphia.⁹ Although there are several criticisms regarding the use of IF as a tool to measure the quality of publication,^10-13 it is still a very simple, convenient and quick indicator for assessing the impact and quality of research.¹⁴ Thus, this parameter is given a lot of authority for recruitments, funding, promotions and rewards, especially in developing countries.¹⁴

Since there is no data that can show the performance of Omani scientists and institutes in biomedical research, this study was conducted to analyze the quantity and quality of biomedical publications emanating from Oman over the last five years (2005-2009).

Methods

Data regarding the biomedical publications from Omani origin were retrieved from Pubmed. The key word entered was "Oman." Display setting was chosen for "Pub date," which sorts the search results in chronological order starting from the latest publications. Study period was defined from January 2005 to December 2009. To avoid false positives, the abstracts of all retrieved articles published during that period were opened and carefully read for inclusion criteria of the publications in this study.

The inclusion criteria included; 1) All studies which were conducted in and published from Oman. This also included regional and international collaborative studies in which Omani researcher(s) were also involved. 2) All studies which were conducted outside Oman but at the time of submission of papers and publication, the author(s) was/were working in Oman, 3) All studies which were conducted in Oman but at the time of publication, the author(s) was/were working outside Oman.

On the contrary, the exclusion criteria included; 1) All studies which were neither conducted in Oman nor belonged to any Omani scientists. The search with key word "Oman" yielded such publications because the name Oman appeared in the publications. This included some authors named "Oman," reference to some Omani family, reference to Bay of Oman or some previous study done in Oman, 2) All studies not related to biomedical sciences for example, publications belonging to pure physics, chemistry and engineering. These included publications unrelated to any of the biological sciences but were retrieved in the search from Pubmed.

Special care was taken for articles when the abstract showed data collected from Oman but the affiliation of the corresponding author was not from any of the Omani institutes. Since abstracts in the Pubmed show affiliation of only the corresponding author, full text articles was retrieved to verify for the presence of Omani author(s) in the publication. After thorough scrutiny for inclusion, the abstract of each article was transferred to Microsoft Word and a comprehensive file was created. Each publication was coded and data regarding the name of journal, year of publication, type of publication, institute, specialty, number of authors and whether the study was conducted in Oman or not, were recorded. The type of publication (original research article, review, case report etc) was documented as classified by the journals themselves. All the health care units working under the administration of the Ministry of Health (MoH) were divided into primary / secondary and tertiary hospitals. Some publications were the result of collaborative studies, in which authors belonged to more than one institute. In such cases, each collaborating institute was credited for that publication. The journal IF for 2008 was applied for every publication. Full-length original articles, review articles and publications mentioning new research methods/new technique were assigned full IF of the journal. However, half of the journal IF was assigned to correspondences/commentaries, case reports/ series and short communications.

The cumulative IF for a particular year was determined by adding IF of all the publications of that year. The mean IF of the year was calculated by dividing the cumulative IF with the total number of publications in that particular year. All these data were then entered into Microsoft Excel and analyzed as number, means and percentages. The associations of total number of publications and the type of publications by year were assessed using the Chi-square test. One way ANOVA followed by Bonferroni’s test was used to observe the differences in mean impact factor by year of publication and mean number of authors by type of publications. A p value of less than 0.05 was treated as statistically significant.

Results

In this study, 752 biomedical publications published from January 1, 2005 to December 31, 2009 were retrieved from the Pubmed. After excluding articles according to the exclusion criteria, 519 publications were included for final analysis. Table 1 shows year-wise distribution of publications from 2005-2009. It is evident from the table that the number of biomedical publications steadily increased over the last five years. However, 2006 saw an increased number of publications compared to 2007 and 2008. Nevertheless, there was no significant difference observed in the number of publications during the five year study period (p=0.10).

Cumulatively, in all five years, original research articles constituted more than half (54.3%) of all the publications, followed by case reports/series (30.1%). Year-wise analysis of the types of publication is also presented in Table 1. Original research articles were approximately 45.8% in the year 2005, and their share increased significantly (p<0.01) in 2006 (60.4%), 2007 (59.6%) and 2008 (58.1%). However, a sharp and statistically significant (p<0.01) decline in original publications was documented in the year 2009. Corresponding to this change in pattern of original publications, case reports/series which accounted for 39.8% of the total publications in year 2005, there was a significant decrease between 2006-2008 (p<0.01). However, once again case reports/series accounted for 37.2% of all the biomedical publications in 2009. The contribution of other types of publications was more or less similar over the last five years.

Table 2 shows the contribution of various institutions in biomedical research between the years 2005-2009. Cumulatively, SQU contributed more than half of the biomedical publications (51.5%), followed by the Ministry of Health (MoH), which contributed just over 38% of the publications. Primary and secondary health units of the MoH were more productive compared to tertiary hospitals. All other institutions in Oman had a share of approximately 10% of the publications. Year-wise distribution of the productivity of various Omani institutes is also presented in Table 2. In the last five years, the observed pattern was similar, while SQU leads in the country in terms of the number of publications, followed by institutions working under the administration of the MoH.

Table 3 shows the specialty-wise analysis of biomedical publication. Medicine accounted for 14.1% of all the biomedical publications, followed by ophthalmology (10.4%) and surgery (10.4%).

Table 1 Year-wise distribution of types of articles published from Oman in biomedical journals from years 2005-09

*p<0.01

Table 2: Contribution of various institutes of Oman in biomedical publications from years 2005-09

*total exceeds from the actual number of publications because in some publications more than one institutes collaborated.

Table 3: Specialty-wise distribution of publications from Oman in years 2005-2009

In addition to the quantity, the quality of publications was also evaluated using journal IF as a determinant. Corresponding to the number of publications, cumulative IF was highest in 2009 (Fig. 1A). Since the number of publications differed each year, the mean IF was calculated in order to gain a better understanding of the quality of publications. Evidently in Fig. 1B, it was found that the mean IF of the publications was not significantly different throughout the years (p=0.48). Further analysis showed that more than a quarter of the publications in all five years appeared in journals having no IF and more than half of all the publications were in journals having IF of <1. (Fig. 2)

Figure 1: Comparative analysis of year-wise distribution of cumulative (A) and mean (B) impact factor of biomedical publications in years 2005-2009.

Figure 2: Distribution of journals with various impact factors in which biomedical publications from Oman were published in years 2005-2009.

Patterns of authorship were also analyzed from the data. The mean number of authors in all the publications was 4.15±0.13 (ranging from 1-28). Further analysis revealed an interesting finding; that the mean number of authors was significantly different in various types of publications. These were 4.81±0.20 (range: 1-28), 3.60±0.17 (range: 1-12), 2.38±0.18 (range: 1-4), 2.91±0.62 (range: 1-14); 2.79±0.51 (range: 1-10) and 4.21±0.61 (range: 1-9) for original articles, case reports/series, review articles, commentaries/correspondences, and research methods /techniques respectively. The number of authors was higher in original articles compared to case reports/series (p<0.01), review articles (p<0.01), commentaries/correspondences (p<0.05), and research methods/techniques (p<0.05). However, no significant difference was observed in the number of authors between original articles and short communications (p=1.0).

Out of the 519 publications analyzed, 65 (12.5%) were the result of collaborative work between Omani scientists and those from other countries. The authorship of a series of seven papers was shared by a research group with equal credence to all. The authorship pattern of the remaining 58 papers was further analyzed. Interestingly, 8 (13.8%), 4 (6.9%) and 25 (43.1%) publications contained Omani scientists as first authors, corresponding authors and both first as well as corresponding authors, respectively.

Discussion

Biomedical research is an important aspect of health care and progress. Research publications in this area are truly a good marker for determining the extent and quality of biomedical research in a given institute or territory.¹ This is the first study which has meticulously analyzed the quantitative and qualitative progress of biomedical research in Oman. There is one such published report, which has included data from all GCC countries including Oman.¹⁵ They also used Pubmed as a search engine but there are some basic differences in the methodology. For example, they did not define any inclusion or exclusion criteria and there was no control over the false positive results. As mentioned in the Results, 233 out of 752 retrieved publications (approximately 31%) were excluded from the analysis because they did not belong to Oman or any discipline of biomedical sciences. Based on the careful scrutiny of all the publications, it is reasonable to conclude that the current study presents near accurate data regarding the biomedical publications in Oman.

It was found that the number of biomedical publications originating from Oman steadily increased over the last five years. If we compare the difference in the number of publications between two extreme points of the study period i.e. 2005 and 2009, then it can be observed that there is approximately 50% increase in the number of publications in 2009 compared to 2005. However, the serial rise in the number of publications over the period 2005-2009 is not statistically significant. Furthermore, 28 out of total of 519 publications (5.4%) contained data which was generated and collected outside Oman but the authors belonged to some institute of Oman at the time of publication. In the context of expansion of already existing and opening of new academic institutes in Oman over the last few years, this level of productivity seems far from satisfaction.

One important change in the trend of publications was noted that in the year 2005, case reports and original articles accounted for approximately 40% and 46% of the total publications. Whereas, in the years 2006-08, original articles increased by approximately 15%, which was statistically significant. Correspondingly, the share of case reports dropped during those years. However, in 2009, the pattern of publication was noted to be similar as in 2005.

Upon evaluating the role of various academic institutions in publication, it was observed that SQU was leading throughout the five years studied. SQU is continuously increasing its share in biomedical publications. SQU is followed by all units of MoH combined. Together, both of these government organizations produced 90% of all the biomedical publications in the last five years. It was a disappointing finding that private institutions including big hospitals and universities jointly contributed less than 2% of the publications emanating from Oman. This finding may be a wake up call for the regulatory authorities of higher education as well as the administration of private institutions. A lot can be done to improve the status of research including the funding opportunities, basic infrastructure for research and incentives to productive faculty in these institutes.

In addition to quantity, another aim of this study was to determine the quality of publications published from Oman. Journal IF was employed as a measure to gauge the quality of publication. The journal IF is a parameter based on the number of times that a paper in a particular journal has been cited by other journals.¹⁶ Although many concerns have been expressed regarding the use of journal IF as an indicator of the quality of research,^11,17 it is still widely used in the absence of any other simple and quantifiable parameter.¹⁴ Some of these concerns however, are more related to the method of calculation of IF than the overall concept of the IF.¹⁸ The most important concern about journal IF is that it strongly varies across different scientific disciplines.^10,19,20 But in the present study, the IF of the publications among various disciplines was not compared. Instead, all the yearly data was pooled together, nullifying the anomaly associated with IF. The controversy concerning IF as a measure of quality can be summed up in the words of Hoeffel (1998), "Impact Factor is not a perfect tool to measure the quality of articles but there is nothing better and it has the advantage of already being in existence and is, therefore, a good technique for scientific evaluation. Experience has shown that in each specialty the best journals are those in which it is most difficult to have an article accepted, and these are the journals that have a high impact factor."²¹ In the present study, it was observed that the mean IF of the publications remained somewhat similar throughout the last five years and did not change significantly. This shows that although the number of publications has increased in Oman albeit statistically insignificant, quality of research and publication has not improved in the last five years. Another significant finding was that the majority of the Omani publications were published in journals either with no IF or with an IF less than 1. One plausible explanation for this finding is that researchers are constantly under pressure to publish due to promotions and other incentives. They want to get as many publications as possible without taking into account the quality of research.

Areas of research were also determined in this study. Most of the publications published were from clinical areas. It was observed that all basic sciences including pharmacology, biochemistry, molecular biology, microbiology, immunology, anatomy, physiology and genetics accounted for only 20.8% (n=108) of the total publications. Interestingly, 104 out of the 108 (96.3%) papers belonging to basic biomedical sciences were contributed by SQU. This shows the lack of infrastructure for experimental research in Omani institutes other than SQU.

There are few limitations in this study. Firstly, we only analyzed the papers published in Pubmed-indexed journals. Other publications which could not be retrieved from Pubmed, were not included in this study. Those publications may change the conclusion as far as the number of publications is concerned. Another limitation was that weightage assigned to various types of publications was oversimplified. Certainly, the time, labor and cost incurred and consequently the importance of a case report or a correspondence would not be equal to that of an original article or a review article. With this concern in mind, 50% of the journal IF was assigned to case reports and correspondences. Admittedly, this assignment is convenient but not flawless. The third limitation was the possible omission of some publications in which the data might have been generated from laboratory-based experiments and the corresponding author did not belong to Oman. In these types of publications, the place of study does not always appear in the abstract and the search with a key word "Oman" would not pick up those publications. The fourth limitation is that based on the inclusion criteria, certain publications may have been analyzed in this paper, for which the work was done outside Oman by graduate students or researchers and at the time of submission, the authors were based in Oman. This may have inflated the number of publications from Oman.

Conclusion

Overall, this study shows that although the number of publication emanating from Oman increased over the five-year study period, the quality of research did not improve. Furthermore, the research in Oman is mainly conducted by the government-run institutions and there is hardly any contribution from the private institutions.

Acknowledgements

The authors reported no conflict of interest and no funding was received for this work.

1. Tadmouri GO, Tadmouri NB. Biomedical research in the Kingdom of Saudi Arabia (1982-2000). Saudi Med J 2002 Jan;23(1):20-24.

2. Tutarel O. Geographical distribution of publications in the field of medical education. BMC Med Educ 2002 Apr;2:3-10.

3. Garfield E. Citation indexing for studying science. Nature 1970 Aug;227(5259):669-671.

4. Garfield E. Citation analysis as a tool in journal evaluation. Science 1972 Nov;178(60):471-479.

5. Taubes G. Measure for measure in science. Science 1993 May;260(5110):884-886.

6. Calza L, Garbisa S. Italian professorships. Nature 1995;374:492 .

7. Vinkler P. Evaluation of some methods for the relative assessment of scientific publications. Scientometrics 1986;10:157-177 .

8. Garfield E. Citation indexes for science; a new dimension in documentation through association of ideas. Science 1955 Jul;122(3159):108-111.

9. Garfield E. The history and meaning of the journal impact factor. JAMA 2006 Jan;295(1):90-93.

10. Seglen PO. Why the impact factor of journals should not be used for evaluating research. BMJ 1997 Feb;314(7079):498-502.

11. Colquhoun D. Challenging the tyranny of impact factors. Nature 2003 May;423(6939):479, discussion 480.

12. Glanzel W, Moed HF. Journal impact measures in bibliometric research. Scientometrics 2002;53:171-193 .

13. Bordons M, Fernandez MT, Gomez I. Advantages and limitations in the use of impact factor measures for the assessment of research performance in a peripheral country. Scientometrics 2002;53:195-206 .

14. Satyanarayana K, Sharma A. Impact factor: time to move on. Indian J Med Res 2008 Jan;127(1):4-6.

15. Deleu D, Northway MG, Hanssens Y. Geographical distribution of biomedical publications from the Gulf Corporation Council countries. Saudi Med J 2001 Jan;22(1):10-12.

16. Opthof T. Sense and nonsense about the impact factor. Cardiovasc Res 1997 Jan;33(1):1-7.

17. Rossner M, Van Epps H, Hill E. Show me the data. J Exp Med 2007 Dec;204(13):3052-3053.

18. Ogden TL, Bartley DL. The ups and downs of journal impact factors. Ann Occup Hyg 2008 Mar;52(2):73-82.

19. Garfield E. Journal impact factor: a brief review. CMAJ 1999 Oct;161(8):979-980.

20. Amin M, Mabe M. Impact factors: use and abuse. Perspective in publishing no. 1. Elsevier Science, Oxfordshire, UK. 2000; http://www1.elsevier.com/homepage/about/ita/editors/perspectives1.pdf. (accessed on March 7, 2010).

21. Hoeffel C. Journal impact factors. Allergy 1998 Dec;53(12):1225.