Assessment of Educational Environment of Surgical Theatre at a Teaching Hospital of a Saudi University: Using Surgical Theatre Educational Environment Measures

Mona Faisal Al-Qahtani^1* and Mona Al-Sheikh²

How to cite this article

Al-Qahtani MF, Al-Sheikh M. Assessment of Educational Environment of Surgical Theatre at a Teaching Hospital of a Saudi University: Using Surgical Theatre Educational Environment Measures. Oman Med J 2012 May; 27(3):217-223.

How to cite this URL

Al-Qahtani MF, Al-Sheikh M. Assessment of Educational Environment of Surgical Theatre at a Teaching Hospital of a Saudi University: Using Surgical Theatre Educational Environment Measures. Oman Med J 2012 May; 27(3):217-223. Available from http://www.omjournal.org/fultext_PDF.aspx?DetailsID=239&type=fultext

Abstract

Objectives: This study was aimed to determine how medical interns perceive the important factors of the learning environment the surgical theatre at the teaching hospital of the medical school, University of Dammam (UoD). The study also investigated the relationships between the learning environment and academic achievements. Finally, it determined the role and significance of gender on the above perceptions and relationships.

Methods: The Surgical Theatre Educational Environment Measure (STEEM) was used to identify the perceptions of interns on the most important factors prevalent in the surgical theatre as an educational environment. STEEM was administered to all interns during the period of June-September 2009. Ninety-one out of 145 students completed the questionnaire representing a response rate of 63%. Non-parametric statistical analysis was performed using Statistical Package for the Social Sciences (SPSS) Version 17.

Results: The STEEM was shown to be internally consistent for the assessment of the overall educational environment in the surgical theatre of UoD. The overall STEEM mean score was 110. For male and female students, the mean scores were 114 and 107 respectively. There were statistically significant gender differences in the perceptions of "learning opportunities" and "teaching and training". Females rated these subscales lower than males. There were no significant associations between academic achievements and perceptions of the educational environment.

Conclusion: The interns perceived the learning environment of the surgical theatre as less than satisfactory. In comparison with the males; the perception of the females was less positive, particularly in the areas of learning opportunities, and teaching and training. The study also revealed some other problematic areas in the learning environment of surgical theatre of the teaching hospital of UoD. The results imply that there is much room for improvement. They also indicate that equal opportunities for learning and training should be given to both sexes.

Keywords: Surgical Theatre Educational Environment Measure (STEEM); Interns; Learning Environment; Medical Education; Saudi Arabia.

Introduction

A tremendous amount of research has been devoted in recent years at exploring the different levels of educational environment. The learning environment is considered one of the key elements in identifying an effective curriculum,¹ the quality of teaching, learning,² and students’ achievement and satisfaction.^3-5

Roff and McAleer claim that if educators are able to define and explore the factors that constitute the learning environment of a given institution or program, and assess how these factors are perceived by students and teachers, they might be able to discover how these factors can be modified and customized to enhance the learning experience to achieve the educators' teaching goals.⁶

A number of studies have been conducted to explore students' perceptions of their learning environment. Roff et al.⁷ developed a survey instrument to assess medical students' perceptions of five dimensions of their learning environment: perceptions of learning; their perception of teachers; academic self-perceptions; the atmosphere; and their social self-perceptions.

Pololi and Price also designed a survey to measure the learning environment in Southeastern medical school, USA, to assess the teacher's attribute as a facilitator of learning.⁸ Dimensions including "trust and positive relationship between students and faculty", "positive regard for students held by faculty members" "student self-efficacy", "learner-centered learning", and "attitudes about the relationship between physician and patient". Within the clinical setting, Holt and Roff developed a specific survey to measure the educational environment for trainee anesthetists in the anesthetic theatre (ATEEM).⁹ It included such dimensions as autonomy, perceptions of atmosphere, Supervision/workload/support, perception of teachers and teaching, and learning opportunities. In addition, another educational environment measure- the Postgraduate Hospital Educational Environment Measure (PHEEM), was developed to assess the clinical learning environment of junior hospital doctors.¹⁰ It involved three aspects of educational environment such as the perception of role autonomy of teaching and social support. Furthermore, Cassar,¹¹ developed Surgical Theatre Educational Environment Measure (STEEM), an instrument that measures the learning environment in the surgical operating theatre. STEEM includes four subscales dealing mainly with teaching and training, learning opportunities, the atmosphere, and supervision/workload/ support. It is hoped that this study will contribute to the existing knowledge based on the clinical educational environment.

To the best of the authors' knowledge, the learning environment in the surgical theatre setting has not been measured officially at the University of Dammam, Saudi Arabia. In this study, the Surgical Theatre Educational Environment Measure (STEEM) was identified on the basis of medical interns' perceptions of the main factors present in the educational environment of the surgical theatre in a teaching hospital of a medical school, at the UoD.

The objectives of the study were threefold: 1) To identify the perception of medical interns towards the learning environment present in the surgical theatre at medical school teaching hospital, UoD; 2) To investigate possible relationships between the learning environment and academic achievements; and 3) To determine whether gender has any significant effect on the above issues.

Methods

The STEEM questionnaire consisted of 40 statements. For each statement a Likert scale ranging from 5 "strongly agreed" to 1 "strongly disagreed" was used.

The scoring was reversed for negative statements (statements number: 8,11,14,16,19,22,23,26,27,28,30,31,33,34,35,36,37,38, and 40) so that for all items; the higher the score meant the more positive the perception. This implies that the overall possible maximum score of STEEM was 200 and a minimum score 40. An overall score of 120 out of 200 would indicate a neutral perception, any value >120 would indicate a more satisfactory environment, while any value of <120 would indicate a less than satisfactory learning environment. The maximum scores for the subscales of STEEM were as follows:

Teaching and training subscale 13×5=65 maximum, statement 1-13

Learning opportunities subscale 11×5=55 maximum, statement 14-24 Atmosphere subscale 8×5=40 maximum, statement 25-32

Supervision/Workload/support subscale 8×5=40 maximum, statement 33-40

The data was analyzed using Statistical Package for the Social Sciences (SPSS) Version 17. Cronbach alpha coefficient was used to assess the internal consistency of the whole questionnaire and of each subscale. Spearman rho coefficient was calculated to assess the association between the score for academic achievement, as measured by the overall grade average, in both "general medical courses" and "specific surgical courses" subscales, and overall STEEM scores. The descriptive statistics were reported as the median and interquartile range. The comparative statistics used the non-parametric analysis of Mann-Whitney test to assess any significant gender differences. Microsoft Office Excel version 2003 was used for the graphic representation of data.

Results

The questionnaires were self-administered to all medical interns during their surgical rotation at the teaching hospital of the University of Dammam, Saudi Arabia, during the period from June-September in the academic year 2009-2010. Approximately 20 minutes was required for the completion of the questionnaire. A total of 91 out of 145 questionnaires were completed by the interns yielding a 63% response rate. There were 45 male students (49%) and 46 female students (51%). Their mean ages were 25.2 years (SD=1.007) and 24.6 years (SD=.771), respectively. Their overall grades average in the courses in surgery were as follows: 66% of the participants obtained a very good to excellent grade, while approximately one-third obtained a good grade. With regards to the overall grade average in medical courses; about 46% obtained a very good to excellent grade, while 43% obtained a good grade. Only 11% obtained an average grade.

The realiability analysis was performed using the Cronbach's alpha coefficient which was computed for the overall questionnaire and for each of the four subscales to measure the internal consistency of the questionnaire. The results of the Cronbach's alpha values were interpreted according to Richardson's suggestion.¹² Cronbach's alpha for the overall questionnaire was 0.793. The following were the Cronbach alpha values for the factors indicated: "teaching and training" subscale -.805; "supervision/workload/support" subscale-.696; "learning opportunities" subscale -.409; "atmosphere’"subscale -.572. The entire Cronbach alpha coefficients were considered to be within the range of high to very high. The last two Cronbach alpha coefficients were considered as average to high.

Correlation analysis was done using the Spearman's rho coefficient which was calculated to assess the association among the four subscales and the overall STEEM score; and any possible associations. All four subscales of STEEM were correlated significantly and positively with the overall STEEM score ranging from .537 to .728. In addition, there was a significant acceptable correlation between "teaching and training" and the "learning opportunities" subscales (r=0.58), but a low significant correlation between "teaching and training" and "atmosphere" subscales (r=0.28). Moreover, there was a significant acceptable correlation between the "atmosphere" and the "supervision/workload/support" subscales (r=0.57), but a low significant correlation between the "learning opportunities" and "atmosphere" subscales (r=0.27).

Spearman's rho coefficient was also calculated for students' academic achievement score in "surgery courses" in relation to their academic achievement in "general medical course". Spearman's rho coefficient was also computed for each academic achievement score in relation to the overall STEEM score and the four subscales scores. There was a significant correlation between students’ academic achievement in surgery in relation to general medical courses (r=0.61). However, there were no significant associations between both Academic achievements and the overall STEEM score or its subscales.

The scores of subscales and individual statements are depicted in Fig. 1 which shows the percentage scores for each of the four subscales and the overall percentage score for the whole group. The highest percentage score was given to the "supervision/workload/support" subscales (56.89%), followed by the "learning opportunities" subscale which had 56.84%. "Teaching and training" had the lowest percentage score of 51.76%. The mean overall score of the whole group was 110 (55.04%).

Table 1 shows the median and interquartile range for each statement in STEEM survey for the whole group (n=91), male (n=45), and female (n=46) separately. The most highly rated statements by the whole group were no. 27 "The nursing staff like it when I operate as the operation takes longer," and statement no. 19 "More senior trainees (do not) take my opportunities to operate". (the medians were 4, and IQR were 3-5 and 3-4, respectively)*. The lowest three rated statements were no. 6 "My trainer’s surgical skills are very good," statement no. 2 "I get on well with my trainer," and statement no. 1 "My trainer has a pleasant personality" (median scores were 2; IQR ranged between 1-2 and 2-2).

Figure 1: Overall percentage score and percentage scores for individual subscales for the whole group (N=91).

Table 1: Median and interquartile range for each statement in STEEM for the whole group (N=91), Male (n=45), and Female (n=4 6) Internship Students.

R= reverse score; IQR= interquartile range

Figure 2: Comparison between the perceptions of male and female interns.

Fig. 2 shows the overall percentage score and percentage scores for individual subscales based on gender. It revealed that there was a divergence between the perceptions of male and female interns toward their educational environment.

For females, the "supervision/workload/support" subscale obtained the highest score, while for males the highest score was for "learning opportunities". However, males and females both held similar views by rating the "teaching and training" subscale the lowest.

Statistically significant differences were found in the analysis for significant gender differences. Table 2 depicts the median, interquartile range and the mean rank of statements and subscales showing statistically significant differences in the perceptions of the male and female interns. The mean ranks were higher in all statements for male students, except no 40. Furthermore, male students scored a higher mean rank for "teaching and training" subscale, "learning opportunities" subscale and the overall STEEM score compared to their female counterparts.

Table 3 showed that the sample size, response rate, and the ratio of male: female were higher in the current study than in Cassar's study. However, the overall STEEM mean score, the overall Cronbach alpha coefficient, and the percentage scores of all four subscales as well as their Cronbach alpha coefficients were smaller in this study than in Cassar's study.

Table 2: Median, Interquartile Range, and the Mean Rank for Individual Statements and Subscales of the STEEM where Significant Statistical Differences were observed between the Perceptions of Male and Female Students.

Table 3: Comparison of STEEM scores at the University of Dammam (current study) and Cassar's (2004) study.

Comparison between the current study and Cassar’s study,¹¹

Table 3 shows a comparison between the current study and Cassar's.¹¹ This study had four times the sample size of that of Cassar's.¹¹ Lower values were achieved for the overall STEEM score (55%) and all four subscale scores in the current study compared with Cassar's, (overall mean score was 74%, indicating a satisfactory score). The overall STEEM mean score in the current study indicates that the learning environment as perceived by all interns was unsatisfactory.

The subscale analysis showed some discrepancies between the interns in the current study and the Scottish trainees in the Cassar study.¹¹ The highest in rank of the subscales for the interns in the current study were "supervision/workload/support", and "learning opportunities". This demonstrates an overall supportive environment in the surgical theatre for the interns. The Scottish trainees ranked "teaching and training" the highest in the subscales. For the Scottish trainees the lowest was "learning opportunities", whereas "teaching and training" was ranked the lowest for the interns in our study. This implies that STEEM has the ability to differentiate between different factors affect the learning environment of the surgical theatre.

The overall internal consistency of STEEM in the current study (Cronbach alpha coefficient =.793) was a little lower than that found in the Cassar study (Cronbach alpha coefficient =.877), which indicates a very high internal consistency of the inventory. The Cronbach alpha coefficients for both the "teaching and training" and "atmosphere" subscales for the current study (Cronbach alpha =.805 and .572 respectively) were similar to those reported in the Cassar study (Cronbach alpha coefficient =0.842 and 0.574, respectively).¹¹ The internal consistency for "learning opportunities" in the current study was lower than in the Cassar study.¹¹ But the Cronbach alpha coefficient for the subscale "supervision/workload/support" in the current study exhibited a higher internal consistency than was reported in the Cassar study.¹¹ In the current study, the individual subscales yielded different values for Cronbach alpha coefficients, which might indicate that each subscale assesses a different component that contributes to the overall learning environment. Moreover, STEEM detected gender significant differences in our study, while the Cassar study reported no gender differences.¹¹

A comparison of the male and female responses revealed significant differences in the perceived learning environment, a finding which is at variance with the work by Cassar,¹¹ but in accord with the finding of Kanashiro et al.¹³ In this study, female scores showed a statistically significant difference (p<.05) on the subscales of "teaching and training", and "learning opportunities", from the males, as shown by a confirmation of the analysis of individual item/statement. Both statements no. 6 "My trainer’s surgical skills are very good," and statement no. 10 "Before the operation, my trainer discusses the part of the procedure I will perform" fall within the 'teaching and training" subscale and were rated significantly lower by female interns than their male counterparts. Also both statements no. 17 "I get enough opportunity to assist", and no. 18 "There are enough theatre sessions per week for me to gain the appropriate experience", which fall within the "learning opportunities" subscale were rated significantly lower by females than the males. Only statement no. 40 "Theatre sessions are (not) too long" was rated significantly higher by female students than the males. It appears that female interns perceive their educational environment in the theatre setting as less pleasant than their male counterparts. They indicated their lack of satisfaction with their trainer's surgical skills by their scores. Moreover, they felt that they had fewer learning opportunities than they should have in the surgical theatre. The reasons for these divergent views of the male and female interns were not explored in this study.

Correlation analysis between each of the four subscales of STEEM and the overall STEEM score revealed significant positive correlations. The two subscales that correlated most strongly with the overall STEEM score were "learning opportunities" (r=.728), and "teaching and training" (r=.728), a finding that accords with the work of Mahoney et al.¹⁴ In addition, correlation analysis between the subscale scores revealed significant acceptable correlations between the subscale scores of "teaching and training" and "learning opportunities", and between "atmosphere" and "supervision/working/support" subscale scores. Moreover, there were significant correlations, albeit low, between the subscale scores of 'atmosphere' and "learning opportunities", a finding that is in accordance with Cassar's study¹¹; and between "atmosphere" and "teaching and training". The low correlation between the various subscales (with the exception of "teaching and training" with "learning opportunities" and that of "atmosphere" with "supervision/workload/support") seems to indicate that each subscale assesses a distinct component of the educational environment of the surgical theatre.

Correlation analysis between students' academic achievements, in both general medical courses and surgical courses, and perceptions of educational environment, revealed no significant associations between any of the academic achievements and subscale of the STEEM. This finding is at variance with the work of Vahala and Winston,¹⁵ Mayya and Roff,³ and Pimparyon et al.⁵ all of whom claimed an association between academic achievement and the perception of educational environment.

This study was limited to medical students during their internship at a Surgical Department of only one of the colleges of the University of Dammam. Therefore, any generalization should be limited to colleges with similar characteristics.

Conclusion

This study has revealed that the STEEM questionnaire is a reliable instrument for assessing the learning environment of the surgical theatre for medical interns. The learning environment of the surgical theatre was perceived as less than satisfactory by interns of the medical school, University of Dammam. The Female medical students considered the learning environment less positive than the males did (p=.020). From the point of view of the female medical students; the teaching and training, and the learning opportunities in the surgical theatre were not as pleasant as the males viewed them (p=.016 vs. p=.005, respectively). This implies that there is much room for improvement. Overall, the results indicated that STEEM could be used as a diagnostic measure to assess the educational environment of the surgical theatre for interns.

Accordingly, it is recommended that equal opportunities for learning and training be given to both sexes in the surgical theatre in order to develop and improve the surgical skills of the students.

Finally, it is hoped that the information provided by this study could be used to improve the educational environment of the surgical theatre in the College of Medicine, Teaching Hospital, University of Dammam, which will consequently have a positive impact on students' learning in Saudi Arabia.

Acknowledgements

References

1. Bassaw B, Roff S, McAleer S, Roopnarinesingh S, De Lisle J, Teelucksingh S, et al. Students’ perspectives on the educational environment, Faculty of Medical Sciences, Trinidad. Med Teach 2003 Sep;25(5):522-526.

2. O’Brien A, Chan T, Cho M. Investigating Nursing Students’ Perceptions of the Changed in a Nursing Curriculum by Means of the Dundee Ready Education Environment Measure (DREEM) Inventory: Results of a Cluster Analysis. Int J Nurs Educ Scholarsh 2008;5(1):25.

3. Mayya SS, Roff S. Students’ perceptions of educational environment: a comparison of academic achievers and under-achievers at kasturba medical college, India. Educ Health (Abingdon) 2004 Nov;17(3):280-291.

4. Genn JM. AMEE Medical Education Guide No. 23 (Part 1): Curriculum, environment, climate, quality and change in medical education-a unifying perspective. Med Teach 2001 Jul;23(4):337-344.

5. Pimparyon P, Roff S, McAleer S, Poonchai B, Pemba S. Educational environment, student approaches to learning and academic achievement in a Thai nursing school. Med Teach 2000;22(4):359-364 .

6. Roff S, McAleer S. What is educational climate? Med Teach 2001 Jul;23(4):333-334.

7. Roff S, McAleer S, Harden R, Al-Qahtani M, Ahmed A, Deza H, et al. Development and validation of the Dundee Ready Education Environment Measure (DREEM). Med Teach 1997;19(4):295-299 .

8. Pololi L, Price J. Validation and use of an instrument to measure the learning environment as perceived by medical students. Teach Learn Med 2000;12(4):201-207.

9. Holt MC, Roff S. Development and validation of the Anaesthetic Theatre Educational Environment Measure (ATEEM). Med Teach 2004 Sep;26(6):553-558.

10. Roff S, McAleer S, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach 2005 Jun;27(4):326-331.

11. Cassar K. Development of an instrument to measure the surgical operating theatre learning environment as perceived by basic surgical trainees. Med Teach 2004 May;26(3):260-264.

12. Richardson R. Assessment and Evaluation 1988. Centre for Medical Education, Ninewells Hospital and Medical School, Dundee, Scotland, UK.

13. Kanashiro J, McAleer S, Roff S. Assessing the educational environment in the operating room-a measure of resident perception at one Canadian institution. Surgery 2006 Feb;139(2):150-158.

14. Mahoney A, Crowe PJ, Harris P. Exploring Australasian Surgical Trainees' Satisfaction with Operating Theatre Learning Using the 'Surgical Theatre Educational Environment Measure'. ANZ Surg 2010;80(12):884-889 .