Outcome-Based Validity and Reliability Assessment of Raters Regarding the Admission Triage Level in the Emergency Department: a Cross-Sectional Study

Abstract

Introduction: Emergency department (ED) is usually the first line of healthcare supply to patients in non-urgent to critical situations and, if necessary, provides hospital admission. A dynamic system to evaluate patients and allocate priorities is necessary. Such a structure that facilitates patients’ flow in the ED is termed triage. Objective: This study was conducted to investigate the validity and reliability of implementation of Emergency Severity Index (ESI) system version 4 by triage nurses in an overcrowded referral hospital with more than 80000 patient admissions per year and an average emergency department occupancy rate of more than 80%. Method: This prospective cross-sectional study was conducted in a tertiary care teaching hospital and trauma center with an emergency medicine residency program. Seven participating expert nurses were asked to assess the ESI level of patients in 30 written scenarios twice within a three-week interval to evaluate the inter-rater and intra-rater reliability. Patients were randomly selected to participate in the study, and the triage level assigned by the nurses was compared with that by the emergency physicians. Finally, based on the patients’ charts, an expert panel evaluated the validity of the triage level. Results: During the study period, 527 patients with mean age of 54 ± 7 years, including 253 (48%) women and 274 (52%) men, were assessed by seven trained triage nurses. The degree of retrograde agreement between the collaborated expert panel’s evaluation and the actual triage scales by the nurses and physicians for all 5 levels was excellent, with the Cohen’s weighted kappa being 0.966 (CI 0.985–0.946, p < 0.001) and 0.813 (CI 0.856–0.769, p<0.001), respectively. The intra-rater reliability was 0.94 (p < 0.0001), and the inter-rater reliability for all the nurses was in perfect agreement with the test result (Cohen’s weighted kappa were as follows: 0.919, 0.956, 0.911, 0.955, 0.860, 0.956, and 0.868; p < 0.001). Conclusion: The study findings showed that there was perfect reliability and, overall, almost perfect validity for the triage performed by the studied nurses.

1. Robinson D. An integrative review: triage protocols and the effect on ED length of stay. J Emerg Nurs. 2013;39(4):398-408.
2. Singer R, Infante A, Oppenheimer C, West C, Siegel B. The use of and satisfaction with the Emergency Severity Index. J Emerg Nurs. 2012;38(2):120-6.
3. Hashemi B, Baratloo A, Rahmati F, Forouzanfar M, Motamedi M, Safari S. Emergency Department Performance Indexes Before and After Establishment of Emergency Medicine. Emergency. 2013;1(1):20-3.
4. Baratloo A, Rahmati F, Forouzanfar MM, Hashemi B, Motamedi M, Safari S. Evaluation of Performance Indexes of Emergency Department. Iran J Emerg Med. 2015;2(1):33-8.
5. Janke A, Brody A, Overbeek D, Bedford J, Welch R, Levy P. Access to care issues and the role of EDs in the wake of the Affordable Care Act. Am J Emerg Med. 2015;33(2):181-5.
6. Farahmand S, Shabestari O, Pakrah M, Hossein-Nejad H, Arbab M, Bagheri-Hariri S. Artificial Intelligence-Based Triage for Patients with Acute Abdominal Pain in emergency Department; a Diagnostic Accuracy Study. Adv J Emerg Med. 2017;1(1):e5.
7. Seyedhosseini-Davarani S, Asle-Soleimani H, Hossein-Nejed H, Jafarbaghdadi R. Do Patients with Chest Pain Benefit from Installing Triage System in Emergency Department? Adv J Emerg Med. 2018;2(1):e8.
8. Murray M, Bullard M, Grafstein E. Revisions to the Canadian Emergency Department Triage and Acuity Scale implementation guidelines. CJEM. 2004;6(6):421-7.
9. Farrohknia N, Castrén M, Ehrenberg A, Lind L, Oredsson S, Jonsson H, et al. Emergency department triage scales and their components: a systematic review of the scientific evidence. Scand J Trauma Resusc Emerg Med. 2011;19:42.
10. Lee J, Oh S, Peck E, Lee J, Park K, Kim S, et al. The validity of the Canadian Triage and Acuity Scale in predicting resource utilization and the need for immediate life-saving interventions in elderly emergency department patients. Scand J Trauma Resusc Emerg Med. 2010;19:68.
11. Baratloo A, Maleki M. Iranian emergency department overcrowding. JEPT. 2015;1(2):39.
12. Momeni M, Vahidi E, Seyedhosseini J, Jarchi A, Naderpour Z, Saeedi M. Emergency Overcrowding Impact on the Quality of Care of Patients Presenting with Acute Stroke. Adv J Emerg Med. 2018;2(1):e3.
13. Ganley L, Gloster A. An overview of triage in the emergency department. Nurs Stand. 2010;26(12):49-56; quiz 8.
14. Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int. 2010;107(50):892-8.
15. Yoon P, Steiner I, Reinhardt G. Analysis of factors influencing length of stay in the emergency department. CJEM. 2003;5(3):155-61.
16. Fatovich D, Hirsch R. Entry overload, emergency department overcrowding, and ambulance bypass. Emerg Med J. 2003;20(5):406-9.
17. Safari S, Rahmati F, Baratloo A, Motamedi M, Forouzanfar MM, Hashemi B, et al. Hospital and pre-hospital triage systems in disaster and normal conditions; a review article. Iran J Emerg Med. 2015;2(1):2-10.
18. Fernandes C, Tanabe P, Gilboy N, Johnson L, McNair R, Rosenau A, et al. Five-level triage: a report from the ACEP/ENA Five-level Triage Task Force. J Emerg Nurs. 2005;31(1):39-50; quiz 118.
19. Shelton R. The Emergency Severity Index 5-level triage system. Dimens Crit Care Nurs. 2009;28(1):9-12.
20. Wuerz R, Milne L, Eitel D, Travers D, Gilboy N. Reliability and validity of a new five-level triage instrument. Acad Emerg Med. 2000;7(3):236-42.
21. Bergs J, Verelst S, Gillet J, Vandijck D. Evaluating implementation of the emergency severity index in a Belgian hospital. J Emerg Nurs. 2014;40(6):592-7.
22.Green N, Durani Y, Brecher D, DePiero A, Loiselle J, Attia M. Emergency Severity Index version 4: a valid and reliable tool in pediatric emergency department triage. Pediatr Emerg Care. 2012;28(8):753-7.
23. Baumann M, Strout T. Evaluation of the Emergency Severity Index (version 3) triage algorithm in pediatric patients. Acad Emerg Med. 2005;12(3):219-24.
24.Buschhorn H, Strout T, Sholl J, Baumann M. Emergency medical services triage using the emergency severity index: is it reliable and valid? J Emerg Nurs. 2013;39(5):e55-63.
25. Castner J. Emergency Department triage: what data are nurses collecting? J Emerg Nurs. 2011;37(4):417-22.
26. Martin A, Davidson CL, Panik A, Buckenmyer C, Delpais P, Ortiz M. An examination of ESI triage scoring accuracy in relationship to ED nursing attitudes and experience. J Emerg Nurs. 2014;40(5):461-8.
27. Oredsson S, Jonsson H, Rognes J, Lind L, Göransson K, Ehrenberg A, et al. A systematic review of triage-related interventions to improve patient flow in emergency departments. Scand J Trauma Resusc Emerg Med. 2011;19:43.
Files
IssueVol 2 No 3 (2018): Summer (July) QRcode
SectionOriginal article
PMCIDPMC6549196
PMID31172095
Keywords
Emergency department Patient outcome assessment Reliability and validity Triage
Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
1.
Seyedhosseini-Davarani S, Nejati A, Hossein-Nejad H, Mousavi S-M, Sedaghat M, Arbab M, Bagheri-Hariri S. Outcome-Based Validity and Reliability Assessment of Raters Regarding the Admission Triage Level in the Emergency Department: a Cross-Sectional Study. Front Emerg Med. 2018;2(3):e32.

Downloads

Download data is not yet available.