TORONTO - A new study suggests Ontario hospitals saw no measurable improvements in their surgical safety records six months after they implemented a World Health Organization-designed safe surgery checklist program.
The lead author of the study said the findings may suggest earlier claims about the patient outcome improvements hospitals could achieve through use of the surgical checklist were overstated.
But other experts, including the leader of the WHO effort, said the new study shows Ontario fumbled the introduction of the surgical safety checklist program.
"The lesson is that it hasn't been implemented properly, not that the checklist is garbage," said Dr. Jason Leitch, clinical director for Scotland's National Health Service, which has achieved a 23 per cent reduction in surgery-related deaths over the past six years using a set of tools which include surgical checklists.
The new study was published Wednesday in the New England Journal of Medicine, which also published the 2009 study that reported large improvements in surgical safety associated with use of a standardized checklist.
The checklist requires operating teams to introduce themselves and outline their roles before surgery begins, confirm the procedure that is to be done, ensure that all necessary equipment is in place and discuss any known concerns. Before the surgery ends, staff confirm all instruments and sponges are accounted for and that the team has discussed how the patient's ongoing care needs to be managed.
The 2009 study reported a 47 per cent reduction in deaths associated with high-risk surgeries when the checklist was used.
The buzz generated by that study prompted many jurisdictions, including Ontario, to order hospitals to adopt the checklist. In Ontario, hospitals were required to begin reporting publicly on how they were adhering to that directive beginning in July 2010.
So several researchers from University of Toronto hospitals decided to see whether the adoption of the checklist was delivering results.
Using hospital records compiled by the Institute for Clinical Evaluative Sciences, they looked at surgery-related deaths and complications — things like surgical site infections and secondary surgeries needed because of problems related to the initial one — for 92 hospitals around the province.
They compared how many of these events happened in the individual hospitals for a three-month period before the hospital introduced its checklist program to the number of events that happened in a three-month period after the checklist was in place.
They found no statistically significant differences in surgery-related deaths or complications when they compared the before and after pictures.
"What we found was making extraordinary improvements in patient safety is probably going to take a lot more work than the introduction of something like these safety checklists," said lead author Dr. David Urbach, a general surgeon at the University Health Network in Toronto.
But Dr. Atul Gawande, who led the WHO program to develop the surgical checklist, suggested he was not surprised Urbach and his co-authors didn't see marked improvements so early into the implementation of the program.
It takes time and effort to effect the culture change required to allow the checklist program to be successful, said Gawande, a surgeon at Brigham and Women's Hospital in Boston and a professor at Harvard School of Public Health.
Leitch said making the checklist work requires effort and training within hospitals, as well as monitoring whether hospitals are actually using the system. But jurisdictions where authorities have simply sent out letters mandating use of the checklist don't see much benefit, he said.