Significant Event Audit (SEA)

Synonyms: critical event audit, critical incident analysis, structured case analysis, facilitated case discussion

Significant event audit (SEA) has been defined as occurring when "individual cases in which there has been a significant occurrence (not necessarily involving an undesirable outcome for the patient) are analysed in a systematic and detailed way to ascertain what can be learnt about the overall quality of care and to indicate changes that might lead to future improvements".¹

Significant events can be very wide-ranging. Examples could include:

• Underage pregnancy
• Coping with staff illness
• Complaints or compliments received by the practice
• Breaches of confidentiality
• A sudden unexpected death or hospitalisation
• An unsent referral letter
• A diabetic registered blind
• A delay in cancer diagnosis

A careful and structured dissection of events involved in the case (clinical, administrative, or organisational) takes place with the central, crucial questions:²

• "How could things have been different?"
• "What can we learn from what happened?"
• "What needs to change?"

The technique of SEA is based on work on the critical incident technique developed during World War II by an aviation psychologist called Flanagan to identify successful and non-successful aspects of combat leadership which could then be applied to training.⁵ Its use has been extended far beyond this field to business, health care, organisational psychology and education, for example. SEAs focus on the team rather than the individual is a key difference compared to critical incident technique.⁶

A key concern of the modern NHS has been the improvement of quality and reduction of risk of harm to patients. Clinical governance was instituted in April 1999 in the wake of the publication of 'A First Class Service'⁷ which was followed by other important government documents, including the Chief Medical Officer's 'An Organisation With a Memory'⁸ and the Department of Health's 'Building a Safer NHS for patients'⁹ which identified opportunities for improving patient safety. Forms of significant event analysis existed within the health service prior to the uptake of SEA including the traditional grand round, clinical-pathology meetings, morbidity and mortality meetings and the various Confidential Inquiries. These all tend to focus on failures and to be based on more traditional hierarchical structures and inquisition. SEA has been strongly promoted, particularly within primary care, as a means of delivering many aspects of clinical governance.
From 2004, SEA was introduced into the Quality and Outcomes Framework (QUOF) as one of the organisational indicators. Increasingly, also, proof of an individual's involvement in SEA and of reflective learning based on it, is being demanded in appraisal and potentially for future revalidation.

The extent of patient safety incidents is hard to gauge, dependant as it is on reporting and identification. This is particularly true in primary care. For Acute, Ambulance and Mental Health NHS trusts in 2004-5, the following statistics were recorded:¹⁰

• 974, 000 reported incidents and near-misses
• Most did not include hospital acquired infections (a further 300,000 cases)
• Most commonly reported - falls, medication error, equipment failure, documentation errors, communication failure
• Cost of increased hospital stays estimated at £2 billion per annum (without hospital acquired infections)
• Clinical negligence cases (2003-4) - £423 million for settled cases and provision of over £2 billion for unsettled cases
• Estimated 10.8% of inpatients suffer an adverse event, of which half are thought preventable and approximately 6% cause permanent impairment and 8% death

Within General Practice, medication error is a very significant cause of patient morbidity and even mortality:¹¹

• 25% patients experience an adverse event within 4 weeks of starting a medicine, of which 11% are considered preventable.
• 7% hospital admissions result from drug-related problems, 59% of which are considered preventable.

In one study looking at the SEA submitted for the QUOF within one PCT:¹²

• 26.7% events involved patient safety
• 6.5% were serious or life-threatening
• 19.9% were potentially serious
• 28.5% involved medicines management

The SEA process:³

• The whole primary health care team ideally should be involved and any other relevant 'stakeholders' should be invited formally by the team.
• A regular meeting (or part of a meeting) should be instituted to discuss significant events, both good and not so good.
• Events should be collected as they occur and recorded in a form/book kept in an accessible place.
• Ahead of the meeting, make an agenda based on priority, availability of personnel and involvement of team members. Circulate the agenda a few days ahead of the meeting.
• At every SEA meeting, run through the record of the last meeting, particularly reviewing action points.
• Identify a facilitator for the meeting.
• Each topic/event should be presented by the key person with description of what first occurred, the subsequent events, and why they perceived the incident to be an example of effective or ineffective practice and followed by more general discussion.⁶
• Try to remember to provide positive comments ahead of criticism.
• Focus on system improvement rather than issues with individuals (though these may arise) to develop a 'no blame' culture.
• Record the key points in a systematic way. Standardised proformas are widely used. Pay attention to any action points agreed and those agreeing to implement/oversee change.
• Outcomes include:
  • Congratulations where a SEA has been well handled.
  • Immediate action to rectify problems revealed by the SEA.
  • Lack of resolution - possibly requiring further work or audit to clarify the situation.
  • No action - sometimes SEA is an appropriate place for the team to air frustrations without requiring systems change, e.g. "life’s like that but I feel better for talking about it."

Important considerations include:^2,6,13

• The selection of time, duration and setting of the meeting. A comfortable, quiet room is essential. Holding the discussions during the day makes it easier for all staff to attend but it can be difficult to avoid routine interruptions. The length of a discussion will vary, but between 20 and 45 minutes will generally be needed for each case. Be aware that cases generating emotive topics may need up to an hour.
• Group size - generally large groups function less well than small groups.
• Choice of facilitator: this is critical to get right. Some advocate the use of a facilitator external to the group. This may not be practical but choice needs to be sensitive to organisational hierarchies and leadership styles etc.
• Rules and guidelines - a clear set of ground rules is needed, particularly at the outset.
• Ownership and commitment. Commitment can only be assumed if the participants' need for safety has been addressed. The evolution of Practice Professional Development Plans (PPDPs) can contribute to formal acknowledgement of SEA for all participants, which will help build corporate ownership.
• Topic choice is important as different team members will have different involvements and interest. Some topics, e.g. those very clinically based, may be appropriate for discussion in more limited groups.
• Management of the process, needs to be active e.g.arrange a follow up meeting, important to ‘close audit loop’, consider the need for individual debriefing after some meetings.

Difficulties and barriers to the process need to be anticipated:^2,6,13

• Time restrictions - where time is too short, participants can feel that quick, easy or superficial solutions are adopted. Finding the time for the meeting can be difficult, especially when part-time staff have to make special arrangements to attend.
• Members also recognise that it can be difficult to be sufficiently honest - beware the fear of a 'Pandora's box'-effect. Staff may fear that lifting the lid will release terrible pressures and unexpected consequences but where SEA is confined to safe or unduly trivial areas, it loses effectiveness.
• The process is emotionally demanding. SEA is often a new and uncomfortable experience for many of the team who may find the critical process generally disconcerting and could be embarrassed by revelations of other members' shortcomings. Debriefing time is often denied by the many other tasks of general practice taking over immediately afterwards without adequate opportunity to talk things over.
• Group dynamics can be difficult: some group members may feel vulnerable in speaking out, especially if their contribution might be seen as critical of those perceived to be of higher status.
• Externally employed staff could be stressed, with loyalties to the GPs and their own management structure sometimes in conflict, which could interfere with SEA
• Motivation - the selection of topics affects motivation. Leaders are more inclined to choose events that involved them with the risk that clinical GP topics can dominate SEA, alienating non-clinical or non-medical staff.
• Issues of confidentiality or fear of increasing the risk of litigation need to be addressed.
• In some aspects, SEA performed within a practice is quite insular and there is a need to be able to assess its effectiveness. In theory, where staff are unable to recognise poor standards of care, no change will be made. Assessment instruments are being developed.¹⁴

Prevention of harm to patients depends on strong clinical governance. Reporting of near misses and significant events is particularly important so that lessons may be learnt but reporting is fragmented within primary care with a variety of reporting mechanisms to different organisations. Currently, GPs report directly to their local PCT Clinical Governance lead or NPSA,⁴ as well as other systems such as the Yellow Card system for adverse medicine events.¹⁵