About the Nurse Telehealth Call Center Industry Quality Assurance/Quality Improvement Survey Report 2022
In May 2022, Schmitt-Thompson Clinical Content (STCC) sent a survey to nurses who work in a telehealth triage call center or clinic/office practice setting. We requested feedback about their quality assurance and quality improvement practices. We received responses to the survey from 57 call centers across the United States and Canada.
The objective of conducting the survey is to provide the STCC network with quantitative and qualitative results about the quality assurance and quality improvement practices in telehealth triage call centers. We hope you find the results useful.
Findings Summary
Seventy-five percent of the respondents work in a large telehealth triage environment with more than 25 nurses. An additional 2% work in a call center with 6-10 nurses, 9% with 11-15 nurses, 5% with 16-20 and 9% with 20-25 nurses. Twenty-two (39%), of respondents indicated they represent a children’s hospital call center.
Eighty-eight percent of call centers have a QA/QI committee or person for their call center with most meeting at least monthly.
The quality indicator identified as the top priority in most call centers is patient safety. Whether the response was stated specifically as “patient safety”, or “correct disposition” or “correct guideline”, it’s clear that patient safety is the number one goal in nurse telehealth triage settings.
The top three metrics measured are:
Correct guideline selected (88%),
Correct disposition selected (88%) and,
Triage call time average (75%).
Call Review
But not everything is about the mechanics of conducting telehealth triage. Several respondents also mentioned soft skills such as voice tone and attitude of the nurse conducting the triage. In most of the cases (74%), written documentation and voice recordings are used in the call review process. In terms of a frequency of review, there was quite an assortment of responses; ranging from one per quarter per RN to three per week per RN. Many comments reflected the fact that new hires have a higher percentage of call reviews.
ED Follow-Up
One-third of call centers follow-up on patients referred to the ED by a telehealth triage nurse. Of those that do follow-up (21 respondents), most (81%) measure if the patient actually went to the ED, the ED diagnosis (48%) and if the call was a potential over referral (43%).
Additional QA Metrics
Over half (64%) indicated they have conducted specific QA/QI projects in their call center in addition to individual monitoring. Other projects mentioned include new service lines, condition upgrades (over and under referrals) and fever > 1 year old. Two-thirds of call centers include peer review as part of their QA/QI process with documentation review (74%) and call recording review (62%) as the predominant methods of peer review.
Staff QA Coaching
A majority (95%) share their QA/QI findings with staff during an individual 1:1 meeting followed by annual RN evaluations (58%) and staff meetings (40%). Respondents were almost evenly split between their nurses feeling the QA process is helpful with learning opportunities and those feeling the process is perceived as punitive.
2022 Nurse Telehealth Call Center Industry Quality Assurance/Quality Improvement Survey Report
Question 1: Do you have a QA/QI committee or person for your call center?
Eighty-eight percent of respondent call centers have a QA/QI committee or person for their call center.
Question 2: If you have a committee, how often do they meet?
Additional responses:
Bi-Monthly, sub-committees meet monthly, leader review occurs every 2 weeks.
Have had the initial meeting only to establish our charter. Will meet quarterly.
Monthly for regular updates, other meetings scheduled as needed.
Meetings suspended during COVID.
The RN manager reads each call taken each day and reviews for outcome and over/under referral.
We do perform monthly peer reviews of triage calls on a rotating basis.
Just have 2-3 people who do QA whenever there is time to do it. Nothing really formal.
We don't have a specific committee for QA/QI but our QA and Education/Training Team meets weekly.
Up until COVID where redeployment of nursing resources was needed. Goal is quarterly.
We only have 1 QA person
We don't really have a committee. The assistant manager assigns random MRNs (3) per person quarterly. We use a form that the nurses fill out regarding their documentation and then I research/make comments and review individually. Very time consuming; however, the nurses find it useful. I also listen to recorded calls occasionally and review with nurses if I find any concerns.
Our team used to do leader to RN review & RN to RN reviews. When the pandemic struck, we were not able to capture that time. We will be reinstating that soon as staffing increases a bit.
Singular QA coach.
Monthly nurse meetings and yearly provider meeting.
Question 3: What are the roles of the committee members? (Check all that apply)
Additional responses:
Staff nurses (7 respondents).
Upcoming committee. Implementing in 2022. Members all invited.
Triage nurses, patient access representatives.
Part of the larger QA committee for the health system.
Account management, staff for each business unit, CEO, COO.
Clinical quality RNs.
Shared governance quality chair.
Question 4: Are your calls recorded?
As indicated in the graph above, approximately 96% of respondent call centers indicated their nurse telehealth calls are recorded with 31% also capturing screen shots and 10% including an embedded QA tool to support the audio.
Question 5: Do staff listen to their own recorded calls? Question 6: If so, in what situations does staff listen to their own calls occur?
Eighty-three percent (83%) of call centers indicated that staff listen to their own recorded calls in the situation where an issue or concern is brought forward or when completing self QA evaluations of their own calls (45%).
Additional responses:
QA performs call reviews and sends them to the RNs with the call findings.
When reviewing completed call evaluations.
Monthly audits by manager.
With the QA coach during the monthly QA sessions with each nurse.
Its recommended staff listen to their calls that have been evaluated by their team lead.
Completing both peer and self QA evaluations of calls.
Reviewing an evaluation.
We also do regular QA on staff as well as our new hires. They are sent the call and their evaluation.
Staff listen to a peer's call each month and they have a brief discussion. From time-to-time staff will listen to their own calls, generally if there is an issue. All quarterly QAs include listening to a live or recorded call as well as reviewing two additional written records.
When examples shared at nurse team meetings. Share good calls and calls where opportunities exist since this provides great insight for learning/coaching and risk mitigation.
They are presented with the opportunity to listen to every call (on their own) that is audited. Issues are reviewed with the Supervisor.
Encouraged to listen to each call with QA Review.
During orientation we have new hires listen to their calls, identify what they felt went well and what they could improve on. The trainer does the same and notes are compared. We have an embedded evaluation scoring system so when calls are audited, we use the evaluation tool to mark which elements were missing from the call (greeting, closing, workflows followed etc.).
They do self-evaluations and they are given the audio with their QI.
The team of nurses are required to do peer reviews on call recordings selected by their manager.
During 1:1's with supervisors weekly, biweekly, monthly depending on experience. Tiered approach based on expertise.
Clinicians can be asked at random to review calls.
Question 7: What quality metrics or organization specific data points do you use in your QA/QI program? (Check all that apply)
Eighty-eight percent (88%) of call center respondents indicated both “correct guideline selection” and “correct disposition selection” as key quality metrics used in their QA/QI program. Additionally, 75% indicated “triage call time average”, “average speed of answer” (61%) and “caller/patient satisfaction” (59%).
Additional responses:
Documentation needs to follow the flow of the call. Amount of time the call was put on hold by the nurse. That the chart is closed in the disposition given, in the right guideline, and with the correct demographics for the person triaged.
Voice and word choice - tone of voice, soft customer service skills.
Customer service.
We mainly focus on whether the nurse followed our workflows and used the appropriate protocol and reached the appropriate disposition. We do have a couple metrics on soft skills such as being professional and empathetic.
We monitor metrics above separately from our QA review. We include the guideline selection and disposition in our quality review. We monitor all components of the nursing call and assessment in our quality review.
Call abandonment rate.
I do post monthly call time averages for awareness only at this time.
Correct provider page process using EHR to enhance clinical call information of the patient
In addition to the above we also review the triager's tone of voice/attitude towards the caller and whether the nurse gauged the level of the caller when providing care advice/recommendations.
Communication, care advice delivery, caller/patient understanding.
Question 8: How do you review calls?
A majority, 74%, of respondent call centers, review telehealth triage calls using both written call documentation and listening to recorded calls.
Additional Responses:
Use both audio and written evaluation PLUS screen recording (assuming it is working for that RN).
Call parents for feedback.
Peer to peer listening/documentation review monthly.
Screen capture and electronic QA form.
While listening to calls, we pull up the patient's chart to also review the documentation taken during the call.
Online triage documentation (in EHR).
Q9: How many calls are reviewed in a normal (not COVID) year? Is it a percentage of total calls? Do you review a set number or a set percentage per staff person? Other?
We review every shift for the first five weeks. Both audio and documentation calls are reviewed for a minimum of five weeks. We progress the RN to monthly reviews which consists of two random calls until they have been here at least a year. Once the RN has been here a year and scores 99% or higher, she can be reviewed quarterly or if a situation arises.
Active QIPS: Survey 10% of the total volume of calls, establish goals, interventions, barriers, and follow up. Maintenance QIPs: Once monthly goals are met, survey five random calls per month or 15 per quarter and still meet goal.
Retired QIPs: Spot checks randomly (we don’t have guideline for when or how). General survey: 80-100 total surveys done per month by team.
4 calls per month - 1 per week.
Our committee is assigned 10 calls per month to evaluate. We just started having direct supervisors also review the calls. The goal is 2 calls per RN per month.
Low performers and new hires = weekly. Mid performers = monthly. High performers = quarterly.
Two per month per RN for triage. A set number per month for medication and health information calls.
At least one per nurse by manager in a complete audit and another QI that looks at component completion of encounter.
Twice a year, five calls each nurse. New hires are audited every month until confident and then move to twice year.
Pre-COVID the aim was to review three calls each month for each nurse. One review was with the nurse present and 1-2 reviews were by the coach, and then an email evaluation was sent to the nurses. New nurses would also get coached more frequently - their 5th independent shift they would have three calls reviewed with them – a documentation review, a service call review and then listen to an assessment call and review that one together. The direct reporting manager would also do a call review with new staff at the one-month mark of independent practice. With about 225 nurses on staff, we usually were able to accomplish 1-2 evaluations with each nurse per month.
New staff start at eight calls per month. As experience increases, the number of calls reviewed decreases to two per month for experienced staff.
Set percent per staff.
1 per week per triager.
Our QA reviews one call per nurse on staff per month.
Ten calls per month per nurse.
Audio - 2 -3 once new nurse out of mentoring. If RN doing well on QA: 1-2 per quarter. Chart review - 5 charts per quarter if RN doing well on QA, more reviews if QA scores are < 85%
25%
Two calls per employee per month.
A number of calls per nurse for triage and an overall number of calls per medication and health information.
Set number of calls per staff.
12 calls per month per nurse
Five per agent per month.
20-24 calls per 8-hour shift.
Set number per staff member. Three calls per month per team member.
Generally, three triage calls monthly plus all ED disposition calls.
We review one call per quarter for our established staff. This can increase if we identify staff that are having performance issues. Our new hires get more calls reviewed; five calls per week then decreasing from there over the first few months after they are done with orientation. We also review on an as needed basis for complaints, etc.
Five calls per nurse per quarter, so 20 calls per year.
Three per RN per quarter.
Three calls per quarter; new hires 3-6 per month x 6 months.
One call per week per RN.
Not nearly enough. We do more on new staff during their training period, and less once established in their practice. 1-2 calls/nurse/quarter. 5-10/month during the first few months. If anyone is found to have a "bad call" we increase frequency until we see improvement.
Set number per week per staff.
10%.
5% total calls, no less than three per nurse.
We review a random sampling of calls per nurse.
We aim to audit at least 10 calls a month for each nurse but I would love to know what is an ideal number of calls that should be reviewed.
Minimum of two per nurse/month.
Three calls per quarter per person.
Monthly for a score of 90%. Twice a month for a score less than 80%. Every other month for a score of 95%.
Did review six triage calls per quarter (additional audits if not passed).
0.6% of total calls are reviewed.
30 peer review and 30 self-review per year.
About 5% of all calls.
About 1-2% of calls/month. Staff with higher FTE have more calls reviewed. We also review more calls for RNs in their 1st year.
Two calls per team member per month and additionally any requested audits.
5% of calls.
2-3 calls per RN per month.
New hires within three months have 10/week audited. After that if there are no concerns, it is 2/week. If safety concerns are identified at any point during employment, audits go back to 10/week while education/training is provided to remedy the concern/trend.
An audit consists of two calls per employee. The score of the audit determines the interval in which the employee is audited again.
Set number on each staff member; four per month.
Four calls per RN per month.
Question 10: What tools do you use to measure QA/QI? (Check all that apply)
Additional Responses:
URAC standards.
Data from recording software.
Built in QA template within call recording application.
I have referenced URAC's standards for abandonment rate and Average Speed of Answer.
Question 11: How often do you perform QA/QI on your staff? (Check all that apply)
Additional Responses:
We do all of the above.
We review 24-hour calls weekly as a leadership team (disposition not ED now, be seen in 4 but ends up being admitted to system within 24 hours of the call).
We will soon have a daily dashboard in our EMR that staff will be able to access.
Our new hires get a higher frequency of listening for the first few months during and after orientation.
New hires have more frequent assessment until they exhibit solid competency.
Our vision is to provide weekly call metrics from (Cisco) software but we struggle to obtain clean and meaningful data that our team can interpret and that is easy for our quality coordinator and supervisor/manager to run. We also aim to consistently audit calls/charting on each nurse but would like to give monthly feedback on this but the software we use is not user friendly for staff or leaders (Calabrio). There is a dashboard that has potential. Annually at performance reviews, we tie in recent call/documentation audit results to discuss overall strengths and areas to improve. We use EPIC EMR.
Currently not doing.... due to pandemic/staffing. Presented monthly call metrics. QA was quarterly from triage notes (1:1's with leader).
QA is completed on a new staff member for 6 months which includes every call via documentation and two calls per week audio.
Staff are sent their "dashboard" on a quarterly basis which shows their individual performance compared to the department.
An audit consists of two calls per employee. The score of the audit determines the interval in which the employee is audited again. The audit schedule shall be: New Hire/While in Trial Period - a trial period of forty (40) working days or three hundred and twenty (320) hours worked, whichever occurs first). First audit shall be completed within first 3 shifts. Proceeding audit schedule based on audit rating: Green- audit again in 2 weeks Yellow- audit again in 1 week Red- audit again next scheduled shift Triage RNs outside of trial period: Green- audit every 3 months Yellow- audit within 1 month Red- audit within 3 shifts
Question 12: Do you have, or have you had, targeted review or specific clinical QA/QI reviews outside of routine individual monitoring?
The data indicates that it is becoming more common for the triage nurse to have access to the patient’s electronic medical record at the time of the call.
Question 13: If so, what are you monitoring?
We monitor for condition upgrades, over and or under referrals. All calls are verified by the HCN and RN for accuracy and HIPAA compliance. Areas are identified by QA for improvement, an action plan is formed and monitoring of the progress is done until completion.
Varies based on department or individual identified needs as well as department goals.
We review 24-hour calls weekly as a leadership team (disposition not ED now, be seen in 4 but ends up being admitted to system within 24 hours of the call).
Proper HIPAA identification.
Any issues.
We do safety reviews of charts of callers that went to ED within 24 hours of calling, but were given a self-care referral when they called. We review and coach on calls that someone has made a complaint about. We review and coach on calls that were referred to another medical department when that department then provides us reviews of feedback on the referral.
Effects of program structural changes.
Usually on a specific guidelines or symptom. Sometimes with a new service line, will also include procedural/process QA.
Not in ministry calls, community-based call numbers, out of state calls, post ER/Hospitalization calls.
New employees, employees with a low feedback score, senior triage RN's review an additional 10 calls per month on a specific outcome that is trending the month prior.
Focus identified as needed.
More frequent monitoring of an individual immediately out of orientation and/or when errors are found.
I want to assure all nurses are taking a brief medical history of patient or copying history from the EMR.
Meets Emergency Criteria.
Performance Improvement Plan.
We currently have a de-centralized model for nurse triage services for our 31 offices. There is no formal QA/QI process. We have been following data from the EMR to monitor if symptom related calls are being managed by protocols to support triage. Historically this was a challenge, but data showing it is improving.
Guideline use for specific body systems competency by target area (chest pain, abdominal pain, etc.).
New hires, any changes in policy or workflow, new initiative workflows.
Triage call/doc times - look for call talk time and documentation time averages [performance coaching opportunities]. ED referral rate - if outside STCC benchmarks or dept benchmark average [performance coaching opportunities].
Sexually harassing callers.
Documentation accuracy & adequacy of assessment.
Various protocols and processes used.
Whenever a new program starts call.
Client workflow adherence. Appropriate escalation.
We try to monitor productivity of how many in basket messages (EPIC EMR) nurses handle, but the reporting tools in Epic are not user friendly and can only access some of this info for past, not present day. A real time dashboard like our call software would be most ideal.
We conduct focus reviews specific to certain trends we see with use of guidelines or dispositions against expectations.
We will address unusual calls at staff meetings and discuss, we do a lot of pediatric refreshers at meetings, new protocols/major changes are reviewed when they come out.
Authenticators, workflow, customer service.
The QI committee will determine a specific focus or guideline per month. For example, one month we might review calls that utilized the Head Injury guideline. We also have a 24-Hour Outcomes Committee where we look at calls where the patient presented to the ED within 24 hours of speaking with one of our triage RNs who did not refer the patient to be seen. We look at the reverse of that as well...good catches where a nurse did refer in and the patient was admitted.
We have dashboards created for every SOP to ensure we are meeting the expectations of the org and patient care.
New specialty triage coverage guidelines specific to that specialty (i.e., cancer guidelines). COVID specific calls. Use of Info or Med guideline in lieu of symptom based guideline.
Anytime something NEW is implemented (protocols, workflows, specialty...) we audit it for 90 days post implementation.
We have the following QIPs (Quality Improvement Projects): ED/Admission after triage - chart reviews for patients coming to the ED or inpatient area within 24 hours of nurse triage. BBFF - newborn calls - specifically looking for breathing, behavior, feeding, and fever assessments and education. Fever > 1 y.o. - do we offer fever handouts to families calling for fever? Do we use the right protocol? Endocrine - ABCD assessment, endo reason for call, blood glucose & last insulin documented, 2nd nurse check for insulin recommendations, appropriate escalation. Community Calls - ABCD assessments, permission to share record, etc. Neuro/Seizures - coming in 2022 - build underway Head Injury - retired Croup - retired Poisoning – retired
Question 14: What criteria do you use to initiate a specific QA/QI project? (Check all that apply)
Additional Responses:
New sub-specialty support, i.e., endocrine - diabetic population, epilepsy protocol use.
New guidance or changed policy. Patient outcome.
Review of calls and possibly triage guidelines may be done if another medical department (i.e., ED physicians, local CDC) are questioning the referrals that are sent to them.
We have standard audits we do every year.
Team member failing QA.
High risk symptoms or guidelines may be reviewed at nurse meetings for education, have provider (specialty service) provide in-service to nursing teams.
When a particular audit element falls below threshold, we focus on it until it improves, and stabilizes.
Whenever a new program starts call; program specific details.
Our internal Triage Process Improvement Committee (TPIC) identifies workflows that are inefficient based on feedback we get from our nurses, our providers and patients or based on errors that have occurred.
Random pull of triage calls. Always have follow up on any concerns whether from callers or colleagues.
Question 15: Do you do any follow-up on patients you refer to the ED?
One third of respondent call centers answered “Yes, per nurse leadership or QA person judgment” to this question. Conversely, two-thirds of respondents do not do any follow-up on patients referred to the ED by a telehealth triage nurse.
Question 16: If so, what do you measure for patients referred to the ED? (Check all that apply)
Twenty-one individuals responded to the question regarding what is measured for patients referred to the ED.
Additional Responses:
We perform routine next day follow-up for all calls to the nurse advice line. Implemented in May - we use outbound text messaging using Patient EXP to replace call when we have number on file in demographics.
Have their needs been met? Are there ongoing concerns?
Whether the guideline selection/use was appropriate.
Do not measure at this time as nurse triage platform different than EHR. Future state, we look to have this ability as it would allow for assessment of ED referrals; appropriate and/or over referred to allow for coaching on guidelines and assessments.
We send the PPC a message from EPIC if we refer to the ED and they no show. Since we are a pediatric facility, we ask if they will be coming to our facility or going to a closer one. If they choose our facility, we put in a referral to the ED.
Was the patient admitted, in which case we highlight as a good catch.
Question 17: Do you include peer review as part of your QA/QI process?
Two-thirds of respondent call centers include peer review as part of their QA/QI process.
Question 18: If yes, what is your process for peer review? (Check all that apply)
Additional Responses:
We instituted a monthly RN Grand Rounds: Will write up the call scenario, without the original outcome, to check with all RNs' "what would you ask and do". Specifically for over-send outcomes for ultimate 1:1 training with the RN who is over sending.
Our Quality Coordinator (who was a triage nurse in our dept and promoted) will provide 1:1 feedback with nurses and review documentation, recorded calls. Our nurses all need to obtain two peer reviews (system template) to be turned in with their own annual performance review but these peer reviews are not based on call recordings they heard of their peers, they are general questions answered about teamwork, etc.
We have yet to implement peer review however this is something that we are considering.
Question 19: What method do you use to share QA/QI findings with your staff? (Check all that apply)
Additional Responses:
Monthly emails to the staff with numbers assigned to the RNs making it anonymous.
Staff meetings present group results.
We don't specifically share QA scores. But if we identify a trend of several people missing same issue, we will review it in a staff meeting. I will work 1:1 with RN if significant issues noted.
Calls are sent to the employees along with the evaluation and comments. Our QA coordinator will meet with staff during orientation or if an established staff is having a particular issue or struggle. If there continues to be issues with staff, during their QA evaluations a supervisor may get involved as needed.
Staff meetings.
We generically (without using employee names) report to the team cases that had potential for harm or actual harm/delays in daily safety huddles and monthly staff meetings. Likewise, if someone on the team has a "good catch" due to being detail oriented or having a questioning attitude and averted a poor patient outcome, we recognize them on team daily safety huddles and at staff meetings and submit their names for system formal recognition programs like our "good catch award" or "breakfast with CNO". Both are monthly nurse recognition programs our system has in place.
Interdepartmental meetings with specific teams to discuss trends and results.
Question 20: What approach do you use in sharing QA/QI findings? (Check all that apply)
Additional Responses:
The auditors use "canned" verbiage in emails to staff to highlight things they did well, and areas they scored low.
Staff have real-time access to evaluations that are completed by the quality committee via a web portal.
Can access them via the dashboard.
Metrics shared on a dept dashboard with names blinded and each RN/Agent assigned a corresponding level. Allows staff to see where they measure according to the group.
1:1 meetings.
Dashboard.
We encourage near misses or occurrences that impact patients, be entered into our safety event data base.
It depends on the situation.
WebEx (video) meeting; 1:1 meeting with individual.
Question 21: Do you have a QA result appeal process for the triage nurse or their supervisor so that they can appeal for another review?
Additional Responses:
The RN is able to appeal and then the call is sent to the medical director to review.
Staff are always welcome to discuss their evaluations and scores with our QA person or their leader.
Any negative findings are discussed 1:1 and if there is disagreement, RN can discuss with leadership.
Informal but yes, review would occur with supervisor/manager and QA program manager.
Nothing formal. But we would review it again if they asked.
It's treated as a learning experience and not punitive.
Not formally. If a nurse or supervisor did not agree with a case finding and has their own input on the matter, we do review the case with a new perspective. When significant issues occur, our hospital's peer review board reviews cases with many departments represented (legal, clinical leadership, patient advocate).
The RN will email the department educator if she disagrees with an audit. The educator will then review the call.
We work in a very collaborative environment.
Question 22: How do the triage nurses perceive the QA/QI activities?
In summary, this is a very mixed response. Slightly more respondents believe the QA/QI process is helpful and provides learning opportunities than they find it punitive. However, there is a still a large contingent of respondents that have mixed reviews of the process and/or find the process to be “intrusive” or “punitive”.
Additional Responses:
Nurses perceive the QA activities as learning opportunities.
Most nurses feel that QI is punitive no matter how hard we have tried to change that perception.
Helpful to listen to themselves and critique call.
Involved/ engaged.
Most are welcoming.
Positive.
Nurses new to the process are nervous about it prior to receiving coaching, but once they have been coached the feedback about the process and what they have learned from it is very positive. They see the value in the coaching process helping them to work towards best practice in their calls. Coaching is a harder process for some nurses not open to feedback or making changes to their practice as evidenced by them being more difficult to engage in conversation during their coaching sessions.
Reviews are mixed.
Learning/improvement exercises.
We recently moved to a new QA process within our unit, it has been an uphill battle so far. Our nurses have overwhelming perceived the QA process as punitive. We continue to work to improve the process and use it for coaching.
Mostly favorable. We have new nurses meet with the QA RN, she reviews the different QA tools and that the purpose is for both patient and RN safety, and for growth of the new RN. She works closely with newly mentored staff to create an atmosphere of safety and trust, and QA is presented in a non-punitive manner. Before sharing a case study in a staff meeting, we obtain permission from the RN that took the call, and the nurse’s name is not given. We encourage staff to be constructive in their feedback as a case is reviewed. For any QA review, positive feedback is always included. We also foster with our lead staff an atmosphere of support and encouragement, so nurses feel comfortable asking questions real time with their triage calls.
With difficulty: Instead of learning, opening the mind, expanding knowledge, they take it personally, with "hurt" feelings.
Not a favorite item to have to complete, but understand the reason for needing to always improve.
They perceive the QA/QI process as involved.
Parts are viewed as useful; others are viewed as routine.
Generally well received.
Several are offended by my use of the person who does the QA. But no one else has offered or is willing to do QA.
Appreciate the feedback and information.
They are very involved and agree with the process. We have the leadership team along with two nurses look at the QA forms quarterly to see if we need any updates or changes.
They demand it.
We have come a long way, but I do feel that our nurses view this as a learning tool and something to help them improve their practice.
They feel that the reviews are too subjective and not consistent among the different reviewers.
As supportive and educational. These processes are not punitive.
Mostly positive, as learning opportunities for performance improvement. Like case study reviews. In past, peer reviews were performed but staff became more defensive with this type of review, would attempt to 'justify' why they took that approach on a call. QA feedback received better from department leadership positions (supervisor, program manager QA, educator, manager, director).
60% find the process to be intrusive and overly strict. It takes nurses quite some time to get comfortable with the level of oversight the call recordings with screen capture produce. This 60% find it "big-brotheresque."
Some like it and find it helpful and enjoy it. Some don't like it and think we are always "watching" them. Overall, I think it is more on the “like it” side, but I'm not 100% sure.
As a tool for improvement.
Some feel "picked on". It depends upon the approach. We sent out a very generic triage audit email saying their call had been audited and to please review and respond to their team leader. They did not like that. They felt it was patronizing and wanted to just be told what to do better. Most didn't like that either though! LOL!
Welcomes.
Most of the time, nurses react negatively to QA/QI activities.
It seems nurses trust the process to be fair and transparent. They readily enter occurrences they find into our safety event software knowing the investigation approach is done for fact finding, process improvement. We are not punitive and use re-teaching/re-education as a first line approach to ensure employee knows what happened, which workflows or protocols were circumvented (if applicable). If re-education is given, we have the nurse do a return demo or verbalize their understanding.
They are usually receptive of the feedback.
The nurses appreciate the QI they view it as a learning tool.
Favorable for the RN-to-RN reviews (viewed as less stressful).
They are open to it and to constructive criticism. They value the suggestions their peers (or manager) provide.
Positive. Appreciate the recordings.
They don't love receiving audits and they don't love doing them.
Our team asks to have more calls audited. They like the feedback.
They don’t like listening to their calls but see the value.
When we stopped manual scoring and it's more comment based (we track metrics in background) it has been well received.
When we started the QI program the team was extremely averse to it. I don't think this is unusual when team members start to be audited, assessed, and scored. We took a huge hit on our engagement survey the first year and the comments were specific to the QI program. Comments regarding being "policed" or just "viewed as numbers" and all we cared about was being able to check safety boxes for external measures or that we were meeting "productivity numbers" which we didn't even have. It took a good two years to bring them around.
Mixed response, some are very open to feedback and strive for a higher score
Question 23: What is the biggest obstacle in performing QA/QI? (Check all that apply)
Additional responses:
High call volume and low staffing numbers.
Will look for a national triage RN call audit approach.
Wanting screen capture to augment QA.
Question 24: What quality indicator do you think is the most important in your call center?
Assessment skill (did the triager ask appropriate questions and remain attentive to clues that might affect the selected disposition).
Call outcome- client satisfaction.
Our chart reviews. We have three safety areas of focus - documented assessment, correct guideline selection and correct disposition. The nurse is given feedback on her overall triage encounter, but specific focus is paid to our three safety areas. The nurse can then focus on these areas during triage.
Quality of documentation to support the outcome, clear dialog between caller and nurse.
Correct guideline/disposition; caller satisfaction.
Customer service skills and listening skills.
Call volume.
We have picking the right guideline bolded as well as that they asked all questions not following this results in an automatic 0.
ED Referral/Utilization is most important for our system but appropriate guideline use and disposition is most important to our department.
Correct use of the protocol and correct disposition was reached.
Correct patient disposition (quality over quantity).
Upper leadership - call metrics; manager and QA team - correct guideline, disposition and accurate assessment documentation.
I don't believe that a single metric can do justice as a measure of the call center's quality. Measuring talk/documentation time important for managing call volume/capacity of team, ED referral rate important to note with provider groups as they typically think the nurses doing phone triage "always send patients to ED", which we know is not the case.
I'm not sure if this is a "quality indicator" but we put a huge emphasis on following the process, which on a high-level means using a symptom-based guideline on every call, and going through all of the questions systematically.
Following protocol; disposition change can only result in a higher acuity disposition.
Safe and effective recommendations based on a thorough and accurate assessment.
Post-call automated patient phone surveys; Press-Ganey patient surveys (don't really capture phone services more than they do office visit experience); staff turnover; call metrics.
Reason of call should be supported in correct guideline use.
Final disposition.
Audio and documentation scores.
Customer satisfaction scores.
Call times, calls/hour, patient/provider satisfaction, under referral rate.
Customer service and personal presence.
Appropriate Disposition. However, there is importance in what we call "turnaround time". It is the amount of time between the initial patient contact and the first contact with a triage RN. 90 seconds is our target for emergent calls, 20 minutes is the target for non-emergent calls.
Use of triage guidelines to drive patient disposition.
Question 25: What quality indicator or recent QA/QI project are you most proud of?
Patient satisfaction (2)
Our rapid onboarding of staff in an effort to decrease call back times during COVID.
The endocrine QIP has shown great promise in identifying areas for focused education and improvement. These are higher risk calls, and not all nurses are comfortable managing the "Sick Day Protocol" provided by our endocrine team. We use an Endocrine/Nurse Advice Line specific protocol for these calls.
Establishing a new QA tool to use across the health care system.
Teach back.
Achievement of targeted scores for health information calls.
ED dispositions reviewed every day.
The last year coaching has returned as a priority in our contact center with each nurse getting two opportunities for a call review each month. The coaching is done by our dedicated coaching team that has resumed this role in the last year, after having other duties during the height of the pandemic.
Restructuring of telephone triage that led to significant reduction in under/over-referrals and near elimination of carrier-initiated quality of care requests.
Moving to an internal program we have seen an increase in quality scores in the recent months.
Not necessarily "most proud of", but our audio reviews over the last two years have been mostly favorable, and proud of the nurses in extreme volume situations that they continue with high caliber calls.
Reductions to ER AND to UC by assisting daytime scheduling department: we are assisting the scheduling department by taking some of their calls and directing when to schedule the patient. (Majority of patients with a same day appointment request do not qualify for same day.)
Project to get staff not to pick information only guideline, they must pick the reason for the call. This has improved greatly after using quality project.
Meeting QIA results after targeted efforts.
High percent of nurses consistently meet expectations and above.
Most QA is high scoring. showing good compliance and understanding of our training and tools.
We are working on a project currently to separate the two functions of Nurse Triage and Transfer Center. Currently our nurses are in both skill sets so toggle back and forth between these calls. The nurse calls suffer because of this. The caller is being placed on hold multiple times resulting in either a frustrated caller or an abandoned call.
Simple ED Audit tool/project.
Correct patient disposition- about 98% for department.
5% abandonment rate, 32 second average speed to answer.
Having the capacity to perform QA in this healthcare landscape (short staff, staffing turnover) is a feat in itself. Sharing of disposition rates with our primary care partners, and top 20 guidelines used so they can understand the types of calls received and the action taken.
Implementation of a new Experience Specialist QA tool.
Patient safety is our biggest concern. I think our quality program is what has led to our success. We want to make sure no matter what nurse you get, you will receive the same consistent information. We are very specific on who we hire to try to decrease quality issues.
Charge nurse and Quality Coordinator role implementation; Low ER referral rate; Aside from Covid surge volume increases, typically very good call metrics; Low employee turnover rate.
911/ED recommendations. We ensure that nurses understand the importance of accurate assessments to guide individuals appropriately. At times, patients deny their own symptoms and our nurses have saved many lives.
When able to actually stay on track with quarterly chart audits.
Correct guideline use.
Survey conducted on callers regarding their personal satisfaction and compliance with the nurse triage and the final disposition. Of the 50 surveys conducted 48 said they were very satisfied and would continue to call for advice.
Customer satisfaction; audio scores and documentation error rate.
Post-call survey results.
The initiation of Good Catches. The RNs love receiving audits that highlight the good work they've done and they enjoy knowing the outcome of their call as they don't often get those details.
This past month, we did not identify any missed scheduling opportunities.
Dashboards to quickly identify issues.
Safety and customer service.
We did a LEAN project on variation in triage encounter times. It was enlightening to see the reasons for variation and to be able to use internal best practices to optimize workflows. We were able to reduce variation in encounter time by 20% within the first 3 months post exercise.
Monitoring results and seeing quality scores improve across the department.
As always, Schmitt-Thompson Clinical Content thanks our respondents for continued support of our research endeavors and making the world of telehealth triage safer.