Frequently Asked Questions (FAQs)

Review some of the most frequently asked questions by ControlCheck users. 

Q: Will a variance get resolved if I go back in and document what was missing?

A: No, ControlCheck uses a grouping logic that includes unique attributes for each of the transactions (including time) making up an event summary. By the time a variance is identified on the Review Audit table, documentation (e.g., eMAR, ADC return/waste, etc.) made to “fix” the variance will not resolve it because it will be outside of the original

event summary attribute parameters.

Q: Can the data be sent in real-time?

A: ControlCheck can accept the necessary reporting data from your source systems via three data delivery options (selection is made by the implementing facility):

• Manual file upload via the application’s web-based user interface, usually on a daily or weekly basis
• Automated file transmission via SFTP, usually daily
• Active HL7 integration or clinical API delivery for real-time data sharing

Based on prior hospital partner experience, ControlCheck recommends SFTP for ease and speed of overall implementation, but the hospital partner ultimately chooses their preferred data delivery method. SFTP is by far the most common method used by our customer base, as real-time data sharing will create a lot of unnecessary noise on the Audit Review table

Q: Does a high IRIS score mean there was an act of diversion?

A: No. Although it is not likely possible to determine if an individual has a potential addiction problem and/or if someone is diverting without involving the human factor of an investigation, ControlCheck does provide activity and metrics trending reports to proactively identify those people who represent the biggest risk and where investigation efforts should be focused. ControlCheck can highlight when a user's behavior is trending in the wrong direction.

Q: Which providers/users and/or how many should I be reviewing and on what cadence?

A: Unless otherwise identified as a suspicious user that requires more immediate review, monthly review of those users

with a red IRIS score (those with the highest degree of behavior risk) is recommended.

Q: How do you get the information for the shift analysis?

A: Data for time and attendance can be provided via flat file if the data exists in discrete fields including the following: 

• Employee Name
• Employee ID
• Department
• Role
• Scheduled
• Schedule Date/Time
• Actual Clock In/Clocked Out Date/Time

Q: What is the meaning behind the IRIS semi-circle visuals?

A: As the IRIS score gets larger (the number inside the semi-circle), the ring grows. In the below graphic, notice that the 8.1 red ring is more complete than the 2.0 yellow ring.

• Red ring >4.6X (99th percentile – high risk)
• Yellow ring 1.39 – 4.6X (75th to 98th percentile – medium risk)
• Green ring <1.39X (<75th percentile – low risk)

Q: Are the emails sent tracked through ControlCheck? These would be coming from the audit table. Are the responses sent back and forth tracked by ControlCheck?

A: Emails generated from ControlCheck are sent and tracked by the email server system employed by the hospital. Emails sent to users have a link that (given user privileges) the user would access ControlCheck from and add the requested response information. Responses added within the audit reconciliation process are tracked by ControlCheck.

Q: What is the time period that ControlCheck processes transaction reconciliation after the data files have been consumed

A: This depends on how the queues are looking, but typically it is as soon as 20-30 mins or as long as a couple hours. KitCheck actively monitors these and addresses processing speed as needed.

Q: How are Pharmacy receiving variances handled if the un-accounted for product is found and the count is corrected in the narcotic vault? Would another variance be created as there would be an amount received in the narc vault that didn't have an associated wholesaler order or does ControlCheck have the ability to put these together somehow

after the fact?

A: In reconciling pharmacy transactions ControlCheck uses Day as a way of connecting transactions. In this situation, it would create another variance. On the patient care module, ControlCheck does aggregate against what’s currently open; however, this is not done in the Pharmacy module due to the minimal data available to match and connect transactions on.

Q: Does ControlCheck aggregate and close out variances when nursing back charts an administration?

A: Yes, if the hospital's query pulls the administration in the next day's file, then yes ControlCheck would auto-reconcile that variance if it was still open.

Q: How does the IRIS algorithm compare to the utilization of standard deviation?

A: Standard Deviation expects the data to fall on a ‘Normal’ curve, meaning that some users don’t divert, many are normal, and some divert (like a bell curve. The issue we’ve seen is the actual event and utilization data falls on an ‘Exponential’ curve. ControlCheck uses analytics and machine learning to highlight outlier behavior and present an

outlier priority assessment. Compared to traditional standard deviation or rank order analysis with counts and averages, ControlCheck improves detection and priority assessment of outlying behavior. ControlCheck’s proprietary methods combine a myriad of data sources and apply advanced pattern recognition analysis to provide a more complete and refined view of behavior patterns.

Q: How does ControlCheck monitor float pool nurses?

A: By default, ControlCheck shows users and their corresponding IRIS score over a 30-day period. Comparisons are done by the department where user activity is created. So a float nurse's activity in a department is compared to other nurses' activity in that department. If a float nurse changes departments every day, all of their activity in a representative

department is compared to the activity of others in that representative department.

Q: Could it result in an abnormal score if a Float nurse has a lower amount of documentation in a department compared to other RNs in that same department?

A: As a general rule, we compare people's activities within the same department, but we also sum their behavior scores in different departments to get the whole hospital view. A float nurse with activity in many departments will have their activity in all departments contribute to their overall score. This assists when they have a lower volume of documentation in specific departments.

An example: Let's say a float nurse wastes 1 full package in department A, 1 full package in department B, and 1 full package in department C. One waste in each of those departments might not be abnormal, but the float nurse's IRIS ranking is going to be influenced by their wasting in all three departments (as it should be since they wasted 3

packages).

How does that float nurse compare to a regular RN with 3 full package wastes in just department A? This is harder to generalize because it will depend on overall behavior in each of those departments (A, B, and C). The float nurse's whole hospital activity is going to influence their metric scores, so there is no concern that the behavior is going to be

hidden because it's a small quantity in each department.

Q: What percentage of users will generate a high-risk score?

A: Most facilities should expect to see 30 - 40% of total measured data assessed as High and

Medium Risk, but there is no requirement that results will include all three priority rankings.

Based on the overall variability of a facility’s data from the facility’s norm, you may have a

cluster of Medium Risk events with no High-Risk events, or a cluster of High Risk events with no

Medium Risk events. Some facilities may find all metrics assessed as Low Risk, even with some

variance rates within the data.

Q: Is there currently logic built in specifically for Children’s Hospitals?

A: Currently there is no specific logic for Children’s Hospitals.

Q: What percentage of investigations result in catching a diverter?

A: In the first year of ControlCheck, our customers reported the true positive rate to be 95%. Since then, we have been receiving known diverter events that were identified as red on IRIS on a monthly occurrence. As we learn, we continue to adjust our algorithms with each true diverter and false positive.

Q: Are my nurses being compared with my anesthesiologists? Are my Radiology nurses being compared with my NICU nurses?

A: Analysis is separated by care area; providers in the nursing care area are not compared with providers in the OR care area. Within each care area, metric scoring is largely department-specific but is sometimes adjusted to some degree to account for care area-wide information. At its core, IRIS compares a provider’s activity in a department with other providers’ activity in that same department. We also bring in care-area-wide trends: The smaller the department, the less confident we can be about department-specific behaviors, and we, therefore, use information from the whole care area to influence what is considered “normal” in that department

Q: What happens to transactions that occur around midnight? Why aren’t they appearing on the audit table?

A: Based on the number of hours listed in the hospital settings for Processing Cutoff, events will be held from processing until the next day's ADC/EMR file is received.

Q: My audit table shows staff that are not mine, why?

A: Those Nurses documented on machines that are in your area of supervision. Recommendation: sort page by User to easily sort pages by your staff vs others

Q: Are all of the metrics contributing to the IRIS analytic score weighted equally?

A: Yes, all of them are..