Levelling-Up Cancer Patient Management with Data Engineering

Integrated Care Boards (ICBs) and cancer alliances both provide operational and strategic support for Trusts in delivering cancer care. However, supporting Trusts can be challenging for system partners who often only have access to scattered data that updates infrequently from isolated systems. This means that identifying capacity bottlenecks often occurs after treatment standards have already been breached, which  adversely affects critical 28-day diagnostic standard and 62-day treatment standards.

The CanCollaborate tool, developed by Edge Health in partnership with the Northern Cancer Alliance (NCA), The North Tees and Hartlepool NHS Foundation Trust, and The South Tees Hospitals NHS Foundation Trust, was named to reflect its purpose of enhancing collaboration between Trusts and system partners. CanCollaborate is a system-wide cancer patient management tool, featuring patient tracking and demand forecasting, that helps users proactively identify and act to mitigate capacity bottlenecks.

What is CanCollaborate?

CanCollaborate is a cloud-based solution featuring seamless data integration, real-time updates, and a secure, user-friendly web application interface for monitoring and managing cancer waiting lists.

• Seamless Data Integration: At the core of CanCollaborate is its robust data integration capability. We established a secure, automated data transfer process with Trusts, with encrypted data uploaded to our system every hour. Our methodology enables us to link patient pathways from disparate systems (e.g. Infoflex/Somerset), enabling an integrated view of complex pathways across both Trusts, while eliminating manual intervention.
• Real-time updates: CanCollaborate takes a proactive rather than a reactive approach to cancer pathway management. Our system ensures that North Tees, South Tees and NCA always have access to the same live information. The ability to monitor the PTL dynamically allows for responses to emerging issues, helping NCA to deliver shared solutions.
• Leveraging cloud technologies for data processing: We developed robust pipelines in Azure, integrated with Databricks, to ensure efficient and secure data handling and timely output of the processed data to a SQL database.
• Web application interface: Leveraging our established data pipeline, CanCollaborate provides AI-enabled predictions of a patient’s risk of missing cancer waiting times standards (e.g. 62-day target), which users can interact with through a variety of tabs, including:
  • ‘Waitlist Summary’ tab which provides a snapshot of the current state of the cancer waiting list across both Trusts, helping users quickly grasp the overall state of the patient pathways in their Trust as well as patients that are being tracked by both Trusts.‘Current Cancer PTL’ tab which allows users to identify specific pathway issues and mitigations in real-time. The tool presents patients’ risk of breaching, helping to prioritise the patients that require immediate actions.
  • ‘Demand & Capacity’ tab utilises patient-level data to forecast demand and capacity 12 weeks into the future, facilitating provider planning and system-wide discussions around mutual aid.

Figure 1. Example of CanCollaborate workflow for system and Trust users, for lung cancer

Based on early testing, if our recommendations are actioned by Trusts, CanCollaborate has the potential to increase the percentage of patients meeting targets by an average of 15% across tumour sites (e.g. from 60% to 75%).

Lessons learned and next steps

• Information governance: Secure and compliant management of data across North Tees and South Tees was crucial in the development of CanCollaborate. Before undertaking this work, we supported a joint information governance process between North Tees, South Tees and the NCA.
• Linking data: At the outset of this work, data reconciliation was one of the main challenges faced by the Trusts due to their data being in separate, unlinked systems. To protect patient-level data we established advanced encryption techniques, ensuring data security and confidentiality throughout the integration process. Finally, we implemented novel linking techniques to integrate the data from the different sources.
• Fostering user engagement: CanCollaborate is a tool aimed at meeting the various needs of different types of users (e.g. cancer managers, system partners, BI teams, clinicians). As CanCollaborate continues to be operationalised, we are continuing to work with the NCA and Trusts to identify the user groups that would most benefit from the tool.