Why NHS Trusts should invest in the Azure Synapse Analytics Platform

A bit of background. I have been providing data management, data warehousing and Business intelligence solutions to the NHS for over 25 years. In that time, I have always steered towards a “Microsoft Stack” solution, not least as in most cases NHS Trusts are keen to make use of the Microsoft data products as part of delivering their information strategy. Widely used are PowerBi from the MS Power Platform, SQL Server RDBMS and of course Excel.

With this background and experience it is my view that the forward thinking NHS organisations can now benefit from adopting an Azure Synapse Analytics solution which in the words of Microsoft is ‘A unified analytics platform that allows you to perform data integration, data exploration, data warehousing, big data analytics, and machine learning tasks from a single, unified environment’.

The NHS Trusts I have worked with recently have, as organisations, readily adopted a full Synapse solution architected to suit their local data assets, be they formal or informal. Solutions that provide powerful data integration from multiple sources, data lake and data warehouse storage and transformation with powerful analytical capabilities in a PowerBi front end. Working with the Trusts has also shown me that the technology solution can be designed and built efficiently with skills transferred as part of the project and more importantly, can be supported by the Organisation post roll out and go-live.

So what is Azure Synapse?

In Microsoft’s words, Azure Synapse is an enterprise analytics service that accelerates time to insight across data warehouses and big data systems. In essence, Azure Synapse brings together the best of SQL technologies used in enterprise data warehousing, Spark technologies used for big data, Data Explorer for log and time series analytics, Pipelines for data integration and ETL/ELT, and deep integration with other Azure services such as Power BI and increasingly, AzureMachineLearning. So for the NHS with their Microsoft Enterprise Agreements – a one stop shop to design and build an innovative modern data management and reporting solution.

Why the Azure Synapse Technology suits the NHS and can be easily adopted?

5 Key Areas:

1. Industry-leading SQL

Synapse SQL is a distributed query system for T-SQL that enables data warehousing and data virtualization scenarios and extends T-SQL to address streaming and machine learning scenarios.

• Synapse SQL offers both serverless and dedicated resource models. For predictable performance and cost, create dedicated SQL pools to reserve processing power for data stored in SQL tables. For unplanned or bursty workloads, use the always-available, serverless SQL endpoint.
• Use built-in streaming capabilities to land data from cloud data sources into SQL tables
• Integrate AI with SQL by using machine learning models to score data

2. Industry-standard Apache Spark

Apache Spark for Azure Synapse deeply and seamlessly integrates Apache Spark--the most popular open-source big data engine used for data preparation, data engineering, ETL, and machine learning.

• ML models with SparkML algorithms and Azure Machine Learning integration for Apache Spark 3.1 with built-in support for Linux Foundation Delta Lake.
• Simplified resource model that frees you from having to worry about managing clusters.
• Fast Spark start-up and aggressive autoscaling.
• Built-in support for .NET for Spark allowing you to reuse your C# expertise and existing .NET code within a Spark application.

3. Working with your Data Lake

Azure Synapse removes the traditional technology barriers between using SQL and Spark together. You can seamlessly mix and match based on your needs and expertise.

• Tables defined on files in the data lake are seamlessly consumed by either Spark or SQL.
• SQL and Spark can directly explore and analyse Parquet, CSV, TSV, and JSON files stored in the data lake.
• Fast, scalable data loading between SQL and Spark databases

4. Built-in data integration

Azure Synapse contains the same Data Integration engine and experiences as Azure Data Factory, allowing you to create rich at-scale ETL pipelines without leaving Azure Synapse Analytics.

• Ingest data from 90+ data sources
• Code-Free ETL with Data flow activities
• Orchestrate notebooks, Spark jobs, stored procedures, SQL scripts, and more

5. Data Explorer (Preview)

Azure Synapse Data Explorer provides customers with an interactive query experience to unlock insights from system-generated logs. To complement existing SQL and Apache Spark analytics runtime engines, Data Explorer analytics runtime is optimized for efficient log analytics using powerful indexing technology to automatically index free-text and semi-structured data commonly found in the system-generated logs.

Use Data Explorer as a data platform for building near real-time log analytics and IoT analytics solutions to:

• Consolidate and correlate your logs and events data across on-premises, cloud, and third-party data sources.
• Accelerate your AI Ops journey (pattern recognition, anomaly detection, forecasting, and more)
• Replace infrastructure-based log search solutions to save cost and increase productivity.
• Build IoT Analytics solution for your IoT data.
• Build Analytical SaaS solutions to offer services to your internal and external stakeholders.

The real benefits of adopting Azure Synapse?

Azure Synapse offers the “toolbox” to create a solution that delivers real information and planning benefits to any NHS Trust. If we look at a recent project with a large Mental Health Trust, Azure Synapse was fully utilised to integrate data from their EPR, HR and Payroll, Rostering, Incidents and Workforce management systems. A data lake was hosted within a DMZ on the Trust side of the Firewall with Azure Synapse Pipelines providing the ETL service to the Azure Cloud based Synapse data warehouse. PowerBi then provides the analytics and reporting outputs to end users of all the key Clinical and Management departments.

The benefits that the solution accrues are many; Users getting the style of reporting and analytics that they really need and ask for, for the first time staff rotas and skill mix is correlated to clinical data and particularly on ward incidents, trend analysis and SPC charting is easily provided and the integrated and rich datasets enable Azure MachineLearning to forecast potential outcome scenarios.

As can be seen, the potential benefits are far reaching for both Clinicians and Managers. Technology innovates at speed (see the latest on MS Fabric, CoPilot and PurView) but there is no doubt in my mind that the Azure Synapse platform can really deliver for the NHS in the area of “managing data” and revolutionising decision support.

Further Resources

Blog - Azure Synapse Analytics: Benefits and Application for the NHS Read here

Case Study - Black Country NHS Trust adopt a Synapse analytics based BI Solution. Read more

Podcast - “Seize the Data! The business of becoming a data-driven organisation”, by Graham James. Listen Here

Insight articles

• So you want to be data-driven but you don’t know what that means. Read how.
• How to plan your data analytics strategy. Read here.
• Key elements to drive your data analytics solution & design. Read more

About Village Software

Village Software specialise in app development and data analytics and reporting. We enable NHS Trusts, Universities, and large private sector organisations to transform through the power of data.

https://villagesoftware.co.uk/ | LinkedIn | Contact