Healthcare organizations are adopting hybrid infrastructure to support artificial intelligence workloads, combining cloud flexibility with on-premises control to address clinical and operational requirements.
Research firm IDC projects that by 2028, 75% of enterprise AI workloads will operate in hybrid environments. This trend stems from the understanding that no single platform can fully meet the diverse demands of healthcare AI, including data privacy and cost management.
Hybrid setups enable IT teams to deploy workloads where they perform most effectively. Public cloud environments work well for experimental or high-capacity AI tasks, while private clouds or on-premises GPU clusters handle sensitive data or large-scale inferencing. The approach also helps control expenses amid ongoing GPU shortages and ensures compliance with regulations like HIPAA and GDPR.
Sana Gutierrez, senior manager of the data and artificial intelligence practice at CDW, said the data center’s future lies in hybrid models. “Organizations must decide how to place workloads—whether in the cloud, on-premises, or in a neocloud—to maximize benefits,” she explained. The decision depends on factors like performance, latency, and data sovereignty.
Related: New Therapies Pose Challenges for Insurers
Many organizations begin in the cloud, where flexibility and access to high-performance computing make it an appealing starting point. As workloads stabilize and costs become more predictable, some move them back on-premises. Eryn Brodsky, server and storage practice lead at CDW, said cost often drives this shift. “They bring those workloads back because maintaining them in the cloud becomes expensive compared to investing in their own infrastructure,” she noted.
Most healthcare organizations will continue using the cloud, but selectively. They may train models in the cloud where massive compute power is available, then shift inferencing to on-premises systems where low latency and data privacy are essential. Mariano Carro, principal field solutions architect for Microsoft hybrid infrastructure at CDW, said this approach is becoming common. “Most of the time, we are going to need some resources in the cloud to do the training and for the high level of compute that we need. But once we get that training complete, we may move a workload on-premises to improve performance or protect data privacy,” he explained.
Decisions about AI workload placement change as organizations gain experience with the technology. Latency, accessibility, and governance requirements all influence where workloads run, and today’s solution may not fit tomorrow’s needs. Brodsky emphasized that quick data access is important, but security and compliance remain critical. “Workload placement is going to take into consideration things like latency as well as accessibility. Organizations need to think about what AI outcomes they want to leverage the data. You need to have quick access to it, but you also have to ensure that you have proper access to it,” she said.
That balancing act frequently leads to movement between environments over time. Many organizations begin their AI journeys in the cloud, drawn by its flexibility and access to large-scale compute. But as workloads stabilize and costs become clearer, some shift those workloads back into their own environments. “It’s not uncommon for us to see customers starting in the cloud, as the majority of our customers are, then bringing those applications back on-premises. We call it repatriation,” Brodsky says. “They repatriate those workloads on-prem because they have to consider the cost of maintaining those applications and those workloads in the cloud versus the cost of being able to invest in the architecture to support them.”
Related: Carbon capture may curb data center emissions
The interaction between CPUs and GPUs has become central to modern data center design. Some applications perform better on GPUs, while others run more efficiently on CPUs. Understanding this difference helps organizations tailor their infrastructure to specific workloads. Gutierrez said advancements in accelerated computing are changing how data centers function. “Certain processes simply work better on GPUs,” she noted. “Matching the right workloads to the right hardware is essential for optimization.”
For healthcare, the stakes remain high. AI can enhance diagnostics, streamline operations, and cut costs—but only if the infrastructure supports it. Poorly designed systems create bottlenecks, increase expenses, and limit potential benefits. The move to hybrid models isn’t just about technology; it aligns infrastructure with clinical and business objectives to ensure AI delivers meaningful value.
Gutierrez concluded, “When organizations implement AI and accelerated computing effectively, the impact on their operations is significant.”
Access to care remains uneven across regions, particularly in rural areas where infrastructure gaps persist. Healthcare access is often limited by these disparities, making hybrid solutions even more relevant for providers serving diverse populations.