How to build, maintain, and secure infrastructure for AI factories

Moving AI into production creates a long-term maintenance obligation for the software behind it. GPU infrastructure can remain in service across several generations of models and frameworks, while the operating system (OS), containers, and application dependencies each follow their own support lifecycle. Enterprise buyers need to establish who will maintain those components and how security updates will reach deployed workloads.

In this blog, we’ll cover the AI factory, as a useful way to frame this investment. It includes the infrastructure and processes used to develop, train, and serve models, so its security requirements extend across the production lifecycle. We’ll give an overview of how Canonical’s portfolio supports this computing foundation; for more details, read our whitepaper, Secure your AI Infrastructure.

Balancing stability and security in your infrastructure

While AI teams do need access to new frameworks and model architectures, changes to the underlying platform introduce their own validation work. Upgrading Kubernetes or replacing a container dependency can affect workloads that have already been tested for production.

Canonical addresses this through long-term security maintenance across Ubuntu and its broader infrastructure portfolio. Ubuntu Pro is Canonical’s enterprise-grade subscription for security maintenance,  and compliance automation. It provides up to 15 years of security patching for the OS and packages in the Ubuntu archive. It also covers Canonical’s infrastructure and application software across AI and data, priced per node with no additional licence fees.

Beyond security maintenance, Ubuntu Pro offers access to Livepatch, which enables you to apply critical kernel security fixes without a reboot, so long-running model training jobs don’t have to be interrupted. Another relevant Ubuntu Pro component is the Ubuntu Security Guide (USG). It applies and audits CIS and DISA-STIG hardening profiles, which helps you show that every environment meets the same standard. FIPS 140-3 certified cryptographic modules are also available for government, finance, and healthcare requirements.

Extend maintenance into the AI container

An inference container can carry its own copies of Python libraries and native dependencies. Updating the host OS leaves those copies unchanged, which means an enterprise can have fully patched hosts while still running vulnerable application images.

Canonical’s Container Build Service addresses this challenge through custom container builds and security maintenance for their dependencies, including AI and machine learning software. This extends maintenance beyond packages installed on the host to the software shipped with the workload.

Rockcraft and Chisel address how those images are constructed. Rockcraft provides declarative Ubuntu-based Open Container Initiative (OCI) compliant builds, while Chisel selects the files an application needs, allowing unnecessary components to be omitted. Component metadata remains available for identifying image contents.

For an enterprise, these capabilities reduce the work involved in building and maintaining its own base images and dependency sets. The organization still needs to validate and deploy updated workloads, but it can obtain maintained container software from Canonical rather than take on all the underlying maintenance itself.

Operate the model pipeline on supported infrastructure

The maintenance requirement also includes the services used to build and deploy models. Charmed Kubeflow provides machine learning pipelines on Kubernetes, with metadata captured during execution and integrations, such as Charmed MLflow for experiment tracking and model management.

“Charmed” refers to how Canonical packages and automates the operation of these applications. A charm is a software operator containing application-specific code for deployment, configuration, supported upgrades, and integrations. Juju, Canonical’s open source orchestration engine, runs these operators and coordinates relationships between applications, reducing the integration scripts and maintenance procedures that platform teams would otherwise need to develop themselves.

Canonical Kubernetes and Charmed Kubeflow bring enterprise maintenance and support to these layers. Together with the charms, they reduce the work needed to operate the upstream infrastructure and Machine Learning Operations (MLOps) components, so they can check compatibility and performance before rolling changes out to production.

Protect sensitive workloads from the hosting infrastructure

Some deployments also require protection from privileged host software. This is a distinct requirement from maintaining packages or separating Kubernetes workloads.

Ubuntu supports confidential virtual machines (VMs) using Intel TDX and AMD SEV-SNP, with hardware-enforced isolation designed to protect guest memory from the host hypervisor. Supported configurations with NVIDIA H100 GPUs extend confidential computing to accelerated AI workloads. Remote attestation provides evidence about the execution environment that can be assessed before sensitive data or keys are released. You can learn more about confidential computing on our webpage.

Canonical’s work in this area connects hardware capabilities with the Ubuntu software needed to deploy and operate them. The guest still requires security maintenance, because protecting its memory from the host does not fix vulnerabilities in the OS or applications running inside it. The Ubuntu and container security maintenance described above therefore remains necessary within confidential VMs.

Read the whitepaper

The maintenance obligation that begins with an AI deployment continues as models are replaced and new workloads enter production. Thinking of the platform as a factory helps explain why: successive models depend on the same build systems, training pipelines and serving infrastructure, whose security needs to be maintained throughout those changes.

Canonical’s portfolio addresses that ongoing work through security maintained packages and container dependencies, operational automation in the Charmed stack, and confidential computing for sensitive workloads. Together, these offerings reduce the software maintenance and integration work enterprises need to take on themselves.

Our whitepaper, Secure your AI Infrastructure, examines how these components fit together and the security responsibilities each addresses. Read it to explore how Ubuntu and Canonical can support the long-term operation of your AI factory.