Evergreen analysis · 14 September 2026

Red Hat AI 3.5: How to Protect Critical AI on Shared GPUs

By the ELYMENT AI editorial team · Free to read

Red Hat made AI 3.5 generally available on 9 September 2026 with controls for priority-aware model serving, multi-tenant fairness, admission and protection against workload starvation. The release makes a practical point for any organisation consolidating inference on shared GPUs: efficiency depends on deciding which requests may wait when demand exceeds capacity. Business leaders should define workload classes, measurable service objectives, tenant limits, queue behaviour and fallback paths before relying on a scheduler to enforce them.

A cyan critical workload lane moves through a shared inference gateway while lower-priority blue lanes queue beneath the headline Red Hat AI 3.5 Puts Critical Work First.
Original ELYMENT.AI editorial illustration.

What Red Hat AI 3.5 changed

Red Hat says AI 3.5 adds generally available priority-aware serving and native multi-tenancy across Red Hat AI Inference Server and Red Hat OpenShift AI. Its controls cover admission, fairness and starvation protection, while controlled deployments add canary validation and rollback. Hosted control planes are available where teams need stronger isolation between environments.

The release also adds generally available inference observability and a GPU dashboard. Token showback for Model as a Service is a Technology Preview, as is agent tracing. That status matters: Red Hat's release notes say Technology Preview features are not supported by its production service-level agreements and are not recommended for production workloads. Buyers should not treat every feature in a launch announcement as equally mature.

Shared GPUs turn capacity into a business queue

A shared pool can improve utilisation, but it also joins real-time customer requests, internal copilots, evaluations and batch jobs to the same constraint. Request counts alone do not reveal pressure: long prompts, large outputs and tool-heavy workflows can consume materially different amounts of compute. Under saturation, an undefined queue becomes an accidental business policy.

Priority therefore needs to express impact, not organisational rank. A payment exception or safety review may deserve protection because delay creates harm; a senior executive's exploratory prompt may not. Fair-share controls should also stop one tenant or workload from consuming the pool simply because it submits more traffic.

Define the workload contract before the scheduler

Create a short operating contract for each workload class before configuring priority bands. Record:

  • Business class and owner: what the workload does, who is accountable and what failure interrupts.
  • Service objective: target success rate, time to first token, end-to-end latency and maximum acceptable queue time.
  • Saturation trigger: the measured condition that activates admission control, with explicit headroom for critical traffic.
  • Fairness rule: tenant caps, weights or reserved capacity that prevent noisy-neighbour behaviour.
  • Expiry and fallback: when a queued request is rejected, degraded, rerouted or completed asynchronously.
  • Cost evidence: tokens, GPU time or another allocation measure that supports showback and capacity decisions.

Test the ugly minute

Red Hat's technical guidance separates admission, which decides when work may enter, from scheduling, which decides where it runs. It recommends testing at measured saturation with at least two tenant streams, then examining latency, success and rejection metrics alongside queues and key-value cache behaviour.

Turn that into a business acceptance test. Run critical traffic, a noisy tenant and a batch surge together. Confirm the protected class stays within its objective, lower classes receive the intended share, expired work fails clearly and the queue drains after demand falls. Repeat the test after model, context-window, quantisation or hardware changes because each can alter the capacity envelope.

What operators should do next

Start with the most consequential shared inference service and map its current queue policy. If nobody can say what happens at saturation, measure the limit in a controlled test before adding more tenants. Configure priority, fairness and expiry against the workload contract, then use canary deployment and rollback evidence for every material change.

ELYMENT AI's workload-routing guide helps separate model choice from runtime priority; our agent-registry analysis covers ownership across shared services; and our outcome-based AI contracts guide shows how to make service evidence enforceable. ELYMENT AI can help turn those controls into an operating model that finance, risk and engineering can all test.

Sources

Continue learning

Frequently asked questions

What is priority-aware serving in Red Hat AI 3.5?

It is a generally available set of controls for admitting and ordering inference work by priority while preserving tenant fairness and protecting workloads from starvation.

Why do shared GPU pools need admission control?

When demand exceeds measured capacity, admission control determines which work may enter the service. Without an explicit policy, critical and delay-tolerant requests compete on accidental queue behaviour.

Which Red Hat AI 3.5 features are Technology Preview?

Red Hat identifies Model as a Service token showback and agent tracing as Technology Preview capabilities. Its release notes say such features are not covered by production service-level agreements and are not recommended for production workloads.

Explore ELYMENT AI