LLMOps and platform engineering: Who should own the AI pipeline?

A few years ago, getting a model into production meant a data scientist, a DevOps engineer, and a narrow set of tools: train it, test it, ship it, watch the dashboards. Large language models broke that picture. The thing being “operated” is now a system that chains prompts, calls vector databases, and produces open-ended text judged on tone and safety, not just accuracy. That’s the gap LLMOps fills, and it lands on top of MLOps and DevOps, all competing for the same pipeline platform engineering is supposed to run. Get the ownership model wrong, and you recreate the shadow-IT problem platform engineering was built to prevent, just with prompts instead of Jenkinsfiles.

What Is LLMOps?

LLMOps, or large language model operations, is the set of practices, tools, and workflows used to develop, deploy, and manage LLMs throughout their production lifecycle: data management, prompt engineering and fine-tuning, deployment and serving, monitoring and evaluation, and security and governance. It isn’t MLOps with a new label. Google Cloud calls it a specialised subset of MLOps, and the gap comes down to scale and evaluation: LLMs cost more to fine-tune and serve, and their output is far harder to score than a clean accuracy number. An LLM has to be more than accurate; it has to be secure and trustworthy, and that’s a much harder thing to measure.

It’s also the ongoing care and feeding of a model once it’s live, not a one-time deployment checklist. Models drift, costs creep, prompts stop working the way they used to, and the integrations tying an LLM to a CRM or an internal knowledge base need constant tending.

The LLMOps lifecycle, end to end

The lifecycle runs from data prep and prompt engineering (treating prompts as versioned artefacts, not throwaway strings), through fine-tuning open foundation models with libraries like Hugging Face Transformers, to model and prompt versioning and lineage tracking, inference serving on GPU-backed endpoints, and monitoring with human feedback for drift and cost. Every stage needs infrastructure, access controls, and a place to run. This is exactly the territory platform engineering already occupies for every other kind of workload.

Who owns the pipeline: DevOps, MLOps, and platform engineering

Software delivery has fractured into three camps: DevOps teams buried under deployment tickets, platform engineering teams building self-service golden paths in response, and MLOps teams who built an entirely parallel stack (MLflow, Kubeflow, Weights & Biases) because DevOps tooling never anticipated data versioning or drift monitoring. Add LLMOps, and you risk a third parallel stack for prompts, vector stores, and RAG pipelines, invisible to whoever is supposed to be governing the rest.

Platform engineering is infrastructure-centric, and MLOps is model-centric. These are different disciplines solving different problems, with platform engineering serving as the backbone the others run on top of. The unhelpful question is “who owns the pipeline?” The useful one is “who owns which layer, and is anyone actually coordinating across them?”

Where LLMOps fits in the platform engineering layer cake

CNCF TAG App Delivery’s Platforms Whitepaper describes a similar three-tier split: Products at the top, Platforms in the middle (the thinnest reasonable integration layer), and Capability Providers at the bottom. Model fine-tuning jobs, vector databases, prompt registries, and inference endpoints are, functionally, just another platform capability, needing the same API, versioning, and clear ownership as anything else.

The tooling already exists across the CNCF ecosystem: Backstage surfaces golden paths at the product layer, Crossplane composes infrastructure at the bottom layer, and orchestration frameworks such as Kratix, KusionStack, and KubeVela sit in the middle, letting teams expose an LLM pipeline through the same self-service interface as everything else. Skip that, and the DevOps-versus-MLOps turf war resurfaces one layer higher.

The real risk isn’t the model, it’s shadow LLMOps

The bigger operational risk isn’t a hallucinating chatbot; it’s the sprawl around it: a team standing up its own RAG pipeline against an unreviewed vector store, invisible to whoever tracks what’s actually running. This “shadow AI” pattern is the same one that made the DevOps-versus-platform split so painful: a capability gets built outside the platform because the platform wasn’t ready, and it never gets folded back in. The fix isn’t slowing teams down. Instead, it’s making the platform say yes fast, with governance built in.

Governing LLMOps through the platform, not around it

Concretely, platform teams should implement:

Governed APIs, not ad hoc scripts. Fine-tuning jobs, prompt deployments, and inference endpoints should be requestable through the same self-service interface developers already use for everything else.

Policy enforcement at request time. Cost limits, data residency rules, and model access controls checked before a job kicks off, not discovered after the cloud bill arrives.

Human approval where the blast radius warrants it. Not every prompt change needs a sign-off, but a model touching customer PII or making autonomous decisions probably does.

An audit trail that answers “what changed, and why.” Regulators and security teams want to know what changed about the model, the prompt, or the data, and who or what approved it.

From a governance standpoint, an LLM pipeline is just another automated consumer of platform capabilities that needs the same guardrails a human developer or an autonomous agent would get. The teams getting this right aren’t picking a side in the DevOps-versus-platform-versus-MLOps argument. Rather, they’re treating the entire pipeline, LLM workloads included, like a product: versioned, observable, cost-aware, and built with feedback loops rather than handed down as a fixed set of rules.

Final thoughts

LLMOps isn’t a rebranding of MLOps; it’s MLOps under real strain: bigger models, higher costs, fuzzier evaluation, and a production surface that didn’t exist a few product cycles ago. But that strain doesn’t mean the ownership question needs reinventing. Build the golden path once, expose it as a governed capability, and let every consumer, be they developer, data scientist, or AI agent, use it through the same API or UI. LLMOps doesn’t need its own kingdom. It needs a well-run platform willing to let it in.

This is a live conversation inside CNCF: TAG App Delivery’s Platforms Working Group is where much of the cross-project thinking on golden paths and platform governance is already happening, and it’s open to anyone who wants to help shape it.

If you want to learn more, check out my deep-dive article, “What Is LLMOps and How Does It Relate to Platform Engineering?” on the Syntasso website.