Cluster-Aware AI Agents for Kubernetes
Most Kubernetes AI tools still behave like remote assistants. They receive copied logs, infer from partial context, and often ask operators to trust a hosted service with production data. A recent CNCF walkthrough, Building a Cluster-Aware AI Agent with Kubernetes, Argo CD, and GitOps, shows a different operating model: run the agent in the cluster, scope its access with RBAC, and deploy it like any other workload.
The example project, local-k8s-ai-agent, is useful because it is small enough to inspect but realistic enough to teach the pattern.
What Is Local K8s AI Agent?
Local K8s AI Agent is a self-hosted Kubernetes assistant built with FastAPI, Ollama, Mistral 7B, Argo CD, and Argo CD Image Updater. It exposes a chat UI and REST endpoints for two modes.
The /ask mode sends a general Kubernetes question to the local LLM. The /diagnose mode reads live cluster state first, then asks the model to reason over pods, events, and logs from the selected namespace. That difference matters. Generic advice says why CrashLoopBackOff happens. Cluster-aware advice can point to the exact pod, event, restart count, or image pull failure that is happening now.
Key Features
- Local model serving: Ollama runs Mistral 7B inside the cluster, so prompts and cluster evidence do not need to leave the environment.
- Read-only RBAC: A dedicated
ServiceAccountcangetandlistresources such as pods, logs, events, services, deployments, replicasets, and namespaces. - GitOps delivery: GitHub Actions builds multi-architecture images, Argo CD Image Updater writes new image tags back to Git, and Argo CD reconciles the cluster.
- Operator-friendly API: The project includes a browser UI,
/ask,/diagnose, and OpenAPI docs for automation experiments.
Installation
The README walks through a local minikube setup. The core path is familiar to platform teams:
git clone https://github.com/MaryamTavakkoli/local-k8s-ai-agent.git
cd local-k8s-ai-agent
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
From there, build and push the image, register the repo with Argo CD, apply the application manifests, and port-forward the API for testing.
Operational Tips
The most important lesson is the permission model. Start agents with read-only access. Let Kubernetes enforce the boundary instead of relying on prompt wording. If the agent gives a bad answer, the blast radius is limited to advice.
Keep prompts, RBAC, image tags, and manifests in Git. That turns agent behavior into reviewable infrastructure rather than a hidden chat configuration. It also gives SRE teams the audit trail they need when incident tooling starts reading production state.
Conclusion
Cluster-aware agents are more interesting than generic chatbots because they ground responses in live infrastructure. The local-k8s-ai-agent project is not a full production SRE platform, but it demonstrates the control plane pattern clearly: observe with scoped credentials, reason with real context, and deliver changes through GitOps.
Looking to automate infrastructure operations? Akmatori helps SRE teams reduce toil with AI agents built for real production workflows. For reliable global infrastructure, check out Gcore.