Making Argo CD and Knative Services Play Nicely

Making Argo CD and Knative Services Play Nicely

Knative and Argo CD are a strong pairing. Knative gives application teams fast serverless-style rollouts on Kubernetes, and Argo CD keeps the desired state auditable in Git. The catch is that a Knative Service is not a plain static Kubernetes object. It is the entry point to a controller-managed system that creates revisions, routes traffic, mutates status, and writes annotations as the service moves through a rollout.

If Argo CD is configured as if a Knative Service were just another Deployment, the two controllers can end up arguing. Argo sees controller-owned fields as drift, keeps trying to self-heal them, and may report healthy services as OutOfSync or Progressing.

We recently tightened this up after seeing slow rollout status, stale OutOfSync applications, and a generated revision blocked by an invalid probe port.

The Failure Mode

The first symptom was that a rollout looked stuck. The service was ready, but Knative was doing a gradual traffic migration while Argo kept reconciling around it. The old revision stayed visible during rollout, and the service sat in RolloutInProgress longer than expected.

Generated agent services also stayed OutOfSync after their pods were healthy. The live Service had fields that did not exist in Git because Knative owns them: status, spec.traffic, serving.knative.dev/creator, and serving.knative.dev/lastModifier.

One generated revision failed before a pod existed. Kubernetes rejected the Deployment that Knative tried to create because the generated probe had an HTTP path but no explicit port. In the Knative revision this became startupProbe.httpGet.port: 0, which is invalid.

What Argo CD Should Ignore

A Knative Service has desired fields that Git should own: image, env vars, resources, concurrency, node selectors, and probes. It also has runtime fields that Knative should own: readiness status and traffic routing between revisions.

Use ignoreDifferences for the runtime-owned fields:

ignoreDifferences:
  - group: serving.knative.dev
    kind: Service
    jsonPointers:
      - /status
      - /spec/traffic
      - /metadata/annotations/serving.knative.dev~1creator
      - /metadata/annotations/serving.knative.dev~1lastModifier

Then make sync respect those ignores:

syncPolicy:
  syncOptions:
    - ServerSideApply=true
    - RespectIgnoreDifferences=true

ServerSideApply=true helps when multiple controllers participate in ownership. RespectIgnoreDifferences=true is what stops Argo from still applying fields it was told to ignore.

Add Knative Health

Argo CD also needs a health check for Knative Service. The practical version reads the Ready condition from .status.conditions and maps it to Healthy, Degraded, or Progressing.

This follows the same shape described in OneUptime's post on Argo CD and Knative Services: https://oneuptime.com/blog/post/2026-02-26-argocd-knative-services/view

Keep Rollout Policy Central

Avoid copying rollout duration annotations into every service:

metadata:
  annotations:
    serving.knative.dev/rollout-duration: "180s"

That makes normal deploys feel stuck. Prefer a cluster default in Knative serving config:

spec:
  config:
    network:
      rollout-duration: "30"

Only add a service-level override when a workload really needs slower migration.

Generate Valid Probes

Generated Knative services should include explicit probe ports:

startupProbe:
  httpGet: {path: /healthz, port: 8000}
  periodSeconds: 5
  timeoutSeconds: 3
  failureThreshold: 60
readinessProbe:
  httpGet: {path: /healthz, port: 8000}
  periodSeconds: 5
  timeoutSeconds: 3
  failureThreshold: 6

Do not rely on implicit port defaulting through the Knative revision path. A missing port can become port: 0, and Kubernetes will reject the Deployment.

Checklist

1. Add Argo CD health customization for serving.knative.dev/Service.
2. Ignore Knative-owned fields: /status, /spec/traffic, and serving annotations.
3. Add RespectIgnoreDifferences=true.
4. Use ServerSideApply=true.
5. Keep rollout duration in Knative cluster config unless a service needs an override.
6. Generate explicit probe ports.
7. Normalize resource values in Git, for example cpu: "1" instead of fighting Kubernetes normalization from 1000m to 1.

The Result

The goal is not to make Argo CD less strict. The goal is to make Argo strict about the fields Git should own and quiet about the fields Knative must own at runtime.

Once those boundaries are clear, Argo reports real drift, Knative manages rollouts cleanly, and operators stop chasing false OutOfSync states while a service is already healthy.