CertGrid CertGrid
Hands-on Lab·Kubernetes and Cloud Native Associate

Installing Argo CD

The documented one-liner fails on this chart with "annotations: Too long". The cause is client-side apply storing your whole manifest in an annotation, and the fix is one flag. A CRD in the same install measures 197,595 bytes of it.

Delivery and GitOps Guide 42 of 46 Intermediate

Written against the versions above. The stable manifests move. The behaviour shown here - the annotation limit, self-heal, prune scope - is not version specific.

Argo CD runs in the cluster it deploys to. Nothing here needs an ingress or a LoadBalancer - the server Service stays ClusterIP.
Server NameIP AddressOSRolesCPURAMHDD
CKA1001192.168.0.175Ubuntu 26.04 LTSControl Plane Node2 Core4 GB50 GB
CKA1001-NODE01192.168.0.176Ubuntu 26.04 LTSWorker Node2 Core4 GB50 GB
CKA1001-NODE02192.168.0.177Ubuntu 26.04 LTSWorker Node2 Core4 GB50 GB
CKA1001-NODE03192.168.0.178Ubuntu 26.04 LTSWorker Node2 Core4 GB50 GB

Before you start

  1. Apply the manifests, and read the last line

    Everything scrolls past looking healthy - service accounts, roles, config maps, seven Deployments, network policies - and then:

    The CustomResourceDefinition "applicationsets.argoproj.io" is invalid:
    metadata.annotations: Too long: may not be more than 262144 bytes

    This is not Argo CD's fault and not a broken cluster. kubectl apply without --server-side stores a copy of the entire object you submitted in the kubectl.kubernetes.io/last-applied-configuration annotation, so it can compute a diff next time. Annotations are capped at 256 KB. A CRD with a large OpenAPI schema exceeds it on its own.

    The dangerous part is that the command partially succeeded. Most objects exist; one CRD does not.

    bash Example session
    kubectl --context cka1001 create namespace argocdnamespace/argocd createdkubectl --context cka1001 apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yamlcustomresourcedefinition.apiextensions.k8s.io/applications.argoproj.io createdcustomresourcedefinition.apiextensions.k8s.io/appprojects.argoproj.io createdserviceaccount/argocd-application-controller createdserviceaccount/argocd-applicationset-controller createdserviceaccount/argocd-dex-server createdserviceaccount/argocd-notifications-controller created

    Expected resultA long list of created objects, then the Too long error, then a non-zero exit.

    Success conditionYou have the error text rather than a vague failure.

  2. Confirm what is actually missing

    Never assume a partial apply left the thing you need. Ask:

    Error from server (NotFound): ... "applicationsets.argoproj.io" not found

    So Application exists but ApplicationSet does not. A cluster in that state will run single Applications fine and fail confusingly the first time someone writes an ApplicationSet - weeks later, with no memory of this install.

    bash Example session
    kubectl --context cka1001 get crd applicationsets.argoproj.io 2>&1 | head -2Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "applicationsets.argoproj.io" not found

    Expected resultA NotFound for the CRD that failed.

    Success conditionYou can name the object that did not get created.

  3. Server-side apply, and why it works

    --server-side changes who computes the diff. The API server tracks field ownership in metadata.managedFields instead of stashing your manifest in an annotation, so there is nothing to exceed the limit.

    Expect conflict warnings on the second run: objects created client-side are now owned by a different manager, and server-side apply says so rather than silently taking over. On a fresh install those are informational. In production, read them - a conflict is the API server telling you two things are fighting over a field.

    After it completes, all three CRDs exist.

    bash Example session
    kubectl --context cka1001 apply --server-side -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml 2>&1 | tail -4* You may co-own fields by updating your manifest to match the existing  value; in this case, you'll become the manager if the other manager(s)  stop managing the field (remove it from their configuration).See https://kubernetes.io/docs/reference/using-api/server-side-apply/#conflictskubectl --context cka1001 get crd -o name | grep argoproj.iocustomresourcedefinition.apiextensions.k8s.io/applications.argoproj.iocustomresourcedefinition.apiextensions.k8s.io/applicationsets.argoproj.iocustomresourcedefinition.apiextensions.k8s.io/appprojects.argoproj.io

    Expected resultConflict guidance from server-side apply, then three argoproj.io CRDs.

    Success conditionapplicationsets.argoproj.io is present.

  4. Measure the annotation that caused it

    Worth doing once so the limit stops being abstract. The Application CRD - the one that *did* apply client-side - carries 197,595 bytes of last-applied-configuration. That is 75% of the 262,144-byte ceiling, for a CRD that fits. Its larger sibling did not.

    The managedFields managers on the server-side-applied CRD name both writers: kubectl and kube-apiserver. That list is how server-side apply keeps track without an annotation at all.

    bash Example session
    kubectl --context cka1001 get crd applications.argoproj.io -o jsonpath='{.metadata.annotations.kubectl\.kubernetes\.io/last-applied-configuration}' | wc -c197595kubectl --context cka1001 get crd applicationsets.argoproj.io -o jsonpath='{.metadata.managedFields[*].manager}'kubectl kube-apiserver

    Expected resultA byte count in the high 190,000s, and two field managers.

    Success conditionYou can state how close the working CRD came to the limit.

  5. What got installed

    Seven Pods, and each one has a distinct job worth knowing:

    • application-controller - the reconcile loop. Compares cluster to Git and acts.
    • repo-server - clones repos and renders manifests (Helm, Kustomize, plain YAML).
    • server - the API and web UI.
    • redis - a cache for rendered manifests.
    • dex - optional SSO.
    • applicationset-controller, notifications-controller - generate Applications, and send alerts.

    Note the Service is ClusterIP. Argo CD does not expose itself, which is a sane default: the UI is an admin surface with cluster-wide write access.

    bash Example session
    kubectl --context cka1001 -n argocd get podsNAME                                                READY   STATUS    RESTARTS   AGEargocd-application-controller-0                     1/1     Running   0          81sargocd-applicationset-controller-579c4c54b8-z29v2   1/1     Running   0          81sargocd-dex-server-744c5d4467-sjvhd                  1/1     Running   0          81sargocd-notifications-controller-dd4ff84c-f8564      1/1     Running   0          81sargocd-redis-84497fb7c5-699gg                       1/1     Running   0          81sargocd-repo-server-5f46d9f598-mtgrj                 1/1     Running   0          81sargocd-server-7f4549bb69-d7grb                      1/1     Running   0          81skubectl --context cka1001 -n argocd get svc argocd-server -o custom-columns=NAME:.metadata.name,TYPE:.spec.type,PORTS:.spec.ports[*].portNAME            TYPE        PORTSargocd-server   ClusterIP   80,443

    Expected resultSeven Running Pods and a ClusterIP Service on 80/443.

    Success conditionAll Argo CD Pods are Running.

Troubleshooting

Official sources