CertTools Logo
CertTools

Kubernetes Certificate Commands

Complete guide to managing certificates, TLS secrets, and CSRs in Kubernetes clusters

🔐 TLS Secrets Management

Create TLS secret from certificate files

kubectl create secret tls my-tls-secret \
  --cert=path/to/cert.pem \
  --key=path/to/key.pem

Creates a TLS secret for use with ingress or services. The secret contains tls.crt and tls.key.

Create TLS secret in specific namespace

kubectl create secret tls my-tls-secret \
  --cert=cert.pem \
  --key=key.pem \
  --namespace=production

Create TLS secret from Let's Encrypt certificates

kubectl create secret tls letsencrypt-tls \
  --cert=/etc/letsencrypt/live/example.com/fullchain.pem \
  --key=/etc/letsencrypt/live/example.com/privkey.pem

Use fullchain.pem (not cert.pem) to include intermediate certificates.

View TLS secret contents

kubectl get secret my-tls-secret -o yaml

Extract certificate from secret

kubectl get secret my-tls-secret -o jsonpath='{.data.tls\.crt}' | base64 -d > cert.pem

Extract private key from secret

kubectl get secret my-tls-secret -o jsonpath='{.data.tls\.key}' | base64 -d > key.pem

⚠️ Warning: Be careful with private keys. Only extract when necessary.

Check certificate expiry date in secret

kubectl get secret my-tls-secret -o jsonpath='{.data.tls\.crt}' | \
  base64 -d | \
  openssl x509 -noout -enddate

Update existing TLS secret

# Delete old secret
kubectl delete secret my-tls-secret

# Create new secret with updated certificate
kubectl create secret tls my-tls-secret \
  --cert=new-cert.pem \
  --key=new-key.pem

Note: Pods using the secret need to be restarted to pick up the new certificate.

List all TLS secrets

kubectl get secrets --field-selector type=kubernetes.io/tls

🌐 Ingress TLS Configuration

Basic ingress with TLS

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
spec:
  tls:
  - hosts:
    - example.com
    secretName: my-tls-secret
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80

Ingress with multiple TLS hosts

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: multi-host-ingress
spec:
  tls:
  - hosts:
    - example.com
    - www.example.com
    secretName: example-tls
  - hosts:
    - api.example.com
    secretName: api-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-service
            port:
              number: 80
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: api-service
            port:
              number: 80

Create ingress with TLS via kubectl

kubectl apply -f ingress-tls.yaml

Check ingress TLS configuration

kubectl get ingress my-ingress -o yaml

# Or describe for more details
kubectl describe ingress my-ingress

Force SSL redirect (nginx ingress)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - example.com
    secretName: my-tls-secret

Test ingress TLS endpoint

# Get ingress IP
kubectl get ingress my-ingress

# Test with curl
curl -v https://example.com

# Test with openssl
openssl s_client -connect example.com:443 -servername example.com

📝 Certificate Signing Requests (CSR)

Create Certificate Signing Request

# First, generate a private key and CSR locally
openssl req -new -newkey rsa:4096 -nodes \
  -keyout user.key -out user.csr \
  -subj "/[email protected]/O=engineering"

# Create Kubernetes CSR object
cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: user-csr
spec:
  request: $(cat user.csr | base64 | tr -d '\n')
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
EOF

List pending CSRs

kubectl get csr

Approve Certificate Signing Request

kubectl certificate approve user-csr

Deny Certificate Signing Request

kubectl certificate deny user-csr

View CSR details

kubectl describe csr user-csr

# View the actual CSR content
kubectl get csr user-csr -o jsonpath='{.spec.request}' | base64 -d | openssl req -text -noout

Retrieve signed certificate

kubectl get csr user-csr -o jsonpath='{.status.certificate}' | base64 -d > user.crt

Delete CSR

kubectl delete csr user-csr

Complete workflow for user authentication certificate

# 1. Generate key and CSR
openssl genrsa -out user.key 4096
openssl req -new -key user.key -out user.csr -subj "/CN=john/O=developers"

# 2. Create and submit CSR to Kubernetes
cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: john-csr
spec:
  request: $(cat user.csr | base64 | tr -d '\n')
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
EOF

# 3. Approve CSR (as cluster admin)
kubectl certificate approve john-csr

# 4. Retrieve signed certificate
kubectl get csr john-csr -o jsonpath='{.status.certificate}' | base64 -d > user.crt

# 5. Create kubeconfig for user
kubectl config set-credentials john \
  --client-certificate=user.crt \
  --client-key=user.key \
  --embed-certs=true

🔄 Certificate Rotation

Check cluster certificate expiration

# On control plane node
kubeadm certs check-expiration

# Or manually check API server certificate
openssl x509 -in /etc/kubernetes/pki/apiserver.crt -noout -enddate

Renew all cluster certificates (kubeadm)

# Renew all certificates
sudo kubeadm certs renew all

# Restart control plane components
sudo systemctl restart kubelet

⚠️ Warning: This should be done on control plane nodes. Ensure you have backups.

Renew specific certificate

sudo kubeadm certs renew apiserver
sudo kubeadm certs renew apiserver-kubelet-client
sudo kubeadm certs renew front-proxy-client

Update kubeconfig after certificate renewal

sudo kubeadm init phase kubeconfig admin

# Copy new kubeconfig
sudo cp /etc/kubernetes/admin.conf ~/.kube/config
sudo chown $(id -u):$(id -g) ~/.kube/config

Rotate application TLS secret

# 1. Get new certificate (e.g., from Let's Encrypt)
# 2. Update secret
kubectl create secret tls my-tls-secret \
  --cert=new-cert.pem \
  --key=new-key.pem \
  --dry-run=client -o yaml | kubectl apply -f -

# 3. Restart pods to pick up new certificate
kubectl rollout restart deployment my-app

Automate certificate rotation with cert-manager

# Install cert-manager
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml

# Create Let's Encrypt issuer
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx

🤖 cert-manager (Automatic Certificate Management)

Install cert-manager

# Install with kubectl
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml

# Or with Helm
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --version v1.13.0 \
  --set installCRDs=true

Create Let's Encrypt issuer (HTTP-01)

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx

Create Let's Encrypt issuer (DNS-01 for wildcards)

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-dns
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-dns
    solvers:
    - dns01:
        cloudflare:
          email: [email protected]
          apiKeySecretRef:
            name: cloudflare-api-key
            key: api-key

Create Certificate resource

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: example-com-tls
  namespace: default
spec:
  secretName: example-com-tls
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  dnsNames:
  - example.com
  - www.example.com

Ingress with automatic certificate (annotation)

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
  tls:
  - hosts:
    - example.com
    secretName: example-com-tls
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-service
            port:
              number: 80

cert-manager will automatically create and manage the certificate based on the annotation.

Check certificate status

kubectl get certificate
kubectl describe certificate example-com-tls

# Check certificate request
kubectl get certificaterequest
kubectl describe certificaterequest example-com-tls-xxxxx

Manually trigger certificate renewal

# Delete certificate request to trigger renewal
kubectl delete certificaterequest -l certificate.cert-manager.io/certificate-name=example-com-tls

# Or use cmctl (cert-manager CLI)
cmctl renew example-com-tls

Check cert-manager logs

kubectl logs -n cert-manager deployment/cert-manager -f

👤 Kubeconfig Client Certificates

View current kubeconfig certificate

# View current context
kubectl config current-context

# View certificate details
kubectl config view --raw -o jsonpath='{.users[0].user.client-certificate-data}' | \
  base64 -d | \
  openssl x509 -text -noout

Check kubeconfig certificate expiry

kubectl config view --raw -o jsonpath='{.users[0].user.client-certificate-data}' | \
  base64 -d | \
  openssl x509 -noout -enddate

Create user with certificate authentication

# 1. Generate user private key
openssl genrsa -out user.key 4096

# 2. Create certificate signing request
openssl req -new -key user.key -out user.csr \
  -subj "/CN=jane/O=developers"

# 3. Submit CSR to Kubernetes (see CSR section above)
# 4. Get signed certificate from Kubernetes
# 5. Configure kubeconfig

kubectl config set-credentials jane \
  --client-certificate=user.crt \
  --client-key=user.key \
  --embed-certs=true

kubectl config set-context jane-context \
  --cluster=my-cluster \
  --user=jane \
  --namespace=default

kubectl config use-context jane-context

Extract certificate from kubeconfig

# Extract client certificate
kubectl config view --raw -o jsonpath='{.users[?(@.name=="jane")].user.client-certificate-data}' | \
  base64 -d > jane.crt

# Extract client key
kubectl config view --raw -o jsonpath='{.users[?(@.name=="jane")].user.client-key-data}' | \
  base64 -d > jane.key

Renew admin kubeconfig certificate (kubeadm)

# Renew admin.conf certificate
sudo kubeadm certs renew admin.conf

# Update kubeconfig
sudo cp /etc/kubernetes/admin.conf ~/.kube/config
sudo chown $(id -u):$(id -g) ~/.kube/config

# Verify
kubectl get nodes

Configure RBAC for certificate-based user

# Create role binding for user (CN from certificate)
kubectl create rolebinding jane-admin \
  --clusterrole=admin \
  --user=jane \
  --namespace=development

# Or create cluster-wide role binding
kubectl create clusterrolebinding jane-cluster-admin \
  --clusterrole=cluster-admin \
  --user=jane

🕸️ Service Mesh TLS (mTLS)

Overview of service mesh mTLS

Service meshes like Istio and Linkerd provide automatic mutual TLS (mTLS) between services within the mesh, encrypting service-to-service communication and providing identity verification.

  • Automatic certificate issuance and rotation
  • Mutual authentication between services
  • Zero-trust security model
  • Transparent to applications (sidecar proxy handles TLS)

Istio: Check mTLS status

# Check if mTLS is enabled
kubectl get peerauthentication --all-namespaces

# Check destination rules
kubectl get destinationrules --all-namespaces

# Verify mTLS for a specific service
istioctl authn tls-check <pod-name>.<namespace> <service-name>.<namespace>.svc.cluster.local

Istio: Enable strict mTLS cluster-wide

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: istio-system
spec:
  mtls:
    mode: STRICT

Istio: View certificate chain in Envoy proxy

# Get certificate info from Envoy sidecar
kubectl exec <pod-name> -c istio-proxy -- \
  curl -s localhost:15000/certs | \
  openssl x509 -text -noout

# Check certificate validity
istioctl proxy-config secret <pod-name> -o json

Linkerd: Check mTLS status

# Check Linkerd mTLS for all pods
linkerd stat deploy -n <namespace>

# View TLS identity for a pod
linkerd identity <pod-name> -n <namespace>

# Check certificate validity
linkerd identity --kubeconfig ~/.kube/config

Linkerd: View trust anchor certificate

# Get trust anchor certificate
kubectl get configmap linkerd-identity-trust-roots -n linkerd -o yaml

# View certificate expiry
kubectl get configmap linkerd-identity-trust-roots -n linkerd \
  -o jsonpath='{.data.ca-bundle.crt}' | \
  openssl x509 -noout -enddate

Rotate service mesh CA certificate (Istio)

# Generate new CA certificates
mkdir new-ca
cd new-ca
make -f ../tools/certs/Makefile.selfsigned.mk root-ca

# Update Istio CA secret
kubectl create secret generic cacerts -n istio-system \
  --from-file=ca-cert.pem \
  --from-file=ca-key.pem \
  --from-file=root-cert.pem \
  --from-file=cert-chain.pem \
  --dry-run=client -o yaml | kubectl apply -f -

# Restart Istio control plane
kubectl rollout restart deployment istiod -n istio-system

🔍 Viewing & Inspecting Certificates

View certificate details from secret

kubectl get secret my-tls-secret -o jsonpath='{.data.tls\.crt}' | \
  base64 -d | \
  openssl x509 -text -noout

Check certificate expiry dates across all secrets

for secret in $(kubectl get secrets --field-selector type=kubernetes.io/tls -o name); do
  echo "=== $secret ==="
  kubectl get $secret -o jsonpath='{.data.tls\.crt}' | \
    base64 -d | \
    openssl x509 -noout -subject -enddate
  echo ""
done

View certificate Subject Alternative Names (SAN)

kubectl get secret my-tls-secret -o jsonpath='{.data.tls\.crt}' | \
  base64 -d | \
  openssl x509 -noout -text | \
  grep -A 1 "Subject Alternative Name"

Test ingress TLS connection

# Get ingress external IP
kubectl get ingress my-ingress

# Test with curl
curl -v https://example.com

# Check certificate chain
echo | openssl s_client -servername example.com -connect example.com:443 2>/dev/null | openssl x509 -noout -text

Compare certificate in secret vs. deployed

# Get fingerprint from secret
kubectl get secret my-tls-secret -o jsonpath='{.data.tls\.crt}' | \
  base64 -d | \
  openssl x509 -noout -fingerprint

# Get fingerprint from live site
echo | openssl s_client -connect example.com:443 2>/dev/null | \
  openssl x509 -noout -fingerprint

🔧 Troubleshooting

Ingress shows "default certificate" instead of custom TLS

Problem: Ingress controller serving default/fallback certificate

Solutions:

  • Verify secret exists: kubectl get secret my-tls-secret
  • Verify secret is in same namespace as ingress
  • Check ingress TLS configuration matches secret name
  • Verify certificate covers the ingress hostname (check SAN)
  • Restart ingress controller: kubectl rollout restart deployment -n ingress-nginx nginx-ingress-controller

cert-manager certificate stuck in "pending" state

Problem: Certificate not being issued

Solutions:

  • Check certificate status: kubectl describe certificate
  • Check challenge status: kubectl get challenge
  • Verify HTTP-01 challenge endpoint is accessible (port 80)
  • For DNS-01, check DNS provider credentials
  • Check cert-manager logs: kubectl logs -n cert-manager deployment/cert-manager

"x509: certificate signed by unknown authority"

Problem: Certificate chain incomplete or self-signed

Solutions:

  • Use fullchain.pem (not cert.pem) for Let's Encrypt
  • Verify certificate chain: openssl verify -CAfile ca-bundle.pem cert.pem
  • Check if using staging vs. production Let's Encrypt
  • For self-signed certs, add CA to trust store

Pods not picking up updated certificate

Problem: Updated secret but pods still use old certificate

Solutions:

  • Secrets mounted as volumes aren't automatically reloaded
  • Restart pods: kubectl rollout restart deployment my-app
  • Or configure app to watch for certificate changes and reload
  • Consider using cert-manager with automatic rotation

Debug certificate issues in pod

# Exec into pod
kubectl exec -it my-pod -- sh

# Check mounted certificate
ls -la /etc/ssl/certs/
cat /etc/ssl/certs/tls.crt

# View certificate details
openssl x509 -in /etc/ssl/certs/tls.crt -text -noout

# Test connection from pod
curl -v https://example.com

✅ Best Practices

Secret management

  • Use namespaces to isolate secrets (secrets don't cross namespaces)
  • Enable encryption at rest for secrets in etcd
  • Use RBAC to restrict access to TLS secrets
  • Never commit secrets to version control (use sealed-secrets or external secret operators)
  • Regularly rotate certificates (every 30-90 days)

Automation

  • Use cert-manager for automatic certificate management
  • Set up monitoring for certificate expiry (30, 15, 7 days warnings)
  • Automate certificate rotation with proper testing
  • Use Let's Encrypt staging for testing before production
  • Document certificate renewal procedures

Security

  • Use strong key sizes (RSA 4096 or ECDSA P-256 minimum)
  • Enable TLS 1.2+ only (disable TLS 1.0 and 1.1)
  • Use secure cipher suites
  • Implement certificate pinning for critical services
  • Regularly audit certificate usage across cluster

Production deployment

  • Always use cert-manager in production (not manual secret updates)
  • Use DNS-01 challenge for wildcard certificates
  • Set up proper monitoring and alerting
  • Have rollback plan for certificate issues
  • Test certificate rotation in staging first
  • Document all certificate configurations in GitOps repo