Kubernetes Production Setup Best Practices for Enterprise Teams

Running Kubernetes in production for enterprise workloads is fundamentally different from getting a cluster up and running. The gap between a working cluster and a production-grade platform spans security hardening, multi-tenancy, observability, compliance, and operational procedures that prevent 3 AM pages. Enterprise teams need battle-tested patterns, not just tutorials.

This guide covers the practices that separate production Kubernetes from demo Kubernetes — drawn from real-world deployments managing hundreds of services across regulated industries.

Cluster Architecture

Control Plane Configuration

Enterprise clusters should use managed control planes (EKS, GKE, AKS) unless you have a dedicated platform team of 5+ engineers. Self-managing etcd, the API server, and controller managers is a full-time job.

For EKS, configure the control plane for high availability:

Key decisions:

• Private API endpoint only. Public API endpoints are the #1 attack vector for enterprise clusters. Use a VPN or bastion for kubectl access.
• Separate system and workload node groups. System components (ingress, monitoring, cert-manager) run on dedicated nodes with taints. Application workloads can't interfere with cluster operations.
• gp3 volumes. 20% cheaper than gp2 with better baseline IOPS (3,000 vs 100/GB).

Node Pool Strategy

Enterprise clusters typically need 3-4 node pools:

Node Pool	Instance Type	Purpose	Taint
system	m6i.xlarge	Ingress, monitoring, cert-manager	CriticalAddonsOnly
workload	m6i.2xlarge	Application services	None
compute	c6i.4xlarge	CPU-intensive batch jobs	workload-type=compute
gpu	g5.xlarge	ML inference	nvidia.com/gpu

Namespace and Multi-Tenancy

Namespace Strategy

Organize namespaces by team and environment, not by application:

Resource Quotas

Every namespace must have resource quotas. Without them, a single team's runaway deployment can starve the entire cluster:

Limit Ranges

Set defaults so that pods without explicit resource requests don't run unbounded:

Security Hardening

Pod Security Standards

Enforce pod security at the namespace level using built-in Pod Security Admission:

The restricted policy requires:

• No privileged containers
• No host networking or host ports
• Read-only root filesystem
• Non-root user
• Seccomp profile set

Network Policies

Default deny all traffic, then explicitly allow what's needed:

RBAC

Create role bindings per team with least-privilege access:

Secrets Management

Never store secrets in Kubernetes Secret objects directly — they're base64-encoded, not encrypted. Use an external secrets operator:

Observability

The Three Pillars

Enterprise Kubernetes requires metrics, logs, and traces working together.

Metrics with Prometheus:

Alerting rules:

Structured Logging

Enforce JSON logging across all services. This makes log aggregation and querying practical at scale:

Deployment Strategies

Rolling Updates with Safety

Key points:

• Three probes: Startup (for slow-starting apps), readiness (for traffic routing), liveness (for stuck processes).
• preStop hook: The 10-second sleep allows load balancers to deregister the pod before it terminates. Without this, you get 502 errors during deployments.
• terminationGracePeriodSeconds: 60: Give in-flight requests time to complete.

Pod Disruption Budgets

Protect services during node drains and cluster upgrades:

With 5 replicas and minAvailable: 3, only 2 pods can be down simultaneously during voluntary disruptions. This prevents cluster upgrades from taking down your service.

Cluster Upgrades and Maintenance

Upgrade Strategy

Enterprise clusters should follow a staged upgrade pattern:

1. Dev cluster: Upgrade immediately when new version is available
2. Staging cluster: Upgrade 1 week after dev (soak test)
3. Production cluster: Upgrade 2-3 weeks after staging
4. DR cluster: Upgrade after production is stable (1 week)

For EKS managed node groups, use a rolling update:

Node Maintenance

Automate node rotation to ensure nodes don't drift:

Disaster Recovery

Backup Strategy

Use Velero for cluster-level backups:

Multi-Cluster Strategy

Enterprise deployments should run at least two clusters:

• Active-active for critical services (traffic split across clusters)
• Active-passive for stateful services (failover on primary failure)

Use an external load balancer (AWS Global Accelerator, Cloudflare) to route between clusters, not in-cluster service mesh.

Conclusion

Production Kubernetes for enterprise teams is a platform engineering discipline, not a deployment target. The patterns covered here — namespace isolation, pod security standards, network policies, observability, and upgrade procedures — represent the minimum bar for enterprise workloads. Skipping any of these creates operational debt that surfaces at the worst possible time.

The most common failure mode isn't a cluster going down. It's a gradual erosion of operational discipline: resource quotas not enforced, network policies not applied, alerts not actionable. Enterprise teams should treat their Kubernetes configuration as production code — reviewed, tested, and continuously validated. GitOps tools like Argo CD or Flux make this practical by ensuring the cluster state always matches what's in version control.