Is your Kubernetes API Server exposed?

David Calvert
July 6, 2022
  # security # kubernetes # aws
        
Picture from Tumisu on pixabay

This article was originally published on Medium.

After reading “Over 380 000 open Kubernetes API servers”, I asked myself if some of the clusters I manage could be exposed. I then decided to check and I think you should too!

What are we talking about?

If you work with Kubernetes, there is a good chance that most of your interaction with it is thru the Kubernetes API Server. Almost each time you use kubectl, k9s, Lens or Octant, you are in fact sending HTTP requests to the API Server.

What’s interesting here is to know:

  • From where are you talking to the API?
  • And what’s in between?

If you are using your Kubernetes client thru a bastion host and/or a virtual private network (VPN), your are probably in a good spot. If not or in doubt, you should definitely check!

Why would it be exposed?

There could be many different answers to this question, but the most common one is probably going to be: because it’s the default!

I’m going to use Amazon Elastic Kubernetes Service (EKS) as an example here. When you create an EKS cluster on AWS, here’s what you will find by default on Step 2 “Specify networking” :

Screenshot: AWS Cluster endpoint access
Screenshot: AWS Cluster endpoint access

This means that all clusters created using the default configuration expose the API Server to the entire internet (0.0.0.0/0)! A bit scary… but let’s not panic!

How to test ?

In order to test your cluster configuration, you will need to get your Kubernetes cluster API Server(s) IP or FQDN. In most cases, you will find this information in your Kubernetes configuration file:

$ grep server ~/.kube/config
server:https://XXXXXXXXX.abc.us-west-1.eks.amazonaws.com
server:https://XXXXXXXXX.abc.us-west-1.eks.amazonaws.com
server:https://XXXXXXXXX.abc.us-west-1.eks.amazonaws.com

Once you have this, you can simply try to query the /version endpoint from your terminal or any web browser:

$ curl -k https://XXXXXXXXX.abc.us-west-1.eks.amazonaws.com/version
{
  "major": "1",
  "minor": "20+",
  "gitVersion": "v1.20.15-eks-a64ea69",
  "gitCommit": "03450cdabfc4162d4e447e6d8c5037efe6d29742",
  "gitTreeState": "clean",
  "buildDate": "2022-05-12T18:44:04Z",
  "goVersion": "go1.15.15",
  "compiler": "gc",
  "platform": "linux/amd64"
}

If you get similar answer, this means that you Kubernetes API Server is exposed on internet.

How to fix it?

There’s many way to get around this, I will share here a quick way to reduce the attack surface easily and also a recommend a better architecture.

The quick fix

This one is straightforward and you should be able to it immediately in your AWS console.

  • For a new cluster : EKS > Clusters > Create EKS cluster > Specify networking
  • For an existing EKS cluster:EKS > Clusters > ${Your-Cluster} > Networking Tab > Manage networking

Once there, unfold Advanced settings and you will be able to add up to 40 source CIDR block. If you want to allow a single IP address, just append /32 after like this:

Screenshot: AWS Cluster endpoint access with CIDR
Screenshot: AWS Cluster endpoint access with CIDR

With this solution, your Kubernetes API server is still exposed publicly on internet, but only the configured CIDR block will be allowed to access it.

To get your public IP, you can use a service like whatismyip.com or use dig:

$ dig +short [email protected]
x.x.x.x

A better approach

Once you have limited your public exposition, you have time to think and work on a more secure and sophisticated architecture to manage your Kubernetes cluster(s).

I personally recommend to:

  • Use a VPN to access your Virtual Private Cloud (VPC)
  • Configure your EKS Cluster endpoint access to Private mode
  • Create a private SSH bastion host in your VPC
  • Update your EKS Security Group to only allow inbound traffic from the bastion host
  • Enable access & audit logs on your bastion

Here’s how the architecture looks like:

AWS architecture made with https://www.cloudcraft.co
AWS architecture made with https://www.cloudcraft.co

While this article used the AWS cloud provider, you should be able to reproduce something similar on most cloud providers.

Final words

I hope that this article has been useful to you! If it’s the case, you should check out my other articles. Feel free to send me some feedback, especially if you find any errors, if you think of any enhancements or if you find something outstanding!

Feel free to follow me on:

👋