Ensure No Other Proxies are Running on the local machine.
Deploying Web Applications with Kubernetes on American Cloud Kubernetes Service (ACKS)
Prerequisites
-
Install
kubectl
by following these instructions -
Install
helm
by following these instructions -
Owned domain with the ability to manage DNS
-
Dockerized application images in a public or private registry (extra steps in section Connecting to Private Image Repositories)
1. Provisioning Kubernetes Cluster
-
Choose a name, project, version, region, and node plan for your ACKS cluster.
2. Connecting to Kubernetes Cluster
-
Once the cluster is in "Running" state:
-
Download the cluster config file by clicking on "Download Config File"
-
Move the
kube.conf
file to a new directory. You'll be creating more files alongside it in order to set up your app.
Note: kube.conf
contains connection details on how your machine will connect and dispatch commands to the cluster. Every action will be of the form: kubectl --kubeconfig kube.conf
unless you set it as the global kube config.
- Set
kube.conf
as the default config by runningexport KUBECONFIG=kube.conf
, or by copying the file to~/.kube/config
Note: Example 1-create-admin-user.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kubernetes-dashboard
This generates one user (the ServiceAccount) and gives it the permissions necessary to access the Dashboard (the ClusterRoleBinding)
- Run
kubectl apply -f 1-create-admin-user.yaml
to create a user profile in order to generate access tokens to log in to the Dashboard.
ac-demo % kubectl apply -f 1-create-admin-user.yaml
serviceaccount/admin-user created
clusterrolebinding.rbac.authorization.k8s.io/admin-user created
-
Run
kubectl proxy
in a new terminal to start the Dashboard UI locally. Leave this running in the background. -
Open this url in your browser: http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
-
Run
kubectl -n kubernetes-dashboard create token admin-user
to get a fresh token, and paste it in the input field of the Dashboard login. -
You will be met with an empty dashboard, and the namespace
default
selected.
3. Creating App Resources
First we want to get our app running in its own pods. Then we can expose it.
We are going to create:
1 Deployment (a prescriptive model of your application including environment variables, port mappings, and scaling details)
1 Service (a way of allowing external access into your application)
If your images are hosted in a private repository, you will need to create 1 Secret as well (a protected resource containing repository access information, assuming your images are in a private registry)
Connecting to Private Image Repositories
Let's continue our example for now by pulling a public image which will run on internal port 8080.
Note: Example 2-demo-app-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-app
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: demo-app
strategy:
type: RollingUpdate
template:
metadata:
labels:
app: demo-app
spec:
containers:
- image: paulbouwer/hello-kubernetes:1.8
imagePullPolicy: IfNotPresent
name: demo-app
env:
- name: MESSAGE
value: Hello world!
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: demo-svc
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 8080
selector:
app: demo-app
Deploy by running kubectl apply -f 2-demo-app-deployment.yaml
You can check on your resources by running kubectl get pods
and kubectl get svc
, or by checking in your Dashboard:
Congratulations! Your application is running in Kubernetes.
4. Exposing Your App
Next, we must create LoadBalancer and Ingress resources to allow external access.
We start by installing the Kubernetes Nginx Ingress Controller
ac-demo % helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" has been added to your repositories
ac-demo % helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "ingress-nginx" chart repository
Update Complete. ⎈Happy Helming!⎈
ac-demo % helm install nginx-ingress ingress-nginx/ingress-nginx --set controller.publishService.enabled=true
NAME: nginx-ingress
LAST DEPLOYED: Tue Oct 25 20:40:16 2022
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status by running 'kubectl --namespace default get services -o wide -w nginx-ingress-ingress-nginx-controller'
An example Ingress that makes use of the controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example
namespace: foo
spec:
ingressClassName: nginx
rules:
- host: www.example.com
http:
paths:
- pathType: Prefix
backend:
service:
name: exampleService
port:
number: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
Take note of the new public ip after a couple minutes by running kubectl --namespace default get services -o wide -w nginx-ingress-ingress-nginx-controller
Now we create an Ingress to point traffic to the LoadBalancer:
Note: Example 3-nginx-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: "demo.your_domain_name"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: demo-svc
port:
number: 80
Before we apply it, we need to ensure that we have a DNS A record pointing your domain to the new public ip of your LoadBalancer.
Apply the Ingress:
kubectl apply -f 3-nginx-ingress.yaml
Go to https://demo.your_domain_name and see the Hello Kubernetes app!
5. Securing Your App
Now we need to get SSL / HTTPS playing nicely.
ac-demo % kubectl create namespace cert-manager
namespace/cert-manager created
ac-demo % helm repo add jetstack https://charts.jetstack.io
"jetstack" has been added to your repositories
ac-demo % helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "jetstack" chart repository
...Successfully got an update from the "ingress-nginx" chart repository
Update Complete. ⎈Happy Helming!⎈
ac-demo % helm install cert-manager jetstack/cert-manager --namespace cert-manager --version v1.6.0 --set installCRDs=true
NAME: cert-manager
LAST DEPLOYED: Tue Oct 25 21:05:28 2022
NAMESPACE: cert-manager
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
cert-manager v1.6.0 has been deployed successfully!
In order to begin issuing certificates, you will need to set up a ClusterIssuer
or Issuer resource (for example, by creating a 'letsencrypt-staging' issuer).
More information on the different types of issuers and how to configure them
can be found in our documentation:
https://cert-manager.io/docs/configuration/
For information on how to configure cert-manager to automatically provision
Certificates for Ingress resources, take a look at the `ingress-shim`
documentation:
https://cert-manager.io/docs/usage/ingress/
Note: Example 4-production-issuer.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
# Email address used for ACME registration
email: your_email_address
server: https://acme-v02.api.letsencrypt.org/directory
privateKeySecretRef:
# Name of a secret used to store the ACME account private key
name: letsencrypt-prod-private-key
# Add a single challenge solver, HTTP01 using nginx
solvers:
- http01:
ingress:
class: nginx
ac-demo % kubectl apply -f 4-production-issuer.yaml
clusterissuer.cert-manager.io/letsencrypt-prod created
Update the Ingress by using a new config file:
Note: Example 5-nginx-ingress-secured.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo-ingress
annotations:
kubernetes.io/ingress.class: nginx
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
tls:
- hosts:
- demo.your_domain
secretName: demo-tls
rules:
- host: "demo.your_domain"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: demo-svc
port:
number: 80
ac-demo % kubectl apply -f 5-nginx-ingress-secured.yaml
ingress.networking.k8s.io/demo-ingress configured
Connecting to Private Image Repositories
In order to connect to a private image or package repository, a token with sufficient access to pull images needs to be encoded and stored in Kubernetes as a Secret.
In this example, we will be connecting to a private registry (GHCR: GitHub Container Registry) which contains a Docker image with a NextJS web application.
We create a new personal access token with scope read:packages
by visiting https://github.com/settings/tokens/new?scopes=read:packages
We are granted a token, in this example: ghp_vMutK7pgmY1d6hOpF9vGeVpcUB34fd0i7O0j
We need a base64 encoded string which contains the username and the token:
ac-demo % echo -n "github-username:ghp_vMutK7pgmY1d6hOpF9vGeVpcUB34fd0i7O0j" | base64<
Z2l0aHViLXVzZXJuYW1lOmdocF92TXV0SzdwZ21ZMWQ2aE9wRjl2R2VWcGNVQjM0ZmQwaTdPMGo=
Create a new file, .dockerconfigjson
, with the following content:
{
"auths": {
"https://ghcr.io/ORGANIZATION_NAME/IMAGE_REPOSITORY_NAME":{
"username":"github-username",
"password":"ghp_vMutK7pgmY1d6hOpF9vGeVpcUB34fd0i7O0j",
"email":"YOUR_EMAIL",
"auth":"Z2l0aHViLXVzZXJuYW1lOmdocF92TXV0SzdwZ21ZMWQ2aE9wRjl2R2VWcGNVQjM0ZmQwaTdPMGo="
}
}
}
Note: This docker config format can be used to authenticate any Docker image repository, not just GHCR
Now encode this entire file, which we will save as the secret.
ac-demo % cat .dockerconfigjson | base64
ewogICAgImF1dGhzIjogewogICAgICAgICJodHRwczovL2doY3IuaW8vT1JHQU5JWkFUSU9OX05BTUUvSU1BR0VfUkVQT1NJVE9SWV9OQU1FIjp7CiAgICAgICAgICAgICJ1c2VybmFtZSI6ImdpdGh1Yi11c2VybmFtZSIsCiAgICAgICAgICAgICJwYXNzd29yZCI6ImdocF92TXV0SzdwZ21ZMWQ2aE9wRjl2R2VWcGNVQjM0ZmQwaTdPMGoiLAogICAgICAgICAgICAiZW1haWwiOiJZT1VSX0VNQUlMIiwKICAgICAgICAgICAgImF1dGgiOiJaMmwwYUhWaUxYVnpaWEp1WVcxbE9tZG9jRjkyVFhWMFN6ZHdaMjFaTVdRMmFFOXdSamwyUjJWV2NHTlZRak0wWm1Rd2FUZFBNR289IgogICAgCX0KICAgIH0KfQ==
This is the configuration file which will be used to create the Secret, along with a Deployment which uses it to connect to the image repository.
apiVersion: v1
kind: Secret
metadata:
name: registry-credentials
namespace: default
type: kubernetes.io/dockerconfigjson
data:
.dockerconfigjson: ewogICAgImF1dGhzIjogewogICAgICAgICJodHRwczovL2doY3IuaW8vT1JHQU5JWkFUSU9OX05BTUUvSU1BR0VfUkVQT1NJVE9SWV9OQU1FIjp7CiAgICAgICAgICAgICJ1c2VybmFtZSI6ImdpdGh1Yi11c2VybmFtZSIsCiAgICAgICAgICAgICJwYXNzd29yZCI6ImdocF92TXV0SzdwZ21ZMWQ2aE9wRjl2R2VWcGNVQjM0ZmQwaTdPMGoiLAogICAgICAgICAgICAiZW1haWwiOiJZT1VSX0VNQUlMIiwKICAgICAgICAgICAgImF1dGgiOiJaMmwwYUhWaUxYVnpaWEp1WVcxbE9tZG9jRjkyVFhWMFN6ZHdaMjFaTVdRMmFFOXdSamwyUjJWV2NHTlZRak0wWm1Rd2FUZFBNR289IgogICAgCX0KICAgIH0KfQ==
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: demo-app
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: demo-app
strategy:
type: RollingUpdate
template:
metadata:
labels:
app: demo-app
spec:
containers:
- image: ghcr.io/ORGANIZATION_NAME/IMAGE_REPOSITORY_NAME
imagePullPolicy: IfNotPresent
name: demo-app
env:
- name: REACT_APP_ENVIRONMENT
value: PROD
ports:
- containerPort: 8080
imagePullSecrets:
- name: registry-credentials
Use Traefik Ingress (Instead of NGINX)
In order to use traefik as an ingress controller, simply run these commands and apply this traefik ingress file instead of using nginx.
Note: You still need to configure an A record to point to your domain.
helm repo add traefik https://helm.traefik.io/traefik
helm repo update
helm install traefik traefik/traefik
traefik-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo-ingress
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: "demo.your_domain"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: demo-svc
port:
number: 80
ac-demo % kubectl apply -f traefik-ingress.yaml
ingress.networking.k8s.io/demo-ingress created
Enable Autoscaling for your App
In order to enable Kubernetes autoscaling follow our Kubernetes Autoscaling Guide.