Kops

Dump a cluster config

there are two types of configs that make up a kops created cluster:

1. the cluster config

2. an instance group config, one for each instance group.

THe easiest way to save a copy of this is to

kops edit cluster

then write the config to a file.

kops edit ig <instance group name>

and save that to a file.

The Shiva

# /bin/sh

# Source:
# https://kubernetes.io/docs/setup/custom-cloud/kops/
# route 53 domain registered and hosted: 

# check existing status:

echo subdomain: ${subdomain}
echo bucketconfig: ${bucketconfig}
echo prefix ${prefix}
echo cn: ${cn}
echo pubkey ${pubkey} 

export subdomain="dev.thedomain.com"
export bucketconfig="dthornton-clusters"
export prefix="lab001"
export cn="${prefix}.${subdomain}" # clustername
export pubkey="~/.ssh/dthornton.pub"

# check again:

echo subdomain: ${subdomain}
echo bucketconfig: ${bucketconfig}
echo prefix ${prefix}
echo cn: ${cn}
echo pubkey ${pubkey} 


# does the bucket exist?
aws s3api list-buckets --output table | grep ${bucketconfig}.${subdomain}
# if not make a bucket:
echo aws s3 mb s3://${bucketconfig}.${subdomain}
# aws s3 mb s3://${bucketconfig}.${subdomain}

# sync a local copy
# aws s3 sync s3://${bucketconfig}.${subdomain} s3bucket

export KOPS_STATE_STORE="s3://${bucketconfig}.${subdomain}"

echo KOPS_STATE_STORE
echo ${KOPS_STATE_STORE}

# example:
# kops create cluster --zones=us-east-1c useast1.dev.quadratic.net

# cluster creation , chicken and egg:
# this command makes the kops cluster object but fails to make all the aws cloud objects because there is no key to give the instances.
kops create cluster --zones="ca-central-1a,ca-central-1b" "${cn}"

kops create cluster \
--zones ca-central-1a,ca-central-1b \
--master-zones ca-central-1a \
--image ami-9526abf1 \ # latest ca-central-1 ubuntu Tue 18 Sep 2018 10:52:50 EDT 
--ssh-public-key ${pubkey} \
--node-size t2.medium \
--node-count 2 \ 
--master-size t2.medium \
--network-cidr 10.10.0.0/16 \
--dns-zone ${subdomain} \
--cloud-labels "owner=dthornton,managedby=kops" \
--name  ${cn} \
--yes

kube config is :  /Users/david/.kube/config

api url: https://api.${cn}/api/v1/nodes?limit=500

optional:

kops update cluster --name $cn --yes

Note that above we explicity tell it what public key to use at the outset.

# now make a kube secret of type public key. This assumes you already have a private public pair and you are giving kops the public part so that it can give it to kubrnetes and AWS.

# kops create secret --name ${cn} sshpublickey admin -i ~/.ssh/${pubkey}
# kops edit cluster ${cn}

# ig = instance group

kops edit ig --name=${cn} nodes
kops edit ig --name=${cn} master-ca-central-1a

#Suggestions:
# * validate cluster: kops validate cluster
# * list nodes: kubectl get nodes --show-labels
# * ssh to the master: ssh -i ~/.ssh/id_rsa admin@api.cacentral.dev.quadratic.net
# * the admin user is specific to Debian. If not using Debian please use the appropriate user based on your OS.
# * read about installing addons at: https://github.com/kubernetes/kops/blob/master/docs/addons.md.

kops update cluster ${cn} --yes

kops rolling-update cluster

# specify some stuff at creation: 

#kops create cluster \
#--master-zones=us-east-1a,us-east-1b,us-east-1c \
#--zones=us-east-1a,us-east-1b,us-east-1c \
#--node-count=2 \
#--image ami-32705b49 \
#${cn}

kubectl -n kube-system get po

set name spae to kube-system
get pods

#  Note that the kops destroy cleans out the kubectl config file: "/Users/david/.kube/config"
kops delete cluster ${cn} --yes

Tricks for making small updates to a host

In a kops environment you might want to make change to the cluster spec that implies a change to a node ( Master or Node ).

normally you would:

kops edit cluster # edit the text description of the cluster as per kops' spec ( see https://github.com/kubernetes/kops/blob/master/docs/cluster_spec.md )
kops update cluster # review your change
kops update cluster–yes # Apply your change to the files in the KOPS bucket.
kops rolling-update cluster # review the list of hosts that will be updated.
kops rolling-update cluster --yes # actually reboot, recreate, restart the nodes that need it.

This can be a long process, especially when kops waits a long time ( 5m ) between per-host changes, to be sure that a given host has had enough time to reboot for example.

There is another way.

( I found it here: https://github.com/kubernetes/kops/issues/3645 )

you can "test" a change easily on a per host basis by logging into the host and telling it to update itself manually ( do this after the "kops update cluster–yes" ).

sudo SKIP_PACKAGE_UPDATE=1 /var/cache/kubernetes-install/nodeup --conf=/var/cache/kubernetes-install/kube_env.yaml --v=8

nodeup is the tool that kops uses to get kubernetes configured on the host.

note that in the example I found the path to the nodeup binary and the kube_env file was different than what we had in uk-blue.

I used locate to find the correct binary and config file.

It's worth looking at the config file to get a better idea of what it is that the nodeup command is doing.

I figure this out in the proces of getting auditing to work. I wanted to define an audit file n the host file system for kube api server to use.

I was able to make this change to one host by hand and examine the effect without interrupting service otherwise.

Also see

• Kubernetes - what you gone do?