1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
|
# Set up KIND cluster with multiple KCP and worker nodes running konnectivity
Change to the `examples/kind-multinode-kcp` folder and run `./quickstart-kind`. This script
performs the following operations:
1. Render config templates in `templates/` using provided values.
2. Create a new `kind` cluster with the desired number of KCP and worker nodes.
3. Changes `kubectl` context to point to the new `kind` cluster.
4. Deploys `konnectivity` proxy servers and agents to the KCP and worker nodes.
## `./quickstart-kind.sh` command-line flags
- `--cluster-name <NAME>`: Name of the `kind` cluster to be created Default: `knp-test-cluster`
- `--overwrite-cluster`: Overwrite existing `kind` cluster if necessary. Default: do not overwrite.
- `--server-image <IMAGE_NAME>[:<IMAGE_TAG>]`: Proxy server image to deploy. Default: `gcr.io/k8s-staging-kas-network-proxy/proxy-server:master`
- `--agent-image <IMAGE_NAME>[:<IMAGE_TAG>]`: Proxy server image to deploy. Default: `gcr.io/k8s-staging-kas-network-proxy/proxy-agent:master`
- `--num-kcp-nodes <NUM>`: Number of control plane nodes to spin up. Default: 2.
- `--num-worker-nodes <NUM>`: Number of worker nodes to spin up. Default: 1.
- `--server-count-override <NUM>`: If this flag is >= 0, override the `--serverCount` flag in the proxy server's configuration to the provided number. Default: set `--serverCount` to equal the number of KCP nodes.
- `--sideload-images`: Use `kind load ...` to sideload custom proxy server and agent images with the names set by `--server-image` and `--agent-image` into the kind cluster. Default: do not sideload.
- Use this if you don't want to publish your custom KNP images to a public registry.
- NOTE: You MUST specify an image tag (i.e. `my-image-name:my-image-tag` and not just `my-image-name`) and the image tag MUST NOT be `:latest` for this to work! See [`kind` docs](https://kind.sigs.k8s.io/docs/user/quick-start/#loading-an-image-into-your-cluster) for why this is necessary.
## Example usage to deploy custom local KNP images
In the repo root, build KNP and its docker images with the following:
```shell
make clean
make certs
make gen
make build
make docker-build
```
Verify that the new images are available in the local docker registry with `docker images`. Then, bring up the cluster:
```shell
cd examples/kind-multinode
# These are the default values of the registry, image name, and tag used by the Makefile.
# Edit them if necessary.
REGISTRY=gcr.io/$(gcloud config get-value project)
TAG=$(git rev-parse HEAD)
TARGET_ARCH="amd64"
SERVER_IMAGE="$REGISTRY/proxy-server-$TARGET_ARCH:$TAG"
AGENT_IMAGE="$REGISTRY/proxy-agent-$TARGET_ARCH:$TAG"
# Bring up the cluster!
./quickstart-kind.sh --cluster-name custom-knp-test --server-image "$SERVER_IMAGE" --agent-image "$AGENT_IMAGE" \
--num-kcp-nodes 3 --num-worker-nodes 2 --sideload-images
```
## Making sure the script worked
Check that the `konnectivity` pods are up and running:
```shell
kubectl --namespace kube-system get pods | grep konnectivity
# Output:
# konnectivity-agent-4db5j 1/1 Running 0 34m
# konnectivity-agent-c7gj5 1/1 Running 0 34m
# konnectivity-agent-h86l9 1/1 Running 0 34m
# konnectivity-server-9bl45 1/1 Running 0 34m
# konnectivity-server-dcfz8 1/1 Running 0 34m
# konnectivity-server-klww5 1/1 Running 0 34m
# konnectivity-server-nrfz8 1/1 Running 0 34m
```
Then create a test pod on a worker node and verify you can get logs from it:
```shell
kubectl run test --image httpd:2
# Output:
# pod/test created
kubectl get pods
# Output:
# NAME READY STATUS RESTARTS AGE
# test 1/1 Running 0 34s
kubectl logs test
# Output:
# AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.244.5.3. Set the 'ServerName' directive globally to suppress this message
# AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.244.5.3. Set the 'ServerName' directive globally to suppress this message
# [Wed Jun 12 20:42:06.471169 2024] [mpm_event:notice] [pid 1:tid 139903660291968] AH00489: Apache/2.4.59 (Unix) configured -- resuming normal operations
# [Wed Jun 12 20:42:06.471651 2024] [core:notice] [pid 1:tid 139903660291968] AH00094: Command line: 'httpd -D FOREGROUND'
```
|
