# Web Terminal

## Problem to solve

Epic: <https://gitlab.com/groups/gitlab-org/-/epics/11015>
Issue: <https://gitlab.com/gitlab-org/gitlab/-/issues/418264>

- As a user of a Kubernetes cluster that is connected to GitLab using the GitLab
  agent, I would like to interact with the cluster via a web terminal interface.

## Personas

- [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer)
- [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops)
- [Ingrid (Infrastructure Operator)](https://about.gitlab.com/handbook/product/personas/#ingrid-infrastructure-operator)

## User experience goal

- An agent administrator can manipulate a Kubernetes cluster that is connected
  via the GitLab agent with a web terminal interface within GitLab.
   - Users with user access may also reach the agent in a future iteration.
- The web terminal includes some commonly used tools to help with
   - debugging workloads
   - managing a cluster
- The user is restricted in the cluster by regular Kubernetes RBAC

The above functionality works without setting up any extra applications or
tokens, but the agent for Kubernetes. It requires the agent be configured with
some form of user access as detailed in [kubernetes user
access](./kubernetest_user_access.md)

Installing the suite of command line tools necessary to interact with the
kubernetes cluster can be hard or impossible in some environments. Corporate
machines may restrict access, and many mobile terminals available are
limited in what tools are available.

## Proposal

For an agent configured with `user_access: access_as: agent: {}`

### High-level flow

```mermaid
sequenceDiagram
  actor U as User
  participant F as GitLab FE
  participant G as GitLab BE
  participant S as KAS
  participant A as Agent
  participant K as Kubernetes Cluster


  U->>F: Select `Terminal` Tab
  U->>F: Press `Launch` Button
  F-)G: Request shell pod
  G-)S: Request shell pod
  S-)A: Deploy shell image pod
  A-)+K: Deploy shell image pod
  create participant P as Pod
  K->>P: Deploy Pod
  K->>A: Pod created
  A->>S: Pod created
  S->>G: Pod created
  G->>F: Pod created
  loop Poll for status
    F->>+S: Get pod status
    S->>+A: Get pod status
    A->>+K: Get pod status
    K->>-A: Return status
    A->>-S: Return status
    S->>-F: Return status
  end
  Note over F,S: When Pod status is Ready

  F-)S: Pod Attach
  S-)A: Pod Attach
  A-)K: Pod Attach
  K-)P: Pod Attach
  P->>F: WebSocket established

  loop User entering commands
    U->F: kubectl get pods --all-namespaces
    F->P: kubectl get pods --all-namespaces
    P->F: NAMESPACE    NAME...
  end

  U->>F: exit
  F->>P: exit
  P->>F: Websocket disconnected

  F-)S: delete Pod
  S-)A: delete Pod
  A-)K: delete Pod
  destroy P
  K-xP: delete Pod

  K-)A: Pod deleted
  A-)S: Pod deleted
  S-)F: Pod deleted
```

### GitLab side

GitLab adds an API endpoint that applies a Kubernetes manifest to the cluster to
create a pod that uses the same service account as the agent to give users
access to the cluster API. This endpoint MAY return before the pod is `status: ready`.

The GitLab backend should be able to call a new API in KAS (below) to
deploy a web terminal pod. This API should return the pod details, specifically
the namespace and pod name. These details should be recorded to be able to
remove pods later.

A sidekiq job should spawn that cleans up web terminal pods by calling a new API
within KAS to check the status of a given pod/namespace and delete it.

```rest
POST /api/v4/internal/project/:project_id/cluster_agents/:agent_id/terminal
Content-Type application/json
Accept: application/json
```

```rest
{ "namespace": "<agent-namespace>", "pod": "<web-terminal-pod-name>" }
```

### KAS

KAS adds an endpoint that is only valid for GitLab BE authenticated calls that
spins up a web terminal, roughly with the below manifest.

It should also record the pod name to clean up the resource once the pod exits,
or otherwise label web-terminal pods to easily find them.

KAS adds an endpoint that is only valid for GitLab BE authenticated calls that
cleans up finished web terminals. Finished web terminals are web terminal Pods
with Statuses other than `Ready` and `Pending`. The endpoint accepts a namespace
and pod name, verifies the status, and deletes the pod.

Manifest:

```yaml
apiVersion: apps/v1
kind: Pod
metadata:
  generateName: web-terminal-
  namespace: :agent-namespace
spec:
  serviceAccountName: :agent-service-account
  restartPolicy: Never
  containers:
    - image: registry.gitlab.com/gitlab-org/cluster-integration/shell-image:<GITLAB MAJOR>.<GITLAB MINOR>
      name: web-terminal
      stdin: true
      stdinOnce: true
      tty: true
```

### Agent side

To accomplish the above, the GRPC endpoint
`/gitlab.agent.agent_tracker.rpc.AgentTracker/GetConnectedAgents()` must be
modified such that it returns the service account name for the agent in the
`AgentMeta` structure. This is called by KAS to get the service name used in the
deployment of the web terminal pod.

### GitLab Frontend

Upon receiving the response from the above GitLab API endpoint, the GitLab
frontend polls the status of the pod, either via the `?watch` endpoint or via
simple polling until the status is `Ready`. After that, it creates a websocket
connection to the pod via the agent's `k8s-proxy`. Then, it writes data to an
`xterm` instance.

Upon closing of the websocket, the frontend will attempt to clean up the pod by
sending a `DELETE` to the `k8s-proxy` to clean up the resource. However, this is
not guaranteed, and so the agent should make a best effort to clean up these
pods once their status is `Completed`.

### On User Impersonation (**PREMIUM**)

When `user_access: access_as: user: {}` is enabled for the agent:

When creating the pod, a short-term personal access token (~6 hour expiry) is
provisioned that is injected into the pod as part of the `kubectl`
configuration.  This token also allows for `glab` access, and should therefore
have the `api` and `k8s_proxy` scopes. The `kubectl` context is also directed at
KAS instead of using a Service Account, to allow the token to authenticate the
user as commands are performed. This means that the `serviceAccountName` in the
above manifest is omitted.

Another option for the authorization is to augment agentk to be able to support
user impersonation locally by validating tokens. This way, `kubectl` calls don't
need to go to KAS and theoretically lower the latency of the calls. Agentk can
potentially do validation on its own, or validate via KAS and cache the result.
The latter would allow us to still use a short-lived GitLab PAT.  Agentk would
have to generate self-signed SSL certificates for kubeconfig to accept the
connection. The public part of the cert would need to be bundled with the
kubeconfig.

## Future iterations

- Custom container, namespace, and service account permissions (**ULTIMATE**)
- Workspaces support (**PREMIUM**)

## GitLab links

### Issues & Epics

- [Main epic](https://gitlab.com/groups/gitlab-org/-/epics/11015)