Specific setups of Disa¶

Disa s a shared (multi-tenant) environment, we aim to provide users with the full power of a Kubernetes environment but certain limits apply here. Below are the specific rules that apply to the Disa cluster and how it may impact your usage.

Namespace and directoryName¶

Kubernetes uses Namespaces to isolate different deployments. All your Helm charts (apps), Pods, PersistentVolumeClaims (storage) live within Namespacess.

In Disa, rancher manages project and user access. You create Namespace using the rancher web interface, and user-created Namespaces always belong to a project. You can create many Namespaces to isolate different applications.

As Namespace is a cluster-wide resource shared by all Disa users, Namespaces are constrained such that you must prefix the project directoryName. For example, if you have a project named c3se416-261, your Namespaces will have to be named like c3se416-261-myapp.

The directoryName is used as a user-friendly key to identify your project in Disa. Other resources below also identifies projects with the directoryName. When you apply a project on Disa, you an choose a unique directoryName.

Storage¶

Project that has a storage allocation linked to it will get a StorageClass assigned. The StorageClass may only be used from Namespaces from the same project.

By default, mutation rule will rewrite PVCs deployed by users and assign a StorageClass according to the project. At the moment, all Disa projects gets persistent CephFS storage from the disk partition of Cephyr. The default StorageClass assigned to each project is named as <directoryName>-disk.

Workload and Kueue¶

Disa manages all user workloads with the kueue scheduler. The compute resource belonging to a project is represented as a ClusetrQueue, typically this will be the ClusterQueue that is named as your project's directorName. A LocalQueue resource named default is created in every Namespace which points to that ClusterQueue.

To effectively schedule workloads in a shared environment, Disa enforces resource requests and limits, this means all containers must request CPU and memory resource, and must have the same limit as requested (containers exceeding their requested memory will be killed). If nothing is specified for a workload, default values will be enforced.

You can have an overview of kueue status at: https://kueue.disa.c3se.chalmers.se/

Networking¶

Namespaces belonging to a project get assigned CiliumNetworkPolicies automatically which only allow internal traffic from within the same Namespace or external traffic through the Gateway API.

Gateway API (Envoy Gateway) is how Disa exposes the shared disa.c3se.chalmers.se domain. Project members can deploy HTTPRoutes and ListenerSets to expose services to the internet. Those resources are constrained to the subdomains <directorName>.disa.c3se.chalmers.se or *.<directorName>.disa.c3se.chalmers.se.

It is possible to use this gateway directly to your pods that serve HTTP traffic. But we recommend users to always expose only HTTPS services. With GatewayAPI, this means deploying ListenerSets which get secret TLS certificates renewed automatically, and attach HTTPRoutes to them.

To protect your endpoint from spams it is also recommended that you set up SecurityPoliciy that limit external access to your application through the gateway.

The Helm charts offered by C3SE have easy options to expose services, set up TLS truncation and SecurityPolicies. If you use a third-party Helm chart or deploy your own application, you can deploy the relevant resources yourself. Below is a minimal example:

apiVersion: gateway.networking.k8s.io/v1
kind: ListenerSet
metadata:
  name: <your-app>
  namespace: <your-namespace>
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-issuer
spec:
  parentRef:
    name: public-gateway
    namespace: gateway
    kind: Gateway
    group: gateway.networking.k8s.io
  listeners:
    - hostname: <your-app>.<project>.disa.c3se.chalmers.se
      protocol: HTTPS
      port: 443
      name: https
      tls:
        certificateRefs:
        - name: <your-app>
          kind: Secret
        mode: Terminate
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: <your-app>
  namespace: <your-namespace>
spec:
  hostnames:
  - <your-app>.<project>.disa.c3se.chalmers.se
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: ListenerSet
    name: <your-app>
    namespace: <your-namespace>
    sectionName: https
  rules:
  - backendRefs:
    - kind: Service
      name: <your-svc>
      port: 8000
    matches:
    - path:
        type: PathPrefix
        value: /
---
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
  name: allow-only-chalmers
  namespace: <your-namespace>
spec:
  targetRefs:
    - kind: HTTPRoute
      name: <your-http-route>
      group:  gateway.networking.k8s.io
  authorization:
    defaultAction: Deny
    rules:
    - action: Allow
      principal:
        clientCIDRs:
        - 129.16.0.0/16