python>> cdap-operator>> 返回
项目作者： cdapio

项目描述 ：
CDAP Kubernetes Operator
高级语言： Go
项目地址: git://github.com/cdapio/cdap-operator.git
创建时间： 2018-12-11T01:51:22Z
项目社区：https://github.com/cdapio/cdap-operator
开源协议：Other
下载


Kubernetes operator for CDAP

Project Status

Alpha

The CDAP Operator is still under active development and has not been extensively tested in production environment. Backward compatibility of the APIs is not guaranteed for alpha releases.

Prerequisites

  • Version >= 1.9 of Kubernetes.
  • Version >= 6.0.0 of CDAP

Quick Start

Build and Run Locally

You can checkout the CDAP Operator source code, build and run locally. To build the CDAP Operator, you need to setup your environment for the Go language. Also, you should have a Kubernetes cluster

  1. Checkout CDAP Operator source
    1. mkdir -p $GOPATH/src/cdap.io
    2. cd $GOPATH/src/cdap.io
    3. git clone https://github.com/cdapio/cdap-operator.git
    4. cd cdap-operator
  2. Generates and install the CRDs
    1. make install
  3. Compiles and run the CDAP Operator
    1. make run
  4. Deploy CDAP CRD to the cluster
    1. kubectl apply -k config/crd
  5. Edit the sample CDAP CR and deploy to the cluster
    1. kubectl apply -f config/samples/cdap_v1alpha1_cdapmaster.yaml

Build Controller Docker Image and Deploy in Kubernetes

You can also build a docker image containing the CDAP controller and deploy it to Kubernetes.

  1. Build the docker image
    1. IMG=cdap-controller:latest make docker-build
    You can change the target image name and tag by setting the IMG environment variable.
  2. Push the docker image
    1. IMG=cdap-controller:latest make docker-push
  3. Deploy CDAP CRD and RBAC to the cluster
    1. make deploy

Using CDAP operator to manage CDAP instances in Kubernetes

A step by step guide of running CDAP in Kubernetes using CDAP operator can be found in the blog post.

Using the Admission Controller

The CDAP operator can be configured to optionally run a webhook server for a mutating admission controller. The mutating admission controller allows the operator to change the following fields in CDAP pods:

  1. Add init containers
  2. Add Node Selectors
  3. Add tolerations

These mutations can be defined using the MutationConfigs field in CDAPMaster.

Prerequisites

Kubernetes requires that the webhook server uses TLS to authenticate with the kube API server. For this you will need to ensure the TLS certificates are present in the /tmp/k8s-webhook-server/serving-certs directory in the cdap-controller pod. To simplify the management of TLS certificates, you can use cert-manager. The following steps assume you are in the root directory of the Git repository and have already deployed the CDAP operator stateful set.

  1. Deploy cert-manager in the cluster.
    1. kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.0/cert-manager.yaml
    You should see 3 pods running for cert-manager.
    1. kubectl get pods -n cert-manager
    2. NAME                                       READY   STATUS    RESTARTS       AGE
    3. cert-manager-655c4cf99d-rbzwr              1/1     Running   0              2m
    4. cert-manager-cainjector-845856c584-csbsw   1/1     Running   0              2m
    5. cert-manager-webhook-57876b9fd-68vgc       1/1     Running   0              2m
  2. Deploy a kubernetes service for the webhook server.
    1. # set the namespace in which CDAPMaster is deployed.
    2. export CDAP_NAMESPACE=default
    3. sed -e 's@{CDAP_NAMESPACE}@'"$CDAP_NAMESPACE"'@g' <"./webhooks/templates/webhook-service.yaml" | kubectl apply -f -
  3. Deploy the cert-manager self-signed issuer.
    1. sed -e 's@{CDAP_NAMESPACE}@'"$CDAP_NAMESPACE"'@g' <"./webhooks/templates/issuer.yaml" | kubectl apply -f -
  4. Deploy the Certificate resource.
    1. sed -e 's@{CDAP_NAMESPACE}@'"$CDAP_NAMESPACE"'@g' <"./webhooks/templates/certificate.yaml" | kubectl apply -f -
    Wait for the certificate to be ready.
    1. kubectl get Certificates
    2. NAME                READY   SECRET                     AGE
    3. cdap-webhook-cert   True    cdap-webhook-server-cert   1d
  5. Add the following fields in the CDAP operator stateful set spec:
    ```yaml

    Filename: cdap-controller.yaml

    spec:
    containers:
    • command:
      • /manager
        args: [“—enable-webhook”, “true”]
        …
        volumeMounts:
      • mountPath: /tmp/k8s-webhook-server/serving-certs
        name: cert
        readOnly: true
        …
        volumes:
    • name: cert
      secret:
      defaultMode: 420
      secretName: cdap-webhook-server-cert
  1. 6. Deploy the mutating webhook resource:
  2. ```bash
  3. sed -e 's@{CDAP_NAMESPACE}@'"$CDAP_NAMESPACE"'@g' <"./webhooks/templates/webhook.yaml" | kubectl apply -f -

The webhook is now configured and it will intercept requests to create new pods made by CDAP.

Example use case: Isolate pods that execute user code in Google Kubernetes Engine.

Assuming task workers are enabled, the pods that execute user code in CDAP are task workers and preview runners. Let us call these pods as “worker pods”. To isolate these worker pods in a dedicated node pool with the help of the admission controller, you follow these steps:

  1. Create a node pool for running only worker pods.
    1. gcloud container node-pools create worker-pool \
    2.  --cluster cdap-cluster --project my-gcp-projet --location us-east1
  2. Add a taint to the new node pool. This will prevent pods from being scheduled on the node pool unless they specify the corresponding toleration.
    1. gcloud beta container node-pools update worker-pool  \
    2. --node-taints="worker-pods-only=true:NoExecute" \
    3. --cluster cdap-cluster --project my-gcp-projet --location us-east1

  3. Add the following configuration to the CDAPMaster:

    1. # Filename: cdapmaster.yaml
    2. spec:
    3. ...
    4. mutationConfigs:
    5. - labelSelector:
    6.    matchExpressions:
    7.    - {key: cdap.twill.app, operator: In, values: [task.worker, preview.runner]}
    8.  podMutations:
    9.    nodeSelectors:
    10.      cloud.google.com/gke-nodepool: worker-pool
    11.    tolerations:
    12.    - effect: NoExecute
    13.      key: worker-pods-only
    14.      operator: Equal
    15.      tolerationSeconds: 3600
    16.      value: "true"

    Now whenever CDAP launches preview runner of task worker pods, the admission controller will mutate the pod specifications before they are deployed to ensure the pods get scheduled only on the node pool “worker-pool”.

    Running Unit Tests

  4. Install kubebuilder.

  5. Install setup-envtest by running:

    1. go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest

  6. After installing setup-envtest, use it to download envtest 1.19.x for kubebuilder and to set your KUBEBUILDER_ASSETS environment variable:

    1. # Downloads envtest v1.19.x and writes the export statement to a temporary file
    2. $(go env GOPATH)/bin/setup-envtest use -p env 1.19.x > /tmp/setup_envtest.sh
    3. # Sets the KUBEBUILDER_ASSETS environment variable
    4. source /tmp/setup_envtest.sh
    5. # Deletes the temporary file
    6. rm /tmp/setup_envtest.sh

  7. Run make test