这是本节的多页打印视图。点击此处打印.

部署指南

1: 部署环境要求
2: All In One 部署与卸载

2.1: All In One 最小化部署
2.2: KubeCube 卸载

3: 在已有K8s上安装KubeCube

3.1: 通过helm安装KubeCube
3.2: 通过helm纳管计算集群

4: 使用 hotplug 进行拓展

4.1: 热插拔 hotplug 介绍
4.2: 通过 hotplug 启用 loggie

5: 使用脚本安装 K8s 和 KubeCube（已废弃）

5.1: 在已有k8s集群中部署KubeCube
5.2: 添加计算集群
5.3: 添加节点
5.4: 多节点高可用部署

6: 已有系统接入

6.1: Prometheus
6.2: ElasticSearch
6.3: 第三方认证系统

1 - 部署环境要求

在进行 All In One 或者多节点部署之前，请按照以下内容确认环境要求

系统版本及硬件要求

操作系统	最低要求
Ubuntu 16.04, 18.04	CPU：4 核，内存：8 G，磁盘空间：20 G
Debian Buster, Stretch	CPU：4 核，内存：8 G，磁盘空间：20 G
CentOS 7.x	CPU：4 核，内存：8 G，磁盘空间：20 G
Kylin v10	CPU：4 核，内存：8 G，磁盘空间：20 G

OS ARCH	硬件要求
AMD 64	Intel 系列，AMD 系列
ARM 64	Phytium 2000（国产化arm64芯片理论上都支持，但未经过充分测试）

以上系统配置要求适用于 KubeCube 默认最小化 All In One 模式安装，如需启动更多可插拔组件和拓展功能，建议机器配置为 8 核 CPU 和 16 G 内存

依赖说明

KubeCube	Kubernetes	Docker	Containerd
v1.0 ~ v1.4	v1.18 ~ v1.23	19.3.12+	1.5.x

容器运行时

KubeCube 支持的 cri 跟随 Kubernetes 标准
KubeCube 部署脚本支持的 cri 如下：

支持的容器运行时	版本
Docker	19.3.12+
Containerd	1.5.x

节点上若无容器运行时，部署脚本将自动安装 containerd 1.5.5 作为容器运行时

网络插件

KubeCube 支持的 cni 跟随 Kubernetes 标准
KubeCube 部署脚本的 cni 目前支持 calico

Kubernetes 版本

KubeCube 支持的 k8s 版本为 v1.18 ~ v1.26 KubeCube 部署脚本支持的 k8s 版本为 v1.18.20、v1.19.13、v1.20.9、v1.21.2、v1.22.2、v1.23.5、v1.24.7、v1.25.3、v1.26.0

关于 K8s 集群安装

KubeCube 的 All In One 安装脚本提供单节点的 K8s 集群安装。如果需要自定义安装 K8s 集群，可以使用 kubeadm、kubez-ansible 等开源工具，之后 kubez-ansible 中也会集成 KubeCube。

前置准备

在使用 All In One 部署脚本开始 KubeCube 的安装时，脚本会检测环境，并提示需要安装缺失的依赖

env-check

监控组件说明

KubeCube 会默认安装 Prometheus 等监控组件，如果选择在已有k8s集群中部署 KubeCube，并且集群中已安装 Mertics Server，安装 KubeCube 后 Prometheus 会和 Mertics Server 产生冲突，导致监控功能不可用。需要在安装 KubeCube 后执行以下步骤：

点击页面右上角【切换到控制台】，点击任意空间，进入到控制台页面；
在左侧菜单栏点击【自定义资源CRD】，进入到集群级别 CRD 列表，可以点击右上方输入 “hotplug” 进行搜索，找到 “hotplugs.hotplug.kubecube.io” CRD，点击【v1】版本进入 CRD 详情页；
选择 common 实例，点击【设置YAML】，找到 spec.component. name=kubecube-monitoring，添加环境变量 prometheusAdapter.enabled=false，如：

  - env: |
      grafana:
        enabled: false
      prometheus:
        prometheusSpec:
          externalLabels:
            cluster: "{{.cluster}}"
          remoteWrite:
          - url: http://10.173.32.129:31291/api/v1/receive
      prometheusAdapter:      
  			enabled: false
    name: kubecube-monitoring
    namespace: kubecube-monitoring
    pkgName: kubecube-monitoring-15.4.10.tgz
    status: enabled

2 - All In One 部署与卸载

2.1 - All In One 最小化部署

对于想要快速开始、快速体验的用户来说，All In One 是最佳的安装方式。All In One 部署脚本将会在执行该脚本的节点上，安装一个单节点的 K8s 集群，并在该 K8s 集群上部署 KubeCube。

v1.8.x

在 Linux 上部署 KubeCube

开始安装

在 Linux 机器上执行部署脚本

export KUBECUBE_VERSION=v1.8

curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/release/v1.3/entry.sh | bash

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

completed-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

v1.4.x

在 Linux 上部署 KubeCube

开始安装

在 Linux 机器上执行部署脚本

KUBECUBE_VERSION=v1.4

curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

completed-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

v1.2.x

在 Linux 上部署 KubeCube

开始安装

在 Linux 机器上执行部署脚本

KUBECUBE_VERSION=v1.2

curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

completed-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

v1.1.x

在 Linux 上部署 KubeCube

开始安装

在 Linux 机器上执行部署脚本

KUBECUBE_VERSION=v1.1

curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

completed-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

v1.0.x

在 Linux 上部署 KubeCube

开始安装

在 Linux 机器上执行部署脚本

KUBECUBE_VERSION=v1.0

curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

completed-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

2.2 - KubeCube 卸载

KubeCube 提供一键卸载脚本

⚠️ 注意，卸载脚本仅用于使用 All In One 部署脚本安装的环境，不应该用于任何正式环境

下载卸载脚本

curl -o cleanup.sh https://kubecube.nos-eastchina1.126.net/hack/cleanup.sh

一键卸载

一键卸载按顺序卸载 kubecube, kubernetes 和 docker

/bin/bash cleanup.sh all

指定卸载

卸载脚本可以指定卸载对象，但需注意卸载依赖

卸载 KubeCube

删除 kubecube chart release，并清理 kubecube 在本机上的相关文件

/bin/bash cleanup.sh kubecube

卸载 Kubernetes

卸载 kubernetes 集群，并清理 kubernetes 在本机上的相关文件

/bin/bash cleanup.sh k8s

卸载 Docker

卸载 docker，并清理 docker 在本机上的相关文件

⚠️注意：需在卸载 kubernetes 后才能卸载 docker，不然会造成 kubernetes 集群异常

/bin/bash cleanup.sh docker

3 - 在已有K8s上安装KubeCube

3.1 - 通过helm安装KubeCube

通过 helm 的方式在管控集群中部署 KubeCube

v1.8.x

要求

已有 K8s 集群环境
helm v3 客户端

下载 KubeCube helm chart 包

KUBECUBE_VERSION=v1.8

curl -s https://kubecube.nos-eastchina1.126.net/kubecube-chart/${KUBECUBE_VERSION}/kubecube-chart.tar.gz | tar -xz

通过 helm 在管控集群上安装 KubeCube

创建 pivot-value.yaml 文件并填写必要的 value 值

# pivot-value.yaml

global:
  # 管控集群的 Node IP，用来暴露 KubeCube 的 NodePort service
  nodeIP: x.x.x.x

  dependencesEnable:
    ingressController: "false" # 如果集群中没有部署 ingress controller，请将此设置为 "true"
    localPathStorage: "false" # 如果集群中没有部署 local path storage，请将此设置为 "true"
    metricServer: "false" # # 如果集群中没有部署 metric server，请将此设置为 "true"
    
  # 如果要启动日志功能，请假以下值设置为 "enabled"
  hotPlugEnable:
    pivot:
      logseer: "disabled" 
      logagent: "disabled"
      elasticsearch: "disabled"    

  localKubeConfig: xx # 管控集群的 kubeconfig 的 base64
  pivotKubeConfig: xx # 管控集群的 kubeconfig 的 base64

warden:
  containers:
    warden:
      args:
        cluster: "pivot-cluster"  # 管控集群名

部署 KubeCube

helm install kubecube -n kubecube-system --create-namespace ./kubecube-chart -f ./pivot-value.yaml

卸载管控集群中的 KubeCube

注意：not found 错误可以被忽略

手动清理 webhook

kubectl delete validatingwebhookconfigurations kubecube-validating-webhook-configuration warden-validating-webhook-configuration kubecube-monitoring-admission

卸载 KubeCube chart

helm uninstall kubecube -n kubecube-system

清理残留资源

kubectl delete sa/kubecube-pre-job -n kubecube-system
kubectl delete clusterRole/kubecube-pre-job 
kubectl delete clusterRoleBinding/kubecube-pre-job
kubectl delete ns kubecube-system hnc-system kubecube-monitoring

3.2 - 通过helm纳管计算集群

使用 helm 的方式在计算集群中安装 Warden（KubeCube 的计算集群 agent），安装完成后，Warden 将主动向管控集群的 KubeCube 注册计算集群信息，完成计算集群的纳管。

v1.8.x

要求

已有 K8s 集群环境
helm v3 客户端

下载 KubeCube helm chart 包

KUBECUBE_VERSION=v1.8

curl -s https://kubecube.nos-eastchina1.126.net/kubecube-chart/${KUBECUBE_VERSION}/kubecube-chart.tar.gz | tar -xz

通过 helm 在计算集群上安装 Warden

创建 member-value.yaml 文件并填写必要的 value 值

# member-value.yaml

global:
  # 管控集群的 Node IP，用来暴露 KubeCube 的 NodePort service
  nodeIP: x.x.x.x

  dependencesEnable:
    ingressController: "false" # 如果集群中没有部署 ingress controller，请将此设置为 true
    localPathStorage: "false" # 如果集群中没有部署 local path storage，请将此设置为 true
    metricServer: "false" # # 如果集群中没有部署 metric server，请将此设置为 true

  # 以下为计算集群指定安装组件，请不要修改
  componentsEnable:
    kubecube: "false"
    warden: "true"
    audit: "false"
    webconsole: "false"
    cloudshell: "false"
    frontend: "false"
    
  # 如果要启动日志功能，请假以下值设置为 "enabled"
  hotPlugEnable:
    common:
      logagent: "disabled"

  localKubeConfig: xx # 当前集群的 kubeconfig 的 base64
  pivotKubeConfig: xx # 管控集群的 kubeconfig 的 base64

warden:
  containers:
    warden:
      args:
        inMemberCluster: true
        cluster: "member-cluster"  # 集群集群名

部署 Warden

helm install warden -n kubecube-system --create-namespace ./kubecube-chart -f ./pivot-value.yaml

卸载计算集群中的 warden

注意：not found 错误可以被忽略

在管控面删除 cluster cr

kubectl delete cluster {计算集群名}

在计算集群手动清理 webhook

kubectl delete validatingwebhookconfigurations kubecube-validating-webhook-configuration warden-validating-webhook-configuration kubecube-monitoring-admission

在计算集群卸载 warden chart

helm uninstall warden -n kubecube-system

再计算集群清理残留资源

kubectl delete sa/kubecube-pre-job -n kubecube-system
kubectl delete clusterRole/kubecube-pre-job 
kubectl delete clusterRoleBinding/kubecube-pre-job
kubectl delete ns kubecube-system hnc-system kubecube-monitoring

4 - 使用 hotplug 进行拓展

4.1 - 热插拔 hotplug 介绍

热插拔是通过修改配置文件实现不停机更新监控、日志、审计等组件的启停和配置。

热插拔

热插拔的实现是基于 helm ，因此集群需要预先安装好 helm3 版本。

登录管控 k8s 集群，执行命令可以查看热插拔配置

# kubectl get hotplug
NAME            PHASE     AGE
common          running   23h
pivot-cluster   running   16d

其中：

common 表示公共的热插拔配置，pivot-cluster 表示 pivot-cluster 这个 k8s 集群的热插拔配置。
允许自定义各个集群的热插拔配置覆盖 common 热插拔配置实现个性化配置 k8s 集群组件热插拔。
热插拔配置的 .metadata.name 要求与k8s集群名称一致，所有 k8s 集群信息查看命令为 kubectl get cluster。

Common配置说明

目前，默认的 common 配置包括以下几个组件：

logseer：日志管理组件，仅在管控集群安装

logagent：日志采集代理组件

kubecube-monitoring：监控 prometheuse 组件

kubecube-thanos：监控 thanos 组件，仅在管控集群安装

示例如下：

apiVersion: hotplug.kubecube.io/v1
kind: Hotplug
metadata:
  annotations:
    kubecube.io/sync: "true"					## 同步信号，kubecube会将这个配置同步到各个集群
  name: common                                  ## 公共配置为cmmon，其余集群特殊配置为集群名字
spec:
  component:
  - name: audit                                 ## 审计日志
    status: disabled
  - name: logseer                               ## 日志管理组件
    namespace: logseer
    pkgName: logseer-v1.0.0.tgz
    status: disabled                            ## 启停标识：这里disabled为禁用
  - name: logagent                              ## 日志采集代理组件
    namespace: logagent
    pkgName: logagent-v1.0.0.tgz
    status: enabled                             ## 启停标识：这里enabled为启用
    env: |                                      ## 环境变量
      clustername: "{{.cluster}}"               ## {{.cluster}} 程序会自动注入集群名字替换
  - name: kubecube-monitoring                   ## 监控组件
    namespace: kubecube-monitoring
    pkgName: kubecube-monitoring-15.4.7.tgz
    status: enabled
    env: |
      grafana:
        enabled: false
      prometheus:
        prometheusSpec:
          externalLabels:
            cluster: "{{.cluster}}"
          remoteWrite:
          - url: http://10.173.32.42:31291/api/v1/receive      
  - name: kubecube-thanos                       ## 监控thanos组件
    namespace: kubecube-monitoring
    pkgName: thanos-3.18.0.tgz
    status: disabled
status:                                         ## message显示各个组件运行状态，phase显示总体运行状态
  message: '{"kubecube-monitoring":"release is running","kubecube-thanos":"release    
    is running","logagent":"release is running","logseer":"release is running"}'
  phase: running

每一个组件基本包含5个要素，在 spec.component 下：

name：组件名称

namespace：指定组件部署的命名空间，若指定的命名空间不存在，会自动以该字段值去创建一个命名空间。

pkgName：安装包名称，安装包默认存放路径为 warden 容器里的 /root/helmchartpkg，以 emptydir 形式存在。

status：组件是否启用

env：环境变量配置

每一个要素都可以使用集群独特配置进行覆盖，未覆盖的要素则依然使用 common 里的配置。

管控集群配置说明

Pivot-cluster 配置是管控集群配置，即 KubeCube 所在的集群。集群独特的配置会与 common 的配置结合，用于个性化配置集群组件，结合时遇到相同字段pivot-cluster 优先。

apiVersion: hotplug.kubecube.io/v1
kind: Hotplug
metadata:
  annotations:
    kubecube.io/sync: "true"            ## 同步信号，kubecube会将这个配置同步到其他集群
  name: pivot-cluster                   ## 与集群名字一致，指明这是pivot-cluster这个集群的热插拔配置
spec:
  component:
  - name: logseer                       ## 日志管理组件
    status: enabled                     ## 结合common设置为disabled，这里设置为enabled，标识其余集群不启用，pivot-cluster集群启用
  - name: kubecube-monitoring
    env: |
      grafana:
        enabled: true
      prometheus:
        prometheusSpec:
          externalLabels:
            cluster: "{{.cluster}}"
          remoteWrite:
          - url: http://thanos-receive:19291/api/v1/receive      
  - name: kubecube-thanos
    status: enabled
    env: |
      receive:
        replicaCount: 1
        replicationFactor: 1      
status:
  message: '{"kubecube-monitoring":"release is running","kubecube-thanos":"release
    is running","logagent":"release is running","logseer":"release is running"}'
  phase: running

4.2 - 通过 hotplug 启用 loggie

KubeCube 使用 hotplug 热插拔方式集成 Loggie，用户可以通过打开 hotplug 中关于日志部分的开关来开启 Loggie。我们有两个时间点可以用来开启 Loggie。

在使用 Helm 安装 KubeCube 时

在使用 Helm 安装 KubeCube 时，我们可以设置以下 values 来开启 Loggie。

安装管控集群时的参数如下：

# pivot-value.yaml
...
global:
  # set "enabled" if wanna open log application.
  hotPlugEnable:
    pivot:
      logseer: "enabled" 
      logagent: "enabled"
      elasticsearch: "enabled"
...

安装计算集群时的参数如下：

# member-value.yaml
...
global:    
  # set "enabled" if wanna open log application.
  hotPlugEnable:
    common:
      logagent: "enabled"
...

在使用过程中

在使用过程中，我们也可以通过直接修改 hotplug 的方式来开启 Loggie。

修改管控集群的 hotplug 如下：

kubectl edit hotplug pivot-cluster

# pivot-cluster
apiVersion: hotplug.kubecube.io/v1
kind: Hotplug
metadata:
  name: pivot-cluster
spec:
  component:
  - name: elasticsearch
    namespace: elasticsearch
    pkgName: elasticsearch-7.8.1.tgz
    status: enabled # 将该值设为 enabled 来开启日志
...
  - name: logseer
    status: enabled # 将该值设为 enabled 来开启日志
  - name: logagent
    status: enabled # 将该值设为 enabled 来开启日志

修改计算集群的 hotplug 如下：

kubectl edit hotplug common

# common
apiVersion: hotplug.kubecube.io/v1
kind: Hotplug
metadata:
  name: common
spec:
  component:
...
  - env: |
      clustername: "{{.cluster}}"
      elasticsearch:
        address: x.x.x.x:32200 # 填写管控集群的 es 的 nodeport svc 访问地址，一般为 {nodeIP}:32200
    name: logagent
    namespace: logagent
    pkgName: logagent-1.3.0.tgz
    status: enabled # 将该值设为 enabled 来开启日志

5 - 使用脚本安装 K8s 和 KubeCube（已废弃）

5.1 - 在已有k8s集群中部署KubeCube

v1.4.x

在 Kubernetes 集群中部署 KubeCube

开始安装

在 Linux 机器上执行部署脚本

KUBECUBE_VERSION=v1.4

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置安装脚本参数

该安装模式下，需要修改以下参数：

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"

# if install k8s
INSTALL_KUBERNETES="false"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

complete-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

v1.2.x

在 Kubernetes 集群中部署 KubeCube

开始安装

在 Linux 机器上执行部署脚本

KUBECUBE_VERSION=v1.2

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置安装脚本参数

该安装模式下，需要修改以下参数：

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"

# if install k8s
INSTALL_KUBERNETES="false"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

complete-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

v1.1.x

在 Kubernetes 集群中部署 KubeCube

开始安装

在 Linux 机器上执行部署脚本

KUBECUBE_VERSION=v1.1

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置安装脚本参数

该安装模式下，需要修改以下参数：

INSTALL_KUBECUBE_MEMBER=“false”

MASTER_IP="${node ip}"

${node ip} 表示你运行脚本所在 node 机器的 ip，该 node 需要可操作 kubectl

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"

# if install k8s
INSTALL_KUBERNETES="false"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

complete-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

v1.0.x

在 Kubernetes 集群中部署 KubeCube

⚠️修改 Kubernetes API-Server 配置

必要性

KubeCube 对多集群提供统一的认证和鉴权服务，需要使用 k8s api-server 的 auth-webhook 能力来做拓展。
KubeCube 提供对 k8s-apiserver 日志进行审计的能力，这需要为 k8s api-server 指定审计服务后端。

修改操作

如果您的 k8s api-server 服务是以 deployment 形式运行的，请直接修改 deployment ；如果您的 k8s api-server 服务是以 static pod 形式运行的，您需要修改对应的 manifest 文件，它的文件路径通常为 /etc/kubernetes/manifests/kube-apiserver.yaml ，修改内容如下：

apiVersion: v1
kind: Pod
metadata:
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
    - command:
        - kube-apiserver
        - --audit-log-format=json
        - --audit-log-maxage=10
        - --audit-log-maxbackup=10
        - --audit-log-maxsize=100
        - --audit-log-path=/var/log/audit
        - --audit-policy-file=/etc/cube/audit/audit-policy.yaml
        - --audit-webhook-config-file=/etc/cube/audit/audit-webhook.config
        - --authentication-token-webhook-config-file=/etc/cube/warden/webhook.config
      name: kube-apiserver
      volumeMounts:
        - mountPath: /var/log/audit
          name: audit-log
        - mountPath: /etc/cube
          name: cube
          readOnly: true
  volumes:
    - hostPath:
        path: /var/log/audit
        type: DirectoryOrCreate
      name: audit-log
    - hostPath:
        path: /etc/cube
        type: DirectoryOrCreate
      name: cube

开始安装

在 Linux 机器上执行部署脚本

KUBECUBE_VERSION=v1.0

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置安装脚本参数

该安装模式下，需要修改以下参数：

INSTALL_KUBECUBE_MEMBER=“false”

MASTER_IP="${node ip}"

${node ip} 表示你运行脚本所在 node 机器的 ip，该 node 需要可操作 kubectl

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# if install k8s
INSTALL_KUBERNETES="false"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# master ip means master node ip of cluster
MASTER_IP="x.x.x.x"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
KUBERNETES_VERSION="1.20.9"

# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

等待部署完成

KubeCube 部署完成后，请根据提示信息登陆 console 管理页面

complete-deploy

使用 admin 账户登陆 console

⚠️请在登陆后修改 admin 用户的密码

5.2 - 添加计算集群

KubeCube 可以添加其它集群作为计算集群，前提是，计算集群能够访问管控集群的 k8s api-server 和 KubeCube，默认情况下 KubeCube 使用 NodePort 对外暴露服务，用户可自行使用 ingress 进行暴露

v1.4.x

方式一：部署新集群并添加

在 linux 机器上，需要构建 Kubernetes 集群并安装 KubeCube 依赖项

开始安装

KUBECUBE_VERSION=v1.4

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置安装脚本参数

该安装模式下，需要修改以下参数：

INSTALL_KUBECUBE_PIVOT=“false”

INSTALL_KUBECUBE_MEMBER=“true”

INSTALL_KUBERNETES=“true”

MEMBER_CLUSTER_NAME=“member-1”

MASTER_IP="${node ip}"

KUBECUBE_HOST="${pivot node ip}"

MEMBER_CLUSTER_NAME 表示计算集群的名字，注意，不能与已有的计算集群名称同名, ${node ip} 表示你运行脚本所在 node 机器的 ip，该 node 需要可操作 kubectl, ${pivot node ip} 表示管控集群 node 机器的 ip，用于向 KubeCube 注册集群

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="true"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST="y.y.y.y"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME="member-1"

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"

方式二：纳管已有集群

添加已有集群只需从 console 页面导入集群信息即可

在 console 页面中导入集群信息

add-member-cluster

在 console 中确认新集群

check-cluster

v1.2.x

方式一：部署新集群并添加

在 linux 机器上，需要构建 Kubernetes 集群并安装 KubeCube 依赖项

开始安装

KUBECUBE_VERSION=v1.2

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置安装脚本参数

该安装模式下，需要修改以下参数：

INSTALL_KUBECUBE_PIVOT=“false”

INSTALL_KUBECUBE_MEMBER=“true”

INSTALL_KUBERNETES=“true”

MEMBER_CLUSTER_NAME=“member-1”

MASTER_IP="${node ip}"

KUBECUBE_HOST="${pivot node ip}"

MEMBER_CLUSTER_NAME 表示计算集群的名字，注意，不能与已有的计算集群名称同名, ${node ip} 表示你运行脚本所在 node 机器的 ip，该 node 需要可操作 kubectl, ${pivot node ip} 表示管控集群 node 机器的 ip，用于向 KubeCube 注册集群

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="true"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST="y.y.y.y"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME="member-1"

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

方式二：纳管已有集群

添加已有集群只需从 console 页面导入集群信息即可

在 console 页面中导入集群信息

add-member-cluster

在 console 中确认新集群

check-cluster

v1.1.x

方式一：部署新集群并添加

在 linux 机器上，需要构建 Kubernetes 集群并安装 KubeCube 依赖项

开始安装

KUBECUBE_VERSION=v1.1

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置安装脚本参数

该安装模式下，需要修改以下参数：

INSTALL_KUBECUBE_PIVOT=“false”

INSTALL_KUBECUBE_MEMBER=“true”

INSTALL_KUBERNETES=“true”

MEMBER_CLUSTER_NAME=“member-1”

MASTER_IP="${node ip}"

KUBECUBE_HOST="${pivot node ip}"

MEMBER_CLUSTER_NAME 表示计算集群的名字，注意，不能与已有的计算集群名称同名, ${node ip} 表示你运行脚本所在 node 机器的 ip，该 node 需要可操作 kubectl, ${pivot node ip} 表示管控集群 node 机器的 ip，用于向 KubeCube 注册集群

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="true"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST="y.y.y.y"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME="member-1"

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

方式二：纳管已有集群

添加已有集群只需从 console 页面导入集群信息即可

在 console 页面中导入集群信息

add-member-cluster

在 console 中确认新集群

check-cluster

v1.0.x

方式一：部署新集群并添加

在 linux 机器上，需要构建 Kubernetes 集群并安装 KubeCube 依赖项

开始安装

KUBECUBE_VERSION=v1.0

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置安装脚本参数

该安装模式下，需要修改以下参数：

INSTALL_KUBECUBE_PIVOT=“false”

INSTALL_KUBECUBE_MEMBER=“true”

INSTALL_KUBERNETES=“true”

MEMBER_CLUSTER_NAME=“member-1”

MASTER_IP="${node ip}"

KUBECUBE_HOST="${pivot node ip}"

MEMBER_CLUSTER_NAME 表示计算集群的名字，注意，不能与已有的计算集群名称同名 ${node ip} 表示你运行脚本所在 node 机器的 ip，该 node 需要可操作 kubectl ${pivot node ip} 表示管控集群 node 机器的 ip，用于向 KubeCube 注册集群

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="true"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME="member-1"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# master ip means master node ip of cluster
MASTER_IP="x.x.x.x"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST="y.y.y.y"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
KUBERNETES_VERSION="1.20.9"

# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

方式二：纳管已有集群

添加已有集群需要从 console 页面获取添加集群的定制脚本

在 console 页面中获取添加集群的脚本

add-member-cluster

使用脚本添加集群

在集群的 node 机器上，使用从 console 中下载的脚本，该机器需要能够执行 kubectl

/bin/bash add_cluster.sh

等待集群添加完成

add-cluster

在 console 中确认新集群

check-cluster

5.3 - 添加节点

KubeCube 提供为已有集群添加节点的能力，同时也支持使用 kubeadm 的原生方式添加节点

⚠️ 注意通过 KubeCube 的脚本添加节点时，node 机器需要能够通过 ssh 访问 master 机器，支持公钥和密码两种 ssh 方式，执行脚本前可以在 node 上 ssh 到 master 测试连通性

v1.4.x

向集群添加工作节点

在新节点上执行部署脚本

KUBECUBE_VERSION=v1.4

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待新节点加入集群

MASTER_IP 为 master 节点 ip

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"

向集群的 control-plane 添加 master 节点

在新节点上执行部署脚本

KUBECUBE_VERSION=v1.4

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待新节点加入 control-plane

MASTER_IP 需要填已有的 master 节点 ip
NODE_MODE 当该模式为 node-join-control-plane 时，需要指定 master 节点们的 CONTROL_PLANE_ENDPOINT(vip)
CONTROL_PLANE_ENDPOINT 为高可用 vip
可以根据需要选择适合自己的 ssh 方式

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP="y.y.y.y"

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

v1.2.x

向集群添加工作节点

在新节点上执行部署脚本

KUBECUBE_VERSION=v1.2

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待新节点加入集群

MASTER_IP 为 master 节点 ip

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

向集群的 control-plane 添加 master 节点

在新节点上执行部署脚本

KUBECUBE_VERSION=v1.2

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待新节点加入 control-plane

MASTER_IP 需要填已有的 master 节点 ip
NODE_MODE 当该模式为 node-join-control-plane 时，需要指定 master 节点们的 CONTROL_PLANE_ENDPOINT(vip)
CONTROL_PLANE_ENDPOINT 为高可用 vip
可以根据需要选择适合自己的 ssh 方式

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP="y.y.y.y"

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

v1.1.x

向集群添加工作节点

在新节点上执行部署脚本

KUBECUBE_VERSION=v1.1

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

tip: 你可以通过预先下载离线包和镜像来减少部署时间 export PRE_DOWNLOAD="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待新节点加入集群

MASTER_IP 为 master 节点 ip

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP="y.y.y.y"

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

向集群的 control-plane 添加 master 节点

在新节点上执行部署脚本

KUBECUBE_VERSION=v1.1

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待新节点加入 control-plane

MASTER_IP 需要填已有的 master 节点 ip
NODE_MODE 当该模式为 node-join-control-plane 时，需要指定 master 节点们的 CONTROL_PLANE_ENDPOINT(vip)
CONTROL_PLANE_ENDPOINT 为高可用 vip
可以根据需要选择适合自己的 ssh 方式

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP="y.y.y.y"

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

v1.0.x

向集群添加工作节点

在新节点上执行部署脚本

KUBECUBE_VERSION=v1.0

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

tip: 你可以通过预先下载离线包和镜像来减少部署时间 export PRE_DOWNLOAD="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待新节点加入集群

MASTER_IP 为 master 节点 ip

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
KUBERNETES_VERSION="1.20.9"

# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

向集群的 control-plane 添加 master 节点

在新节点上执行部署脚本

KUBECUBE_VERSION=v1.0

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待新节点加入 control-plane

MASTER_IP 需要填已有的 master 节点 ip
CONTROL_PLANE_ENDPOINT 为高可用 vip

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-control-plane"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.10" #{ip}:{port} , dns

# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
KUBERNETES_VERSION="1.20.9"

# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

5.4 - 多节点高可用部署

本文提供 Kubernetes 的高可用部署和 KubeCube 的高可用部署方案，VIP 的实现需要用户自行提供

v1.4.x

主机规划

IP 地址	主机名	角色
10.173.32.2	lb1	Keepalived & HAproxy
10.173.32.3	lb2	Keepalived & HAproxy
10.173.32.4	master1	master, etcd
10.173.32.5	master2	master, etcd
10.173.32.6	master3	master, etcd
10.173.32.7	worker1	worker
10.173.32.8	worker2	worker
10.173.32.9	worker3	worker
10.173.32.10		vip 地址

⚠️master2、master3、worker1、worker2、worker3 需要能够通过密钥或者密码 ssh 访问 master1

部署高可用 Kubernetes

KubeCube 部署脚本提供部署高可用 k8s 的能力，当然，你也可以使用其他工具搭建高可用的 k8s 集群

开始安装

在 master1 上执行部署脚本

KUBECUBE_VERSION=v1.4

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 Kubernetes 安装完成，master2 和 master3 加入 control-plane 的方式与之相同

CONTROL_PLANE_ENDPOINT 为高可用 k8s-apiserver 的 vip，在此我们用任意 master 节点的 ip 代替

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.4" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

worker1 作为工作节点加入集群

在 worker1 上执行部署脚本

KUBECUBE_VERSION=v1.4

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 worker1 加入集群，worker2 和 worker3 加入集群的方式与之相同

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"

部署高可用 KubeCube

在 master1 上执行部署脚本

KUBECUBE_VERSION=v1.4

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 KubeCube 部署完成

install.conf

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"

# if install k8s
INSTALL_KUBERNETES="false"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="docker"

v1.2.x

主机规划

IP 地址	主机名	角色
10.173.32.2	lb1	Keepalived & HAproxy
10.173.32.3	lb2	Keepalived & HAproxy
10.173.32.4	master1	master, etcd
10.173.32.5	master2	master, etcd
10.173.32.6	master3	master, etcd
10.173.32.7	worker1	worker
10.173.32.8	worker2	worker
10.173.32.9	worker3	worker
10.173.32.10		vip 地址

⚠️master2、master3、worker1、worker2、worker3 需要能够通过密钥或者密码 ssh 访问 master1

部署高可用 Kubernetes

KubeCube 部署脚本提供部署高可用 k8s 的能力，当然，你也可以使用其他工具搭建高可用的 k8s 集群

开始安装

在 master1 上执行部署脚本

KUBECUBE_VERSION=v1.2

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 Kubernetes 安装完成，master2 和 master3 加入 control-plane 的方式与之相同

CONTROL_PLANE_ENDPOINT 为高可用 k8s-apiserver 的 vip，在此我们用任意 master 节点的 ip 代替

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.4" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

worker1 作为工作节点加入集群

在 worker1 上执行部署脚本

KUBECUBE_VERSION=v1.2

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 worker1 加入集群，worker2 和 worker3 加入集群的方式与之相同

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

部署高可用 KubeCube

在 master1 上执行部署脚本

KUBECUBE_VERSION=v1.2

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 KubeCube 部署完成

install.conf

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"

# if install k8s
INSTALL_KUBERNETES="false"

# k8s cni, support now is calico only
CNI="calico"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.19.13, 1.20.9, 1.21.2, 1.22.2, 1.23.5
KUBERNETES_VERSION="1.23.5"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# +optional
# KUBERNETES_BIND_ADDRESS generally is node_ip
# can be set when NODE_MODE="master" ot "control-plane-master"
# default value is $(hostname -I |awk '{print $1}')
KUBERNETES_BIND_ADDRESS="" #{node_ip}

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

#######################################################################
# container runtime config
# if value is docker, then use docker as container runtime
# else if value is containerd, then use containerd as container runtime
#######################################################################
CONTAINER_RUNTIME="containerd"

v1.1.x

主机规划

IP 地址	主机名	角色
10.173.32.2	lb1	Keepalived & HAproxy
10.173.32.3	lb2	Keepalived & HAproxy
10.173.32.4	master1	master, etcd
10.173.32.5	master2	master, etcd
10.173.32.6	master3	master, etcd
10.173.32.7	worker1	worker
10.173.32.8	worker2	worker
10.173.32.9	worker3	worker
10.173.32.10		vip 地址

⚠️master2、master3、worker1、worker2、worker3 需要能够通过密钥或者密码 ssh 访问 master1

部署高可用 Kubernetes

KubeCube 部署脚本提供部署高可用 k8s 的能力，当然，你也可以使用其他工具搭建高可用的 k8s 集群

开始安装

在 master1 上执行部署脚本

KUBECUBE_VERSION=v1.1

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 Kubernetes 安装完成，master2 和 master3 加入 control-plane 的方式与之相同

CONTROL_PLANE_ENDPOINT 为高可用 k8s-apiserver 的 vip，在此我们用任意 master 节点的 ip 代替

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.4" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP=""

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

worker1 作为工作节点加入集群

在 worker1 上执行部署脚本

KUBECUBE_VERSION=v1.1

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 worker1 加入集群，worker2 和 worker3 加入集群的方式与之相同

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

部署高可用 KubeCube

在 master1 上执行部署脚本

KUBECUBE_VERSION=v1.1

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 KubeCube 部署完成

install.conf

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"

# if install k8s
INSTALL_KUBERNETES="false"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
# support now is: 1.20.9, 1.19.13, 1.18.20, 1.21.2
KUBERNETES_VERSION="1.20.9"

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

#######################################################################
# member cluster config
# used when INSTALL_KUBECUBE_MEMBER="true"
#######################################################################

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

#######################################################################
# ssh config
# used when NODE_MODE="node-join-master" or node-join-control-plane
#######################################################################

# +optional
# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"

# +optional
# the user who can access master node, it can be empty
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

#######################################################################
# offline config
# used when offline install choose, must lift offline pkg first
#######################################################################

OFFLINE_INSTALL="false"

OFFLINE_PKG_PATH=""

cube.conf

将kubecube_replicas设置为3，使得 KubeCube 使用 3 副本部署，并且由于podAntiAffinity

# custom values for kubecube

kubecube_replicas=3
kubecube_args_logLevel="info"

v1.0.x

主机规划

IP 地址	主机名	角色
10.173.32.2	lb1	Keepalived & HAproxy
10.173.32.3	lb2	Keepalived & HAproxy
10.173.32.4	master1	master, etcd
10.173.32.5	master2	master, etcd
10.173.32.6	master3	master, etcd
10.173.32.7	worker1	worker
10.173.32.8	worker2	worker
10.173.32.9	worker3	worker
10.173.32.10		vip 地址

⚠️master2、master3、worker1、worker2、worker3 需要能够通过密钥或者密码 ssh 访问 master1

部署高可用 Kubernetes

开始安装

在 master1 上执行部署脚本

KUBECUBE_VERSION=v1.0

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 Kubernetes 安装完成

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.10" #{ip}:{port} , dns

# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
KUBERNETES_VERSION="1.20.9"

# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

master2 节点加入 control-plane

在 master2 上执行部署脚本

KUBECUBE_VERSION=v1.0

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 master2 加入 control-plane

master3 加入 control-plane 与此类似，仅需修改 LOCAL_IP为10.173.32.6

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-control-plane"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="10.173.32.10" #{ip}:{port} , dns

# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
KUBERNETES_VERSION="1.20.9"

# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

worker1 作为工作节点加入集群

在 worker1 上执行部署脚本

KUBECUBE_VERSION=v1.0

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 worker1 加入集群

worker2 和 worker3 加入集群的方式与之类似，仅需修改LOCAL_IP为本机 IP 即可

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="false"

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# if install k8s
INSTALL_KUBERNETES="true"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="node-join-master"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
KUBERNETES_VERSION="1.20.9"

# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

部署高可用 KubeCube

在 master1 上执行部署脚本

KUBECUBE_VERSION=v1.0

export CUSTOMIZE="true";curl -fsSL https://kubecube.nos-eastchina1.126.net/kubecube-installer/${KUBECUBE_VERSION}/entry.sh | bash

设置脚本参数，并按照提示继续运行安装脚本并等待 KubeCube 部署完成

install.conf

# if install kubecube on pivot cluster
INSTALL_KUBECUBE_PIVOT="true"

# if install kubecube on member cluster
INSTALL_KUBECUBE_MEMBER="false"

# if install k8s
INSTALL_KUBERNETES="false"

# there are four node mode below:
# "master" : node will be installed as a master of cluster
# "node-join-master" : node will be install as a worker of cluster to join master
# "control-plane-master" : node will be installed as a master to control plane of cluster
# "node-join-control-plane" : node will be installed as a master to join control plane
NODE_MODE="control-plane-master"

# +optional
# must be set when INSTALL_KUBECUBE_MEMBER="true"
# this value is the name of member cluster you
# want to take over
MEMBER_CLUSTER_NAME=""

# +optional
# must be set when NODE_MODE="control-plane-master"
# or "node-join-control-plane"
CONTROL_PLANE_ENDPOINT="" #{ip}:{port} , dns

# master ip means master node ip of cluster
MASTER_IP="10.173.32.4"

# +optional
# KUBECUBE_HOST must be set when as a member cluster to
# join pivot cluster, the value is pivot node ip
KUBECUBE_HOST=""

# zone has two choice
# 1. "cn" : in mainland
# 2. "others" : out of mainland
ZONE="cn"

# k8s version you want to install
KUBERNETES_VERSION="1.20.9"

# +optional
# the user who can access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_USER="root"

# +optional
# the port specified to access master node, it can be empty
# when NODE_MODE="master" or "control-plane-master"
SSH_PORT=22

# +optional
# must be empty when ACCESS_PRIVATE_KEY_PATH set
# password for master user to access master node
ACCESS_PASSWORD=""

# +optional
# must be empty when ACCESS_PASSWORD set
# ACCESS_PRIVATE_KEY for master user to access master node
ACCESS_PRIVATE_KEY_PATH="/root/.ssh/id_rsa"

cube.conf

将kubecube_replicas设置为3，使得 KubeCube 使用 3 副本部署，并且由于podAntiAffinity，它们会运行在非controlPlane的节点上，并且每个节点仅运行单个副本

# custom values for kubecube

kubecube_replicas=3
kubecube_args_logLevel="info"

6 - 已有系统接入

6.1 - Prometheus

KubeCube 在部署时会自动安装 Prometheus 等监控组件，以实现监控功能。本文档介绍了如何在 KubeCube 中接入用户已有的 Prometheus。

准备工作

在集群部署好 Prometheus，Prometheus 可以正常监控到集群资源的数据；
在集群中部署好 KubeCube；
说明：部署好 KubeCube 后，由于集群内已有 Prometheus operator，多个 operator 会导致集群内 Prometheus 相关功能不可用，需要卸载 KubeCube 监控组件或删除本地 operator。
以平台管理员角色登录 KubeCube 管控集群。

步骤

1、添加 Label

由于 KubeCube 需要实现多集群监控，因此在 KubeCube 查询监控数据时，都会在 query 表达式中添加 cluster={clusterName} 来进行集群过滤。用户需要在 Prometheus 的 exporter 中添加这一 label，前端查询监控数据时才能查询到结果。

2、卸载 KubeCube 监控组件

方式一页面操作：

点击页面右上角【切换到控制台】，点击任意空间，进入到控制台页面；
在左侧菜单栏点击【自定义资源CRD】，进入到集群级别 CRD 列表，可以点击右上方输入 “hotplug” 进行搜索，找到 “hotplugs.hotplug.kubecube.io” CRD，点击【v1】版本进入 CRD 详情页；
选择 common 实例，点击【设置YAML】，找到 spec.component. name=kubecube-monitoring，将 “status” 改成 “disabled”，即卸载 KubeCube 自带监控组件。

方式二命令行操作：

kubectl edit hotplug common
找到 spec.component. name=kubecube-monitoring，将 “status” 改成 “disabled”。

详细配置说明见热插拔。

3、部署 ServiceMonitor

查看集群资源

查看控制台内监控数据，需要部署两个 ServiceMonitor：kubelet 和 kube-state-metrics 的 ServiceMonitor，样例如下：

部署 kubecube-monitoring-kubelet

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: kubecube-monitoring-kubelet
  namespace: kubecube-monitoring
spec:
  endpoints:
  - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
    honorLabels: true
    port: https-metrics
    relabelings:
    - sourceLabels:
      - __metrics_path__
      targetLabel: metrics_path
    scheme: https
    tlsConfig:
      caFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      insecureSkipVerify: true
  - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
    honorLabels: true
    path: /metrics/cadvisor
    port: https-metrics
    relabelings:
    - sourceLabels:
      - __metrics_path__
      targetLabel: metrics_path
    scheme: https
    tlsConfig:
      caFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      insecureSkipVerify: true
  - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
    honorLabels: true
    path: /metrics/probes
    port: https-metrics
    relabelings:
    - sourceLabels:
      - __metrics_path__
      targetLabel: metrics_path
    scheme: https
    tlsConfig:
      caFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
      insecureSkipVerify: true
  jobLabel: k8s-app
  namespaceSelector:
    matchNames:
    - kube-system
  selector:
    matchLabels:
      k8s-app: kubelet

部署 kubecube-monitoring-kube-state-metrics

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: kubecube-monitoring-kube-state-metrics
  namespace: kubecube-monitoring
spec:
  endpoints:
  - honorLabels: true
    port: http
  selector:
    matchLabels:
      app.kubernetes.io/instance: kubecube-monitoring
      app.kubernetes.io/name: kube-state-metrics

查看组件监控

当前 KubeCube 平台支持对组件的监控视图可视化查询，详细说明见平台组件监控。接入外部监控后，用户可按需在集群内部署对应组件的 ServiceMonitor。

各个 ServiceMonitor 的 yaml 可参考 https://github.com/kubecube-io/charts/tree/main/kubecube-monitoring/templates/exporters。

4、部署 Dashboard

查看集群资源（控制台内监控数据），需要部署：

查看组件监控，可以部署对应的 Dashboard：

5、修改 Nginx 配置

使用命令行：kubectl edit configmap nginx-config -n kubecube-system
找到原有的地址配置，修改为自有 Prometheus 地址
```
upstream monitoring {
    server kubecube-thanos-query.kubecube-monitoring:9090;
}
```
即将kubecube-thanos-query.kubecube-monitoring:9090 替换为外部地址。
重启 pod：kubectl delete pod frontend-xxxxxx-xxxxx -n kubecube-system

6.2 - ElasticSearch

KubeCube 提供了日志服务和操作审计服务，默认关闭。用户在开启后，日志服务和操作审计服务均会将日志发送到 ElasticSearch 进行存储，由 ElasticSearch 对日志进行管理。用户可以在热插拔中修改配置，安装内部 ElasticSearch，也可以配置外部的 ElasticSearch 地址，对接已有的 ElasticSearch。下面分别介绍如何在这两个功能中接入外部 ElasticSearch。

日志

方式一页面操作：

点击页面右上角【切换到控制台】，点击任意空间，进入到控制台页面；
在左侧菜单栏点击【自定义资源CRD】，进入到集群级别 CRD 列表，可以点击右上方输入 “hotplug” 进行搜索，找到 “hotplugs.hotplug.kubecube.io” CRD，点击【v1】版本进入 CRD 详情页；
选择 common 实例，点击【设置YAML】，找到 spec.component. name=logseer，添加环境变量，如：

- name: logseer
  namespace: logseer
  pkgName: logseer-v1.0.0.tgz
  status: disabled
  env: |
    address: elasticsearch-master.elasticsearch.svc

方式二命令行操作：

kubectl edit hotplug pivot-cluster
找到 spec.component. name=logseer，添加环境变量，同上。

详细配置说明见热插拔。

操作审计

kubectl edit deploy audit -n kubecube-system

添加环境变量：AUDIT_WEBHOOK_HOST、AUDIT_WEBHOOK_INDEX、AUDIT_WEBHOOK_TYPE，如

env:
- name: AUDIT_WEBHOOK_HOST
  value: http://elasticsearch-master.elasticsearch:9200
- name: AUDIT_WEBHOOK_INDEX
  value: audit
- name: AUDIT_WEBHOOK_TYPE
  value: logs

注：如果同时配置了内部和外部 ElasticSearch，审计日志将优先发到外部 ElasticSearch。

其他详细说明见：操作审计。

6.3 - 第三方认证系统

外部认证系统接入

KubeCube 中包含一套自有的认证系统，同时也支持多种类型的外部认证系统的接入。本文档介绍了如何在 KubeCube 中接入 GitHub、Ldap 以及通用认证接口三种认证系统的操作步骤。

GitHub 认证

1. 登记应用信息

在 GitHub 注册一个 Oauth 应用，Homepage URL 和 Authorization callback URL 均填写 http://{kubecube_host}/#/login，创建成功后 Github 生成一个 ClientId，再手动创建一个 Client secret。

2. 修改配置文件

在管控集群修改 configmap：kubectl edit cm kubecube-auth-config -n kubecube-system，修改内容如下：

apiVersion: v1
kind: ConfigMap
data:
  github: |
    enabled: true
    clientId: 80b802dc59eeb847ed00
    clientSecret: 83dc8eb788f706de3449d45e61f45ebdca433de2
    host: http://10.219.196.107:30080

参数说明如下：

参数	说明	类型	默认值
enabled	是否开启 Github 登录	boolean	false
clientId	GitHub 授权的 ClientId	string
clientSecret	GitHub 授权的 Client secret	string
host	KubeCube 服务器地址	string

3. 访问前端登录页面

访问 KubeCube 前端登录页面，选择使用GitHub账号登录；

external-auth

授权应用，点击 Authorize xxxapp，即可登录到 KubeCube。

KubeCube 会使用 Github 返回的信息自动在集群中创建该用户，并标记该用户的 “LOGINTYPE” 为 “github”。

Ldap 认证

KubeCube 支持接入用户已部署的 LDAP 来进行认证。具体步骤如下。

1. 添加启动参数

为 KubeCube Deployment 添加启动参数，参数说明如下：

参数	说明	类型	是否必填	默认值	示例
ldap-is-enable	是否开启 Ldap 登录	boolean	是	false	true
ldap-server	Ldap 服务器地址	string	是		10.219.196.107
ldap-port	Ldap 服务器端口号	string	否	389	389
ldap-base	Ldap 查询分区	string	是		dc=example,dc=com
ldap-admin-user-account	Ldap 管理员账号	string	是		cn=admin,dc=example,dc=com
ldap-admin-password	Ldap 管理员密码	string	是		admin123456
ldap-object-class	Ldap 对象类	string	否	person	person
ldap-login-name-config	用户名所在配置	string	否	uid	cn
ldap-object-category	Ldap objectcategory	string	否		dc=example,dc=com

示例如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kubecube
  namespace: kubecube-system
spec:
  template:
    spec:
      containers:
      - args:
        - -ldap-is-enable=true
        - -ldap-object-class=person
        - -ldap-server=10.219.196.107
        - -ldap-base=dc=example,dc=com
        - -ldap-admin-user-account=cn=admin,dc=example,dc=com
        - -ldap-admin-password=admin123456
        - -ldap-login-name-config=cn

2. 请求登录

接口路径

/api/v1/cube/login

接口方法

POST

请求参数

参数名称	参数说明	参数来源	参数类型	是否必须	备注
name	用户名	`body`	string	是
password	密码	`body`	string	是
loginType	登录方式	`body`	string	是	ldap

响应

如果 Ldap 返回认证成功，KubeCube 在集群中创建该用户，并标记该用户的 “LOGINTYPE” 为 “ldap”。

响应码	状态	描述	响应体
200	OK	请求成功，返回在集群中创建的User信息	User

数据模型

User

参数名称	参数说明	参数类型	是否必须
kind	User	string	是
apiVersion	user.kubecube.io/v1	string	是
metadata	元数据	Metadata	是
spec		UserSpec	是
status		UserStatus	是

Metadata

参数名称	参数说明	参数类型	是否必须	备注
name	该用户在集群中的名称	string	是
labels	标签	map[string]string	否	Ldap返回的真实用户名保存在标签中
creationTimestamp	创建此对象时的时间戳	Time	否
generation	所需状态的特定生成的序列号	integer	否
uid	资源对象在集群中的唯一标识	string	否
selfLink	资源关联的url	string	否

UserSpec

参数名称	参数说明	参数类型	是否必须	备注
loginType	登录方式	string	是	ldap

UserStatus

参数名称	参数说明	参数类型	是否必须	备注
lastLoginTime	上次登录时间	Time	否
lastLoginIP	上次登录IP	string	否

请求示例

curl https://0.0.0.0:7443/api/v1/cube/login -X POST -d '{"name": "test123","password":"123456","loginType":"ldap"}' --header "Content-Type: application/json"

返回示例

{
	"kind": "User",
	"apiVersion": "user.kubecube.io/v1",
  "metadata": {
		"labels": {
			"name": "test123"
		},
	"spec": {
		"loginType": "ldap",
	},
	"status": {
		"lastLoginTime": "2022-07-06T06:25:37Z",
		"lastLoginIP": "10.219.196.107"
	}
}

通用认证

通用认证指的是通过请求第三方接口的方式进行认证。KubeCube 提供了一种通用的接入方式，但对接口的返回有一定要求。具体步骤如下。

1. 接口准备

第三方认证接口要求：

请求 url 为配置的固定值，KubeCube 会将请求携带的 header 进行转发，因此第三方接口对 header 进行认证；
接口返回格式为 map[string]interface{}，其中包含一组值为 key=“name”，value=“{用户名}”。

2. 添加启动参数

为 KubeCube Deployment 添加启动参数，参数说明如下：

参数	说明	类型	是否必填	默认值	示例
generic-auth-is-enable	是否开启通用认证方式	boolean	是	false	true
generic-auth-url	第三方认证url	string	是		https://kubecube123.com/api/v1/demo/auth
generic-auth-method	第三方认证请求方式	string	是		GET
generic-auth-scheme	请求协议	string	否	http	https
generic-auth-insecure-skip-verify	是否跳过安全校验	string	否		true
generic-auth-tls-cert	tls证书	string	否		LS0NGc9PQotLS0tLUVOFURS0tLS0t
generic-auth-tls-key	tls密钥	string	否		LS0NsfGcQotfdLS0tLUVOFURtLS0ta1

示例如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kubecube
  namespace: kubecube-system
spec:
  template:
    spec:
      containers:
      - args:
        - -generic-auth-is-enable=true
        - -generic-auth-url=https://kubecube123.com/api/v1/demo/auth
        - -generic-auth-method=GET
        - -generic-auth-scheme=https
        - -generic-auth-insecure-skip-verify=true

开启通用认证方式后，用户可跳过登录直接访问 KubeCube。KubeCube 会将请求携带的 header 转发给配置的第三方认证平台，由第三方认证平台返回的结果决定请求是否通过认证。