TipsOnKubesprayUpgrading
Dec 10, 2018Technology
目的
Rong部署框架随kubespray社区升级策略。
前提
下载kubespray升级包，如v2.8.0:
将此包解压到某目录，这里解压到:
# cd /var1/myimages/RongUpgrade/280
# ls
kubespray-2.8.0
快速同步镜像基准制作
设置一个Rong CentOS7.5 ISO安装后的基础环境, (Base/Rong_Base.qcow2). 制作方法如下（如果已经制作完毕，则可忽略下面步骤):
激活用于存储cache的rpm包:
# vim /etc/yum.conf
keepcache=1
改回原有的仓库配置:
cd /etc/yum.repos.d
mv back/* .
mv kubespray_centos7.repo back/
安装必要的包用于GFW：
yum install -y gcc libevent-devel
redsocks配置，这里就不说了
保存为/var1/myimages/RongUpgrade/Base/Rong_Base.qcow2 , 以后的每次升级都使用该源包，现在创建一个用于升级2.8.0的镜像:
$ qemu-img create -f qcow2 -b Base/Rong_Base.qcow2 v280.qcow2
6核7G的虚拟机配置，选择default网络, 注意更改MAC地址为与上面定义时一致:
获得包/镜像
配置一个用于部署的环境:
# cp deploy.key .
# cp -r inventory/sample inventory/rong
# vim inventory/rong/hosts.ini
[all]
allinone ansible_host=192.168.122.166 ansible_ssh_user=root ansible_ssh_private_key_file=./deploy.key
[kube-deploy]
allinone
[kube-master]
allinone
[etcd]
allinone
[kube-node]
allinone
[k8s-cluster:children]
kube-master
kube-node
部署一次:
# ansible-playbook -i inventory/rong/hosts.ini cluster.yml
在全FQ的环境下，部署应该可以成功，此时备份容器镜像:
# docker images
.....
# docker images | sed -n '1!p' | awk {'print $1":"$2'}
# docker save -o docker.tar gcr.io/google-containers/kube-proxy:v1.12.3 gcr.io/google-containers/kube-apiserver:v1.12.3 gcr.io/google-containers/kube-controller-manager:v1.12.3 gcr.io/google-containers/kube-scheduler:v1.12.3 coredns/coredns:1.2.6 gcr.io/google_containers/cluster-proportional-autoscaler-amd64:1.3.0 gcr.io/google-containers/coredns:1.2.2 gcr.io/google_containers/kubernetes-dashboard-amd64:v1.10.0 quay.io/coreos/etcd:v3.2.24 quay.io/calico/node:v3.1.3 quay.io/calico/ctl:v3.1.3 quay.io/calico/kube-controllers:v3.1.3 quay.io/calico/cni:v3.1.3 nginx:1.13 gcr.io/google-containers/pause:3.1 gcr.io/google_containers/pause-amd64:3.1
# cp docker.tar.gz /mnt
拷贝出的docker.tar即含有所有的容器镜像.
备份安装包,rpm包:
# yum install -y nethogs
# cd /var/cache/
# mkdir -p /root/rpms
# find . | grep rpm$ | xargs -I % cp % /root/rpms/
# cd /root/rpms/
# createrepo .
# cp rpms.tar.gz /mnt
kubespray源码更改
更改配置脚本:
# cp ~/roles/kube-deploy ./roles/kube-deploy
# 替换rpms文件和docker.tar文件
# cd kubespray-2.8.0/roles/kube-deploy/files
# rm -rf kubespray_centos7_rpms
# cp /mnt/rpms ./kubespray_centos7_rpms
# rm -f kubespray_images.tar.xz
# cp /mnt/docker.tar ./kubespray_images.tar
# xz kubespray_images.tar.xz
# vim tag_and_push.sh
更改这里，为我们的docker images名称->私有仓库名称
更改download角色:
# vim ./roles/download/defaults/main.yml
# Download URLs
#kubeadm_download_url: "https://storage.googleapis.com/kubernetes-release/release/{{ kubeadm_version }}/bin/linux/{{ image_arch }}/kubeadm"
#hyperkube_download_url: "https://storage.googleapis.com/kubernetes-release/release/{{ kube_version }}/bin/linux/amd64/hyperkube"
#etcd_download_url: "https://github.com/coreos/etcd/releases/download/{{ etcd_version }}/etcd-{{ etcd_version }}-linux-amd64.tar.gz"
#cni_download_url: "https://github.com/containernetworking/plugins/releases/download/{{ cni_version }}/cni-plugins-{{ image_arch }}-{{ cni_version }}.tgz"
kubeadm_download_url: "http://portus.gggg.com:8888/kubeadm"
hyperkube_download_url: "http://portus.gggg.com:8888/hyperkube"
etcd_download_url: "https://github.com/coreos/etcd/releases/download/{{ etcd_version }}/etcd-{{ etcd_version }}-linux-amd64.tar.gz"
cni_download_url: "http://portus.gggg.com:8888/cni-plugins-{{ image_arch }}-{{ cni_version }}.tgz"
更改私有镜像:
# vim inventory/rong/group_vars/k8s-cluster/k8s-cluster.yml
得到镜像名称:
# cat ./roles/download/defaults/main.yml | grep _image_repo:|grep -v kube_image_repo
如:
etcd_image_repo: "quay.io/coreos/etcd"
flannel_image_repo: "quay.io/coreos/flannel"
flannel_cni_image_repo: "quay.io/coreos/flannel-cni"
calicoctl_image_repo: "quay.io/calico/ctl"
calico_node_image_repo: "quay.io/calico/node"
加前缀:
etcd_image_repo: "xxxxx/quay.io/coreos/etcd"
....
更改cluster.yml,省略docker-ce的安装:
- hosts: k8s-cluster:etcd:calico-rr:!kube-deploy
any_errors_fatal: "{{ any_errors_fatal | default(true) }}"
gather_facts: false
- { role: kubernetes/preinstall, tags: preinstall }
#- { role: "container-engine", tags: "container-engine", when: deploy_container_engine|default(true) }
- { role: download, tags: download, when: "not skip_downloads" }
environment: "{{proxy_env}}"
更改deploy-centos的配置:
# roles/bootstrap-os/tasks/bootstrap-centos.yml
- name: Configure intranet repository
shell: mkdir -p /etc/yum.repos.d/back && mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/back; curl http://portus.teligen.com:8888/kubespray_centos7.repo>/etc/y
um.repos.d/kubespray_centos7.repo && yum makecache && systemctl stop firewalld ; systemctl disable firewalld
- name: Install packages requirements for bootstrap
yum:
.....
验证：
