WorkingTipsOnIstioDev

Sample SVC

Create a sample svc using minikube:

# sudo docker save jrelva/nginx-autoindex>autoindex.tar
# eval $(minikube docker-env)
# docker load<autoindex.tar
# kubectl run --image=jrelva/nginx-autoindex:latest nginx-autoindex --port=80 --image-pull-policy=IfNotPresent
deployment "nginx-autoindex" created
# kubectl get deployment
NAME               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
nginx-autoindex    1         1         1            1           6s
# kubectl expose deployment nginx-autoindex --name nginx-autoindex-svc
# kubectl get svc | grep nginx
nginx-autoindex-svc   ClusterIP   10.107.181.75    <none>        80/TCP           29s

Istio Configuration:

# kubectl get svc --all-namespaces | grep istio-ingress
istio-system   istio-ingress          LoadBalancer   10.100.152.241   <pending>     80:30336/TCP,443:32004/TCP  

Istio Ingress

kismatic110tips

preparation

Deployment machine, Download the packages:

# mkdir deploy
# cd deploy
# wget https://github.com/apprenda/kismatic/releases/download/v1.10.0/kismatic-v1.10.0-linux-amd64.tar.gz
# git clone https://github.com/apprenda/kismatic.git
# tar xzvf *.tar.gz;
# ls 
ansible  helm  kismatic  kismatic-master  kismatic-master.zip  kismatic-v1.10.0-linux-amd64.tar.gz  kubectl  provision

Target node(all-in-one), install python-pip, shadowsocks, redsocks, gcc, etc, for acrossing the fucking GFW!

plan

plan the cluster

./kismatic install plan
Plan your Kubernetes cluster:
=> Number of etcd nodes [3]: 1
=> Number of master nodes [2]: 1
=> Number of worker nodes [3]: 1
=> Number of ingress nodes (optional, set to 0 if not required) [2]: 0
=> Number of storage nodes (optional, set to 0 if not required) [0]: 0
=> Number of existing files or directories to be copied [0]: 0

Generating installation plan file template with: 
- 1 etcd nodes
- 1 master nodes
- 1 worker nodes
- 0 ingress nodes
- 0 storage nodes
- 0 files

Wrote plan file template to "kismatic-cluster.yaml"
Edit the plan file to further describe your cluster. Once ready, execute the "install validate" command to proceed.

An empty kismatic-cluster.yaml will be generated, later we will edit it.

validate

validate with detailed information:

./kismatic install validate -o raw

Error:

ansible/bin/ansible-playbook -i ansible/inventory.ini -s ansible/playbooks/preflight.yaml --extra-vars @ansible/clustercatalog.yaml -vvvv
Traceback (most recent call last):
  File "ansible/bin/ansible-playbook", line 36, in <module>
    import shutil
  File "/usr/lib/python3.6/shutil.py", line 10, in <module>
    import fnmatch
  File "/usr/lib/python3.6/fnmatch.py", line 14, in <module>
    import re
  File "/usr/lib/python3.6/re.py", line 142, in <module>
    class RegexFlag(enum.IntFlag):
AttributeError: module 'enum' has no attribute 'IntFlag'
error running playbook: error running ansible: exit status 1

Seems because the python is python3 rather than python2.

Edit the python definition:

# vim ansible/bin/ansible-playbook
    #!/usr/bin/python2

Then your validation will be OK.

install apply

Via following command:

# ./kismatic install apply

WorkingTipsOnPlayWithK8s

Aim

To Write an tutorial for colleagues for learning, they only have to open the browser, by clicking then they could get an automated dev environment.

Environment

play-with-kubernetes blog:

# git clone https://github.com/play-with-docker/play-with-kubernetes.github.io.git
# cd play-with-kubernetes.github.io/
# vim _config.yml
pwkurl: http://192.168.189.114
# docker-compose up

Then open your browser http://192.168.189.114:4000, and you will see the play-with-k8s webpages.

For using the local infrastructure, to configure the play-with-docker with following steps:

# cd /root/go/src/github.com/play-with-docker/play-with-docker
# vim config/config.go
	//flag.StringVar(&DefaultDinDImage, "default-dind-image", "franela/dind", "Default DinD image to use if not specified otherwise")
	flag.StringVar(&DefaultDinDImage, "default-dind-image", "franela/k8s", "Default DinD image to use if not specified otherwise")

While the image we specified here could be the one you added your changes, but default we will use franela/k8s.

The webpage is showed as:

WorkingTipsOnPlayWithDocker2

migration

Really migrate this image into the inner intranet, without any internet connection.

Registry Changing

You have to comment the proxy definition, or your registry instance will restart frequently, thus your dind won’t get working using registry.

# vim /root/data/config.yml
	#proxy:
		# remoteurl: https://registry-1.docker.io
# docker restart docker-registry-proxy-2

systemd definition

Define following two systemd units:

# vim /etc/systemd/system/playwithdocker.service 
[Unit]
Description=playwithdocker
After=docker.service
Requires=docker.service

[Service]
Environment=GOPATH=/root/go/
ExecStart=/usr/bin/docker-compose -f /root/go/src/github.com/play-with-docker/play-with-docker/docker-compose.yml up -d

[Install]
WantedBy=multi-user.target

This unit will start blog service automatically.

# vim /etc/systemd/system/playwithdockerblog.service 
[Unit]
Description=playwithdockerblog
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/bin/docker-compose -f /root/Code/play-with-docker.github.io/docker-compose.yml up -d

[Install]
WantedBy=multi-user.target
# systemctl enable playwithdocker.service
# systemctl enable playwithdockerblog.service

Next time the service will automatically start.

Offline CSS/js

bootstrap fonts:

# wget https://github.com/twbs/bootstrap/archive/v3.3.7.zip
# unzip bootstrap-3.3.7.zip
# cd fonts
# mkdir ~/Code/play-with-docker.github.io/_site/fonts/
# cp * ~/Code/play-with-docker.github.io/_site/fonts/

Then your image will display correctly.

Google Fonts

Download the Fonts description from the website, then put all of the related fonts under your local folder.

dnsmasq

Download the rpm package via:

# yum install yum-plugin-downloadonly
# yum reinstall --downloadonly --downloaddir=/root/rpms dnsmasq

Transfer the package to intranet and install it. Then edit the configuration file of dnsmasq:

# vim /etc/dnsmasq.conf
address=/192.192.189.114/192.192.189.114
# systemclt enable dnsmasq && systemctl start dnsmasq

WorkingTipsOnPlayWithDocker

Environment

Qemu Image

Qemu image preparation:

# mkdir offline-play-with-docker
# cd offline-play-with-docker 
# qemu-img create -f qcow2 offline-play-with-docker.qcow2 200G
Formatting 'offline-play-with-docker.qcow2', fmt=qcow2 size=214748364800 cluster_size=65536 lazy_refcounts=off refcount_bits=16

Virt-manager Networking

Network Name:

Definition of IPV4:

Warning:

Isolation(Could be adjust to isolated):

Create vm and specify vmworks:

Install CentOS 7.4, partition like following:

System

Set the hostname via:

Install mate desktop(for debugging purpose):

# wget -O /etc/yum.repos.d/epel.repo http://mirrors.aliyun.com/repo/epel-7.repo
# yum --enablerepo=epel -y groups install "MATE Desktop"

Install some tools:

# yum install -y vim wget python-pip gcc git nethogs
# pip install shadowsocks
# pip inststall --upgrade pip
# yum install -y libevent-devel
# build the redsocks for crossing the gfw!!!

Now you could cross the firewall for installing the go or other staffs(crossing the gfw WILL let everything more smoothly).

Install docker-ce:

#  yum install -y yum-utils \
  device-mapper-persistent-data \
  lvm2
# yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo
# yum install -y docker-ce

Install docker-compose via pip:

# pip install docker-compose

Start and enable docker:

# systemctl enable docker
# systemctl start docker
# docker swarm init

You have to run docker swarm init, or you won’t login into the play-with-docker.

Install golang:

# yum install -y golang
# mkdir ~/go
# vim ~/.bashrc
export GOPATH=/root/go
export PATH=$PATH:$GOPATH/bin

Create the directory and clone the source code:

# mkdir ~/Code
# cd Code
# git clone https://github.com/play-with-docker/play-with-docker.git

Build the play-with-go:

# go get -u github.com/golang/dep/cmd/dep
# which dep
/root/go/bin/dep
# cd ~/Code/play-with-docker/
# go get -v -d -t ./...
# cd /root/go/src/github.com/play-with-docker/play-with-docker
# dep ensure

Fixed IP

Fixed IP address then we could manually build the dind adjusting to this IP address:

Local Registry

In order to work offline, we have to use local repository.

# mkdir ~/data
# cd ~/data
# docker run -it --rm --entrypoint cat registry:2 /etc/docker/registry/config.yml > config.yml
# vim config.yml
proxy:
      remoteurl: https://registry-1.docker.io
# mkdir ~/data/data
# docker run -d --restart=always -p 5000:5000 --name docker-registry-proxy-2 -v /root/data/config.yml:/etc/docker/registry/config.yml -v /root/data/data:/var/lib/registry registry:2

Now examine the docker registry running:

# docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
0ea38c808e8d        registry:2          "/entrypoint.sh /etc…"   2 seconds ago       Up 1 second         0.0.0.0:5000->5000/tcp   docker-registry-proxy-2

Build dind

dind is docker in docker, which is a docker image for running docker instance inside docker, we build it manually:

# vim
/root/go/src/github.com/play-with-docker/play-with-docker/dockerfiles/dind/daemon.json
    "insecure-registries": ["https://192.192.189.114:5000"],
# vim
/root/go/src/github.com/play-with-docker/play-with-docker/dockerfiles/dind/Dockerfile
......
    /usr/sbin/sshd -o PermitRootLogin=yes -o PrintMotd=no 2>/dev/null && \
    dockerd --registry-mirror=http://192.192.189.114:5000 &>/docker.log 
......

Build the image:

# docker build -t franela/dind:latest .
# docker images | grep dind
franela/dind        latest              7832d23a42c7        About a minute ago   439MB
docker              stable-dind         d303f49c92a7        2 weeks ago          147MB

This dind image could use local registry, so we only need to sync once, then we could let it running really offlinely.

Local play-with-docker

Change the source code:

# cd /root/go/src/github.com/play-with-docker/play-with-docker
# vim handlers/bootstrap.go +64
- return false
+ return true
# vim config/config.go
 flag.StringVar(&PlaygroundDomain, "playground-domain", "192.192.189.114", "Domain to use for the playground")

Make sure your dnsmasq is running, then edit the /etc/dnsmasq.conf via:

.................
# Add local-only domains here, queries in these domains are answered
# from /etc/hosts or DHCP only.
#local=/localnet/
address=/192.192.189.114/192.192.189.114
address=/localhost/127.0.0.1
.................

Run play-with-docker:

# cd /root/go/src/github.com/play-with-docker/play-with-docker
# docker-compose up

Then use a browser to access this website:

Examine the registry now:

Then in host terminal, examine the downloaded registry cache:

curl http://192.192.189.114:5000/v2/_catalog
{"repositories":["library/alpine","library/ubuntu"]}

play-with-docker classroom

Clone the repository:

# cd ~/Code
# git clone https://github.com/play-with-docker/play-with-docker.github.io.git
# cd play-with-docker.github.io/
# vim _config.yml
pwdurl: http://192.192.189.114

# mkdir _site
# groupadd jekyll
# useradd jekyll -m -g jekyll
# chown jekyll:jekyll -R .
# docker-compose up

Now open the browser and see the result:

tips for docker-nginx

# docker run --name docker-nginx -p 8333:80 -d -v /root/gcr:/usr/share/nginx/html jrelva/nginx-autoindex
root@playwithdocker:/etc/systemd/system# cat mynginx.service 
[Unit]
Description=mynginx
Requires=docker.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker start -a docker-nginx
ExecStop=/usr/bin/docker stop -t 2 docker-nginx

[Install]
WantedBy=multi-user.target