WorkingTipsOnmvisor

Hardware/OS/Software

nuc11 running Ubuntu 22.04:

dash@dash-NUC11PAHi5:~$ lscpu | grep 1135
型号名称:                          11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
dash@dash-NUC11PAHi5:~$ uname -a
Linux dash-NUC11PAHi5 6.2.0-31-generic #31~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Wed Aug 16 13:45:26 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
dash@dash-NUC11PAHi5:~$ cat /etc/issue
Ubuntu 22.04.3 LTS \n \l

Build Steps

Install prerequisite packages:

sudo apt update -y
sudo apt install -y build-essential git meson protobuf-c-compiler autoconf automake libtool curl make g++ unzip protobuf-compiler  cmake uuid-dev pkg-config libyaml-cpp-dev libspice-client-glib-2.0-dev libpixman-1-dev libzstd-dev libasound2-dev libsdl2-dev libepoxy-dev

Clone/build/install virglrender:

unzip virglrenderer-main.zip
cd virglrenderer-main/
meson -Dprefix=/usr build
cd build/
sudo ninja install

Clone/build/install mvisor:

unzip mvisor-master.zip 
cd mvisor-master/
vim meson_options.txt
    option('vgpu',
      type: 'boolean',
      value: true,
      description: 'Enable VGPU device'
meson setup build
meson compile -C build
./build/mvisor --version
     MVisor: 2.5.2
sudo cp build/mvisor /usr/bin/

VM Operations

Folder content:

$ pwd
/home/dash/mvisorwin
$ ls
virtio-win-0.1.240.iso  win10.qcow2  zh-cn_windows_10_consumer_editions_version_22h2_updated_sep_2023_x64_dvd_4cde879b.iso

Create the yaml via:

$ cat default.yaml 
name: Default configuration
base: i440fx.yaml

machine:
  memory: 8G
  vcpu: 4
  # Set vcpu thread priority value [-20, 19]
  # A higher value means a lower priority
  priority: 1
  # Turn on BIOS output and performance measurement
  debug: No
  # Turn on hypervisor to lower CPU usage (Hyper-V is used for Windows)
  hypervisor: Yes

objects:
  - name: cmos
    # gmtime for linux, localtime for windows
    rtc: localtime

  - class: qxl
  - class: spice-agent
  - class: qemu-guest-agent
  - class: usb-tablet

  - class: virtio-network
    backend: uip
    mac: 00:50:00:11:22:33
    map: tcp:0.0.0.0:8022-:22

  - class: ata-cdrom 
    image: /home/dash/mvisorwin/zh-cn_windows_10_consumer_editions_version_22h2_updated_sep_2023_x64_dvd_4cde879b.iso
  
  - class: ata-cdrom
    image: /home/dash/mvisorwin/virtio-win-0.1.240.iso

  - class: virtio-block
    image: /home/dash/mvisorwin/win10.qcow2
    snapshot: No
  
  # - class: floppy
  #   image: /data/images/floppy.img

  # - class: virtio-block
  #   image: /data/empty.qcow2
  #   snapshot: No

  # - class: virtio-fs
  #   path: /tmp/fuse
  #   disk_name: mvisor-fs
  #   disk_size: 2G
  #   inode_count: 200

  # - class: vfio-pci
  #   sysfs: /sys/bus/mdev/devices/c2e088ba-954f-11ec-8584-525400666f2b
  #   debug: Yes

  - class: virtio-vgpu
    memory: 1G
    staging: Yes
    blob: No
    node: /dev/dri/renderD128

Start the machine via:

mvisor -c default.yaml

/images/2023_10_09_22_36_53_1788x941.jpg

After installation:

/images/2023_10_09_22_44_45_399x442.jpg

Install virtio drivers:

/images/2023_10_09_22_45_10_744x685.jpg

Install qxl driver:

/images/2023_10_09_22_46_10_896x735.jpg

Qxl ready:

/images/2023_10_09_22_46_29_482x236.jpg

virtio-vgpu:

/images/2023_10_09_22_49_15_762x737.jpg

Enable the test sign driver:

/images/2023_10_09_22_52_41_631x156.jpg

Reboot to make the driver take effect, install driver:

/images/2023_10_09_22_54_13_1092x559.jpg

Result:

/images/2023_10_09_22_54_30_1029x484.jpg

The Mvisor VGPU:

/images/2023_10_09_22_55_03_1017x689.jpg

but the gpu won’t work

HostapdOnRTL8188EE

lspci for getting the wireless card mode:

# lspci | grep -i wireless
01:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8188EE Wireless Network Adapter (rev 01)

Install the script:

yay -S linux-wifi-hotspot

Create the wifi via:

systemctl enable --now create_ap
create_ap wlp1s0 enp4s0 xxx xxxxxxx --hidden

BuildKernelTheDockerWay

编译镜像准备

使用rockylinux:9的容器镜像创建一个容器实例:   

sudo docker run -it rockylinux:9 bash

在容器实例中,运行以下命令准备内核的编译环境:    

sed -e 's|^mirrorlist=|#mirrorlist=|g'     -e 's|^#baseurl=http://dl.rockylinux.org/$contentdir|baseurl=https://mirrors.ustc.edu.cn/rocky|g'     -i.bak     /etc/yum.repos.d/rocky-extras.repo     /etc/yum.repos.d/rocky.repo
yum makecache
dnf install -y 'dnf-command(config-manager)'
dnf config-manager --set-enabled crb
yum install -y vim rpm-build python3-devel elfutils-devel  openssl-devel perl-generators pesign yum-utils bc bison bpftool dwarves flex gcc gcc-c++ git-core hmaccalc kmod m4 make net-tools perl-devel gcc-plugin-devel  rpm-build rpmdevtools  dnf-plugins-core ncurses-devel make gcc bc bison flex elfutils-libelf-devel openssl-devel grub2 rpm-build rsync gcc vim yum-utils perl systemd-udev  asciidoc audit-libs-devel binutils-devel clang dwarves fuse-devel gcc-c++ gcc-plugin-devel git-core glibc-static java-devel kabi-dw kernel-rpm-macros libbabeltrace-devel libbpf-devel libcap-devel libcap-ng-devel libmnl-devel libnl3-devel libtraceevent-devel libtracefs-devel lld llvm lvm2 net-tools newt-devel numactl-devel pciutils-devel perl-devel python3-docutils system-sb-certs tpm2-tools xmlto elfutils-devel nss-tools perl-generators pesign python3-devel xz-devel
# download the following packages offlinely 
yum install -y WALinuxAgent-cvm-2.7.0.6-9.el9_2.1.rocky.0.noarch.rpm systemd-boot-unsigned-252-14.el9_2.1.x86_64.rpm
useradd -m mock

新建一个终端,在该终端上将运行中且已做上述修改的容器实例提交为容器镜像以便下次使用:     

[root@dellnew ~]# docker ps 
CONTAINER ID   IMAGE          COMMAND   CREATED         STATUS         PORTS     NAMES
f7eb549f3d44   rockylinux:9   "bash"    7 minutes ago   Up 7 minutes             wonderful_sinoussi
[root@dellnew ~]# docker commit wonderful_sinoussi buidrockykernel:latest
[root@dellnew ~]# docker images
REPOSITORY        TAG       IMAGE ID       CREATED         SIZE
buidrockykernel   latest    207a4b57059e   5 seconds ago   1.94GB

2. 使用容器编译内核

使用上节创建的编译镜像编译内核:

[root@text ~]# docker run --name=testrocky -v /root/buildout:/buildout -it buidrockykernel:latest /bin/bash
[root@fa4d8f532c21 /]# cp /buildout/kernel-5.15.113-200.el9.src.rpm /home/mock/
[root@fa4d8f532c21 /]# su - mock
[mock@fa4d8f532c21 ~]$ rpm -Uvh kernel-5.15.113-200.el9.src.rpm 
[mock@fa4d8f532c21 ~]$ cd rpmbuild/SPECS/
[mock@fa4d8f532c21 SPECS]$ time rpmbuild -ba kernel.spec 2>&1 | tee build.log

编译出的内核rpm包位于/home/mock/rpmbuild下,可以通过find /home/mock/rpmbuild | grep rpm$命令找到。

BuildKernelImage

Create a docker instance:

sudo docker run -it rockylinux:9 bash

In docker, run:

sed -e 's|^mirrorlist=|#mirrorlist=|g'     -e 's|^#baseurl=http://dl.rockylinux.org/$contentdir|baseurl=https://mirrors.ustc.edu.cn/rocky|g'     -i.bak     /etc/yum.repos.d/rocky-extras.repo     /etc/yum.repos.d/rocky.repo
yum makecache
dnf install -y 'dnf-command(config-manager)'
dnf config-manager --set-enabled crb
yum install -y vim rpm-build python3-devel elfutils-devel  openssl-devel perl-generators pesign yum-utils bc bison bpftool dwarves flex gcc gcc-c++ git-core hmaccalc kmod m4 make net-tools perl-devel gcc-plugin-devel  rpm-build rpmdevtools  dnf-plugins-core ncurses-devel make gcc bc bison flex elfutils-libelf-devel openssl-devel grub2 rpm-build rsync gcc vim yum-utils perl systemd-udev  asciidoc audit-libs-devel binutils-devel clang dwarves fuse-devel gcc-c++ gcc-plugin-devel git-core glibc-static java-devel kabi-dw kernel-rpm-macros libbabeltrace-devel libbpf-devel libcap-devel libcap-ng-devel libmnl-devel libnl3-devel libtraceevent-devel libtracefs-devel lld llvm lvm2 net-tools newt-devel numactl-devel pciutils-devel perl-devel python3-docutils system-sb-certs tpm2-tools xmlto elfutils-devel nss-tools perl-generators pesign python3-devel xz-devel
# download the following packages offlinely 
yum install -y WALinuxAgent-cvm-2.7.0.6-9.el9_2.1.rocky.0.noarch.rpm systemd-boot-unsigned-252-14.el9_2.1.x86_64.rpm
useradd -m mock

Then in host machine, do :

[root@dellnew ~]# docker ps 
CONTAINER ID   IMAGE          COMMAND   CREATED         STATUS         PORTS     NAMES
f7eb549f3d44   rockylinux:9   "bash"    7 minutes ago   Up 7 minutes             wonderful_sinoussi
[root@dellnew ~]# docker commit wonderful_sinoussi buidrockykernel:latest
[root@dellnew ~]# docker images
REPOSITORY        TAG       IMAGE ID       CREATED         SIZE
buidrockykernel   latest    207a4b57059e   5 seconds ago   1.94GB

Next time using this latest commited kernel you could directly build kernel src.

test images

Build via following command:

[root@text ~]# docker run --name=testrocky -v /root/buildout:/buildout -it buidrockykernel:latest /bin/bash
[root@fa4d8f532c21 /]# cp /buildout/kernel-5.15.113-200.el9.src.rpm /home/mock/
[root@fa4d8f532c21 /]# su - mock
[mock@fa4d8f532c21 ~]$ rpm -Uvh kernel-5.15.113-200.el9.src.rpm 
[mock@fa4d8f532c21 ~]$ cd rpmbuild/SPECS/
[mock@fa4d8f532c21 SPECS]$ time rpmbuild -ba kernel.spec 2>&1 | tee build.log

Build515KernelRPM

Using a docker instance:

$ dnf config-manager --set-enabled crb
$ yum install -y vim rpm-build python3-devel elfutils-devel  openssl-devel perl-generators pesign yum-utils bc bison bpftool dwarves flex gcc gcc-c++ git-core hmaccalc kmod m4 make net-tools perl-devel gcc-plugin-devel
$ vim /etc/yum.repos.d/kernellongterm.repo
[copr:copr.fedorainfracloud.org:kwizart:kernel-longterm-5.15]
name=Copr repo for kernel-longterm-5.15 owned by kwizart
baseurl=https://download.copr.fedorainfracloud.org/results/kwizart/kernel-longterm-5.15/epel-9-$basearch/
type=rpm-md
skip_if_unavailable=True
gpgcheck=1
gpgkey=https://download.copr.fedorainfracloud.org/results/kwizart/kernel-longterm-5.15/pubkey.gpg
repo_gpgcheck=0
enabled=1
enabled_metadata=1
$ yum makecache

As mock, do following:

$ yumdownloader --source kernel-longterm
$ ls
kernel-longterm-5.15.124-200.el9.src.rpm
$ rpm -Uvh kernel-longterm-5.15.124-200.el9.src.rpm
$ ls
kernel-longterm-5.15.124-200.el9.src.rpm  rpmbuild

Replace to 5.15.113-1, then modify as following:

$ cp linux-5.15.tar.xz ./rpmbuild/SOURCES/linux-5.15.tar.xz
$ vim rpmbuild/SPECS/kernel.spec
Line 135? 
# Do we have a -stable update to apply?
#%define stable_update 124
%define stable_update 113

%define rpmversion %{kversion}.%{stable_update}
%define patchversion 5.15
#%define pkgrelease 200
%define pkgrelease 1
1400 # released_kernel with possible stable updates
1401 # This is special because the kernel spec is hell and nothing is consistent
1402 #xzcat %{SOURCE5000} | patch -p1 -F1 -s
1403 #xzcat %{SOURCE5000} | patch -p1 -F1 -s
1404 git commit -a -m "Stable update"
1405 
1406 # Note: Even in the "nopatches" path some patches (build tweaks and compile
1407 # fixes) will always get applied; see patch defition above for details
1408 
1409 #git am %{patches}                                                                                                                                                                                                                                      
1410 #git am %{patches}