ConfigurationOn4GDecoding

Above 4G Decoding (Available if the system supports 64-bit PCI decoding) Select Enabled to decode a PCI device that supports 64-bit in the space above 4G Address. The options are Enabled and Disabled.

翻译:

G以上解码（如果系统支持64位PCI解码，则可用）选择 “已启用 “来解码一个支持64位的PCI设备，在4G地址以上的空间。选项是已启用和已禁用。

禁用的情况:

root@mac:~# lspci -v | grep "Memory.*64-bit"
	Memory at d0000000 (64-bit, prefetchable) [disabled] [size=256M]
	Memory at e0000000 (64-bit, prefetchable) [disabled] [size=2M]
	Memory at fcf60000 (64-bit, non-prefetchable) [size=16K]
	Memory at fce04000 (64-bit, non-prefetchable) [size=4K]
	Memory at fce00000 (64-bit, non-prefetchable) [size=16K]
	Memory at b0000000 (64-bit, prefetchable) [size=256M]
	Memory at c0000000 (64-bit, prefetchable) [size=2M]
	Memory at fca00000 (64-bit, non-prefetchable) [size=1M]
	Memory at fc900000 (64-bit, non-prefetchable) [size=1M]

开启的情况:

idv@idv-TC-9070:~$ sudo lspci -v | grep "Memory.*64-bit"
	Memory at de000000 (64-bit, non-prefetchable) [size=16M]
	Memory at c0000000 (64-bit, prefetchable) [size=256M]
	Memory at df110000 (64-bit, non-prefetchable) [size=64K]
	Memory at df12d000 (64-bit, non-prefetchable) [size=4K]
	Memory at df120000 (64-bit, non-prefetchable) [size=16K]
	Memory at df100000 (64-bit, non-prefetchable) [size=64K]
	Memory at df12a000 (64-bit, non-prefetchable) [size=256]
	Memory at df000000 (64-bit, non-prefetchable) [size=4K]
	Memory at d0000000 (64-bit, prefetchable) [size=16K]

Other options in bios

peci:

PECI是用于监测CPU及芯片组温度的一线总线(one-wire bus)，全称是Platform Environment Control Interface。

OSXKVMGVTPASSTHROUGH

Install Prerequisite packages:

$ sudo apt-get install qemu uml-utilities virt-manager git \
    wget libguestfs-tools p7zip-full make dmg2img -y
$ cat kvm.conf 
options kvm_intel nested=1
options kvm_intel emulate_invalid_guest_state=0
options kvm ignore_msrs=1 report_ignored_msrs=0
$ sudo cp kvm.conf /etc/modprobe.d/
$ echo 1 | sudo tee /sys/module/kvm/parameters/ignore_msrs
$ sudo usermod -aG kvm $(whoami)
$ sudo usermod -aG libvirt $(whoami)
$ sudo usermod -aG input $(whoami)

Clone related repository:

git clone https://github.com/vivekmiyani/OSX_GVT-D
git clone --recursive https://github.com/kholia/OSX-KVM.git
cd OSX-KVM
git checkout 88154b5bac079473660afc2a89704874cc7edf03

Choose Monterey for downloading:

$ ./fetch-macOS-v2.py 
1. High Sierra (10.13)
2. Mojave (10.14)
3. Catalina (10.15)
4. Big Sur (11.6) - RECOMMENDED
5. Monterey (latest)

Choose a product to download (1-5): 5
Monterey (latest)

Format the disk and install macos in kvm:

dmg2img -i BaseSystem.dmg BaseSystem.img
qemu-img create -f qcow2 mac_hdd_ng.img 128G
./OpenCore-Boot.sh

IDVRelease

Install cubic(ubuntu 22.04.2):

sudo apt-add-repository universe
sudo apt-add-repository ppa:cubic-wizard/release
sudo apt update
sudo apt install --no-install-recommends cubic

Then start cubic and following the guideline.

In terminal do:

root@cubic:~# history
    1  dpkg -l
    2  gsettings set org.gnome.desktop.session idle-delay 0
    3  gsettings set org.gnome.desktop.screensaver ubuntu-lock-on-suspend false
    4  vi /etc/default/apport 
    5  passwd
    6  which vim
    7  apt install openssh-server
    8  vim /etc/ssh/sshd_config 
    9  vi /etc/ssh/sshd_config 
   10  dpkg -l | grep thunderbird
   11  sudo apt remove thunderbird
   12  dpkg -l | grep libreoffice
   13  sudo apt remove libreoffice
   14  sudo apt remove libreoffice*
   15  sudo apt remove libreoffice-core
   16  dpkg -l | more
   17  apt install -y sddm
   18  apt-cache search sddm
   19  apt update
   20  vi /etc/apt/sources.list
   21  ap tupdate
   22  apt update
   23  apt install sddm
   24  apt-cache search sddm
   25  apt-cache search build-essential
   26  apt-get policy build-essential
   27  apt-get search build-essential
   28  apt-cache policy build-essential
   29  vi /etc/apt/sources.list
   30  apt-get update
   31  apt-cache search sddm
   32  sudo apt install -y sddm
   33  df -h
   34  history
   35  useradd -m ctyunidv
   36  passwd ctyunidv
   37  mkdir -p /etc/sddm.conf.d/
   38  vim /etc/sddm.conf.d/autologin.conf
   39  vi /etc/sddm.conf.d/autologin.conf
   40  apt install -y iotop
   41  history
   42  clear
   43  ls
   44  vi /etc/default/grub 
   45  update-grub2 
   46  vim /etc/initramfs-tools/modules 
   47  vi /etc/initramfs-tools/modules 
   48  update-initramfs -u -k all
   49  which scp
   50  scp ctyunidv@172.23.119.211:~/CtyunDesktopIDV_1.0.2_101000200_x64_03-17-11-20.deb .
   51  scp ctyunidv@172.23.119.211:~/ctgcd-clouddesktop-idvagent.war .
   52  ls
   53  sudo apt install -y ./CtyunDesktopIDV_1.0.2_101000200_x64_03-17-11-20.deb 
   54  ssh-keygen 
   55  cat /root/.ssh/id_rsa.pub 
   56  scp root@172.23.119.211:/root/ljr/linux-image-5.10.90-c1dc2c9a39ac_5.10.90-c1dc2c9a39ac-17_amd64.deb .
   57  apt install -y ./linux-image-5.10.90-c1dc2c9a39ac_5.10.90-c1dc2c9a39ac-17_amd64.deb 
   58  ls
   59  history

idvworkingtips2

System Installation

Ubuntu 22.04.2 iso, Install with HWE Kernel Selected.
English-> Ubuntu Server(mimimized) -> Skip Ubuntu Pro -> Install OpenSSH server.

sudo apt update
sudo apt install -y vim sddm libvirt-daemon qemu libvirt-daemon-system-systemd libvirt-daemon-driver-qemu libvirt-daemon-system libvirt0

Examine the version:

$ qemu-system-x86_64 --version
QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.6)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

System Configuration

Kernel and modules:

# sudo vim /etc/default/grub
......
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on intel_iommu=pt kvm.ignore_msrs=1"
......
# sudo update-grub2
# sudo vim /etc/initramfs-tools/modules
......
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
# update-initramfs -u -k all
# sudo reboot

Use mate for desktop session:

$ sudo apt install -y mate

libvirt hook should be the same.

sddm autologin :

# mkdir -p /etc/sddm.conf.d
# vim /etc/sddm.conf.d/autologin.conf
[Autologin]
User=idv
Session=mate

while the X Session could be viewd as:

l /usr/share/xsessions/
ubuntu.desktop  ubuntu-xorg.desktop

idvworkingtips

0. Hardware

Detailed info:

11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
16GB DDR4-3200(dual channel)
500GB PCIe4.0 ssd

1. System Installation

Downloads iso and write to flash disk(Above 8G):

wget https://mirrors.ustc.edu.cn/rocky/9.1/isos/x86_64/Rocky-9.1-20221214.1-x86_64-dvd.iso
sudo dd if=./Rocky-9.1-20221214.1-x86_64-dvd.iso of=/dev/sdb bs=1M && sudo sync

Install RockLinux 9.1 to nuc.

Installation configurations:

Select English->English(United States)
Set user passwd(Allow root SSH login with password)
Software Selection(Minimal Install)
Partition: Delete /home, shrink swap to 2GiB(example layout----/boot: 1024MiB, /:462.26GiB,/boot/efi: 512MiB, swap: 2GiB)
Create user: idv(make this user to be administrator)
Set hostname: rockyIDV

After installation, change default repo configuration:

$ sudo su
# sed -e 's|^mirrorlist=|#mirrorlist=|g' \
    -e 's|^#baseurl=http://dl.rockylinux.org/$contentdir|baseurl=https://mirrors.ustc.edu.cn/rocky|g' \
    -i.bak \
    /etc/yum.repos.d/rocky-extras.repo \
    /etc/yum.repos.d/rocky.repo
# dnf makecache

Instal mate desktop enviroment as the idv graphical UI:

sudo dnf config-manager --set-enabled crb
sudo dnf install epel-release
sudo dnf install NetworkManager-adsl NetworkManager-bluetooth NetworkManager-libreswan-gnome NetworkManager-openvpn-gnome NetworkManager-ovs NetworkManager-ppp NetworkManager-team NetworkManager-wifi NetworkManager-wwan adwaita-gtk2-theme alsa-plugins-pulseaudio atril atril-caja atril-thumbnailer caja caja-actions caja-image-converter caja-open-terminal caja-sendto caja-wallpaper caja-xattr-tags dconf-editor engrampa eom firewall-config gnome-disk-utility gnome-epub-thumbnailer gstreamer1-plugins-ugly-free gtk2-engines gucharmap gvfs-fuse gvfs-gphoto2 gvfs-mtp gvfs-smb initial-setup-gui libmatekbd libmatemixer libmateweather libsecret lm_sensors marco mate-applets mate-backgrounds mate-calc mate-control-center mate-desktop mate-dictionary mate-disk-usage-analyzer mate-icon-theme mate-media mate-menus mate-menus-preferences-category-menu mate-notification-daemon mate-panel mate-polkit mate-power-manager mate-screensaver mate-screenshot mate-search-tool mate-session-manager mate-settings-daemon mate-system-log mate-system-monitor mate-terminal mate-themes mate-user-admin mate-user-guide mozo network-manager-applet nm-connection-editor p7zip p7zip-plugins pluma seahorse seahorse-caja xdg-user-dirs-gtk pciutils vim
sudo dnf install lightdm-settings lightdm
sudo systemctl set-default graphical.target
sudo reboot

Enable the lightdm autologin:

sudo nano /etc/lightdm/lightdm.conf
autologin-user=idv

2. Virtualization Configuration

Install and configure libvirtd:

sudo dnf install libvirt-daemon qemu-kvm virt-top libguestfs-tools virt-install
sudo systemctl enable libvirtd
sudo virsh net-start default

view qemu version:

# /usr/libexec/qemu-kvm --version
QEMU emulator version 7.0.0 (qemu-kvm-7.0.0-13.el9_1.2)
Copyright (c) 2003-2022 Fabrice Bellard and the QEMU Project developers

Kernel default options:

# vi /etc/default/grub
......
GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on intel_iommu=pt kvm.ignore_msrs=1"
# grub2-mkconfig -o /boot/efi/EFI/rocky/grub.cfg
# cat /proc/cmdline
Examine the iommu
# sudo dmesg | grep iommu
# sudo dmesg | grep -i dmar

Edit dracut for adding vfio modules:

[idv@rockyIDV ~]$ sudo vim /etc/dracut.conf.d/vfio.conf
add_drivers+="vfio vfio_iommu_type1 vfio_pci vfio_virqfd"
[idv@rockyIDV ~]$ sudo dracut -f --kver  `uname -r`
[idv@rockyIDV ~]$ sudo reboot

3. libvirtd Hooks

Refers to other material.

4. issue

On rocky linux, we got following issues:

[idv@rockyIDV ~]$ ./test_10thFV_win10.sh 
2023-03-28T05:53:46.994085Z qemu-system-x86_64: -device vfio-pci,host=0000:00:02.0,x-igd-gms=2,id=hostdev0,bus=pci.0,addr=2,rombar=0,x-igd-opregion=on: IGD device 0000:00:02.0 does not support OpRegion access,legacy mode disabled
2023-03-28T05:53:46.995006Z qemu-system-x86_64: -device vfio-pci,host=0000:00:02.0,x-igd-gms=2,id=hostdev0,bus=pci.0,addr=2,rombar=0,x-igd-opregion=on: vfio 0000:00:02.0: does not support requested IGD OpRegion feature: No such device

Now change back to Ubuntu22.04.

Also spice-server is removed for rhel9, so got lots of time for building spice related libs.