|
一、软件环境1.kvm部署发行版环境:
root@deepin-PC:/opt/zxh/kvm_20210823/qemu-sw# cat /etc/product-info
UnionTech OS Server 20 Enterprise 1030 20210513 sw_64
2.软件版本:
linux-stable-sw-v4.19.180-sw64-2107.tar.gz
qemu-sw-v3.1.0-sw-2107-rc2.tar.gz
libvirt-v5.0.0-sw-2107-rc2.tar.gz
二、环境部署及验证
(一)、 编译内核
1.使用默认宿主机内核配置文件xuelang,编译内核二进制后,生成initrd.img,机器可以正常启动;
2.使用新内核二进制启动,物理机可以支持kvm_reserved_mem参数设置内存预留大小;
- root@deepin-PC:/opt/zxh/kvm_20210823/qemu-sw# cat /proc/cmdline
- kvm_reserved_mem=128G root=UUID=f67d588a-a134-4ba6-bb0a-1c77929b9943 ro quiet splash DEEPIN_GFXMODE=
- root@deepin-PC:/opt/zxh/kvm_20210823/qemu-sw# free -m
- total used free shared buff/cache available
- Mem: 128488 2991 122488 10 3008 123948
- Swap: 16383 0 16383
- root@deepin-PC:/opt/zxh/kvm_20210823/qemu-sw#
复制代码
(二)、 编译qemu
1) # ./run_kvm_config.sh
报错与解决:
- root@deepin-PC:/opt/zxh/kvm_20210823/qemu-sw# ./run_kvm_config.sh
- ERROR: User requested feature libseccomp
- configure was not able to find it.
- Install libseccomp devel >= 2.3.0
- 类似的错误,还有:libpixman-1-dev、libspice-server-dev、libcap-dev
- ERROR: pixman >= 0.21.8 not present.
- Please install the pixman devel package.
- ERROR: User requested feature spice
- configure was not able to find it.
- Install spice-server(>=0.12.0) and spice-protocol(>=0.12.3) devel
- ERROR: VirtFS requires libcap devel and libattr devel
复制代码
====》 解决方法:root@deepin-PC:/opt/zxh/kvm_20210823/qemu-sw# apt install libseccomp-dev libpixman-1-dev libspice-server-dev libcap-dev
2)make clean
3)make -j16
在目录sw64-softmmu下生成qemu运行的二进制文件,文件目录为sw64-softmmu/qemu-system-sw64
4)make install
(三)、编译libvirt
1) # ./swBuild.sh
报错如下:
- root@deepin-PC:/opt/zxh/kvm_20210823/libvirt# ./swBuild.sh
- configure: error: You must install the gnutls >= 3.2.0 pkg-config module to compile libvirt
- error: configure failed
复制代码
类似的错误还有一些,按照报错安装对应的包即可;
=======》解决方法是: apt -y install libgnutls-dane0 libgnutls-openssl27 libgnutls28-dev libgnutlsxx28 libnl-3-dev libnl-route-3-dev libpciaccess-dev libxml2-utils xsltproc libdevmapper-dev libyajl-dev libyajl2
其中,仓库里面的软件包依赖有问题,可以通过如下包安装解决(此部分包是统信提供的gmp.tar.gz包):
- -rw-r--r-- 1 root root 292912 4月 1 20:37 libgmp10_6.1.2+dfsg-4+0eagle1_sw_64.deb
- -rw-r--r-- 1 root root 917828 4月 1 20:37 libgmp10-dbgsym_6.1.2+dfsg-4+0eagle1_sw_64.deb
- -rw-r--r-- 1 root root 172556 4月 1 20:37 libgmp10-doc_6.1.2+dfsg-4+0eagle1_all.deb
- -rw-r--r-- 1 root root 15776 4月 1 20:37 libgmp3-dev_6.1.2+dfsg-4+0eagle1_sw_64.deb
- -rw-r--r-- 1 root root 698112 4月 1 20:37 libgmp-dev_6.1.2+dfsg-4+0eagle1_sw_64.deb
- -rw-r--r-- 1 root root 23156 4月 1 20:37 libgmpxx4ldbl_6.1.2+dfsg-4+0eagle1_sw_64.deb
- -rw-r--r-- 1 root root 137648 4月 1 20:37 libgmpxx4ldbl-dbgsym_6.1.2+dfsg-4+0eagle1_sw_64.deb
复制代码
完成后,提示如下:
- configure: DTrace: yes
- configure: numad: no
- configure: Init script: systemd
- configure: Char device locks: /var/lock
- configure: Default Editor: vi
- configure: Loader/NVRAM:
- configure: virt-login-shell: yes
- configure: virt-host-validate: yes
- configure: TLS priority: NORMAL
- configure:
- configure: Developer Tools
- configure:
- configure: wireshark_dissector: no
- configure:
- configure: Privileges
- configure:
- configure: QEMU: root:root
- configure:
- Now type 'make' to compile libvirt.
复制代码
2) # make -j32
3) # make install
(四)问题及异常:
1) 统信uos原先系统自带了libvirt5.5版本,现在切换到5.0版本,如何启动刚编译的libvirtd-5.0.0?
==============》 解决方法:重新加载libvirtd.service服务(原先的libvirtd暂未卸载)
systemctl daemon-reload
systemctl start libvirtd
=================================
- root@deepin-PC:/opt/zxh/kvm_20210823/libvirt# systemctl status libvirtd
- ● libvirtd.service - Virtualization daemon
- Loaded: loaded (/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
- Active: active (running) since Thu 2021-08-26 15:10:07 CST; 3 days ago
- Docs: man:libvirtd(8)
- https://libvirt.org
- Main PID: 1836 (libvirtd)
- Tasks: 19 (limit: 32768)
- Memory: 30.4M
- CGroup: /system.slice/libvirtd.service
- ├─1836 /usr/sbin/libvirtd
- ├─2099 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/li
- └─2100 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/libexec/li
- Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
复制代码
2)执行kvm启动脚本网桥报错
root@deepin-PC:/opt/zxh/run_files_zxh# ./run_uos830_ty_zxh_2107.sh
failed to parse default acl file `/usr/local/etc/qemu/bridge.conf'
qemu-system-sw64: bridge helper failed
==============》 解决方法:新建qemu网桥目录和配置文件
brctl addbr virbr0
ifconfig virbr0 192.168.122.1
mkdir -pv /usr/local/etc/qemu # 若该目录不存在,则手动创建
cat >> /usr/local/etc/qemu/bridge.conf << EOF # 编辑网桥配置文件
allow virbr0 > EOF
如,
mkdir -pv /usr/local/etc/qemu
echo "allow virbr0 " > /usr/local/etc/qemu/bridge.conf
3) 执行脚本报错:no core3-reset provided
root@deepin-PC:/opt/zxh/run_files_zxh# ./run_2107_uos830_zxh.sh
hwaddr = 0x804910000000
qemu-system-sw64: no core3-reset provided
==============》 解决方法:core3-reset文件可在qemu源码的pc-bios目录下找到,拷贝到/usr/local/share/qemu/core3-reset即可。
4)执行脚本报错:Bus 'pci.0' not found
root@deepin-PC:/opt/zxh/run_files_zxh# ./run_2107_uos830_zxh.sh
hwaddr = 0x804910000000
qemu-system-sw64: -device virtio-net-pci,netdev=tap0,bus=pci.0: Bus 'pci.0' not found
==============》 解决方法:qemu-V2107发布版,需要将bus=pci.0改成pcie.0
5)报错:“not found (required by virsh)”未找到virsh命令
==============》 解决方法:卸载原来的libvirt相关包后,重新编译libvirtd源码(make、make install)再重启libvirtd服务;
# apt purge libvirt-clients libvirt-daemon libvirt-daemon-system libvirt0
# 进入libvirt源码目录,make -j32;make install
# systemctl restart libvirtd
6)报错:未在路径中找到 'pm-is-supported': 没有那个文件或目录
8月 31 17:25:37 opsk libvirtd[23752]: Failed to get host CPU cache info
8月 31 17:25:37 opsk libvirtd[23752]: 未在路径中找到 'pm-is-supported': 没有那个文件或目录
==============》 解决方法:安装pm-utils包解决pm-is-supported找不到问题;
7)报错:初始化失败
8月 31 18:04:37 deepin-PC systemd[1]: Failed to start Virtualization daemon.
==============》 解决方法:通过源码./src/libvirtd.service文件,确认/usr/sbin/libvirtd可以命令行启动服务,据此得到具体错误信息如下:
root@deepin-PC:/opt/zxh/kvm_20210823/libvirt/src# /usr/sbin/libvirtd
/usr/sbin/libvirtd: /lib/libvirt.so.0: version LIBVIRT_PRIVATE_5.0.0' not found (required by /usr/sbin/libvirtd)
/usr/sbin/libvirtd: /lib/libvirt.so.0: versionLIBVIRT_PRIVATE_5.0.0' not found (required by /lib/libvirt-lxc.so.0)
/usr/sbin/libvirtd: /lib/libvirt.so.0: version `LIBVIRT_PRIVATE_5.0.0' not found (required by /lib/libvirt-qemu.so.0)
再次检查环境,发现当前环境有系统自带的qemu4.0,通过如下命令卸载qemu4.0:
./libvirt/src# apt purge qemu qemu-efi qemu-guest-agent
然后,再重新找到qemu源码,再次编译安装qemu3.1后,再验证libvirtd
另,
发现,如果libvitd默认链接/usr/lib目录下libvirt.so和libvirt.so.0链接的是原来的libvirt.so.0.5000.0旧版本,那么即使make install重新安装,也不会更新的。
要先删除,再systemctl daemon-reload,再libvirt源码目录make install,再重启libvirtd服务
8)报错:操作失败: 网络 'default' 已有 uuid 426325cc-cece-443a-b55c-469508c08680
8月 31 18:35:49 deepin-PC libvirtd[5726]: 操作失败: 网络 'default' 已有 uuid 426325cc-cece-443a-b55c-469508c08680
==============》 解决方法:vim /etc/libvirt/qemu/networks/default.xml,修改uuid即可;
1)“因为系统默认自带virbr0网口,当systemctl restart libvirtd服务时,/var/lib/libvirt/dnsmasq/default.conf这个配置文件中也会重启一个virbr0网口,重复了;
我们将系统默认virbr0网口down后(ifconfig virbr0 down),再重启libvirtd服务就不会出现网络错误;” ------ 不是这个原因,而且即使down掉virbr0网口,异常现象还会复现;
2)再次验证发现,通过deepin用户启动libvirtd时,没有异常报错,切换到root用户启动就会有此报错;
9)报错: '/etc/libvirt/fake_sysinfo' 失败: 没有那个文件或目录
8月 31 18:50:37 deepin-PC libvirtd[6313]: 打开文件 '/etc/libvirt/fake_sysinfo' 失败: 没有那个文件或目录
8月 31 18:50:37 deepin-PC libvirtd[6313]: 内部错误:打开 /etc/libvirt/fake_sysinfo 失败
==============》 解决方法:fake_sysinfo在libvirt源码的src/qemu/fake_sysinfo目录,将文件拷贝到/etc/libvirt/目录(cp fake_sysinfo /etc/libvirt/),再次重启libvirtd服务;
10)报错:Failed to get host CPU cache info
8月 31 19:02:10 deepin-PC libvirtd[6573]: Failed to get host CPU cache info
==============》 解决方法:此信息不影响使用,暂且不管
11) 报错:没有可用JSON解析程序、探测功能失败等
==============》 解决方法:apt安装libyajl-dev、dnsmasq安装包;
12)xml编写报错:but model='pci-root' was found instead
root@deepin-PC:/opt/zxh/run_files_zxh# virsh define uos830-2107_1030-ty.xml
错误:从 uos830-2107_1030-ty.xml 定义域失败
错误:XML 错误:The PCI controller with index='0' must be model='pcie-root' for this machine type, but model='pci-root' was found instead
==============》 解决方法:将xml中model='pci-root'修改为model='pcie-root'
13)xml编写文件如何添加超过2个及以上的网口?
==============》 解决方法:脚本中的id、mac可自由修改。进入虚拟机后使用ifconfig -a找到虚拟网络设备,并绑定与宿主机网桥同一网段的ip地址,即可完成网络配置。如果同时对多台虚拟机、多个虚拟网卡进行网络配置,需注意将mac地址改为互不相同,否则将出现虚拟机网络故障。
-netdev tap,br=brname,helper=/usr/local/libexec/qemu-bridge-helper,id=tap0-device virtio-net-pci,netdev=tap0,id=net0,mac=fa:16:3e:1a:93:22,bus=pcie.0 #第一个虚拟网卡-netdev tap,br=brname,helper=/usr/local/libexec/qemu-bridge-helper,id=tap1-device virtio-net-pci,netdev=tap1,id=net1,mac=fa:16:3e:1a:93:23,bus=pcie.0 #第二个虚拟网
14)xml文件写得vnc指定端口port='5',但是virsh define xml后,virsh edit看到的域文件是‘-1’,vncview远程连接用的是5900
==============》解决方法:暂未解决
设置成port=‘5905’还是一样。virsh edit进入,vnc默认port=‘-1’,对应的端口是5900。
例子:
<graphics type='vnc' port='5905' autoport='yes' listen='0.0.0.0'>
<listen type='address' address='0.0.0.0'/>
<image compression='auto_glz'/>
<jpeg compression='always'/>
<zlib compression='always'/>
<streaming mode='all'/>
</graphics>
15)virsh define xxx.xml出现报错
root@deepin-PC:/opt/zxh/run_files_zxh# virsh define uos830-2107_1030-ty.xml
错误:连接到管理程序失败
错误:将插槽连接到 '/var/run/libvirt/libvirt-sock' 失败: 没有那个文件或目录
==============》解决方法:应该是virbr0没有绑定并启动
需要通过命令定义虚拟网口:
如:
brctl addbr virbr0
ifconfig virbr0 192.168.122.1
16)xml中需要指定machine为core3,目前1621和3231都是指定core3,当前qemu会支持core3和core4
root@deepin-PC:/opt/zxh/run_files_zxh# qemu-system-sw64 --machine help
Supported machines are:
core3 core3 BOARD
core4 CORE4 BOARD
none empty machine
17)xml中指定声卡驱动,虚拟机中无声音输入、输出
==============》解决方法:hda的驱动在内核打开后(建议使用hda:HD audio,目前不支持ac97驱动和ICH9驱动),脚本参数设置如下:
--soundhw hda \
-spice port=5900,disable-ticketing \
启动后,使用spice访问虚拟机
18)xml中指定虚拟机支持显卡驱动报错:“不支持的配置: this QEMU does not support 'qxl' video device”
例子:
<video>
<model type='qxl' heads='1' primary='yes' />
<alias name='video0' />
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0' />
</video>
执行: virsh start uos-test
报错:开始域uos-test失败
错误:不支持的配置:this QEMU does not support 'qxl' video device
==============》解决方法:目前hmcode-V1.1.3及以上版本是支持qxl驱动了。在qemu源码中的default-configs/sw64-softmmu.mak文件中加入一行:CONFIG_QXL=$(CONFIG_SPICE),然后重新配置configure、编译qemu,可解决此qxl驱动支持问题;
19)是否可以提供一份libvirt支持哪些domain xml的说明文档?
=============> 解答:可以参考libvirt官方规范;
20)libvirt是否支持虚拟机挂载CD ROM?
=============> 解答:支持的
例子:
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<source file='/var/qemu-kvm/iso/server-vm.iso'/>
<target dev='sdb' bus='sata'/>
<readonly/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
21)虚拟机是否支持热插拔?
=============> 解答:目前最新的V2107版本支持热插拔
22)libvirt是否支持external开机快照?
=============> 解答:从V2107版本往后的版本是支持的。
23)是否可以提供virt-manager源码?
=============> 解答:目前此产品还在内部验证中,因未经过大规模测试,暂未发布(可咨询申威主流操作系统厂商统信、麒麟,他们已有发布版本);
24)kvm虚拟机是否支持iso引导安装?
=============> 解答:目前qemu V2107发布版还不支持,但尝鲜版master分支是支持的,可以在申威社区下载;
25)kvm虚拟机支持哪些格式的镜像?
=============> 解答:支持raw、qcow2、iso格式镜像;
26)使用libvirt源码本地编译时,报错:需要依赖automake1.15版本(系统默认安装的是1.15版本),安装后,再次make还是报错:
- configure.ac:27: error: version mismatch. This is Automake 1.15.1,
- configure.ac:27: but the definition used by this AM_INIT_AUTOMAKE
- configure.ac:27: comes from Automake 1.15. You should recreate
- configure.ac:27: aclocal.m4 with aclocal and run automake again.
- WARNING: 'automake-1.15' is probably too old.
- You should only need it if you modified 'Makefile.am' or
- 'configure.ac' or m4 files included by 'configure.ac'.
- The 'automake' program is part of the GNU Automake package:
- <http://www.gnu.org/software/automake>
- It also requires GNU Autoconf, GNU m4 and Perl in order to run:
- <http://www.gnu.org/software/autoconf>
- <http://www.gnu.org/software/m4/>
- <http://www.perl.org/>
- make[2]: *** [Makefile:6581: Makefile.in] Error 63
- make[2]: Leaving directory '/opt/zxh/kvm_20210915/libvirt/src'
- make[1]: *** [Makefile:2132: all-recursive] Error 1
- make[1]: Leaving directory '/opt/zxh/kvm_20210915/libvirt'
- make: *** [Makefile:2025:all] 错误 2
- libvirt版本commit: 5ce47653b9640a42e3848c5794ab7d016a69e216
复制代码
==============》解决方法:此原因是aclocal版本和automake版本不匹配导致,可以使用命令# autoreconf -f -i,然后重新make -j 40,再make install
27)重启libvirtd服务,报错信息如下:
- 9月 24 17:39:54 deepin-PC dnsmasq[57475]: started, version 2.80 cachesize 150
- 9月 24 17:39:54 deepin-PC dnsmasq[57475]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ip
- 9月 24 17:39:54 deepin-PC dnsmasq-dhcp[57475]: DHCP, IP range 192.168.122.2 -- 192.168.122.254, lease time 1h
- 9月 24 17:39:54 deepin-PC dnsmasq-dhcp[57475]: DHCP, sockets bound exclusively to interface virbr0
- 9月 24 17:39:54 deepin-PC dnsmasq[57475]: no servers found in /etc/resolv.conf, will retry
复制代码 ==============》解决方法:vim /etc/resolv.conf,写入了如下信息:
Generated by NetworkManager
nameserver 112.4.0.55
nameserver 223.108.108.108
nameserver 8.8.8.8
重启libvirtd服务后就不报错了,感觉应该重启NetworkManager服务。
28)启动xml报错:kvm_set_phys_mem不能分配内存
root@deepin-PC:/opt/zxh/run_files_zxh# virsh start uos830-2107_1030-ty
错误:开始域 uos830-2107_1030-ty 失败
错误:内部错误:process exited while connecting to monitor: kvm_set_phys_mem: error registering slot: Cannot allocate memory
==============》解决方法:目前先限制cma=32G或者64G以内,此bug在修复中
|
|