1 概述

virtio framework是一种适用于pv hypervisor的设备抽象框架,最初Rusty Russell开发virtio是为了给lguest虚拟化平台用的,后面kvm也是使用virtio作为其主要的pv设备模型,virtio跟xen上的pv drivers是类似的,都属于para-virtualized设备模型。para-virtualized设备模型是相对Full-virtualized设备模型而讲的,比如QEMU模拟的E1000网卡,IDE磁盘就属于full-virtualization,full-virtualized设备的坏处就是性能差,通信效率低,原因在于full-virtualizad device的datapath上经常会访问设备register,导致过多的vmexit,双向通知没有利用生产者-消费者模型中常用的按需通知导致太多VMEXIT和中断注入,另外就是一些高级特性支持上可能比较弱。

Para-virtualized设备是专门为虚拟化而生的,在提高数据传输效率,降低per-I/O开销,以及新特性(比如网卡的某些高级offloading特性,也有些新特性可能是物理世界没有的)支持,稳定性方面都做了增强。

virtio涉及前端部分(guest driver)和后端部分(qemu&vhost),大致示意图如下所示

virtio framework分析 - 图1

在上图中主要包括两部分的内容,一是virtio设备模型,一个是前后端通信机制(包括数据传递和通知),后面会分别介绍这部分内容。

2 设备模型

virtio设备模型又分为两部分,一部分是qemu侧的设备模拟,一部分是guest侧的设备驱动。

2.1 设备模拟

2.2 设备驱动

guest os中virtio的bus-device-driver之间的关系如下图所示

virtio framework分析 - 图2

上图中,virtio-pci-device起到桥梁的作用,连接了virtio pci设备和virtio设备。

virtio设备驱动整个初始化流程大概包括以下几个步骤

  • virtio-bus注册
  • virtio pci设备发现与probe
  • virtio设备的生成与注册
  • virtio设备驱动(virtio-net/virtio-blk/virtio-serial/…)probe相应的virtio设备

接下来分别介绍以上几个步骤。

2.2.1 virtio-bus注册

virtio-bus是在guestos启动时作为一个core_initcall注册的,代码如下所示

  1. static int virtio_init(void)
  2. {
  3. if (bus_register(&virtio_bus) != 0)
  4. panic("virtio bus registration failed");
  5. return 0;
  6. }
  7. static void __exit virtio_exit(void)
  8. {
  9. bus_unregister(&virtio_bus);
  10. ida_destroy(&virtio_index_ida);
  11. }
  12. core_initcall(virtio_init);
  13. module_exit(virtio_exit);

2.2.2 virtio pci设备probe

2.2.3 virtio设备的生成与注册

2.2.4 驱动virtio设备

3 前后端通信机制

4 后端性能优化

4.1 vhost

4.2 vhost-user

4 典型设备

4.1 virtio net

4.2 virtio-blk

4.3 virtio-serial

参考资料

https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.pdf

virtio framework

https://www.redhat.com/en/blog/virtio-devices-and-drivers-overview-headjack-and-phone

https://www.redhat.com/en/blog/virtqueues-and-virtio-ring-how-data-travels

https://www.redhat.com/en/blog/packed-virtqueue-how-reduce-overhead-virtio

https://www.cnblogs.com/ck1020/p/6066007.html

virtio networking/vhost/vhost-net

https://www.redhat.com/en/blog/virtio-networking-first-series-finale-and-plans-2020

https://www.redhat.com/en/blog/introducing-virtio-networking-combining-virtualization-and-networking-modern-it

http://blog.vmsplice.net/2011/09/qemu-internals-vhost-architecture.html

https://www.redhat.com/en/blog/introduction-virtio-networking-and-vhost-net

https://www.redhat.com/en/blog/deep-dive-virtio-networking-and-vhost-net

https://www.redhat.com/en/blog/hands-vhost-net-do-or-do-not-there-no-try

vhost-user

https://www.redhat.com/en/blog/how-vhost-user-came-being-virtio-networking-and-dpdk

https://www.redhat.com/en/blog/journey-vhost-users-realm

https://www.redhat.com/en/blog/hands-vhost-user-warm-welcome-dpdk

https://access.redhat.com/solutions/3394851

http://www.virtualopensystems.com/en/solutions/guides/snabbswitch-qemu/

https://wiki.qemu.org/Features/VirtioVhostUser

https://www.redhat.com/en/blog/amazing-new-observability-features-open-vswitch

https://spdk.io/doc/vhost_processing.html

vhost_net polling

https://www.mail-archive.com/netdev@vger.kernel.org/msg99188.html

https://www.mail-archive.com/netdev@vger.kernel.org/msg112365.html

vDPA

https://www.redhat.com/en/blog/achieving-network-wirespeed-open-standard-manner-introducing-vdpa

https://www.redhat.com/en/blog/how-deep-does-vdpa-rabbit-hole-go

https://www.redhat.com/en/blog/introduction-vdpa-kernel-framework

https://www.redhat.com/en/blog/vdpa-kernel-framework-part-1-vdpa-bus-abstracting-hardware

https://www.redhat.com/en/blog/vdpa-kernel-framework-part-2-vdpa-bus-drivers-kernel-subsystem-interactions

https://www.redhat.com/en/blog/vdpa-kernel-framework-part-3-usage-vms-and-containers