本文首发于泊浮目的专栏：https://segmentfault.com/blog/camile

背景

在上篇文章中（ZStack源码剖析之二次开发——可扩展框架
），我们简单的了解了一下ZStack核心引擎的二次开发技巧。在这篇文章中，我们将一起来了解[ZStack-Utility](https://github.com/zstackio/zstack-utility)（即ZStack的Agent端）的二开姿势。

例子

我们以ZStack管理节点调用startVm这个api为例子，一起来看一下在agent上的执行逻辑。

  1.     def start(self):
  2.         http_server = kvmagent.get_http_server()
  4.         http_server.register_async_uri(self.KVM_START_VM_PATH, self.start_vm)

首先，得注册一个http path用来接受reqeust。

  1.     @kvmagent.replyerror
  2.     def start_vm(self, req):
  3.         cmd = jsonobject.loads(req[http.REQUEST_BODY])
  4.         rsp = StartVmResponse()
  5.         try:
  6.             self._record_operation(cmd.vmInstanceUuid, self.VM_OP_START)
  8.             self._start_vm(cmd)
  9.             logger.debug('successfully started vm[uuid:%s, name:%s]' % (cmd.vmInstanceUuid, cmd.vmName))
  10.         except kvmagent.KvmError as e:
  11.             e_str = linux.get_exception_stacktrace()
  12.             logger.warn(e_str)
  13.             if "burst" in e_str and "Illegal" in e_str and "rate" in e_str:
  14.                 rsp.error = "QoS exceed max limit, please check and reset it in zstack"
  15.             elif "cannot set up guest memory" in e_str:
  16.                 logger.warn('unable to start vm[uuid:%s], %s' % (cmd.vmInstanceUuid, e_str))
  17.                 rsp.error = "No enough physical memory for guest"
  18.             else:
  19.                 rsp.error = e_str
  20.             err = self.handle_vfio_irq_conflict(cmd.vmInstanceUuid)
  21.             if err != "":
  22.                 rsp.error = "%s, details: %s" % (err, rsp.error)
  23.             rsp.success = False
  24.         return jsonobject.dumps(rsp)

直接进入主干逻辑，self._start_vm(cmd)。

  1.     @lock.lock('libvirt-startvm')
  2.     def _start_vm(self, cmd):
  3.         try:
  4.             vm = get_vm_by_uuid_no_retry(cmd.vmInstanceUuid, False)
  6.             if vm:
  7.                 if vm.state == Vm.VM_STATE_RUNNING:
  8.                     raise kvmagent.KvmError(
  9.                         'vm[uuid:%s, name:%s] is already running' % (cmd.vmInstanceUuid, vm.get_name()))
  10.                 else:
  11.                     vm.destroy()
  13.             vm = Vm.from_StartVmCmd(cmd)
  14.             vm.start(cmd.timeout)
  15.         except libvirt.libvirtError as e:
  16.             logger.warn(linux.get_exception_stacktrace())
  17.             if "Device or resource busy" in str(e.message):
  18.                 raise kvmagent.KvmError(
  19.                     'unable to start vm[uuid:%s, name:%s], libvirt error: %s' % (
  20.                     cmd.vmInstanceUuid, cmd.vmName, str(e)))
  22.             try:
  23.                 vm = get_vm_by_uuid(cmd.vmInstanceUuid)
  24.                 if vm and vm.state != Vm.VM_STATE_RUNNING:
  25.                     raise kvmagent.KvmError(
  26.                        'vm[uuid:%s, name:%s, state:%s] is not in running state, libvirt error: %s' % (
  27.                         cmd.vmInstanceUuid, cmd.vmName, vm.state, str(e)))
  29.             except kvmagent.KvmError:
  30.                 raise kvmagent.KvmError(
  31.                     'unable to start vm[uuid:%s, name:%s], libvirt error: %s' % (cmd.vmInstanceUuid, cmd.vmName, str(e)))

关键逻辑：

  1.             vm = Vm.from_StartVmCmd(cmd)
  2.             vm.start(cmd.timeout)

先看from_StartVmCmd

  1.     @staticmethod
  2.     def from_StartVmCmd(cmd):
  3.         use_virtio = cmd.useVirtio
  4.         use_numa = cmd.useNuma
  6.         elements = {}
  8.         def make_root():
  9.             root = etree.Element('domain')
  10.             root.set('type', 'kvm')
  11.             # self._root.set('type', 'qemu')
  12.             root.set('xmlns:qemu', 'http://libvirt.org/schemas/domain/qemu/1.0')
  13.             elements['root'] = root
  15.         def make_cpu():
  16.             if use_numa:
  17.                 root = elements['root']
  18.                 e(root, 'vcpu', '128', {'placement': 'static', 'current': str(cmd.cpuNum)})
  19.                 # e(root,'vcpu',str(cmd.cpuNum),{'placement':'static'})
  20.                 tune = e(root, 'cputune')
  21.                 e(tune, 'shares', str(cmd.cpuSpeed * cmd.cpuNum))
  22.                 # enable nested virtualization
  23.                 if cmd.nestedVirtualization == 'host-model':
  24.                     cpu = e(root, 'cpu', attrib={'mode': 'host-model'})
  25.                     e(cpu, 'model', attrib={'fallback': 'allow'})
  26.                 elif cmd.nestedVirtualization == 'host-passthrough':
  27.                     cpu = e(root, 'cpu', attrib={'mode': 'host-passthrough'})
  28.                     e(cpu, 'model', attrib={'fallback': 'allow'})
  29.                 elif IS_AARCH64:
  30.                     cpu = e(root, 'cpu', attrib={'mode': 'host-passthrough'})
  31.                     e(cpu, 'model', attrib={'fallback': 'allow'})
  32.                 else:
  33.                     cpu = e(root, 'cpu')
  34.                     # e(cpu, 'topology', attrib={'sockets': str(cmd.socketNum), 'cores': str(cmd.cpuOnSocket), 'threads': '1'})
  35.                 mem = cmd.memory / 1024
  36.                 e(cpu, 'topology', attrib={'sockets': str(32), 'cores': str(4), 'threads': '1'})
  37.                 numa = e(cpu, 'numa')
  38.                 e(numa, 'cell', attrib={'id': '0', 'cpus': '0-127', 'memory': str(mem), 'unit': 'KiB'})
  39.             else:
  40.                 root = elements['root']
  41.                 # e(root, 'vcpu', '128', {'placement': 'static', 'current': str(cmd.cpuNum)})
  42.                 e(root, 'vcpu', str(cmd.cpuNum), {'placement': 'static'})
  43.                 tune = e(root, 'cputune')
  44.                 e(tune, 'shares', str(cmd.cpuSpeed * cmd.cpuNum))
  45.                 # enable nested virtualization
  46.                 if cmd.nestedVirtualization == 'host-model':
  47.                     cpu = e(root, 'cpu', attrib={'mode': 'host-model'})
  48.                     e(cpu, 'model', attrib={'fallback': 'allow'})
  49.                 elif cmd.nestedVirtualization == 'host-passthrough':
  50.                     cpu = e(root, 'cpu', attrib={'mode': 'host-passthrough'})
  51.                     e(cpu, 'model', attrib={'fallback': 'allow'})
  52.                 elif IS_AARCH64:
  53.                     cpu = e(root, 'cpu', attrib={'mode': 'host-passthrough'})
  54.                     e(cpu, 'model', attrib={'fallback': 'allow'})
  55.                 else:
  56.                     cpu = e(root, 'cpu')
  57.                 e(cpu, 'topology', attrib={'sockets': str(cmd.socketNum), 'cores': str(cmd.cpuOnSocket), 'threads': '1'})
  59.         def make_memory():
  60.             root = elements['root']
  61.             mem = cmd.memory / 1024
  62.             if use_numa:
  63.                 e(root, 'maxMemory', str(68719476736), {'slots': str(16), 'unit': 'KiB'})
  64.                 # e(root,'memory',str(mem),{'unit':'k'})
  65.                 e(root, 'currentMemory', str(mem), {'unit': 'k'})
  66.             else:
  67.                 e(root, 'memory', str(mem), {'unit': 'k'})
  68.                 e(root, 'currentMemory', str(mem), {'unit': 'k'})
  70.         def make_os():
  71.             root = elements['root']
  72.             os = e(root, 'os')
  73.             if IS_AARCH64:
  74.                 e(os, 'type', 'hvm', attrib={'arch': 'aarch64'})
  75.                 e(os, 'loader', '/usr/share/edk2.git/aarch64/QEMU_EFI-pflash.raw', attrib={'readonly': 'yes', 'type': 'pflash'})
  76.             else:
  77.                 e(os, 'type', 'hvm', attrib={'machine': 'pc'})
  78.             # if not booting from cdrom, don't add any boot element in os section
  79.             if cmd.bootDev[0] == "cdrom":
  80.                 for boot_dev in cmd.bootDev:
  81.                     e(os, 'boot', None, {'dev': boot_dev})
  83.             if cmd.useBootMenu:
  84.                 e(os, 'bootmenu', attrib={'enable': 'yes'})
  86.         def make_features():
  87.             root = elements['root']
  88.             features = e(root, 'features')
  89.             for f in ['acpi', 'apic', 'pae']:
  90.                 e(features, f)
  91.             if cmd.kvmHiddenState == True:
  92.                 kvm = e(features, "kvm")
  93.                 e(kvm, 'hidden', None, {'state': 'on'})
  95.         def make_devices():
  96.             root = elements['root']
  97.             devices = e(root, 'devices')
  98.             if cmd.addons and cmd.addons['qemuPath']:
  99.                 e(devices, 'emulator', cmd.addons['qemuPath'])
  100.             else:
  101.                 e(devices, 'emulator', kvmagent.get_qemu_path())
  102.             tablet = e(devices, 'input', None, {'type': 'tablet', 'bus': 'usb'})
  103.             e(tablet, 'address', None, {'type':'usb', 'bus':'0', 'port':'1'})
  104.             if IS_AARCH64:
  105.                 keyboard = e(devices, 'input', None, {'type': 'keyboard', 'bus': 'usb'})
  106.             elements['devices'] = devices
  108.         def make_cdrom():
  109.             devices = elements['devices']
  111.             MAX_CDROM_NUM = len(Vm.ISO_DEVICE_LETTERS)
  112.             EMPTY_CDROM_CONFIGS = None
  114.             if IS_AARCH64:
  115.                 # AArch64 Does not support the attachment of multiple iso
  116.                 EMPTY_CDROM_CONFIGS = [
  117.                     EmptyCdromConfig(None, None, None)
  118.                 ]
  119.             else:
  120.                 # bus 0 unit 0 already use by root volume
  121.                 EMPTY_CDROM_CONFIGS = [
  122.                     EmptyCdromConfig('hd%s' % Vm.ISO_DEVICE_LETTERS[0], '0', '1'),
  123.                     EmptyCdromConfig('hd%s' % Vm.ISO_DEVICE_LETTERS[1], '1', '0'),
  124.                     EmptyCdromConfig('hd%s' % Vm.ISO_DEVICE_LETTERS[2], '1', '1')
  125.                 ]
  127.             if len(EMPTY_CDROM_CONFIGS) != MAX_CDROM_NUM:
  128.                 logger.error('ISO_DEVICE_LETTERS or EMPTY_CDROM_CONFIGS config error')
  130.             def makeEmptyCdrom(targetDev, bus, unit):
  131.                 cdrom = e(devices, 'disk', None, {'type': 'file', 'device': 'cdrom'})
  132.                 e(cdrom, 'driver', None, {'name': 'qemu', 'type': 'raw'})
  133.                 if IS_AARCH64:
  134.                     e(cdrom, 'target', None, {'dev': 'sdc', 'bus': 'scsi'})
  135.                 else:
  136.                     e(cdrom, 'target', None, {'dev': targetDev, 'bus': 'ide'})
  137.                     e(cdrom, 'address', None,{'type' : 'drive', 'bus' : bus, 'unit' : unit})
  138.                 e(cdrom, 'readonly', None)
  139.                 return cdrom
  141.             if not cmd.bootIso:
  142.                 for config in EMPTY_CDROM_CONFIGS:
  143.                     makeEmptyCdrom(config.targetDev, config.bus, config.unit)
  144.                 return
  146.             notEmptyCdrom = set([])
  147.             for iso in cmd.bootIso:
  148.                 notEmptyCdrom.add(iso.deviceId)
  149.                 cdromConfig = EMPTY_CDROM_CONFIGS[iso.deviceId]
  150.                 if iso.path.startswith('ceph'):
  151.                     ic = IsoCeph()
  152.                     ic.iso = iso
  153.                     devices.append(ic.to_xmlobject(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit))
  154.                 elif iso.path.startswith('fusionstor'):
  155.                     ic = IsoFusionstor()
  156.                     ic.iso = iso
  157.                     devices.append(ic.to_xmlobject(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit))
  158.                 else:
  159.                     cdrom = makeEmptyCdrom(cdromConfig.targetDev, cdromConfig.bus , cdromConfig.unit)
  160.                     e(cdrom, 'source', None, {'file': iso.path})
  162.             emptyCdrom = set(range(MAX_CDROM_NUM)).difference(notEmptyCdrom)
  163.             for i in emptyCdrom:
  164.                 cdromConfig = EMPTY_CDROM_CONFIGS[i]
  165.                 makeEmptyCdrom(cdromConfig.targetDev, cdromConfig.bus, cdromConfig.unit)
  167.         def make_volumes():
  168.             devices = elements['devices']
  169.             volumes = [cmd.rootVolume]
  170.             volumes.extend(cmd.dataVolumes)
  172.             def filebased_volume(_dev_letter, _v):
  173.                 disk = etree.Element('disk', {'type': 'file', 'device': 'disk', 'snapshot': 'external'})
  174.                 e(disk, 'driver', None, {'name': 'qemu', 'type': linux.get_img_fmt(_v.installPath), 'cache': _v.cacheMode})
  175.                 e(disk, 'source', None, {'file': _v.installPath})
  177.                 if _v.shareable:
  178.                     e(disk, 'shareable')
  180.                 if _v.useVirtioSCSI:
  181.                     e(disk, 'target', None, {'dev': 'sd%s' % _dev_letter, 'bus': 'scsi'})
  182.                     e(disk, 'wwn', _v.wwn)
  183.                     e(disk, 'address', None, {'type': 'drive', 'controller': '0', 'unit': str(_v.deviceId)})
  184.                     return disk
  186.                 if _v.useVirtio:
  187.                     e(disk, 'target', None, {'dev': 'vd%s' % _dev_letter, 'bus': 'virtio'})
  188.                 elif IS_AARCH64:
  189.                     e(disk, 'target', None, {'dev': 'sd%s' % _dev_letter, 'bus': 'scsi'})
  190.                 else:
  191.                     e(disk, 'target', None, {'dev': 'sd%s' % _dev_letter, 'bus': 'ide'})
  192.                 return disk
  194.             def iscsibased_volume(_dev_letter, _v):
  195.                 def blk_iscsi():
  196.                     bi = BlkIscsi()
  197.                     portal, bi.target, bi.lun = _v.installPath.lstrip('iscsi://').split('/')
  198.                     bi.server_hostname, bi.server_port = portal.split(':')
  199.                     bi.device_letter = _dev_letter
  200.                     bi.volume_uuid = _v.volumeUuid
  201.                     bi.chap_username = _v.chapUsername
  202.                     bi.chap_password = _v.chapPassword
  204.                     return bi.to_xmlobject()
  206.                 def virtio_iscsi():
  207.                     vi = VirtioIscsi()
  208.                     portal, vi.target, vi.lun = _v.installPath.lstrip('iscsi://').split('/')
  209.                     vi.server_hostname, vi.server_port = portal.split(':')
  210.                     vi.device_letter = _dev_letter
  211.                     vi.volume_uuid = _v.volumeUuid
  212.                     vi.chap_username = _v.chapUsername
  213.                     vi.chap_password = _v.chapPassword
  215.                     return vi.to_xmlobject()
  217.                 if _v.useVirtio:
  218.                     return virtio_iscsi()
  219.                 else:
  220.                     return blk_iscsi()
  222.             def ceph_volume(_dev_letter, _v):
  223.                 def ceph_virtio():
  224.                     vc = VirtioCeph()
  225.                     vc.volume = _v
  226.                     vc.dev_letter = _dev_letter
  227.                     return vc.to_xmlobject()
  229.                 def ceph_blk():
  230.                     if not IS_AARCH64:
  231.                         ic = IdeCeph()
  232.                     else:
  233.                         ic = ScsiCeph()
  234.                     ic.volume = _v
  235.                     ic.dev_letter = _dev_letter
  236.                     return ic.to_xmlobject()
  238.                 def ceph_virtio_scsi():
  239.                     vsc = VirtioSCSICeph()
  240.                     vsc.volume = _v
  241.                     vsc.dev_letter = _dev_letter
  242.                     return vsc.to_xmlobject()
  244.                 if _v.useVirtioSCSI:
  245.                     disk = ceph_virtio_scsi()
  246.                     if _v.shareable:
  247.                         e(disk, 'shareable')
  248.                     return disk
  250.                 if _v.useVirtio:
  251.                     return ceph_virtio()
  252.                 else:
  253.                     return ceph_blk()
  255.             def fusionstor_volume(_dev_letter, _v):
  256.                 def fusionstor_virtio():
  257.                     vc = VirtioFusionstor()
  258.                     vc.volume = _v
  259.                     vc.dev_letter = _dev_letter
  260.                     return vc.to_xmlobject()
  262.                 def fusionstor_blk():
  263.                     ic = IdeFusionstor()
  264.                     ic.volume = _v
  265.                     ic.dev_letter = _dev_letter
  266.                     return ic.to_xmlobject()
  268.                 def fusionstor_virtio_scsi():
  269.                     vsc = VirtioSCSIFusionstor()
  270.                     vsc.volume = _v
  271.                     vsc.dev_letter = _dev_letter
  272.                     return vsc.to_xmlobject()
  274.                 if _v.useVirtioSCSI:
  275.                     disk = fusionstor_virtio_scsi()
  276.                     if _v.shareable:
  277.                         e(disk, 'shareable')
  278.                     return disk
  280.                 if _v.useVirtio:
  281.                     return fusionstor_virtio()
  282.                 else:
  283.                     return fusionstor_blk()
  285.             def volume_qos(volume_xml_obj):
  286.                 if not cmd.addons:
  287.                     return
  289.                 vol_qos = cmd.addons['VolumeQos']
  290.                 if not vol_qos:
  291.                     return
  293.                 qos = vol_qos[v.volumeUuid]
  294.                 if not qos:
  295.                     return
  297.                 if not qos.totalBandwidth and not qos.totalIops:
  298.                     return
  300.                 iotune = e(volume_xml_obj, 'iotune')
  301.                 if qos.totalBandwidth:
  302.                     e(iotune, 'total_bytes_sec', str(qos.totalBandwidth))
  303.                 if qos.totalIops:
  304.                     # e(iotune, 'total_iops_sec', str(qos.totalIops))
  305.                     e(iotune, 'read_iops_sec', str(qos.totalIops))
  306.                     e(iotune, 'write_iops_sec', str(qos.totalIops))
  307.                     # e(iotune, 'read_iops_sec_max', str(qos.totalIops))
  308.                     # e(iotune, 'write_iops_sec_max', str(qos.totalIops))
  309.                     # e(iotune, 'total_iops_sec_max', str(qos.totalIops))
  311.             volumes.sort(key=lambda d: d.deviceId)
  312.             scsi_device_ids = [v.deviceId for v in volumes if v.useVirtioSCSI]
  313.             for v in volumes:
  314.                 if v.deviceId >= len(Vm.DEVICE_LETTERS):
  315.                     err = "exceeds max disk limit, it's %s but only 26 allowed" % v.deviceId
  316.                     logger.warn(err)
  317.                     raise kvmagent.KvmError(err)
  319.                 dev_letter = Vm.DEVICE_LETTERS[v.deviceId]
  320.                 if v.useVirtioSCSI:
  321.                     dev_letter = Vm.DEVICE_LETTERS[scsi_device_ids.pop()]
  323.                 if v.deviceType == 'file':
  324.                     vol = filebased_volume(dev_letter, v)
  325.                 elif v.deviceType == 'iscsi':
  326.                     vol = iscsibased_volume(dev_letter, v)
  327.                 elif v.deviceType == 'ceph':
  328.                     vol = ceph_volume(dev_letter, v)
  329.                 elif v.deviceType == 'fusionstor':
  330.                     vol = fusionstor_volume(dev_letter, v)
  331.                 else:
  332.                     raise Exception('unknown volume deviceType: %s' % v.deviceType)
  334.                 assert vol is not None, 'vol cannot be None'
  335.                 # set boot order for root volume when boot from hd
  336.                 if v.deviceId == 0 and cmd.bootDev[0] == 'hd' and cmd.useBootMenu:
  337.                     e(vol, 'boot', None, {'order': '1'})
  338.                 volume_qos(vol)
  339.                 devices.append(vol)
  341.         def make_nics():
  342.             if not cmd.nics:
  343.                 return
  345.             def nic_qos(nic_xml_object):
  346.                 if not cmd.addons:
  347.                     return
  349.                 nqos = cmd.addons['NicQos']
  350.                 if not nqos:
  351.                     return
  353.                 qos = nqos[nic.uuid]
  354.                 if not qos:
  355.                     return
  357.                 if not qos.outboundBandwidth and not qos.inboundBandwidth:
  358.                     return
  360.                 bandwidth = e(nic_xml_object, 'bandwidth')
  361.                 if qos.outboundBandwidth:
  362.                     e(bandwidth, 'outbound', None, {'average': str(qos.outboundBandwidth / 1024 / 8)})
  363.                 if qos.inboundBandwidth:
  364.                     e(bandwidth, 'inbound', None, {'average': str(qos.inboundBandwidth / 1024 / 8)})
  366.             devices = elements['devices']
  367.             for nic in cmd.nics:
  368.                 interface = e(devices, 'interface', None, {'type': 'bridge'})
  369.                 e(interface, 'mac', None, {'address': nic.mac})
  370.                 if nic.ip is not None and nic.ip != "":
  371.                     filterref = e(interface, 'filterref', None, {'filter':'clean-traffic'})
  372.                     e(filterref, 'parameter', None, {'name':'IP', 'value': nic.ip})
  373.                 e(interface, 'alias', None, {'name': 'net%s' % nic.nicInternalName.split('.')[1]})
  374.                 e(interface, 'source', None, {'bridge': nic.bridgeName})
  375.                 if use_virtio:
  376.                     e(interface, 'model', None, {'type': 'virtio'})
  377.                 else:
  378.                     e(interface, 'model', None, {'type': 'e1000'})
  379.                 e(interface, 'target', None, {'dev': nic.nicInternalName})
  381.                 nic_qos(interface)
  383.         def make_meta():
  384.             root = elements['root']
  386.             e(root, 'name', cmd.vmInstanceUuid)
  387.             e(root, 'uuid', uuidhelper.to_full_uuid(cmd.vmInstanceUuid))
  388.             e(root, 'description', cmd.vmName)
  389.             e(root, 'on_poweroff', 'destroy')
  390.             e(root, 'on_crash', 'restart')
  391.             e(root, 'on_reboot', 'restart')
  392.             meta = e(root, 'metadata')
  393.             zs = e(meta, 'zstack', usenamesapce=True)
  394.             e(zs, 'internalId', str(cmd.vmInternalId))
  395.             e(zs, 'hostManagementIp', str(cmd.hostManagementIp))
  396.             clock = e(root, 'clock', None, {'offset': cmd.clock})
  397.             if cmd.clock == 'localtime':
  398.                 e(clock, 'timer', None, {'name': 'rtc', 'tickpolicy': 'catchup'})
  399.                 e(clock, 'timer', None, {'name': 'pit', 'tickpolicy': 'delay'})
  400.                 e(clock, 'timer', None, {'name': 'hpet', 'present': 'no'})
  401.                 e(clock, 'timer', None, {'name': 'hypervclock', 'present': 'yes'})
  403.         def make_vnc():
  404.             devices = elements['devices']
  405.             if cmd.consolePassword == None:
  406.                 vnc = e(devices, 'graphics', None, {'type': 'vnc', 'port': '5900', 'autoport': 'yes'})
  407.             else:
  408.                 vnc = e(devices, 'graphics', None,
  409.                         {'type': 'vnc', 'port': '5900', 'autoport': 'yes', 'passwd': str(cmd.consolePassword)})
  410.             e(vnc, "listen", None, {'type': 'address', 'address': '0.0.0.0'})
  412.         def make_spice():
  413.             devices = elements['devices']
  414.             spice = e(devices, 'graphics', None, {'type': 'spice', 'port': '5900', 'autoport': 'yes'})
  415.             e(spice, "listen", None, {'type': 'address', 'address': '0.0.0.0'})
  416.             e(spice, "image", None, {'compression': 'auto_glz'})
  417.             e(spice, "jpeg", None, {'compression': 'always'})
  418.             e(spice, "zlib", None, {'compression': 'never'})
  419.             e(spice, "playback", None, {'compression': 'off'})
  420.             e(spice, "streaming", None, {'mode': cmd.spiceStreamingMode})
  421.             e(spice, "mouse", None, {'mode': 'client'})
  422.             e(spice, "filetransfer", None, {'enable': 'no'})
  423.             e(spice, "clipboard", None, {'copypaste': 'no'})
  425.         def make_usb_redirect():
  426.             if cmd.usbRedirect == "true":
  427.                 devices = elements['devices']
  428.                 e(devices, 'controller', None, {'type': 'usb', 'model': 'ich9-ehci1'})
  429.                 e(devices, 'controller', None, {'type': 'usb', 'model': 'ich9-uhci1', 'multifunction': 'on'})
  430.                 e(devices, 'controller', None, {'type': 'usb', 'model': 'ich9-uhci2'})
  431.                 e(devices, 'controller', None, {'type': 'usb', 'model': 'ich9-uhci3'})
  433.                 chan = e(devices, 'channel', None, {'type': 'spicevmc'})
  434.                 e(chan, 'target', None, {'type': 'virtio', 'name': 'com.redhat.spice.0'})
  435.                 e(chan, 'address', None, {'type': 'virtio-serial'})
  437.                 redirdev2 = e(devices, 'redirdev', None, {'type': 'spicevmc', 'bus': 'usb'})
  438.                 e(redirdev2, 'address', None, {'type': 'usb', 'bus': '0', 'port': '2'})
  439.                 redirdev3 = e(devices, 'redirdev', None, {'type': 'spicevmc', 'bus': 'usb'})
  440.                 e(redirdev3, 'address', None, {'type': 'usb', 'bus': '0', 'port': '3'})
  441.                 redirdev4 = e(devices, 'redirdev', None, {'type': 'spicevmc', 'bus': 'usb'})
  442.                 e(redirdev4, 'address', None, {'type': 'usb', 'bus': '0', 'port': '4'})
  443.                 redirdev5 = e(devices, 'redirdev', None, {'type': 'spicevmc', 'bus': 'usb'})
  444.                 e(redirdev5, 'address', None, {'type': 'usb', 'bus': '0', 'port': '6'})
  445.             else:
  446.                 # make sure there are three default usb controllers, for usb 1.1/2.0/3.0
  447.                 devices = elements['devices']
  448.                 e(devices, 'controller', None, {'type': 'usb', 'index': '0'})
  449.                 if not IS_AARCH64:
  450.                     e(devices, 'controller', None, {'type': 'usb', 'index': '1', 'model': 'ehci'})
  451.                     e(devices, 'controller', None, {'type': 'usb', 'index': '2', 'model': 'nec-xhci'})
  453.         def make_video():
  454.             devices = elements['devices']
  455.             if IS_AARCH64:
  456.                 video = e(devices, 'video')
  457.                 e(video, 'model', None, {'type': 'virtio'})
  458.             elif cmd.videoType != "qxl":
  459.                 video = e(devices, 'video')
  460.                 e(video, 'model', None, {'type': str(cmd.videoType)})
  461.             else:
  462.                 for monitor in range(cmd.VDIMonitorNumber):
  463.                     video = e(devices, 'video')
  464.                     e(video, 'model', None, {'type': str(cmd.videoType)})
  467.         def make_audio_microphone():
  468.             if cmd.consoleMode == 'spice':
  469.                 devices = elements['devices']
  470.                 e(devices, 'sound',None,{'model':'ich6'})
  471.             else:
  472.                 return
  474.         def make_graphic_console():
  475.             if cmd.consoleMode == 'spice':
  476.                 make_spice()
  477.             else:
  478.                 make_vnc()
  480.         def make_addons():
  481.             if not cmd.addons:
  482.                 return
  484.             devices = elements['devices']
  485.             channel = cmd.addons['channel']
  486.             if channel:
  487.                 basedir = os.path.dirname(channel.socketPath)
  488.                 linux.mkdir(basedir, 0777)
  489.                 chan = e(devices, 'channel', None, {'type': 'unix'})
  490.                 e(chan, 'source', None, {'mode': 'bind', 'path': channel.socketPath})
  491.                 e(chan, 'target', None, {'type': 'virtio', 'name': channel.targetName})
  493.             cephSecretKey = cmd.addons['ceph_secret_key']
  494.             cephSecretUuid = cmd.addons['ceph_secret_uuid']
  495.             if cephSecretKey and cephSecretUuid:
  496.                 VmPlugin._create_ceph_secret_key(cephSecretKey, cephSecretUuid)
  498.             pciDevices = cmd.addons['pciDevice']
  499.             if pciDevices:
  500.                 make_pci_device(pciDevices)
  502.             usbDevices = cmd.addons['usbDevice']
  503.             if usbDevices:
  504.                 make_usb_device(usbDevices)
  506.         def make_pci_device(addresses):
  507.             devices = elements['devices']
  508.             for addr in addresses:
  509.                 if match_pci_device(addr):
  510.                     hostdev = e(devices, "hostdev", None, {'mode': 'subsystem', 'type': 'pci', 'managed': 'yes'})
  511.                     e(hostdev, "driver", None, {'name': 'vfio'})
  512.                     source = e(hostdev, "source")
  513.                     e(source, "address", None, {
  514.                         "domain": hex(0) if len(addr.split(":")) == 2 else hex(int(addr.split(":")[0], 16)),
  515.                         "bus": hex(int(addr.split(":")[-2], 16)),
  516.                         "slot": hex(int(addr.split(":")[-1].split(".")[0], 16)),
  517.                         "function": hex(int(addr.split(":")[-1].split(".")[1], 16))
  518.                     })
  519.                 else:
  520.                     raise kvmagent.KvmError(
  521.                        'can not find pci device for address %s' % addr)
  523.         def make_usb_device(usbDevices):
  524.             next_uhci_port = 2
  525.             next_ehci_port = 1
  526.             next_xhci_port = 1
  527.             devices = elements['devices']
  528.             for usb in usbDevices:
  529.                 if match_usb_device(usb):
  530.                     hostdev = e(devices, "hostdev", None, {'mode': 'subsystem', 'type': 'usb', 'managed': 'yes'})
  531.                     source = e(hostdev, "source")
  532.                     e(source, "address", None, {
  533.                         "bus": str(int(usb.split(":")[0])),
  534.                         "device": str(int(usb.split(":")[1]))
  535.                     })
  536.                     e(source, "vendor", None, {
  537.                         "id": hex(int(usb.split(":")[2], 16))
  538.                     })
  539.                     e(source, "product", None, {
  540.                         "id": hex(int(usb.split(":")[3], 16))
  541.                     })
  543.                     # get controller index from usbVersion
  544.                     # eg. 1.1 -> 0
  545.                     # eg. 2.0.0 -> 1
  546.                     # eg. 3 -> 2
  547.                     bus = int(usb.split(":")[4][0]) - 1
  548.                     if bus == 0:
  549.                         address = e(hostdev, "address", None, {'type': 'usb', 'bus': str(bus), 'port': str(next_uhci_port)})
  550.                         next_uhci_port += 1
  551.                     elif bus == 1:
  552.                         address = e(hostdev, "address", None, {'type': 'usb', 'bus': str(bus), 'port': str(next_ehci_port)})
  553.                         next_ehci_port += 1
  554.                     elif bus == 2:
  555.                         address = e(hostdev, "address", None, {'type': 'usb', 'bus': str(bus), 'port': str(next_xhci_port)})
  556.                         next_xhci_port += 1
  557.                     else:
  558.                         raise kvmagent.KvmError('unknown usb controller %s', bus)
  559.                 else:
  560.                     raise kvmagent.KvmError('cannot find usb device %s', usb)
  562.         # TODO(WeiW) Validate here
  563.         def match_pci_device(addr):
  564.             return True
  566.         def match_usb_device(addr):
  567.             if len(addr.split(':')) == 5:
  568.                 return True
  569.             else:
  570.                 return False
  572.         def make_balloon_memory():
  573.             devices = elements['devices']
  574.             b = e(devices, 'memballoon', None, {'model': 'virtio'})
  575.             e(b, 'stats', None, {'period': '10'})
  577.         def make_console():
  578.             devices = elements['devices']
  579.             serial = e(devices, 'serial', None, {'type': 'pty'})
  580.             e(serial, 'target', None, {'port': '0'})
  581.             console = e(devices, 'console', None, {'type': 'pty'})
  582.             e(console, 'target', None, {'type': 'serial', 'port': '0'})
  584.         def make_sec_label():
  585.             root = elements['root']
  586.             e(root, 'seclabel', None, {'type': 'none'})
  588.         def make_controllers():
  589.             devices = elements['devices']
  590.             e(devices, 'controller', None, {'type': 'scsi', 'model': 'virtio-scsi'})
  592.         make_root()
  593.         make_meta()
  594.         make_cpu()
  595.         make_memory()
  596.         make_os()
  597.         make_features()
  598.         make_devices()
  599.         make_video()
  600.         make_audio_microphone()
  601.         make_nics()
  602.         make_volumes()
  603.         make_cdrom()
  604.         make_graphic_console()
  605.         make_usb_redirect()
  606.         make_addons()
  607.         make_balloon_memory()
  608.         make_console()
  609.         make_sec_label()
  610.         make_controllers()
  612.         root = elements['root']
  613.         xml = etree.tostring(root)
  615.         vm = Vm()
  616.         vm.uuid = cmd.vmInstanceUuid
  617.         vm.domain_xml = xml
  618.         vm.domain_xmlobject = xmlobject.loads(xml)
  619.         return vm

显然，上述逻辑是在组装一份xml，便于之后的libvirt使用。

然后是

  1.  vm.start(cmd.timeout)

可以看到，这里是直接调用了libvirt的sdk。

这仅仅是一个调用流程。而在很多地方，来自MN的请求会直接调用linux的shell命令，详情见linux.py。（获取云盘大小、主存储容量等）。

问题

在基于扩展ZStack的Agent时，如果是一个全新的功能模块，可能并不会造成和原有代码的深度耦合。但如果在原有功能上的增强， 对原有代码进行修改可能会导致我们的业务逻辑和Utility的上游代码耦合。而在没有足够人力来维护、开发ZStack时，我们会将目标定为能够及时跟上发布版本。 因此，我们要尽量减少冲突。

举个例子：我们要对启动vm的逻辑进行增强，添加一个自己的配置写入xml。这段代码如果写进了vm_plugin.py，那么就是一个耦合。耦合多了以后，跟上发布版本就会很困难。

解决方案

这是一个参考方案：

如果是引入一个全新的功能模块，建议重写一个项目。无论是代码规范还是自动化测试，都可以有一个很好的实践。

如果是基于Utility的扩展，比如对于扩展的api——APIStartVmInstanceExMsg。由上游发送http request时，将指定v2版本的agent。比如原有start vm会发送至path：AGENT_IP:7070/vm/start；而如果我们增强了这部分逻辑，将这段代码copy至vm_plugin_ex.py，并注册一个path，ex/vm/start。当然port也要重新注册一个，就像这样：：AGENT_IP:7071/ex/vm/start。

同样的，对linux.py扩展时，复制一个linux2.py来存放属于我们自己的扩展逻辑。