Multiplexer

avocado_varianter_yaml_to_mux.mux

Multiplexer or simply Mux is an abstract concept, which was the basic idea behind the tree-like params structure with the support to produce all possible variants. There is a core implementation of basic building blocks that can be used when creating a custom plugin. There is a demonstration version of plugin using this concept in avocado_varianter_yaml_to_mux which adds a parser and then uses this multiplexer concept to define an Avocado plugin to produce variants from yaml (or json) files.

Multiplexer concept

As mentioned earlier, this is an in-core implementation of building blocks intended for writing Varianter plugins based on a tree with Multiplex domains defined. The available blocks are:

  • MuxTree - Object which represents a part of the tree and handles the multiplexation, which means producing all possible variants from a tree-like object.

  • MuxPlugin - Base class to build Varianter plugins

  • MuxTreeNode - Inherits from TreeNode and adds the support for control flags (MuxTreeNode.ctrl) and multiplex domains (MuxTreeNode.multiplex).

And some support classes and methods eg. for filtering and so on.

Multiplex domains

A default avocado-params tree with variables could look like this:

Multiplex tree representation:
 ┣━━ paths
 ┃     → tmp: /var/tmp
 ┃     → qemu: /usr/libexec/qemu-kvm
 ┗━━ environ
     → debug: False

The multiplexer wants to produce similar structure, but also to be able to define not just one variant, but to define all possible combinations and then report the slices as variants. We use the term Multiplex domains to define that children of this node are not just different paths, but they are different values and we only want one at a time. In the representation we use double-line to visibly distinguish between normal relation and multiplexed relation. Let’s modify our example a bit:

Multiplex tree representation:
 ┣━━ paths
 ┃     → tmp: /var/tmp
 ┃     → qemu: /usr/libexec/qemu-kvm
 ┗━━ environ
      ╠══ production
      ║     → debug: False
      ╚══ debug
            → debug: True

The difference is that environ is now a multiplex node and it’s children will be yielded one at a time producing two variants:

Variant 1:
 ┣━━ paths
 ┃     → tmp: /var/tmp
 ┃     → qemu: /usr/libexec/qemu-kvm
 ┗━━ environ
      ┗━━ production
            → debug: False
Variant 2:
 ┣━━ paths
 ┃     → tmp: /var/tmp
 ┃     → qemu: /usr/libexec/qemu-kvm
 ┗━━ environ
      ┗━━ debug
            → debug: False

Note that the multiplex is only about direct children, therefore the number of leaves in variants might differ:

Multiplex tree representation:
 ┣━━ paths
 ┃     → tmp: /var/tmp
 ┃     → qemu: /usr/libexec/qemu-kvm
 ┗━━ environ
      ╠══ production
      ║     → debug: False
      ╚══ debug
           ┣━━ system
           ┃     → debug: False
           ┗━━ program
                 → debug: True

Produces one variant with /paths and /environ/production and other variant with /paths, /environ/debug/system and /environ/debug/program.

As mentioned earlier the power is not in producing one variant, but in defining huge scenarios with all possible variants. By using tree-structure with multiplex domains you can avoid most of the ugly filters you might know from Jenkins sparse matrix jobs. For comparison let’s have a look at the same example in Avocado:

Multiplex tree representation:
 ┗━━ os
      ┣━━ distro
      ┃    ┗━━ redhat
      ┃         ╠══ fedora
      ┃         ║    ┣━━ version
      ┃         ║    ┃    ╠══ 20
      ┃         ║    ┃    ╚══ 21
      ┃         ║    ┗━━ flavor
      ┃         ║         ╠══ workstation
      ┃         ║         ╚══ cloud
      ┃         ╚══ rhel
      ┃              ╠══ 5
      ┃              ╚══ 6
      ┗━━ arch
           ╠══ i386
           ╚══ x86_64

Which produces:

Variant 1:    /os/distro/redhat/fedora/version/20, /os/distro/redhat/fedora/flavor/workstation, /os/arch/i386
Variant 2:    /os/distro/redhat/fedora/version/20, /os/distro/redhat/fedora/flavor/workstation, /os/arch/x86_64
Variant 3:    /os/distro/redhat/fedora/version/20, /os/distro/redhat/fedora/flavor/cloud, /os/arch/i386
Variant 4:    /os/distro/redhat/fedora/version/20, /os/distro/redhat/fedora/flavor/cloud, /os/arch/x86_64
Variant 5:    /os/distro/redhat/fedora/version/21, /os/distro/redhat/fedora/flavor/workstation, /os/arch/i386
Variant 6:    /os/distro/redhat/fedora/version/21, /os/distro/redhat/fedora/flavor/workstation, /os/arch/x86_64
Variant 7:    /os/distro/redhat/fedora/version/21, /os/distro/redhat/fedora/flavor/cloud, /os/arch/i386
Variant 8:    /os/distro/redhat/fedora/version/21, /os/distro/redhat/fedora/flavor/cloud, /os/arch/x86_64
Variant 9:    /os/distro/redhat/rhel/5, /os/arch/i386
Variant 10:    /os/distro/redhat/rhel/5, /os/arch/x86_64
Variant 11:    /os/distro/redhat/rhel/6, /os/arch/i386
Variant 12:    /os/distro/redhat/rhel/6, /os/arch/x86_64

Versus Jenkins sparse matrix:

os_version = fedora20 fedora21 rhel5 rhel6
os_flavor = none workstation cloud
arch = i386 x86_64

filter = ((os_version == "rhel5").implies(os_flavor == "none") &&
          (os_version == "rhel6").implies(os_flavor == "none")) &&
         !(os_version == "fedora20" && os_flavor == "none") &&
         !(os_version == "fedora21" && os_flavor == "none")

Which is still relatively simple example, but it grows dramatically with inner-dependencies.

MuxPlugin

avocado_varianter_yaml_to_mux.mux.MuxPlugin

Defines the full interface required by avocado.core.plugin_interfaces.Varianter. The plugin writer should inherit from this MuxPlugin, then from the Varianter and call the:

self.initialize_mux(root, paths, debug)

Where:

  • root - is the root of your params tree (compound of TreeNode -like nodes)

  • paths - is the Parameter Paths to be used in test with all variants

  • debug - whether to use debug mode (requires the passed tree to be compound of TreeNodeDebug-like nodes which stores the origin of the variant/value/environment as the value for listing purposes and is __NOT__ intended for test execution.

This method must be called before the Varianter’s second stage. The MuxPlugin’s code will take care of the rest.

MuxTree

This is the core feature where the hard work happens. It walks the tree and remembers all leaf nodes or uses list of MuxTrees when another multiplex domain is reached while searching for a leaf.

When it’s asked to report variants, it combines one variant of each remembered item (leaf node always stays the same, but MuxTree circles through it’s values) which recursively produces all possible variants of different multiplex domains.