Muzzle is a safety feature of the Java agent that prevents applying instrumentation when a mismatch
between the instrumentation code and the instrumented application code is detected.
It ensures API compatibility between symbols (classes, methods, fields) on the application classpath
and references to those symbols made by instrumentation advices defined in the agent.
In other words, muzzle ensures that the API symbols used by the agent are compatible with the API
symbols on the application classpath.

Muzzle是Java代理的一个安全功能，当检测到仪器代码和被检测的应用代码不匹配时，它可以防止应用agent instrumentation。它确保应用程序classpath上的符号（类、方法、字段）与代理中定义的仪表建议对这些符号的引用之间的API兼容性。换句话说，muzzle确保代理使用的API符号与应用程序classpath上的API符号兼容。

Muzzle will prevent loading an instrumentation if it detects any mismatch or conflict.

如果检测到任何不匹配或冲突，Muzzle将阻止加载一个仪器。

How it works

Muzzle has two phases:

  • at compile time it collects references to the third-party symbols and used helper classes;

    在编译时，它收集了对第三方符号和使用的辅助类的引用。

  • at runtime it compares those references to the actual API symbols on the classpath.

    在运行时，它将这些引用与classpath上的实际API符号进行比较。

Compile-time reference collection 编译时引用收集

The compile-time reference collection and code generation process is implemented using a ByteBuddy
plugin (called MuzzleCodeGenerationPlugin).

编译时的引用收集和代码生成过程是通过一个ByteBuddy插件（称为MuzzleCodeGenerationPlugin）实现的。

For each instrumentation module the ByteBuddy plugin collects symbols referring to both internal and third party APIs used by the currently processed module’s type instrumentations (InstrumentationModule#typeInstrumentations()). The reference collection process starts from advice classes (values of the map returned by the TypeInstrumentation#transformers() method) and traverses the class graph until it encounters a reference to a non-instrumentation class (determined by InstrumentationClassPredicate and the InstrumentationModule#isHelperClass(String) predicate). Aside from references, the collection process also builds a graph of dependencies between internal instrumentation helper classes - this dependency graph is later used to construct a list of helper classes that will be injected to the application classloader (InstrumentationModule#getMuzzleHelperClassNames()). Muzzle also automatically generates the InstrumentationModule#getMuzzleContextStoreClasses() method.

对于每个Instrumentation模块，ByteBuddy插件都会收集引用当前处理的模块的类型工具所使用的内部和第三方API的符号（InstrumentationModule#typeInstrumentations()）。引用收集过程从Advice类开始（由 TypeInstrumentation#transformers()方法返回的映射值），并遍历类图，直到遇到对非instrumentation类的引用（由 InstrumentationClassPredicate 和 InstrumentationModule#isHelperClass(String)谓词决定）。除了引用，收集过程还建立了内部仪表帮助类之间的依赖关系图—这个依赖关系图后来被用来构建一个将被注入应用程序类加载器的帮助类列表（InstrumentationModule#getMuzzleHelperClassNames() ）。Muzzle 还自动生成 InstrumentationModule#getMuzzleContextStoreClasses() 方法。

If you extend any of these getMuzzle...() methods in your InstrumentationModule, the muzzle
compile plugin will not override your code: muzzle will only override those methods that do not have
a custom implementation.

如果你在你的 InstrumentationModule 中扩展了任何这些 getMuzzle…() 方法，muzzle 编译插件不会覆盖你的代码：muzzle 只会覆盖那些没有自定义实现的方法。

All collected references are then used to create a ReferenceMatcher instance. This matcher
is stored in the instrumentation module class in the method InstrumentationModule#getMuzzleReferenceMatcher()
and is shared between all type instrumentations. The bytecode of this method (basically an array of
Reference builder calls) and the getMuzzleHelperClassNames() is generated automatically by the
ByteBuddy plugin using an ASM code visitor.

然后，所有收集到的引用被用来创建一个ReferenceMatcher实例。这个匹配器存储在InstrumentationModule#getMuzzleReferenceMatcher()方法中的仪表模块类中，并在所有类型的仪表中共享。这个方法的字节码（基本上是对Reference builder的调用数组）和getMuzzleHelperClassNames()是由ByteBuddy插件使用ASM代码访问者自动生成。

The source code of the compile-time plugin is located in the javaagent-tooling module,
package io.opentelemetry.javaagent.tooling.muzzle.collector.

Runtime reference matching

The runtime reference matching process is implemented as a ByteBuddy matcher in InstrumentationModule.
MuzzleMatcher uses the getMuzzleReferenceMatcher() method generated during the compilation phase
to verify that the class loader of the instrumented type has all necessary symbols (classes,
methods, fields). If the ReferenceMatcher finds any mismatch between collected references and the
actual application classpath types the whole instrumentation is discarded.

It is worth noting that because the muzzle check is expensive, it is only performed after a match
has been made by the InstrumentationModule#classLoaderMatcher() and TypeInstrumentation#typeMatcher()
matchers. The result of muzzle matcher is cached per classloader, so that it is only executed
once for the whole instrumentation module.

The source code of the runtime muzzle matcher is located in the javaagent-tooling module,
in the class Instrumenter.Default and under the package io.opentelemetry.javaagent.tooling.muzzle.

Muzzle gradle plugin

The muzzle gradle plugin allows to perform the runtime reference matching process against different
third party library versions, when the project is built.

Muzzle gradle plugin is just an additional utility for enhanced build-time checking
to alert us when there are breaking changes in the underlying third party library
that will cause the instrumentation not to get applied.
Even without using it muzzle reference matching is always active in runtime,
it’s not an optional feature.

The gradle plugin defines two tasks:

  • muzzle task runs the runtime muzzle verification against different library versions: 

    1. ./gradlew :instrumentation:google-http-client-1.19:muzzle


    If a new, incompatible version of the instrumented library is published it fails the build.

  • printMuzzleReferences task prints all API references in a given module: 

    1. ./gradlew :instrumentation:google-http-client-1.19:printMuzzleReferences

The muzzle plugin needs to be configured in the module’s .gradle file.
Example:

  1. muzzle {
  2.   // it is expected that muzzle fails the runtime check for this component
  3.   fail {
  4.     group = "commons-httpclient"
  5.     module = "commons-httpclient"
  6.     // versions from this range are checked
  7.     versions = "[,4.0)"
  8.     // this version is not checked by muzzle
  9.     skip('3.1-jenkins-1')
  10.   }
  11.   // it is expected that muzzle passes the runtime check for this component
  12.   pass {
  13.     group = 'org.springframework'
  14.     module = 'spring-webmvc'
  15.     versions = "[3.1.0.RELEASE,]"
  16.     // except these versions
  17.     skip('1.2.1', '1.2.2', '1.2.3', '1.2.4')
  18.     skip('3.2.1.RELEASE')
  19.     // this dependency will be added to the classpath when muzzle check is run
  20.     extraDependency "javax.servlet:javax.servlet-api:3.0.1"
  21.     // verify that all other versions - [,3.1.0.RELEASE) in this case - fail the muzzle runtime check
  22.     assertInverse = true
  23.   }
  24. }
  • Using either pass or fail directive allows to specify whether muzzle should treat the
    reference check failure as expected behavior;
  • versions is a version range, where [] is inclusive and () is exclusive. It is not needed to
    specify the exact version to start/end, e.g. [1.0.0,4) would usually behave in the same way as
    [1.0.0,4.0.0-Alpha);
  • assertInverse is basically a shortcut for adding an opposite directive for all library versions
    that are not included in the specified versions range;
  • extraDependency allows putting additional libs on the classpath just for the compile-time check;
    this is usually used for jars that are not bundled with the instrumented lib but always present
    in the runtime anyway.

The source code of the gradle plugin is located in the buildSrc directory.

Covering all versions and assertInverse

Ideally when using the muzzle gradle plugin we should aim to cover all versions of the instrumented
library. Expecting muzzle check failures from some library versions is a way to ensure that the
instrumentation will not be applied to them in the runtime - and won’t break anything in the
instrumented application.

The easiest way it can be done is by adding assertInverse = true to the pass muzzle
directive. The plugin will add an implicit fail directive that contains all other versions of the
instrumented library.
It is worth using assertInverse = true by default when writing instrumentation modules, even for
very old library versions. The muzzle plugin will ensure that those old versions won’t be
accidentally instrumented when we know that the instrumentation will not work properly for them.
Having a fail directive forces the authors of the instrumentation module to properly specify
classLoaderMatcher() so that only the desired version range is instrumented.

In more complicated scenarios it may be required to use multiple pass and fail directives
to cover as many versions as possible.