SBOM.exe: Countering Dynamic Code Injection based on Software Bill of Materials in Java

Abstract

Software supply chain attacks have become a significant threat as software development increasingly relies on contributions from multiple, often unverified sources. The code from unverified sources does not pose a threat until it is executed. [0]Log4Shell.. is a recent example of a supply chain attack that processed a malicious input at runtime, leading to remote code execution. It exploited the dynamic class loading facilities of Java to compromise the runtime integrity of the application. Traditional safeguards can mitigate supply chain attacks at build time, but they have limitations in mitigating runtime threats posed by dynamically loaded malicious classes. This calls for a system that can detect these malicious classes and prevent their execution at runtime.

This paper introduces SBOM.exe, a proactive system designed to safeguard Java applications against such threats. SBOM.exe constructs a comprehensive allowlist of permissible classes based on the complete software supply chain of the application. This allowlist is enforced at runtime, blocking any unrecognized or tampered classes from executing. We assess SBOM.exe’s effectiveness by mitigating 3 critical CVEs based on the above threat. We run our tool with 3 open-source Java applications and report that our tool is compatible with real-world applications with minimal performance overhead. Our findings demonstrate that SBOM.exe can effectively maintain runtime integrity with minimal performance impact, offering a novel approach to fortifying Java applications against dynamic classloading attacks.
Publicly-available repository - https://github.com/chains-project/sbom.exe

1 Introduction

Developers reuse a lot of third-party dependencies [1, 2, 3] to build software applications. The process of building the application using the set of all dependencies is known as the software supply chain of an application. While this practice is good as it avoids reinventing the wheel [4], it is challenging for developers to keep track of the reliability, maintainability and security of their software supply chain [5]. In the latter case, it has recently been observed, with high-profile attacks, that third-party libraries can be exploited by malicious actors, leading to so-called “software supply chain attacks” [6, 7, 8].

Software supply chain attacks are a significant threat to the software security landscape as acknowledged by ENISA [9] and the White House [10]. In 2023, there were twice as many software supply chain attacks as in 2019-2022 combined [11]. A few reactive and proactive techniques have been proposed in the literature to mitigate software supply chain attacks [6]. Reactive techniques are based for example on bots [12] to update dependency regularly and tools to scan for Common Vulnerabilities and Exposures (CVE)s [13] in dependencies. However, these only act after the exploit has been discovered and reported. The main proactive technique to help prevent software supply chain attacks is to systematically create Software Bill of Materials (SBOM). In a nutshell, an SBOM is a complete list of dependencies and tools used to build a software application. SBOMs increase transparency in the software supply chain, and are good for accountability [10], but they cannot prevent attacks at runtime.

In this paper, we propose a novel technique to mitigate a class of software supply chain attacks based on code injection in third-party dependencies at runtime [7]. This type of attack received major attention when the high-profile attack [0]Log4Shell.. (CVE-2021-44228 [14]) was released, demonstrating that the dependency [0]Log4j.., which is part of millions of Java applications’ software supply chain [15], was vulnerable. The attack was possible because the dependency had a vulnerability enabling remote code execution when processing a malicious input. More importantly, the attack was entirely based on dynamic class loading facility that exists in Java.

Conceptually, if an attacker knows that a dependency uses Java dynamic features, then they can exploit these features to compromise any dependent application. The solution is obviously not to forbid these dynamic features, because they are used in virtually all mission and business critical applications, incl. all Spring Java web apps. Also, neither SBOM, reproducible builds, code signing, nor version pinning would prevent these attacks as the malicious code in these attacks appears only while the application is running.

We propose SBOM.exe, a system that ensures the integrity of the Java runtime, by monitoring the software supply chain of an application at runtime, in order to detect and prevent the execution of injected malicious code. SBOM.exe works by building a comprehensive allowlist of classes that are allowed to be executed. We call this list the Bill of Material Index (BOMI). This first task is hard due to the widespread usage of dynamically generated code in Java applications. Second, the BOMI contains the checksums of all allowed classes, both from the application and its supply chain. This is also hard because runtime code generation may be non-deterministic, and differs because of various reasons, incl. Java version [16]. To overcome this problem and make the BOMI robust, we propose a novel technique of bytecode canonicalization that mitigates all sources of non-deterministic features in Java bytecode in order to compute reliable and secure checksum. Then, SBOM.exe ensures that no unknown class is executed at runtime by comparing the checksum of the class about to be loaded with the reference checksum that is in the BOMI. Finally, and most importantly, the tool stops the execution if it intercepts any unknown or tampered class. To our knowledge, SBOM.exe is the first system that reasons and embeds the software supply chain at runtime in order to block malicious code injection in Java.

We evaluate the effectiveness of SBOM.exe by testing it against 3 real-world, critical vulnerabilities in Java libraries: [0]Log4j.. [14], [0]H2.. [17], and [0]Apache Commons Configuration.. [18]. First, we replicate these 3 critical CVEs in a lab setting, next, we show that SBOM.exe fully mitigates them by stop** the execution of the application before the malicious class is executed. We also demonstrate that our SBOM.exe does not break real-world software by running 3 open-source Java applications with nominal workloads - [0]PDFBox.. [19], [0]Ttorrent.. [20], and [0]GraphHopper.. [21]. Finally, our performance measurements with state-of-the-art microbenchmarking show that the overhead incurred by SBOM.exe is atmost 1% which is negligible.

The main contributions of this paper are:

• SBOM.exe, a novel system that ensures the integrity of the Java runtime against code injection. It works by computing a comprehensive allowlist of classes using the software supply chain of the application.

• A robust algorithm and tool to compute checksums of Java bytecode, removing non-deterministic features, and enabling sound malicious code detection.

• A series of experiments showing 1) the effectiveness of SBOM.exe in mitigating real-world Java vulnerabilities including [0]Log4Shell.., 2) the compatibility of SBOM.exe with existing applications, and 3) the absence of significant overhead.

• A publicly available tool and consolidated reproducible attacks on GitHub [22] for future research on this topic.

2 Background

2.1 Dynamicity of the Java Runtime

Java is a dynamic programming language, in the sense that it features different capabilities to load and modify code at runtime [23]. A classloader in Java is responsible for loading classes into the Java Virtual Machine (JVM). A typical classloader takes in the name of the class and finds binary code corresponding to it on disk. Dynamic class loading does not require the binary code to exist on disk since the start of JVM. For example, it enables downloading binary code on the fly or generation of binary code at runtime. In the following, we enumerate the main mechanisms through which a class can be loaded dynamically in Java.

First, classloaders can be extended to execute classes from a remote source [24] or compiled code at runtime [25]. Both of these APIs are open for public usage and extensively used by the native SDK.

Proxies [26] are runtime generated classes that add common functionalities to some classes in the application. For example, a proxy class can have functions to record the time taken for execution. The code for proxies is generated at runtime. Within Java, a major usage of proxies is to generate the code corresponding to Java annotations [27].

Java allows classes to introspect themselves. This is called reflection in Java and this feature also leverages dynamic code generation. Internally, Java creates subclasses of [0]MagicAccessorImpl.. [28] which grants access to the Java runtime to members of the class which otherwise would be inaccessible. Introspection is an essential feature of frameworks. For example, JUnit uses reflection to find the test methods in a class.

Finally, Java bytecode has an instruction called [0]invokedynamic.. [29] that allows bootstrap** methods at runtime. It bootstraps methods by dynamically generating a ‘hidden class’ [30] that contains the implementation of the method. This instruction is used to implement lambda expressions in Java, which is essential in modern versions of Java. Java records also rely on this feature to implement their members like [0]equals...

To sum up, Java is a highly dynamic platform, and its dynamic features are foundational for most notable Java capabilities and usages. However, this dynamicity can be used for malicious purposes, which is the problem we address in this paper. We will study real-world instances of such attacks in section 3 and subsection 6.3.

2.2 The Software Supply Chain

Software Supply Chain refers to the sequence of steps and inputs resulting in the creation of a software artefact [31]. We define the important terms used in the paper regarding the software supply chain.

Build System. A build system is a piece of software that takes in the source code and all its dependencies to create a package or a software artefact. Some examples of build systems are [0]mvn.., [0]npm.., and [0]pip...

Package Registry. A package registry is a trusted central service that hosts packages, dependencies, or software components. Packages are deployed to registries so that other developers can use them. Some examples of package registries are Maven Central, the npm registry, and PyPI.

SBOM. SBOM – software bill of material – is a formal, machine-readable inventory of software components, information about those components, and their hierarchical relationships [32]. Its goal is to enable transparency of the software supply chain. This enables multiple uses to multiple stakeholders [33], such as license compliance analysis or vulnerability management.

One of the key features of SBOM is to report the complete list of software components or dependencies of a software package [34]. The dependencies here refer to both direct and indirect dependencies. The direct dependencies refer to the ones declared by the developers to leverage the additional functionalities. The indirect dependencies are the ones that are required by the direct dependencies and are implicitly trusted by the developers.

3 Threat Model

Our goal is to protect against malicious usage of dynamic features in Java, which were presented in subsection 2.1. Recall that, in Java, a class can be downloaded from a remote source or generated at runtime [35, §3] and executed. In most cases, the developer is unaware of those dynamic classes as they are unknown when they are writing and building the application.

The threat we are defending against is the loading of malicious classes by the Java runtime, which is considered a kind of code injection attack [36, §"Network Class Loaders and Security Issues"]. This class of attack has been studied in the literature [37].

In our context, we equate ‘injected code’ with unknown classes. These classes are neither part of the application, nor of its dependencies, nor the Java standard library. Note that an unknown class can also be a modified version of a known class, triggered by malicious actor tampering with application classes. Per the terminology of Holzinger et al. [37], we mitigate against the vulnerability “Loading of arbitrary classes”.

Figure 1: Overview of the Log4Shell vulnerability.

We will study real-world instances of these attacks in subsection 6.2. For example, [0]Log4Shell.. [14] is one infamous example where unknown code is downloaded from a remote server and executed. Figure 1 gives an overview of a [0]Log4Shell.. attack. The attacker, first, creates a malicious class file [0]Exploit.class.. and hosts it on a server controlled by them. The malicious class file is hosted on the HTTP server and the LDAP server stores a reference to the HTTP server via an HTTP URL. Then, the attacker sends a crafted request with expression [0]${jndi:ldap://hacker.com/o=referenceToExploit}.. in the header. JNDI stands for Java Naming and Directory Interface and it allows one to look up resources in a directory. The LDAP server with domain name [0]hacker.com.. is used to store the reference to the HTTP server. [0]http://hacker.com/Exploit.class.. is queried and it returns the malicious class file. The enterprise application is vulnerable to this as it is using [0]Log4j... [0]Log4j.. logs the value in the User-Agent header. This is normally done to record the telemetry data of the application. However, in this case, the expression is interpreted and the JVM running loads [0]Exploit.class... Finally, the malicious class can execute arbitrary offensive commands, e.g. to steal private data or perform ransomware attacks.

To sum up, in this paper, we propose a technique to mitigate malicious code execution in Java that exploits the dynamic features of the platform.

4 Design & Implementation

Figure 2: Overview of SBOM.exe, a novel system to detect and mitigate code injection attacks in Java systems.

We now present the design of SBOM.exe, our novel system for ensuring supply chain integrity at runtime in Java.

4.1 System Overview

Figure 2 presents an overview of SBOM.exe, a system that verifies the binary integrity of an application’s dependencies at runtime. The system operates in two different phases.

At build time, the indexing phase consists of building an allowlist of all binary classes that are allowed to be executed by the JVM in production. We call this list the [0]BOMI.., which stands for Bill Of Material Index. Then at runtime, SBOM.exe verifies that no unknown class is executed using the BOMI as the ground truth of accepted classes. The [0]SBOM Runtime Watchdog.. is the key component at this stage. It verifies the acceptability of a class by comparing the checksum of a canonical version of a Java class. This checksum is computed exactly the same way in the indexing phase.

There are two major novel concepts in our system.

BOMI. A Bill of Material Index, shortened as ‘BOMI’, is an allowlist of binary classes that are allowed to be executed by the JVM. The core idea behind maintaining an allowlist is to restrict the execution of classes that are not known by the developers. SBOM.exe fully automates this process of indexing all the classes, not requiring the developer to do any manual work. The BOMI captures the identity of allowed classes by computing a checksum over a canonical version of Java bytecode (subsection 4.4).

SBOM Runtime Watchdog. The goal of SBOM Runtime Watchdog is to ensure that no unknown class is executed. It enforces the BOMI at runtime by controlling the loading of classes. In other words, it checks for equivalence of the checksum of the class to load with the checksum of the class in the BOMI. Since some classes are non-deterministically generated at runtime, we canonicalize (subsection 4.4) them to ensure that the checksum equivalence is correct.

In the [0]Indexing.. phase (top part of Figure 2), SBOM.exe lists all Java classes necessary to run the application before it is deployed to production. In Java, default class loaders allow loading classes from filesystem [23, §2]. These can be used to load environment classes and supply chain classes. The default class loaders can be extended to download classes from a remote source or generate classes at runtime. We refer to those classes as dynamically loaded classes.

Now, we define each of the three sets of classes. First, environment classes are the built-in classes provided by the Java standard library. Second, supply chain classes are classes written by the developers of the application, as well as the classes declared as dependencies to the application. Finally, dynamically-loaded classes are classes that only appear at runtime. They are either downloaded from a remote source or generated.

The bottom part of Figure 2 is about the [0]Runtime.. phase of SBOM.exe. In production, a JVM is spawned to run an application. We propose to attach an SBOM Runtime Watchdog to the JVM. The SBOM Runtime Watchdog is attached to the application during startup with the Java agent mechanism. It takes as input the BOMI, which will be used to detect unknown or tampered binary classes. To sum up, for a class to be accepted and used in production, it must be present in the BOMI, and must have the same canonical checksum as recorded in the [0]Indexing.. phase.

4.2 Indexing

The core idea in the [0]Indexing.. phase is to list all Java classes necessary to run the application.

SBOM.exe has three indexing components: environment, supply chain, and dynamic code indexer. All these processes execute sequentially and write the classes to the BOMI. Thus, the BOMI is the union of all classes obtained from the three indexers. We now describe each of the indexers in detail.

4.2.1 Environment Indexer

The environment index contains information about the internal classes of the runtime environment, Java in our paper. SBOM.exe builds this index by scanning the Java standard library and recording the checksum of each standard class in the BOMI. This forms the first part of the BOMI which is referred to as BOMI-Environment.

We rely on ClassGraph [38] which has APIs to scan the Java standard library. It locates all Java classes packaged inside the Java distribution and forwards them to the environment indexer for checksum computation. This index is generated statically and needs to be updated only when the Java version of the application changes. The only requirement of this process is to have Java installed on the system. The index always contains the same classes for a given Java version and operating system and its generation is reproducible.

1 shows an excerpt from the environment index. Each line of the excerpt contains a map of a class from the Java standard library to the checksum of its bytecode. The map helps to quickly look up the checksum of a class based on its name. Note that this is an excerpt and the actual index can contain classes in the order of tens of thousands. Although not all of them are used by the application, we record all of them so that we can guard against classes masking as internal Java standard library classes. For example, a malicious actor can create a class [0]jdk.internal.MaliciousClass.. that could be downloaded at runtime. It could be deemed safe if we don’t record all the internal classes of the Java standard library.

⬇

1 {"java/lang/ClassLoader":{"checksum":"041c0b7"}}

2 {"java/lang/Exception":{"checksum":"50c1c7d"}}

3 {"java/lang/String":{"checksum":"5f158f2"}}

4 {"java/lang/Math":{"checksum":"d3210a9"}}

5 {"java/lang/System":{"checksum":"89b5804"}}

6 ...

Listing 1: Excerpt from the environment index of the Java standard library.

4.2.2 Supply Chain Indexer

The supply chain index is created from the SBOM of the application. The SBOM is taken as input from the SBOM producer. An SBOM maps all the classes in the supply chain of the application to a dependency in a trusted package registry. This step is outside the threat model since the production of SBOM is carried out by the developer of the dependency who uploads it to the trusted package registry. From the trusted package registry, SBOM.exe downloads a JAR file for each dependency. The indexer extracts the JAR file and analyzes all the classes. Finally, the indexer computes the checksum and writes the class name and checksum to the BOMI. This forms the second part of the BOMI which is referred to as BOMI-SupplyChain.

An excerpt from the supply chain index of a Java application is shown in 2. This Java application has a single class that performs logging operations with [0]Log4j.. dependency. Hence, the first line in the excerpt is the class written by the author of the application. The next three classes are from the [0]Log4j.. dependency. The last class is from the [0]Log4j.. dependency but has two checksums. This is because [0]Log4j.. is a multi-release JAR and some classes can have different implementations based on the Java version.

⬇

1 {"org/example/Main":[{"checksum":"f8423f5"}]}

2 {"org/apache/logging/log4j/core/lookup/JndiLookup":[{"checksum":"dacd441"}]}

3 {"org/apache/logging/log4j/LogManager":[{"checksum":"4dafd50"}]}

4 {"org/apache/logging/log4j/core/Logger":[{"checksum":"85a7729"}]}

5 {"org/apache/logging/log4j/util/StackLocator":[{"checksum":"0cd3eb5"}, {"checksum":"2d62281"}]}

6 ...

Listing 2: Excerpt from the supply chain index in the BOMI.

4.2.3 Dynamic Code Generation Indexer

Most Java applications [39],[40] depend on classes that are generated during runtime or are downloaded from a remote source. By construction, those classes are unknown statically and are not indexed by the environment and supply chain indexers. In SBOM.exe, we take special care of supporting binary classes that are downloaded or generated at runtime. The dynamic code generation indexer is responsible for tracing them and for generating the dynamic code index. This index contains unique information about all classes that are neither part of the supply chain nor the Java standard library. This forms the third and final part of the BOMI which is referred to as BOMI-Runtime.

Let us delve into dynamic code generation in Java. Internally, Java uses five mechanisms to generate runtime classes: proxy classes, classes generated using CGLIB, annotation, reflective invocation, and lambda expressions [41]. Also, developers use libraries like ByteBuddy [42], ASM [43], and Javassist [44] to generate classes at runtime.

To capture these classes, SBOM.exe runs the tests of the application and records all classes that are downloaded or generated during test execution. This assumes that the application has a good test suite that exercises the core functionalities of an application. Our experiment (see subsection 6.3) demonstrates that this assumption does not miss any dynamic code that is required to run real-world applications in production.

The dynamic code indexer records the checksum of each class dynamically loaded or generated during test suite execution. We show an excerpt from the dynamic code index in 3. All the classes are generated at runtime. The first class is generated to implement annotations in Java. Since annotations are defined by the developer as interfaces, Java generates implementations of them at runtime. The second class is generated by the Java standard library to get reflective access to the constructor. This is needed by Java for garbage collection. The third class is generated to store argument values for methods that are invoked later in the runtime. The fourth class is generated by the Nashorn JavaScript engine to evaluate the JavaScript code. The last class is an example of a class generated by a Java framework to manage server configuration. Finally, this information is integrated into the final BOMI.

⬇

1 {"com/sun/proxy/$Proxy14":[{"version":"49.0","checksum":"2807adf"}]}

2 {"jdk/internal/reflect/GeneratedConstructorAccessor5":[{"checksum":"c636864"}]}

3 {"java/lang/invoke/BoundMethodHandle$Species_LLL":[{"checksum":"4f886a8"}]}

4 {"jdk/nashorn/internal/scripts/Script$\^eval\_":[{"checksum":"6523d19"}]}

5 {"io/dropwizard/jersey/DropwizardResourceConfig$SpecificBinder5e093dc6-5884-44cc-9901-1417d447e561":[{"hash":"41f6c89"}]}

6 ...

Listing 3: Excerpt from the dynamic code index containing a runtime generated class in the BOMI.

The dynamic code generation indexer is run on projects verified with application level integrity checks: the study subjects provide release checksum and signed releases so that they can be verified.

4.3 SBOM Runtime Watchdog

SBOM.exe is meant to detect unknown code before it is being executed in production, in order to fail-fast under attack, see section 3. The SBOM Runtime Watchdog is responsible for terminating the application when this happens. It is a Java agent that is attached to the application during startup and takes as input the BOMI generated by the above three indexers. At runtime, it intercepts all classes that are loaded by the JVM, computes the checksum of the classes and verifies if the class is in the BOMI. If the class exists in the BOMI and the checksum matches, it means that the class is known and the application continues to execute. If the class is unknown, it means that either the application is under attack or that the indexing was incomplete. Our systematic analysis of the Java software stack is made to exclude the latter (see our experiments in subsection 6.3).

The core duty of the SBOM Runtime Watchdog is to disallow unknown classes and terminate the application when an unknown class is detected. In addition, the incident is reported to the security team.

4.4 Bytecode Canonicalization

A major problem with runtime generated code is that it is non-deterministic. This means that running the same application twice would give slightly different generated bytecodes. To the best of our knowledge, the Java standard library cannot be forced to be fully deterministic [45]. It is capable of producing non-deterministic code [16]. For example, 4 shows the difference between two decompiled versions of the same proxy class in different executions. To account for that problem, SBOM.exe preprocesses all generated classes to obtain a deterministic canonical version, which we describe below.

The first addressed non-determinism feature is the class name. The names of generated classes, like [0]Proxy.. classes include a number that depends on the order of the class loaded, this order is non-deterministic due to e.g. parallelism. Hence this number could be different between runs even though the content of the bytecode is equivalent. In order to ensure that these classes are not considered as different classes, we rewrite the class names to a fixed string constant. For example, the class [0]$Proxy21.. and [0]$Proxy14.. in line number 1 and 2 in 4 is rewritten to [0]foo.. in line number 1 in 5.

The second non-addressed non-determinism feature is type references in the class code. The names of type references are derived from the class name and are also non-deterministic. For example, if class [0]$Proxy21.. has a field called [0]m3... It can be referred to as [0]$Proxy21.m3.., where [0]$Proxy21.. is the type reference. This reference can be used in the bytecode of the same class as shown in line number 12 of 4 or a different class where [0]$Proxy21.. is imported. Thus, we also need to rewrite these references to a fixed string constant [0]foo.. like in line number 6 of 5.

Finally, a third problem lies in the map** of fields and methods in generated classes between different runs. For example, [0]m3.. is mapped to [0]visualUpdate.. in one run and [0]expert.. in another run. This also affected the order of statements in the methods as shown in line number 30 to 33 of 4. This is fixed by sorting the byte array of the bytecode just before computing the checksum in the next step.

After performing the above three transformations, we have a canonical, stable, and deterministic view of the class. SBOM.exe then computes the SHA-256 checksum of the constant pool and the JVM opcodes. The constant pool of a bytecode reflects all the APIs and their exact arguments. The JVM opcodes reflect how these APIs and the arguments are combined and executed.

To sum up, bytecode canonicalization is an essential component of SBOM.exe. Without it, it is impossible to reason about the integrity of binary code across executions. To our knowledge, we are the first to propose such a canonicalization of runtime generated code for the JVM.

⬇

1 -public final class $Proxy21 extends Proxy implements BeanProperty {

2 +public final class $Proxy14 extends Proxy implements BeanProperty {

3 // [Truncated field declarations]

4 - public $Proxy21(InvocationHandler var1) {

5 + public $Proxy14(InvocationHandler var1) {

6 super(var1);

7 }

8

9 - public final boolean visualUpdate() {

10 + public final boolean expert() {

11 try {

12 - return (Boolean)super.h.invoke(this, $Proxy21.m3, (Object[])null);

13 + return (Boolean)super.h.invoke(this, $Proxy14.m3, (Object[])null);

14 } catch (RuntimeException | Error var2) {

15 throw var2;

16 } catch (Throwable var3) {

17 }

18 }

19

20 - public final boolean expert() {

21 + public final boolean visualUpdate() {

22 try {

23 - return (Boolean)super.h.invoke(this, $Proxy21.m4, (Object[])null);

24 + return (Boolean)super.h.invoke(this, $Proxy14.m4, (Object[])null);

25 } catch (RuntimeException | Error var2) {

26 throw var2;

27 } catch (Throwable var3) {

28

29 try {

30 - $Proxy21.m3 = Class.forName("java.beans.BeanProperty", false, var0).getMethod("visualUpdate");

31 + $Proxy14.m3 = Class.forName("java.beans.BeanProperty", false, var0).getMethod("expert");

32 - $Proxy21.m4 = Class.forName("java.beans.BeanProperty", false, var0).getMethod("expert");

33 + $Proxy14.m4 = Class.forName("java.beans.BeanProperty", false, var0).getMethod("visualUpdate");

34 } catch (NoSuchMethodException var2) {

35 throw new NoSuchMethodError(var2.getMessage());

36 } catch (ClassNotFoundException var3) {

Listing 4: Difference between decompiled output of the same proxy class based on java.beans.BeanProperty, showing non-deterministic naming conventions.

⬇

1 public final class foo extends Proxy implements BeanProperty {

2 // [Truncated field declarations]

3

4 public final boolean visualUpdate() {

5 try {

6 return (Boolean)super.h.invoke(this, foo.m3, (Object[])null);

7 } catch (RuntimeException | Error var2) {

8 throw var2;

9 } catch (Throwable var3) {

10 throw new UndeclaredThrowableException(var3);

11 }

12 }

13

14 public final boolean expert() {

15 try {

16 return (Boolean)super.h.invoke(this, foo.m4, (Object[])null);

17 } catch (RuntimeException | Error var2) {

18 throw var2;

19 } catch (Throwable var3) {

20 throw new UndeclaredThrowableException(var3);

21 }

22 }

23

24 // [Truncated methods]

25}

Listing 5: SBOM.exe canonicalization allows for stable checksums in the BOMI.

4.5 BOMI Integrity

The BOMI is a critical component of SBOM.exe. We take the following special measures to ensure the integrity of the BOMI. We ensure that all the indexing processes are running in a trusted environment. We create a Docker image from the base image of the official OpenJDK [46] and then install SBOM.exe in the container.

First, the environment indexer should analyze a safe version of the Java standard library. This is ensured by our selection of the official Java distribution.

Second, we ensure that the indexers interact with trusted sources. For the supply chain index, we only consider the classes that are downloaded from Maven Central. The Maven Central repository has many sub repositories, we ensure that the URLs of the JAR belong to one of the sub repositories of Maven Central. We also install the SSL certificates required to interact with Maven Central so that dependency resolution uses verified URLs. Enforcing SSL certificate verification also prevents DNS hijacking attacks.

4.6 Implementation

We have implemented a prototype of the design which is made available on GitHub [22] under the same name SBOM.exe.

5 Experimental Methodology

To evaluate SBOM.exe, we run experiments with 6 study subjects in order to answer the following research questions:

• RQ1: What is the scale of BOMI for all the study subjects? (Scale)

• RQ2: To what extent can SBOM.exe mitigate high-profile attacks in Java? (Effectiveness)

• RQ3: Is SBOM.exe compatible with real-world applications? (Applicability)

• RQ4: What is the overhead of SBOM.exe? (Performance)

5.1 Study Subjects

Table I: Study subjects used in the evaluation of SBOM.exe.

Study Subjects	Java Version	Dependencies	Workload
[0]Log4Shell.. - CVE-2021-44228[14]	17.0.10 Temurin	2	String input for logging
CVE-2021-42392[17]	17.0.10 Temurin	1	Authentication with server
CVE-2022-33980[18]	11 GA OpenJDK	5	JS Script to print date
apache/pdfbox[19]	21.0.2 OpenJDK	12	10 PDF manipulation commands
mpetazzoni/ttorrent[20]	21.0.2 OpenJDK	6	Torrent downloading
graphhopper/graphhopper[21]	21.0.2 OpenJDK	165	Server initialization and 5 routing requests

We document the study subjects in Table I and these will be used to evaluate SBOM.exe. The first three subjects are high-profile CVE with severity rating critical associated with them. They fall under our threat model and are used to evaluate the effectiveness of SBOM.exe. The last three subjects are real-world applications that are used to evaluate the applicability and performance of SBOM.exe. These Java applications are open-source and popular repositories on GitHub. The next column shows the Java version used to run the application. The versions are the latest LTS, but for CVE, older LTS versions are used to replicate the vulnerability. Next, we show the number of dependencies that are part of the application. Finally, we show the workload that is used for three different purposes. 1) For replicating the vulnerability in the case of high-profile attacks. 2) For running the study subject under the influence of SBOM Runtime Watchdog to evaluate the applicability of SBOM.exe. And finally, 3) for measuring the overhead of SBOM.exe.

5.2 RQ1: Scale of BOMI

To answer this question, we study BOMIs generated by SBOM.exe for the 3 CVEs and 3 real-world applications that will be studied in RQ2 and RQ3. We create the BOMI-Environment statically using the Java Standard Library. We select [0]build-info-go.. as the SBOM producer to generate the BOMI-SupplyChain. It is deemed to be the most precise way to capture the dependencies for a Java application [5]. Finally, test suites are used by Dynamic Code Indexer to create the BOMI-Runtime. Then, we manually analyze them and comment on their characteristics. We also, comment on how deterministic the generation of BOMI is. The BOMI is deterministic if the same classes and checksums corresponding to them are generated for the same application across different runs.

5.3 RQ2: Effectiveness of SBOM.exe

For RQ1, we collect three high-profile attacks to show that SBOM.exe can successfully stop them. High-profile attacks are those that have a CVE with a severity rating critical associated with them. We first replicate the vulnerability in a proof-of-concept application and we show how it can be exploited using a malicious payload. Next, we create a BOMI for the application and pass it to SBOM.exe for enforcement at runtime. Finally, we deploy the application with SBOM.exe’s SBOM Runtime Watchdog and we run the attack using the same malicious payload as in the initial reproduction. We check that SBOM.exe’s approach works and that the application terminates before the malicious class is executed, completely mitigating the attack.

5.4 RQ3: Applicability

We ensure that SBOM.exe can be used for providing runtime integrity to real-world applications without any false positives, i.e. without terminating the application wrongfully. To answer this question, we run SBOM.exe on a set of 3 real-world applications - [0]PDFBox.. (an application for manipulating PDF files), [0]GraphHopper.. (an open source navigation engine that powers openstreetmap.org) [0]Ttorrent.. (a peer-to-peer file downloading tool based on the BitTorrent protocol). Then we construct the BOMI for each application. Finally, we run the application with the workload, while attaching the Runtime SBOM Watchdog to the JVM. If the application executes successfully without inappropriate termination by the watchdog (false positive), we consider the application to be compatible with SBOM.exe.

5.5 RQ4: Overhead of SBOM.exe

We measure the overhead of SBOM.exe caused by the SBOM Runtime Watchdog by running all study subjects with appropriate workloads. The workload is the same as used in RQ3 to evaluate the applicability of SBOM.exe. These workloads are also appropriate for measuring the overhead of SBOM.exe as they are representative of the real-world usage of the application. Per the best practices of overhead measurement in Java, the evaluation is done using a microbenchmarking framework Java Microbenchmark Harness (JMH)  [47] and it reports two metrics - end-to-end time with warmup and workload time excluding warmup. End-to-end time with warmup is the measure of the sum of how long it takes for the JVM to warm up and then run the application. Warming up of JVM is the process of profiling and compiling the bytecode to machine code, it is significant because the JVM has two JIT compilers. We configure JMH to spawn 5 forks of JVM wherein it executes 5 warm-up executions first so that the just-in-time compilers can perform optimizations and compile bytecode to machine code. After that, it runs 5 measurement executions of the workload. End-to-end time with warmup reports the average of 5 warm-up runs and 5 measurement runs over 5 forks. Workload time excluding warmup reports the mean of the runtime of the application over all executions and it excludes the JVM warm-up time. Hence, it gives the execution time when all code has been compiled to machine code and the application is running in its most optimized state. So, times excluding warmup are always lower than end-to-end times with warmup. Finally, we report the percentage overhead for the two metrics. We use the following formula to calculate the overhead:

$$Actual\ Value=\frac{t_{\text{w/}}-t_{\text{w/o}}}{t_{\text{w/o}}}$$

$$Error=\frac{t_{\text{w/}}-t_{\text{w/o}}}{t_{\text{w/o}}}\sqrt{\left(\frac{\Delta t_{\text{w/}}+\Delta t_{\text{w/o}}}{t_{\text{w/}}-t_{\text{w/o}}}\right)^{2}\pm\left(\frac{\Delta t_{\text{w/o}}}{t_{\text{w/o}}}\right)^{2}}$$

$$Overhead=\left(Actual\ Value\pm Error\right)\times 100\%$$

where

• $t_{\text{w/}}$ is the time reported with SBOM.exe,

• $t_{\text{w/o}}$ is the workload time without SBOM.exe,

• $\Delta t_{\text{w/}}$ is the error in time with $t_{\text{w/}}$,

• $\Delta t_{\text{w/o}}$ is the error in time without $t_{\text{w/o}}$,

• $Actual\ Value$ is the relative overhead incurred by SBOM.exe,

• $Error$ is the error in the overhead calculation.

6 Experimental Results

We present the results for all the research questions in the following subsections. All the experiments are run on Azure’s virtual machine called Standard D2as v4. It runs Ubuntu 22.04 LTS (64-bit). The underlying hardware consists of 8GB RAM and 2 virtual cores of Intel® Xeon® Platinum 8272CL. We also host all the results on the GitHub repository [22].

6.1 RQ1: Scale of BOMI

Table II: BOMI for all the study subjects.

Study Subjects	BOMI-Environment	BOMI-SupplyChain	BOMI-Runtime	BOMI	Reproducible
CVE-2021-44228	24085	1268	27	25381	✓
CVE-2021-42392	24085	944	20	25049	✓
CVE-2022-33980	24278	788	34	25100	✓
apache/pdfbox	25309	17760	83	43152	✓
mpetazzoni/ttorrent	25309	801	9	26119	✓
graphhopper/graphhopper	25309	23685	266	49260	✗

Table II presents the details of the BOMI for all the study subjects. The first column lists the study subjects that will evaluate the effectiveness and applicability of SBOM.exe in RQ2 and RQ3 respectively. The second column is the number of classes in BOMI-Environment. It is dependent upon the Java version. The third column is the number of classes in BOMI-SupplyChain. The classes in the supply chain include the dependencies and the project itself. The fourth column lists the number of classes in BOMI-Runtime. BOMI shows the total number of classes by adding the classes from BOMI-Environment, BOMI-SupplyChain, and BOMI-Runtime. Finally, the last column reports if the BOMI is reproducible or not. The BOMI is reproducible if all classes and checksums in BOMI-Environment, BOMI-SupplyChain, and BOMI-Runtime are deterministic.

For CVE-2021-44228, we build the BOMI-Environment using the Java standard library version 17.0.10 Temurin and we capture 24085 classes. The BOMI-SupplyChain contains 1269 classes from the source code for replication on this CVE and its 2 dependencies. The BOMI-Runtime contains 27 classes that are generated during the execution of the test suite of replication. The total number of classes in the BOMI is 25381. We generated the BOMI twice and it contained the same classes and checksums. Hence, we consider the BOMI for CVE-2021-44228 to be reproducible.

We discuss characteristics of BOMI-Environment, BOMI-SupplyChain, and BOMI-Runtime in the following subsections.

BOMI-Environment

The BOMI-Environment contains classes from the Java standard library. Its size is solely dependent upon two factors - the exact version and the vendor of the Java standard library. In our experiments, the classes in BOMI-Environment contributed significantly to BOMI no matter the size of the application.

BOMI-SupplyChain

The BOMI-SupplyChain varies significantly across the study subjects as it depends on the number of dependencies and the size of the project. The size of classes in BOMI-SupplyChain in the CVEs is smaller compared to the real-world applications because we replicate the vulnerability in a small proof-of-concept application. For example, we only need to include one library, com.h2database:h2:1.4.200, and 1 class to replicate the vulnerability in CVE-2021-42392. However, the number of classes in BOMI-SupplyChain is not directly determined by the number of dependencies. [0]PDFBox.. has 12 dependencies and the number of classes in BOMI-SupplyChain is 17760. [0]Ttorrent.. has half the number of dependencies compared to [0]PDFBox.. but the number of classes is 20 times less. This implies that the supply chain of [0]Ttorrent.. contains smaller dependencies compared to [0]PDFBox... Finally, the number of classes in BOMI-SupplyChain of [0]GraphHopper.. is comparable to the classes in BOMI-Runtime, but they have very different characteristics. Classes in BOMI-Runtime are provided by the Java standard library. All those classes are maintained by the same organization. It is consumed by many more stakeholders and each class is under high scrutiny for quality by developers of Java standard library. However, classes in BOMI-SupplyChain are provided by different organizations or open-source developers and are developed independently of each other. Moreover, the classes in BOMI-SupplyChain come from a huge dependency tree containing direct and indirect dependencies. For example, developers of [0]GraphHopper.. declare 4 dependencies and those dependencies have 161 dependencies. The relationships between these 165 dependencies result in the tree being 5 levels deep where only the first level dependencies are declared by the developers. This makes the classes in BOMI-SupplyChain very diverse compared to the classes in BOMI-Runtime.

BOMI-Runtime

BOMI-Runtime is much smaller compared to the last two indices, but these classes are extremely important to capture. Our threat model targets the attacks that are possible because dynamic classes are executed. These classes are not present in Java application JARs nor in the SBOM and they only appear at runtime. A proof-of-concept application for CVE-2021-44228 has 27 classes in BOMI-Runtime. If we enforce that only those classes are allowed to run, we can prevent the loading of malicious classes and attack can be mitigated.

Reproducibility

Reproducibility is an important property for BOMI. The BOMI must list the exact classes and checksums for a specific version of the application. Otherwise, the enforcement of BOMI could cause inconsistent termination of the application. We observe that BOMI-Environment and BOMI-SupplyChain are deterministic for all classes. However, BOMI-Runtime is non-reproducible for [0]GraphHopper.. because one of its dependencies, dropwizard, generates classes based on a random UUID. These classes are different each time the BOMI-Runtime is generated and hence the overall BOMI for [0]GraphHopper.. is also not reproducible. There is also non-determinism in other dynamic classes but is taken care of by our bytecode canonicalization process subsection 4.4. The BOMI for all the other study subjects is reproducible.

Answer to RQ1: What is the scale of BOMI for all the study subjects?
BOMI contains thousands of classes and their checksums. The BOMI-Environment contributes significantly to the BOMI size for all the study subjects due to the sheer complexity of the JVM Standard Library. The size of the BOMI-SupplyChain varies significantly across the study subjects depending on the application supply chain. Finally, the BOMI-Runtime is much smaller compared to the other two, but, as we demonstrate later, the classes captured there are essential to prevent attacks due to the execution of dynamic classes at runtime while remaining compatible with existing applications.

6.2 RQ2: SBOM.exe Effectiveness

Table III: CVEs proven to be fully mitigated by SBOM.exe.

CVE	Vulnerable Dependency	Use Case	Vulnerable API	Malicious Class
CVE-2021-44228 ([0]Log4Shell..)	org.apache.logging.log4j:log4j-core:2.14.1	Logging framework	[0]org.apache.logging.log4j.Logger#error(String)..	[0]xExportObject..
CVE-2021-42392	com.h2database:h2:1.4.200	Database engine	[0]org.h2.util.JdbcUtils#getConnection (String, String,Properties)..	[0]xExportObject..
CVE-2022-33980	org.apache.commons:commons-configuration2:2.7	Parsing configuration files	[0]org.apache.commons.configuration2.interpol.Lookup#lookup(String)..	[0]jdk.nashorn.internal.scripts.Script$\^eval\_..

Table III presents the attacks we aim to mitigate in our experiment. We first mention the CVEs, then we report the vulnerable dependency that includes the CVE, as deployed on Maven Central. The dependency name is the concatenation of the group ID, artefact ID, and version. We also mention the use case for the dependency, to give some context about where the vulnerability is exploited. Next, we mention the particular API entrypoint for the attack and thus, make the dependency vulnerable. Finally, the column ‘Malicious Class’ is the classname that is injected during the attack, and which should be detected and blocked by SBOM.exe. All the attacks are available in our repository [22] and are fully reproducible.

For example, consider the first row of the table which has details about CVE-2021-44228, also known as [0]Log4Shell... The vulnerable dependency group ID, artefact ID, and version are [0]org.apache.logging.log4j.., [0]log4j-core.., and [0]2.14.1.. respectively. This dependency is one of the most popular logging frameworks in Java [48]. The API that processes the malicious input is [0]org.apache.logging.log4j.Logger#error(String)... and the malicious class we use in our experiment is called [0]xExportObject...

CVE-2021-44228 (Log4Shell)

Replication

We first replicate the [0]Log4Shell.. vulnerability in a proof-of-concept application. The application has a single class that has a method containing the vulnerable API whose normal usage is to log messages. Next, we setup the server hosting a classfile that contains bash commands as explained in section 3. We call this classfile [0]xExportObject... To trigger remote class loading, we pass [0]${jndi:ldap://localhost:1389/o=reference}.. as the malicious input to the application. This looks up the LDAP entry corresponding to the name “[0]o=reference..”. The entry has attributes, [0]javaCodebase.. and [0]javaFactory.. [49], that are used to load the malicious class [0]xExportObject... The name of the malicious class is arbitrary and can be any name. Finally, the malicious class executes the bash command and compromises the application.

Mitigation

The mitigation process in SBOM.exe is a two phase process as shown in Figure 2. The first phase happens offline, and SBOM.exe creates the BOMI with the classes from the Java standard library, the supply chain classes and the dynamically generated classes. For this example, the BOMI-Environment gathers all the 24085 classes from the Java standard library version 17.0.10+7 Temurin. The BOMI-SupplyChain contains the classes from the supply chain of the example application, which enumerates to 1269. For building the BOMI-Runtime, we execute a minimal test suite in order to capture dynamic classes. After running the tests, the dynamic code indexer reports 27 classes. All of these classes are either Proxy classes or subclasses of [0]MagicAccessorImpl.. used for introspection (see subsection 2.1). One of the proxy class implements the interface [0]org.apache.logging.log4j.core.config.plugins.Plugin... Recall that the BOMI needs to contain all the non-malicious runtime classes otherwise our agent can terminate the process over a benign class that was not recorded in the BOMI-Runtime.

The second phase of the mitigation procedure of SBOM.exe happens at runtime. In this part of the experiment, we run the application with the same malicious input as before $jndi:ldap://localhost:1389/o=reference, and we attach the SBOM Runtime Watchdog to the application. The JVM loads classes but it is terminated just before loading the malicious class [0]xExportObject.., demonstrating the success of SBOM.exe in mitigating the attack.

We explain this in more detail using 6. It begins with the (1) main method of the application that attempts to (2) log a message, a malicious input in this case. The log message is processed by the internal [0]Log4j.. classes that invoke the (3) lookup method inside the [0]Log4j.. dependency. This lookup method is delegated to the (4) [0]LdapCtx.. class which is part of the Java standard library and it queries for the entry [0]o=reference.. in the LDAP server. This query triggers class loading of the malicious class [0]xExportObject.. by invoking (5) [0]DirectoryManager.getObjectInstance()... The method reads two attributes of the LDAP entry [0]o=reference.. - [0]javaCodebase.. and [0]javaFactory.., then (6) [0]VersionHelper.loadClass().. initiates class loading [49]. At this point, the class is integrated into the runtime by the (7) Classloader and is ready for execution. However, the SBOM Runtime Watchdog (8) intercepts the malicious class, [0]xExportObject.. before execution and terminates the application. This demonstrates that SBOM.exe can integrate into the complex workflow of modern Java applications and mitigate this high-profile real-world attack.

⬇

SBOM_EXE.isLoadedClassAllowlisted$() (8)

ClassLoader.defineClass1$() (7)

// [...] internal java classes

VersionHelper.loadClass$() (6)

DirectoryManager.getObjectInstance$() (5)

LdapCtx.c_lookup$() (4)

JndiLookup.lookup$() (3)

// [...] internal Log4j classes

Logger.log$() (2)

Main.main$() (1)

Listing 6: Call stack at the time when the vulnerable application using Log4j is terminated.

CVE-2021-42392 (H2)

Replication

This CVE corresponds to the [0]H2.. database engine. To replicate an attack, we create the main class of an application that starts the database engine indefinitely. The default page is a login form that requires authentication as shown in Figure 3. We enter the malicious input in the fields "Driver Class" and "JDBC URL" with values [0]javax.naming.InitialContext.. and [0]ldap://localhost:1389/o=reference.. respectively. This leads to remote class loading of the malicious class [0]xExportObject.. from the same LDAP server as in the previous attack.

Mitigation

First, we generate the BOMI for this application. The BOMI-SupplyChain for [0]com.h2database:h2:1.4.200.. contains 944 classes. We create a new test to capture the dynamic classes generated during the execution of the application. The test that starts the database engine, authenticates the default user with username [0]sa.. and no password as shown in Figure 3, and then shuts down the server. Based on this test, the BOMI-Runtime records 20 runtime classes that are similar in nature to the ones we found in the previous application.

In the second phase, we start the server again but with SBOM Runtime Watchdog attached. Next, we go to the local host and pass the malicious inputs in the fields "Driver Class" and "JDBC URL" with values [0]javax.naming.InitialContext.. and [0]ldap://localhost:1389/o=reference.. respectively as shown in Figure 3. The driver class [0]javax.naming.InitialContext.. triggers the subset of the call stack from (4) to (8) as shown in 6. Similar to before, the application is terminated before the malicious class [0]xExportObject.. is executed. SBOM.exe successfully mitigates CVE-2021-42392.

Figure 3: Login screen of the H2 database engine with the malicious input and the default user credentials.

CVE-2022-33980 (Apache Commons)

Replication

Finally, we consider the CVE-2022-33980 which applies to the [0]Apache Commons Configuration.. dependency. This vulnerability leverages the Nashorn JavaScript engine which was bundled with the Java standard library until Java 15. The proof of concept application has a single class, which invokes the vulnerable API. The API takes a String input in the form of [0]prefix:name... For example, [0]java.. and [0]date.. [50] can be used to fetch information about the Java runtime and the current date respectively. We pass the malicious input [0]javascript:var clazz = java.lang.invoke.MethodHandles.lookup().defineClass(<malicious bytecode>); m = clazz.getMethod(’main’); m.invoke(null);... The input triggers the Nashorn engine, which generates the class [0]jdk.nashorn.internal.scripts.Script$\^eval\_.. and executes the malicious bytecode.

Mitigation

To experiment with SBOM.exe against this attack, we first create the BOMI for the application. The BOMI-Environment contains the 24278 classes of Java 11+28 OpenJDK. The BOMI-SupplyChain contains 788 classes from the [0]org.apache.commons:commons-configuration2:2.7.. and its dependencies. To create the BOMI-Runtime, we run a test that evaluates a JavaScript code printing date and time to the terminal. Based on this test, the BOMI-Runtime contains 34 classes.

⬇

SBOM_EXE.isLoadedClassAllowlisted$() (7)

ClassLoader.defineClass1$() (6)

// [...] internal java classes

CompilationPhase$InstallPhase.transform$() (5)

Compiler.compile$() (4)

// [...] internal nashorn classes

NashornScriptEngine.eval$() (3)

StringLookupAdapter.lookup$() (2)

Main.main$() (1)

Listing 7: Call stack at the time when the vulnerable application using commons-configuration is terminated when SBOM.exe detects the malicious class.

We run the application with a malicious JavaScript code as input in the form [0]"prefix:name"... The input contains a malicious bytecode array and JavaScript APIs that would be loaded and executed. We explain the mitigation of vulnerability using the call stack in 7. (1) Main code of the application takes in the input and (2) the class [0]StringLookupAdapter.. in [0]org.apache.commons:commons-configuration2:2.7.. processes it. Since the prefix is [0]javascript.., it triggers the (3) Nashorn engine to evaluate the JavaScript code. The Nashorn engine (4) compilation has 13 phases. The bytecode corresponding to the JavaScript code is generated in the 12th phase,‘Bytecode Generation’, and then in the (5) final phase, the bytecode class loading is initiated. The class is then integrated into the runtime by the (6) Classloader and is ready for execution. However, the SBOM Runtime Watchdog (7) intercepts the malicious class before execution and terminates the application. SBOM.exe successfully stops the attacker of CVE-2021-42392.

Answer to RQ2: To what extent can SBOM.exe mitigate high-profile attacks in Java?
SBOM.exe successfully indexes a comprehensive allowlist for high-profile CVEs that are exploitable. At runtime, the SBOM Watchdog of SBOM.exe does detect and prevent the execution of malicious classes, because they are not in the BOMI, terminating the vulnerable application before it is infected. SBOM.exe mitigates [0]Log4Shell.., one of the most devastating software supply chain attacks in years.

6.3 RQ3: SBOM.exe Applicability

Table IV: Replications used to evaluate whether SBOM.exe is compatible with real-world code, without spurious terminations.

Project	Module	Use Case	Release	kLOC	Tests	Stars
apache/pdfbox	[0]pdfbox-app..	PDF manipulation	3.0.2	117	1805	2.5K
mpetazzoni/ttorrent	[0]ttorrent-cli..	Torrent client	1.5	6	4	1.4K
graphhopper/graphhopper	[0]graphhopper-web..	Navigation engine	9.1	56	2878	4.8K

We select the versions of the real-world applications as shown in Table IV to evaluate the compatibility of SBOM.exe.

The first column ‘Project’ is the name of the GitHub repository of the application. The module column is the exact maven module that we consider. The use case column describes the functionality of the application that we evaluate. The release column is the latest version of the application as of 1^st June 2024, hosted on Maven Central. kLOC indicates the number of thousands of lines of code in the module. The number of tests is the total number of tests that are executed to capture the dynamic classes. Finally, we indicate the number of stars the repository has on GitHub, as an indicator of the popularity of the application.

For example, in the first row, we consider the [0]PDFBox.. application. We consider the module [0]pdfbox-app.. as it is the main module for performing PDF manipulations. The main module in maven projects produces the executable JAR. We take the latest release [0]3.0.2.. of the module. The application has 117 kLOC and 907 tests. The repository has 2.5K stars on GitHub.

In the following subsections, we present the results for each application. We run our experiments using OpenJDK 21.0.2. We organize the results of each application in four parts.

First, we describe the creation of a workload so that we can prepare a baseline usage for the application, as described in Table I. Then we describe the generated BOMI and we report the classes detected by SBOM.exe that are not part of the BOMI and the reasons those classes are not there. Finally, we discuss the changes that we made to the application to make it compatible with SBOM.exe.

Successful run of PDFBox

The workload for [0]PDFBox.. includes 10 PDF manipulations that can be done using the [0]pdfbox-app.. module [19]. They consist of manipulating a sample PDF file and verifying the output.

Before running the workload, we create the BOMI for the application. The BOMI-Environment contains 25309 classes from the Java standard library version 21.0.2+13-58 OpenJDK. The BOMI-SupplyChain contains 17760 classes from all dependencies of [0]PDFBox... The BOMI-Runtime contains 83 [0]PDFBox.. classes that are dynamically loaded during the execution of the test suite.

Next, we run the same workload with the SBOM Runtime Watchdog. We notice that it reports 4 classes that are not allowlisted as shown in 8. These proxy classes are generated because every manipulation command in PDFBox invokes APIs related to the [0]debugger.. module of [0]PDFBox... These APIs are not tested in the existing test suite and hence do not appear in the BOMI. This is a consistent, expected behavior.

⬇

1 [NOT ALLOWLISTED]: jdk/proxy1/$Proxy11

2 [NOT ALLOWLISTED]: jdk/proxy1/$Proxy12

3 [NOT ALLOWLISTED]: jdk/proxy1/$Proxy13

4 [NOT ALLOWLISTED]: jdk/proxy1/$Proxy14

Listing 8: False positives reported by PDFBox.

To make the application fully compatible with SBOM.exe’s approach, we write an additional test case. The test invokes the PDFBox Debugger as GUI and then shuts it down, this test makes sure that the classes generated by the debugger are captured during Dynamic Code Indexing. Once again, we run the application with the SBOM Runtime Watchdog and it runs to completion because all the classes loaded during normal execution are part of the BOMI. Thus, SBOM.exe is compatible with the [0]PDFBox.. application.

Successful run of Ttorrent

The workload for the [0]Ttorrent.. application is to download a torrent file and verify that the download has completed. We use a torrent file that has 1 text file in it.

The BOMI-SupplyChain contains 801 classes from the [0]com.turn:ttorrent-cli:1.5.. and its dependencies. BOMI-Runtime consists of 9 classes.

Upon initial run of Ttorrent, the SBOM Runtime Watchdog reports 11 classes that are not part of the BOMI. One proxy class is generated because of the torrent download. The rest of the 10 classes are labelled as modified because Maven Central hosts the dependencies of [0]Ttorrent.. with an outdated version of Java bytecode. The JAR that we execute contains a more recent version hence there is a difference in checksums.

We make two necessary changes in order to eliminate these initial false positives. First, we add a test that downloads the torrent file, and this captures the missing proxy class in BOMI-Runtime. Second, we self-host [0]ttorrent-cli:1.5.. and its dependencies. This ensures that the classes in BOMI-SupplyChain of [0]Ttorrent.. are compiled to Java 8 bytecode. After these changes, we are able to download the torrent and no false positives are reported. This demonstrates that for SBOM.exe to work: 1) tests must trigger the generation of runtime classes and 2) package managers have to be kept up to date.

Partially successful run of GraphHopper

We design the workload for the [0]GraphHopper.. application that starts up the server and makes 5 routing requests within Berlin, Germany. They involve multiple paths between source, destination, and intermediate points. For example, one of the requests is a route from [0](13.40, 52.55).. to [0](13.50, 52.50)... The first floating point number in the tuple is the longitude and the second is the latitude.

The BOMI-Environment is the exact same as the one in [0]PDFBox.. because we do not change the Java version. The BOMI-SupplyChain contains 23685 classes, and the BOMI-Runtime contains 266 classes.

Before requesting the routes, the server initialization reports 5 classes that are not part of the BOMI. The first 3 are proxy classes and are generated in order to start up the server. The next 2 are related to the configuration of the server.

We investigate all classes and find two clear reasons why they are not included in BOMI. First, there is no test that initializes the server. We add a test that initializes the server and then shuts it down when the server is ready, this captures all proxy classes.

Finally, there is a case where the names of the generated classes are based on a random UUID. Since we do a lookup based on the class name, we are not able to find these classes in the BOMI. Because of this, SBOM.exe is not fully compatible with [0]GraphHopper... Note that it does not invalidate the soundness of bytecode canonicalization. We believe that the whole JVM architecture has not been designed with determinism in mind, yielding this kind of integrity problems. In our current landscape dominated by cybersecurity concerns, this is being taken care of now in other ecosystems. For example, determinism is part of the architecture in Go ecosystem [51].

Answer to RQ3: Is SBOM.exe compatible with real-world applications?
SBOM.exe is fully compatible with real-world applications like [0]PDFBox.., and [0]Ttorrent.. with minimal changes. This validates the two key concepts of indexing and bytecode canonicalization. Our experiments also show the importance of appropriate testing for capturing all dynamically generated classes in the BOMI.

6.4 RQ4: SBOM.exe Overhead

Table V: Performance and Overhead Measurement for SBOM.exe.

Study Subjects	Performance (seconds / number of execution of workload)
	End-to-end time with warmup			Workload time excluding warmup
	W/O SBOM.exe	W/ SBOM.exe	Overhead	W/O SBOM.exe	W/ SBOM.exe	Overhead
apache/pdfbox	$5.949\pm 0.281$	$7.380\pm 1.456$	$(24.1\pm 29.2)\%$	$1.822\pm 0.027$	$1.840\pm 0.025$	$(1.0\pm 2.9)\%$
mpetazzoni/ttorrent	$1.225\pm 0.465$	$1.305\pm 0.108$	$(6.5\pm 46.8)\%$	$0.952\pm 0.032$	$0.954\pm 0.032$	$(0.2\pm 6.7)\%$
graphhopper/graphhopper	$0.900\pm 0.098$	$1.338\pm 0.264$	$(48.7\pm 40.6)\%$	$0.056\pm 0.003$	$0.056\pm 0.008$	$(0.0\pm 19.6)\%$

Table V shows the performance of the study subjects, as recorded by state-of-the-art tool JMH [47]. It is given in seconds per execution of the workload. The second column and the third column list the end-to-end time with warmup and workload time excluding warmup with and without SBOM.exe, as well as the overhead induced by SBOM Runtime Watchdog.

For example, [0]PDFBox.. takes $5.949\pm 0.281$ and $7.380\pm 1.456$ seconds to run the workload without and with SBOM.exe respectively when considering the end-to-end time with warmup. The overhead is computed as follows $\frac{3.404-2.551}{2.551}\pm\frac{3.404-2.551}{2.551}\sqrt{\left(\frac{0.393+0.204}{3.404-2.551}\right)^{2}\pm\left(\frac{0.204}{2.551}\right)^{2}}$. This evaluates to $(24.1\pm 29.2)\%$. Similarly, times after warmup are $1.822\pm 0.027$ and $1.840\pm 0.025$ seconds without and with SBOM.exe. The overhead incurred is $(1.0\pm 2.9)\%$.

In the following subsections, we discuss the end-to-end time with warmup and workload time excluding warmup metrics.

End-to-end time with warmup

The end-to-end time with warmup is the time taken by the JVM to warm up and execute the workload. While warming up, classes in the order of tens of thousands Table II are intercepted by the SBOM Runtime Watchdog. They are verified by comparing their checksums with the BOMI reference and then allowed to be executed. This adds a significant overhead to the end-to-end time with warmup. For example, more than an overhead of $50.0\%$ can be observed in all three study subjects. End-to-end time with warmup shows that the overhead is not negligible as a result of 1) bytecode canonicalization, and 2) additional classloading work. We argue it is acceptable in the context of long running applications like [0]GraphHopper...

Workload time excluding warmup

We notice that the overhead is less than 6.9% in all the study subjects, which is small. This is because, once the classes are verified by the SBOM Runtime Watchdog, they are stored in the JVM cache and are not verified anymore. Hence, the subsequent runs do not have the overhead of verification. All cases have negligible overhead and that also is incurred during the startup of the application. Thus, SBOM.exe is suitable for all study subjects, especially for long running applications.

Variability in the measurement

We notice that errors in overhead computed in Table V are significant. This is because of the measurement variability by JMH. The execution time of the workload is dependent upon a variety of factors like CPU allocation, JVM interpretation of bytecode, JIT compilation, profiling code, optimizations, garbage collection, etc. To minimise fluctuations because of these factors, JMH runs the benchmark in multiple forks of the JVM. Also, a benchmark could be optimized by the compiler and its execution time reported could be much less than the actual execution time. This factor does not affect the overhead because the same benchmark is run with and without SBOM.exe.

Answer to RQ4: What is the overhead of SBOM.exe?
The overhead introduced by SBOM.exe is negligible after warm-up. SBOM.exe can be used in production environments, especially with long-running applications, such as web and enterprise application servers, where the warm up time is a one-time cost which is not a concern.

7 Related Work

Four strategies have been proposed to protect against malicious code execution at runtime. In the following subsections, we discuss them.

7.1 Permission Managers

Permission managers allow developers to define access permissions for the application at varying granularities. Developers define policies, which specify what resources can be accessed by the application, libraries or system calls. The permission manager then enforces the policies at runtime. The integrity of the application is protected, as the sensitive resources can only be accessed per the rules defined in the policies.

Amusuo et al. [52] map network, filesystem, and process permissions to each dependency in the policy file. At runtime, the tool enforces the policy by allowing or denying access to the resources based on which access permissions are defined for the dependency. The enforcement of permissions is done by adding hooks to methods handling resource access in the Java standard library. Jamrozik et al. [53] map the complete application to the Android permissions for location, microphone, etc. They get the list of resources accessed by running the application against the test generated using fuzzing. Only the resources that were accessed during the tests are allowed to be accessed at runtime. Java also provides native support for writing permissions using Java Security Manager (JSM), which is a built-in permission manager that allows the developer to define access permissions for each Java class. JSM is deprecated for removal [54].

There is a body of literature on permission management for JavaScript. Ohm et al. [55], Ferreira et al. [56], Ahmadpanah et al. [57], and De Groef et al. [58] propose permission managers for Node.js applications. Vasilakis et al. [59] propose enforcing read, write, and execute permissions on each field and method in JavaScript. Finally, one can enforce permissions on every single system call invoked by an application, such as in SWAT4J [60], HODOR [61] and Confine [62], which leverage Seccomp BPF to restrict invocation of system calls at runtime.

Our approach is fundamentally different because it does not require the developer to define access permissions for the application, libraries, or system calls. To that extent, our approach is more lightweight and more smoothly applicable in practice. Moreover, enforcing permissions requires modification in the runtime environment like JVM or Node.js which is not required in our approach.

7.2 Compartmentalization

Compartmentalization is a method where different parts of an application are executed in different protection domains. This ensures integrity as one part of the application can be protected against malicious behavior in other parts. Application partitioning and third-party code isolation are methods that have been studied to compartmentalize applications at runtime.

Application partitioning can be done at the hardware level or the software level. Intel Software Guard Extensions (SGX) is a hardware-based security feature that runs untrusted code in an isolated environment, called enclave. Uranus [63] and Montsalvat [64] are two frameworks that enable developers to use SGX capabilities in their applications. The developers can use these libraries to indicate trusted and untrusted parts of the application. The JVM will then execute the trusted and untrusted parts in different regions of the CPU. Another hardware-based mechanism is proposed by Tsampas et al. [65]. They propose a source to source compiler that compiles the code in such a way that it is executed in isolated memory segments. The isolation of memory segments is done by unforgeable pointers that are protected by hardware.

At software level, the trusted and untrusted part of the application is executed in separate processes. Another line of work by Song et al. [66] and Gudka et al. [67] relocates the untrusted part of the application to a separate process. This process has limited access permissions, in the Linux sense.

Isolation of third-party code is another method to compartmentalize applications. [68], [69], [70], and [71] consider third-party code as untrusted because the developer cannot control its code. They propose running the third-party code in an isolated environment which has limited access to the system resources. Then it becomes a challenge for the developer to define the access permissions for the third-party code as we saw in the previous section - access permissions for dependency requires modification in runtimes like Node.js or JVM.

SBOM.exe runs the application in a single process so that the performance overhead at runtime is minimal. Moreover, compartmentalization approaches require manual work to split the application into trusted and untrusted parts.

7.3 Integrity Measurement

Integrity Measurement comes from the Linux kernel and means measuring the application in terms of its call, memory or any kind of execution behavior. This measurement data is then used at runtime to ensure that the application is executing as expected. These measurement based techniques can further be divided into two subsections based on the enforcement mechanism.

Manual verification of integrity

These approaches require the user to verify the integrity of the application by analyzing the measurement list. Hence, the verification of the measurement list is manual.

RIM4J [41] and JMF [72] propose a remote attestation protocol that verifies the integrity of applications running on the cloud. Upon request by the client, the cloud application sends the hash of classes to be written in the heap memory to the client. The client then verifies whether an observed hash is expected and stops the process if the measurement list is tampered.

On a similar line of work, Benedicts et al. [73] and Nauman et al. [74] propose remote attestation protocol for Docker containers and Android platforms respectively. They leverage techniques similar to Integrity Measurement Architecture (IMA) to measure all the loaded executables. This measurement list is hosted on a trusted platform and can be used for verification of loaded executables.

Automatic verification of integrity

These approaches attach the measurement list to the process running. At runtime, the execution behavior is verified automatically based on the measurement list.

RSDS [75] and Prof-gen [76] are two approaches that create an allowlist of system calls that can be invoked by containers. They employ static and dynamic analysis to get all the invoked system calls and then merge the results to get the final allowlist. This allowlist is then converted to a seccomp profile and enforced at runtime.

Nomura et al. [77] propose a method to generate an allowlist of SQL queries that can be invoked by the application. They run tests of the web application to capture all the SQL queries in an allowlist and then enforce it at runtime.

Finally, Anna et al. [78] propose an approach that uses dynamic analysis to create an allowlist of source code and associated file hashes. At runtime, they get the source code of the assembly instruction being executed and verify it against the allowlist. Then they use dynamic taint tracking to enforce the list at runtime.

SBOM.exe can be considered as an integrity measurement based system, specialized for Java where the measurement happens while running the test suite of the application and then enforced at runtime. Thus, it completely removes the manual work of verifying the integrity by client. SBOM.exe is novel in working at the granularity of dependencies, with an automated process of verifying the integrity of the application at runtime and stop** the process when the application loads a malicious class. Such a high level of granularity helps developers to understand the reasons behind integrity violations.

7.4 Other Kinds of Runtime Integrity

Control flow integrity is about checking that the application executes according to the control flow graph as intended. The survey by Burrow et al. [79] provides a comprehensive overview of recent related work on control flow integrity for runtime protection. SBOM.exe is different from control flow integrity: 1) it is at a higher level of abstraction: dependencies; 2) it protects against a different class of attacks (see section 3).

Oblivious Hashing [80] is a technique where the side effects of executed code are verified. The hashing function instruments the code to add guards at the end of every control flow path. At runtime, the guards ensure that the code is executing as expected by verifying the hash of the control flow path. SBOM.exe focuses on the prevention of execution of malicious classes rather than verifying the resulting state of the application. Moreover, SBOM.exe does not require any source code modification in the application whose runtime integrity needs to be preserved.

Deserialization attacks are common in Java applications [81]. In this class of attacks, the attacker sends a serialized object to the application and expects the application to deserialize it. Upon deserialization, the malicious code is executed. Filtering mechanisms [82] have been proposed which allow and deny certain classes to be deserialized. However, Sayar et al. [81] find that these mechanisms make the allowlist contain all possible serializable classes and the deny list is empty. A solution proposed by the authors is to allowlist only the used classes of the application and SBOM.exe is exactly on those lines.

8 Conclusion

In this paper, we proposed SBOM.exe, a novel approach to prevent software supply chain attacks that exploit the dynamic class loading capabilities in Java. SBOM.exe builds a comprehensive Bill of Materials Index (BOMI) which includes all classes that are expected to be loaded at runtime. In production, if the application loads a class that is not part of the BOMI, then SBOM.exe terminates the application to prevent the injection of malicious code. We demonstrate the effectiveness of SBOM.exe by stop** 3 high-profile software supply chain attacks in Java libraries - [0]Log4j.., [0]H2.., and [0]Apache Commons Configuration... We also demonstrate that the construction and monitoring of a BOMI is compatible with real-world applications like [0]PDFBox.., [0]ttorrent.., and [0]GraphHopper.., with limited overhead.

As future work, we plan to extend SBOM Runtime Watchdog to detect hidden classes [30]. As of today, no technology in the Java ecosystem can intercept the loading of hidden classes, but Dynamic Code Indexing can be extended to capture them. Finally, our approach has goals similar to GraalVM [83]. Both techniques aim to know all the classes before the application starts executing. However, GraalVM relies on static analysis and SBOM.exe relies on dynamic analysis and a thorough comparison of both approaches is on the research agenda.

9 Acknowledgment

We acknowledge the work of Elias Lundell, a research assistant at KTH Royal Institute of Technology, who helped in the replication of [0]Log4Shell... This work was supported by the CHAINS project funded by Swedish Foundation for Strategic Research (SSF), as well as by the Wallenberg Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. The icons in the figure are from https://www.flaticon.com/.