Where the Polyglots Are:
How Polyglot Files Enable Cyber Attack Chains and Methods for Detection & Disarmament

Bridges et al. conducted an in-depth evaluation of four leading COTS tools [Bridges et al.(2020)]. Among the test data were 199 malicious JPG+JAR polyglots that went completely undetected by all 4 tools. While we can not prove why these tools failed across the board, we can surmise—based on the unusual 0% detection rate—that the failure occurred in the file-type identification that must occur prior to feature extraction. If the files were interpreted as JPG (the benign component) rather than JAR (the malicious component), it is unlikely that the malicious JAR content was analyzed. This provides a plausible explanation for the complete detection failure. Therefore, to solve the problem of malicious polyglot detection, the problem of correct file type identification should first be solved.

The machine learning models FiFTy and Sceadan—a support vector machine (SVM) and a deep learning model, respectively—were released by researchers for file-format identification [Mittal et al.(2020), Beebe et al.(2013)]. However, neither tool was designed with polyglots in mind or trained on a dataset containing them.

Jana and Shmatikov demonstrated a number of attacks that exploited discrepancies in file type inference and file parsing. Specifically, they found that polyglots—referred to as "ambiguous files conforming to multiple formats" [Jana and Shmatikov(2012)]—evaded detection by 20 out of 36 malware detectors. Using the open-source ClamAV tool as an example, they point out that malware detectors may terminate format inference at the first match, extracting features and/or checking malware signatures only for the first format detected in the polyglot. Jana and Shmatikov argue that exhaustively testing incoming files against all possible formats would introduce an unacceptable overhead. Flexibility in format specification and parser tolerance of malformed files are also presented as reasons why simply improving existing tools is difficult.

Ange Albertini demonstrated that a wide variety of files can be combined into polyglots [Albertini(2015)]. He created an open-source tool known as mitra that can create 4 types of polyglot from a wide range of files. He defined the four types thusly:

• •

  Stack: File B is appended to the end of file A

• •

  Parasite: File B is placed inside comment markers of file A

• •

  Zipper: Both files are placed within one another’s comment markers

• •

  Cavity: File B is placed inside a padding area of file A

A number of previous academic works demonstrated the risk polyglots may pose. For example, a DICOM file is an image archive format designed for medical use. The format was designed to be flexible so medical staff could combine a variety of image formats into a single file for a patient [Graham et al.(2005)]. However, that flexibility means a DICOM file will tolerate combination with a Windows Portable Executable (PE) file to create a malicious polyglot [Ortiz(2019)]. This polyglot could allow an adversary to propagate their malicious PE through a medical network, activating the PE component through a second stage of the attack.

In an attack on data integrity, Popescu demonstrated that a PDF+TIFF polyglot can bypass digital certification verification [Popescu(2012a)]. In this scenario, the attacker sends a valid request (PDF file) for a bank transfer to a target. The attacker’s goal is to change the amount authorized in the PDF without invalidating the certification applied by the target. When the victim opens the file, auto-launch settings intepret the file as a PDF and present the legitimate PDF contents to the victim.

The victim then applies a digital signature that protects the file contents from any future change, and returns the file to the attacker. However, the file is also a TIFF file. The TIFF is an image of the same PDF, albeit with a much larger money transfer authorized. The attacker does not edit the contents of the file (which would break the signature). They merely change the file extension, switching the auto-launch behavior from opening the PDF contents to opening the TIFF contents, before sending the file on to a hypothetical bank.

Since the file contents have not changed, the digital certification is still valid. When the bank opens the file, auto-launch behavior shows the larger fraudulent TIFF transaction rather than the proper PDF amount the victim agreed to when they signed the file.

The survey, performed between November 2022 and January 2023, focused on identifying the role of a polyglot file within a threat actor’s cyber-attack chain. We used publicly available independent sources, general search engines and threat intelligence feeds (e.g., ORKL, X) to gather a wide range of information security reports and articles. Those sources were searched using the following terms: polyglot, combined, and contained. We found that the term polyglot is not always utilized in reports. We therefore had to manually distinguish between reports of true polyglots (two or more valid formats in one file) and other forms of digital steganography. A number of reports described malware that contained a valid format along with an oft-encrypted set of malicious instructions. We do not consider these files as polyglots because the malicious instruction can only be correctly interpreted when passed as input to another component of the malware rather than a parser conforming to a published standard.

For each true polyglot found, we used our knowledge of threat operations to determine the role the polyglot played in the cyber-attack chain. Lastly, the online malware databases, VirusTotal and MalwareBazaar, were used to obtain the actual polyglot samples whenever hashes of the polyglot were provided in a report. The file hashes and sources from our survey of open-source intelligence can be found in the appendix in Tables 3 and  4, respectively.

The survey discovered fifteen examples of a threat actor using a polyglot file in their cyber-attack chain, along with 30 distinct polyglot files. According to MITRE’s Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) framework, polyglots are primarily utilized for Defense Evasion (MITRE ATT&CK TA0005). Polyglot files also fall under the Obfuscated Files or Information (MITRE ATT&CK T1027) heading since these files conceal hidden functionality by appearing to conform to only one file format. We obtained 30 polyglot samples from VirusTotal and MalwareBazaar using the file hashes specified in the reports.

For the purpose of establishing a formal taxonomy for polyglot files, we refer to polyglots as having an overt format and a covert format. The overt format is the format the file presents as (e.g., matches the extension) while the covert format is not apparent without analysis. In most cases, a polyglot consists of a malicious file combined with a benign one; however, in some cases we found that both file formats play a role in advancing the malicious attack chain, as in the HTA+CHM polyglot utilized by IcedID in Section 3.2.1. Therefore, we instead refer to polyglots as combining an overt format with a covert format. A summary of the found-in-the-wild samples is provided in Table 1. In Appendix 9.2 we discuss the capabilities of interest that each file format provides to the malware author (camouflage, non-standard execution path, etc.) to understand why these combinations exist in the wild and how they fill a desired role in attack chains.

We selected two cyber attack chains to demonstrate how well-known APTs utilize polyglots to reach the next step in their cyber attack chains. A third attack chain can be found in Appendix 9.1. CVE numbers and MITRE ATT&CK references are provided where applicable.

The Fazah framework is a modular tool written in Python that can currently generate $46$ format combinations using $8$ covert formats. The combination method—stack and a variety of parasites—is derived from reports of malicious use in our survey and varies between covert format. As discussed in the survey, malicious actors use polyglots either to disguise malicious content using a less suspicious format (images) or add hidden functionality (scripts). Since image formats typically use comment markers, parasites are commonly used by malicious actors. Stacks, meanwhile, are the simplest and easiest method for malicious actors to implement, working well with script and archive formats. Files with distinct comment markers (necessary for zippers) are quite rare. Of the common (but by no means exhaustive) set of formats we tested, only DCM combined with either PDF/GIF/ISO could result in a zipper. Similarly, we found that only ISO paired with PE/PNG/GIF yielded cavities. This does not preclude their use in malicious campaigns, but places them beyond scope for our goal of emulating known attack chains. Table 1 provides the format pairings that Fazah can turn into polyglots. Given the possibility for malicious abuse of the framework, Fazah will not be published publicly at this time.

Our first objective was to determine which machine learning architecture and feature set were most effective at detecting polyglots. Toward this end, we created a small ($\sim 70,000$ files) initial data set using the mitra tool (described in Section 2.2) prior to the development of our Fazah tool. On this preliminary data set, we tested a Support Vector Machine, Random Forest, GradBoost, CatBoost, LightGBM, and MalConv. With the exception of MalConv [Raff et al.(2018)], these models used the byte histogram as their only feature. The byte histogram is a vector of length 256 where the value stored at each index corresponds to the number of times that byte value occurs in the input file. This feature vector is agnostic with respect to file formats since all digitally stored files are a string of bytes. We found that, on this preliminary data set, MalConv and CatBoost were the top performers.

We focused further development on MalConv and CatBoost, labeling our improved versions PolyConv and PolyCat, respectively. At this point, we trained and tested both models on our survey-informed Wild Polyglots data set; all results and figures reported in this paper refer to the Wild Polyglots data set. We found that, for PolyCat but not PolyConv, adding the mime-type output of the file utility improved results. Although file was not competitive at detecting polyglots (see Section 6.1) or at identifying both formats contained within, it was extremely accurate at identifying the first format contained in the file. Therefore, we augmented PolyCat’s feature space with a $1$-hot encoding of the mime-type output from file. We found further improvement by adding the $8000$ most common bigrams and trigrams extracted from each file using an overlap** window. Thus, the final feature space for PolyCat consisted of the byte histogram, the 1-hot encoding of the mime-type from file, and the most common bigrams and trigrams.

MalConv is an oft-cited deep learning classifier designed to detect malware [Raff et al.(2018)]. We trained the model from scratch to identify polyglots rather than to identify malware. None of the polyglots in our data set were malicious in order to guarantee that the model learned to detect multiple formats rather than malicious content. Since the model is trained on raw bytes rather than format-specific features (e.g., the EMBER feature set for PE files [Anderson and Roth(2018)]), MalConv’s architecture is well-suited to the polyglot detection problem which requires a format-agnostic approach. In lieu of a fixed feature-extraction routine, the model takes in raw bytes and learns an encoding (first layer) as well as a set of filters (the convolution layers) to recognize significant byte patterns. MalConv also features an attention and gating mechanism intended to filter out extraneous information in the raw bytes.

We experimented with changes to the architecture in order to make it more effective at our novel task, yielding the PolyConv model mentioned above. The original architecture of MalConv is presented in Figure 7 while PolyConv’s architecture is presented in Figure 8.

We established a baseline for performance by testing existing file-format identification tools on the Wild Polyglots data set: file [Darwin et al.(2019)], binwalk [ReFirmLabs(2021)], TrID [Pontello(2020)], and polydet [pol(2018)]. We also evaluated polyfile [of Bits(2022)], a DARPA-funded tool developed by Trail of Bits for detecting unusual files. Of the aforementioned tools, file and TrID are well-established signature-based utilities for file-format identification. VirusTotal, a widely used anti-virus aggregator (www.virustotal.com), utilizes TrID when reporting detected formats. Binwalk is a file-carving tool that has been used by analysts to find and extract hidden files. We selected these tool to establish a baseline because of their wide-spread use (file), cybersecurity application (binwalk,TrID), and polyglot-awareness (polyfile, polydet). We leave as future work a comparison to FiFTy [Mittal et al.(2020)] and Sceadan [Beebe et al.(2013)], as these detectors do not appear to be polyglot-aware, but might be re-trained in order to properly label polyglot files. Since file outputs labels and not probabilities, the precision-recall curve is not an appropriate metric when comparing our deep learning model to existing tools. Instead, we used the F1 score that balances recall and precision. For any cybersecurity system deployable in the real-world, the ability to detect malware/polyglots (recall) must be tempered by a low probability of false positives (precision) to prevent red-flag fatigue. Therefore, we use F1 to provide a balanced evaluation.

Content disarmament and reconstruction (CDR) tools present an alternative approach to the pre-processing filtering approach for which we have provided solutions. CDR tools allow an end user to strip all but the most trustworthy content from certain formats. Where highly flexible formats, like PDF, have proliferated, these tools have emerged to provide secure use of files that abuse the format flexibility.

Although we have not exhaustively examined this approach, we have developed an image sanitization tool to demonstrate the potential of CDR in disarming polyglots. Our tool, ImSan, disarms image-based polyglots by strip** away all file contents that are not required to display the image. The process is quite straightforward:

1. 1.

   The image file is loaded into Pillow, a fork of the Python Imaging Library

2. 2.

   The image contents are then written to a new file with the option to strip all metadata activated

3. 3.

   The new image file has no extraneous content before/after the image contents (stack/cavity polyglot) or inserted into comment areas (parasite/zipper polyglot)

ImSan can disarm any of the formats that are fully supported (read/write) by Pillow: BLP, BMP, DDS, DIB, EPS, GIF, ICNS, ICO, IM, JPEG, JPEG 2000, MSP, PCX, PNG, PPM, SGI,SPIDER, TGA, TIFF, WebP, XBM. Note, ImSan should be run in an isolated environment to ensure that no vulnerability in Pillow (2 CVE’s reported in 2022) could allow a malicious image to gain execution when the image is parsed.

ImSan disarmed 100% of the image polyglots in a subset (n=392) of image polyglots drawn randomly from the benign Wild Polyglots data set. A small subset was chosen so we could manually verify disarmament through visual inspection of the image’s code rather than relying on one of our detectors. An evaluation of commercial CDR tools against polyglots (including those that are not image based) and the potential methods of circumventing CDR solutions, while out of scope for this work, would be a valuable direction for future work to explore.

We presented the first, to our knowledge, survey of polyglot usage by malicious actors in the wild, demonstrating that polyglot files are an actively used TTP by well-known malicious actors, answering RQ1.

In order to answer RQ2-RQ4, we created a novel data set of polyglot and monoglot files based on file formats and file-combination methods utilized by malicious actors in the wild.

Using this Wild Polyglots data set, we evaluated a number of different machine learning models before focusing on the top two performers, PolyConv and PolyCat. we improved these two models via alterations to their architecture and feature space, respectively.

We found that PolyConv, in both binary and multi-label versions, was effective at detecting polyglots and correctly labeling their contents RQ2, providing analysts with a tool to detect, reroute, and investigate potential polyglots.

PolyConv only slightly outperformed the model upon which it was based, MalConv, demonstrating that MalConv effectively learned to distinguish between polyglot and monoglot files when trained on this objective, despite being designed to detect malware. This is a novel use of MalConv considering that the model was designed to detect PE malware.

Based on our experiments, the improvement from MalConv to PolyConv was due to the reduction of the window and stride size as well as increasing the number of filters and layers. We theorize that the much smaller window/stride allowed the model to learn filters that register even small areas of code with a distinct byte pattern. The need for more filters may be due to the wide variety of formats, each with their own distribution of unique byte patterns, upon which we trained. On the other hand, removing the attention and gating mechanism did not reduce the model’s classification performance.

We answered RQ3 by demonstrating that existing tools do not reliably detect polyglot files, even when designed with an awareness of polyglot files.

To answer RQ4, we produced a set of YARA rules for detecting extraneous content in image files, but found their performance lacking. The rules are available upon request. We then created ImSan, a content disarmament and reconstruction tool that sanitizes image files, demonstrating that it disarmed all of the image-based polyglots with which we tested it.

We cannot guarantee that our deep learning models will perform well on polyglots formed from file formats not included in the training data. File formats based on Open XML (Microsoft Office) are a common malware vector that have not yet been thoroughly explored as polyglot components. PolyConv is format agnostic so we hope that this model’s release will prompt further research using additional file formats.

File size is a limiting factor for malware classification models trained on raw bytes rather than extracted features. Detection could be evaded by utilizing a polyglot whose overt format exceeds the full input capacity of the model, meaning the covert format would not be ingested. We tested head and tail scanning on our data set, but found that this did not improve results since the vast majority of our data set is within the maximum capacity of our PolyConv model. Head and tail scanning could still be evaded if an adversary inserted the second file near the middle of a particularly large first file or appended a large amount of data after the second file contents.

We also tested the YARA rule approach, but found it A) limited by the need to write novel rules for each possible combination of file formats and B) the lack of required signatures in many flexible file formats.

Given the ubiquity of signature-based tools, the development and demonstration of a set of signatures or rules that detect a very high percentage of functional script files as well as the ability to differentiate between different scripting languages would be valuable. While JavaScript (JS) lacks a strict signature, it might be possible to craft a set of rules, perhaps using a tool such as AutoYARA [Raff et al.(2020)], that detects all JS files that have enough control flow instructions to be functional/malicious. This would require the collection of a large number of files for each language, along with some level of assurance that the data set is extremely diverse with respect to the file contents/functionality. If Libmagic were able to reliably distinguish JavaScript and Powershell files from other forms of text, the performance of polyfile and polydet could improve markedly. We are not confident the rule-based approach is tractable, yet cannot rule it out.

Since PolyConv utilizes a global max pooling layer, it is translation invariant. That said, a demonstration of its ability to generalize to novel insertion areas remains future work. We consider translation invariance an important feature in order to future-proof a polyglot detector. Given the flexibility in file formats, it is possible that novel polyglot creation methods will emerge in the future that hide the second file in a novel area of the first file. Therefore, a demonstration that PolyConv is resilient in the face of novel combination methods would demonstrate that future models for polyglot detection should also be translation invariant.

Future work should include the implementation of an intelligent method for subselecting or compressing large input files so they fit within the maximum capacity of a model trained on raw bytes. Head and tail scanning would catch data appended to the very end of the file, but could be evaded by inserting data earlier in the file or appending more benign content after the additional malicious content. Therefore, a more robust input reduction method should not follow a fixed pattern such as always scanning N bytes from the head and M bytes from the tail. Such a method may exist in other domains; we look forward to developments in this area.

Finally, PolyConv needs to be trained and tested on the same wide variety of files as the ubiquitous file utility in order to see widespread adoption.

Batloader and Zloader are two very similar pieces of malware that are used to gain initial access [Kiat et al.(2021), Team(2022b)]. The full attack chain is presented in Figure 12; however, our discussion will focus on the role of the polyglot within that chain. This polyglot is formed by combining an HTA file with a Windows PE file.

Windows PE files are the default executable for the Windows ecosystem. Since their format specification requires the bytes "MZ" to be present at offset zero, this format must be the first—by offset—ingredient in a polyglot in order to preserve functionality. PE polyglots can be created via the cave or stack method. The cave method places the second file in a slack region of the PE. Candidate locations include the DOS Stub, after the last section table entry, or in the padding space after each section assuming the chosen region is large enough to contain the second file. The stack method simply appends the second file to the end (also referred to as the overlay) of the PE file.

In this particular example, an HTA file is added to the signature section of the PE file. Rather ironically, CVE-2020-1599 allows malware authors to add contents to the signature section without invalidating the signature since the contents of this area need to be writable in order to store the calculated signature.

The following sections detail how each overt format was used in combination with a covert format to surreptitiously execute or stage a malicious payload. This is followed by details on the roles filled by polyglot files in notable cyber-attack chains.