Security / / 4 min read
Golang Detection Notes
Experiments with Go binaries, string storage, and YARA signatures across platforms and CPU architectures.
Source: on_analysis ↗
Tl;dr - writing cross-platform Yara signatures for Golang binaries requires understanding how the runtime uses strings and bytes to produce durable signatures. This blog post dives into these details to help you write better Yara signatures for Golang binaries. Finally, I present a containerized Jupyter notebook for experimenting with writing signatures against a Golang program with known values to understand where and how Yara will detect them. All of the code discussed can be found on github at https://github.com/mrichard91/go_yara_detection.
Key Takeaways:
- golang stores "string" types in .rdata (or equiv) in a table of strings, they are not null terminated and no delimeter between strings - "foo", "bar" and "baz" as unique strings may appear as "foobarbaz"
- byte[] are often inlined into code by moving their values into registers, how this happens changes based on cpu and thus these values tend to make poor detection tokens
- compiled go code frequently contains references to the internal function names unless obfuscated with other tools, so for example, "Main.C2function" would appear in all platforms/cpu arch
Background
Golang is a cross-platform language with a robust package ecosystem and easy-to-use concurrency, making it a helpful tool for malware authors. Since its inception in 2009, the Golang malware ecosystem has steadily grown and now includes usage by ransomware, cybercrime, and nation-state actors in their operations globally. I found this article from Unit42, which provides a good survey of Golang malware in the wild.
Ground Truth
When writing Yara signatures to detect or classify malware, we often have only a small set of samples to work from. The small sample size requires us to make assumptions about how compilers will behave, such as that strings will always be stored with the characters in consecutive addresses, allowing us to use the entire string in our signature. I will propose and execute an experiment to write a program containing many attributes commonly used for signatures, compile them on every available operating system platform and CPU architecture, and then validate our signature assumptions. This process will use a Jupyter notebook to build the binaries, execute YARA with signatures for each artifact, and report the results for evaluation.
Our sample program will contain the following artifacts:
- Embedded IPv4, IPv6, and domain names (C2 addresses)
- Unique error string
- A four-byte magic value used in network communications
- HTTP GET request string
We can write Yara signatures to detect these across platforms since we control the source code and know the specific artifacts.
Findings
Using the notebook, we can see problematic detections that we should dive deep into to understand how cross-platform and different architectures impact our signature performance.
First, the compiler replicates all Golang strings from our source code as complete strings across all target builds. These make strong candidates for durable signature elements since CPU or OS do not affect them. We must be careful not to use the Yara "fullword" modifier in our rules since the Golang compiler packs strings without separators. These string values are in the .text section of Windows PE binaries and equivalent locations in ELF and Mach-O.
Second, the compiler does not reliably map strings, and magic values that we have defined as byte slices. These values appear in the binary in truncated form for strings and in alternative byte ordering for magic values. This breaking of the bytes reflects the compiler optimizing the loading of these bytes into memory for different CPU architectures, with the endianness often determining the specifics. For most platforms, you will know these values are bytes because they appear in an executable section of PE, elf, or Mach-o binary. To help identify these I propose using a Yara wrapper that finds the offset of the value in a signature and maps it to an RVA in the binary, then to the section represented by that RVA.
For Windows PE files there is a hack you can use to identify the section (credit to Wes Shields) using the Yara console module. The following rule will detect the unique error string and when the yara rule is run, print the PE section it was found in.
import "pe"
import "console"
// these rules attempt to use the console.log to print the pe section for the match
// credit to wxs for the rules
rule network_detect_string {
strings:
$a1 = "Error connecting baz:"
condition:
for all of ($a*): (
$ and (console.log(pe.sections[pe.section_index(@)].name))
)
}