Masters-projects
Mitigating Instruction Set Incompatibilities in Heterogeneous HPC Systems

Mitigating Instruction Set Incompatibilities in Heterogeneous HPC Systems

Have you ever encountered an "Illegal instruction -- Core dumped" error, or are you curious about how SIGILL works? This thesis is for you!

Being a master’s student
List of projects
Register as a master’s student

Modern computing environments, particularly High-Performance Computing (HPC) clusters, often consist of diverse hardware with varying Instruction Set Architectures (ISAs). This heterogeneity presents significant challenges for binary compatibility. For instance, Intel Xeon Skylake processors support AVX512 instructions, while many AMD Epyc processors (Naples, Rome, Milan) do not, though newer "Genoa/GenoaX" models do. Similarly, Nvidia GH200 nodes with Arm ISA 9.0 are incompatible with Huawei Kunpeng920 nodes, which implement Arm 8.2+ ISA. These discrepancies mean binaries compiled on one platform might crash with "illegal instruction" errors when executed on another. This issue is not unique to x86 and Arm; it's a pervasive problem in heterogeneous systems. For example, we have seen that PyTorch distributes aarch64 Wheels (WHLs) built on Amazon EC2 Graviton2/3 instances (Arm ISA 8.2+), which fail on Nvidia Bluefield-2 DPUs (Arm A72 cores, ISA 8.0). Older PyTorch (< 1.5) distributions built on Graviton (Arm 8.0 ISA) work correctly on Bluefield-2 and Cavium ThunderX2. Operating systems like Linux and macOS employ various strategies to handle binary execution across different architectures. Linux uses its binfmt system to execute different binary formats, such as JVM for Java bytecode. Apple's M1-M5 processors utilize "Static Binary Translation" (SBT) or "Ahead-of-Time Binary Translation" (AOTBT), akin to Java's Just-In-Time (JIT) translation, to run x86_64 software. For native Arm binaries, Apple leverages its own binary translation technology to ensure compatibility across its diverse Arm ISAs. The LLVM-LIPO tool within Clang also facilitates the creation of "universal binaries" that can run on multiple architectures.

Goal

The primary goal of this master's thesis is to investigate and develop methods to ensure binary compatibility on hardware lacking specific instructions. This includes enabling machines with older processor architectures to execute code compiled for newer ones. A key focus will be on addressing scenarios where binaries compiled with advanced instruction sets (e.g., AVX512 or newer Arm ISAs) can still execute efficiently on older hardware. Your research could involve exploring and potentially implementing solutions such as e.g. a Kernel Module for Illegal Instruction Trapping: Developing a Linux kernel module that traps SIGILL (illegal instruction) signals. When an unsupported instruction is encountered, this module would attempt to translate and execute it using a sequence of supported instructions on the native processor. For example, complex AVX512 instructions could be decomposed and executed using a combination of AVX2 instructions. Dynamic Binary Translation (DBT) Techniques: Investigating and potentially implementing aspects of DBT, similar to how Apple's Rosetta 2 or Java's JIT compilers operate, to translate unsupported instruction sequences at runtime. Analysis of Existing Solutions: Deeply studying existing solutions like Linux's binfmt, FatELF, PTX Fat Binaries, and methodologies used by projects like Wine/Proton (for running Win32 binaries on Linux) or OpenBLAS (for optimized library generation across ISAs). Backward Binary Compatibility for Linux ISAs: Proposing and demonstrating a strategy to ensure backward binary compatibility in Linux for various Arm and x86_64 ISAs. Building a new ex3-modules environment with Universal binaries for each class of ISAs, e.g. Aarch64, x86_64/amd64, ao.

Learning outcome

Upon completion of this thesis, you will gain a profound understanding of:

  • Linux kernel modules and their interaction with the operating system
  • ELF binary formats and their execution lifecycle
  • The intricacies of Instruction Set Architectures (ISAs) and their impact on software compatibility
  • Just-In-Time (JIT) and Ahead-of-Time (AOT) binary translation technologies
  • Universal binaries using llvm-lipo (Clang)
  • Practical solutions for addressing challenges in heterogeneous computing environments

Qualifications

  • While prior knowledge of Linux operating systems and kernel modules is beneficial, it is not mandatory. However, a strong interest in learning fundamental concepts of the Linux kernel and innovative solutions for HPC challenges is essential.

Supervisors

  • Håkon Kvale Stensland
  • Tore H. Larsen (day-to-day collaborator)
  • Masoud Hemmatpour

Collaboration partners

  • UiO

References