Fat binary

In 1988, Apollo Computer's Domain/OS SR10.1 introduced a new file type, "cmpexe" (compound executable), that bundled binaries for Motorola 680x0 and Apollo PRISM executables.[3]

A fat-binary scheme smoothed the Apple Macintosh's transition, beginning in 1994, from 68k microprocessors to PowerPC microprocessors. Many applications for the old platform ran transparently on the new platform under an evolving emulation scheme, but emulated code generally runs slower than native code. Applications released as "fat binaries" took up more storage space, but they ran at full speed on either platform. This was achieved by packaging both a 68000-compiled version and a PowerPC-compiled version of the same program into their executable files. The older 68K code (CFM-68K or classic 68K) continued to be stored in the resource fork, while the newer PowerPC code was contained in the data fork, in PEF format.[4]

Fat binaries were larger than programs supporting only the PowerPC or 68k, which led to the creation of a number of utilities that would strip out the unneeded version. In the era of small hard drives, when 80 MB hard drives were a common size, these utilities were sometimes useful, as program code was generally a large percentage of overall drive usage, and stripping the unneeded members of a fat binary would free up a significant amount of space on a hard drive.

Fat binaries were a feature of NeXT's NeXTSTEP/OPENSTEP operating system, starting with NeXTSTEP 3.1. In NeXTSTEP, they were called "Multi-Architecture Binaries". Multi-Architecture Binaries were originally intended to allow software to be compiled to run both on NeXT's Motorola 68k-based hardware and on Intel IA-32-based PCs running NeXTSTEP, with a single binary file for both platforms. It was later used to allow OPENSTEP applications to run on PCs and the various RISC platforms OPENSTEP supported. Multi-Architecture Binary files are in a special archive format, in which a single file stores one or more Mach-O subfiles for each architecture supported by the Multi-Architecture Binary. Every Multi-Architecture Binary starts with a structure (struct fat_header) containing two unsigned integers. The first integer ("magic") is used as a magic number to identify this file as a Fat Binary. The second integer ("nfat_arch") defines how many Mach-O Files the archive contains (how many instances of the same program for different architectures). After this header, there are nfat_arch number of fat_arch structures (struct fat_arch). This structure defines the offset (from the start of the file) at which to find the file, the alignment, the size and the CPU type and subtype which the Mach-O binary (within the archive) is targeted at.

The version of the GNU Compiler Collection shipped with the Developer Tools was able to cross-compile source code for the different architectures on which NeXTStep was able to run. For example, it was possible to choose the target architectures with multiple '-arch' options (with the architecture as argument). This was a convenient way to distribute a program for NeXTStep running on different architectures.

It was also possible to create libraries (e.g. using libtool) with different targeted object files.

Apple Computer acquired NeXT in 1996 and continued to work with the OPENSTEP code. Mach-O became the native object file format in Apple's free Darwin operating system (2000) and Apple's Mac OS X (2001), and NeXT's Multi-Architecture Binaries continued to be supported by the operating system. Under Mac OS X, Multi-Architecture Binaries can be used to support multiple variants of an architecture, for instance to have different versions of 32-bit code optimized for the PowerPC G3, PowerPC G4, and PowerPC 970 generations of processors. It can also be used to support multiple architectures, such as 32-bit and 64-bit PowerPC, or PowerPC and x86, or x86-64 and ARM64.[5]

Apple Universal binary logo

In 2005, Apple announced another transition, from PowerPC processors to Intel x86 processors. Apple promoted the distribution of new applications that support both PowerPC and x86 natively by using executable files in Multi-Architecture Binary format. Apple calls such programs "Universal applications" and calls the file format "Universal binary" as perhaps a way to distinguish this new transition from the previous transition, or other uses of Multi-Architecture Binary format.

Universal binary format was not necessary for forward migration of pre-existing native PowerPC applications; from 2006 to 2011, Apple supplied Rosetta, a PowerPC (PPC)-to-x86 dynamic binary translator, to play this role. However, Rosetta had a fairly steep performance overhead, so developers were encouraged to offer both PPC and Intel binaries, using Universal binaries. The obvious cost of Universal binary is that every installed executable file is larger, but in the years since the release of the PPC, hard-drive space has greatly outstripped executable size; while a Universal binary might be double the size of a single-platform version of the same application, free-space resources generally dwarf the code size, which becomes a minor issue. In fact, often a Universal-binary application will be smaller than two single-architecture applications because program resources can be shared rather than duplicated. If not all of the architectures are required, the lipo and ditto command-line applications can be used to remove versions from the Multi-Architecture Binary image, thereby creating what is sometimes called a thin binary.

In addition, Multi-Architecture Binary executables can contain code for both 32-bit and 64-bit versions of PowerPC and x86, allowing applications to be shipped in a form that supports 32-bit processors but that makes use of the larger address space and wider data paths when run on 64-bit processors.

In versions of the Xcode development environment from 2.1 through 3.2 (running on Mac OS X 10.4 through Mac OS X 10.6), Apple included utilities which allowed applications to be targeted for both Intel and PowerPC architecture; universal binaries could eventually contain up to four versions of the executable code (32-bit PowerPC, 32-bit x86, 64-bit PowerPC, and 64-bit x86). However, PowerPC support was removed from Xcode 4.0 and is therefore not available to developers running Mac OS X 10.7 or greater.

In 2020, Apple announced another transition, this time from Intel x86 processors to Apple silicon. To smooth the transition Apple added support for the Universal 2 binary format. This allows the creation of binaries that run natively on both 64-bit Intel and 64-bit Apple silicon (an AArch64 variant).

CP/M-80, MP/M-80, Concurrent CP/M, CP/M Plus and Personal CP/M-80 executables for the Intel 8080 (and Z80) processor families use the same .COM file extension as DOS-compatible operating systems for Intel 8086 binaries.[nb 1] In both cases programs are loaded at offset +100h and executed by jumping to the first byte in the file. As the opcodes of the two processor families are not compatible, attempting to start a program under the wrong operating system leads to incorrect and unpredictable behaviour.

In order to avoid this, some methods have been devised to build fat binaries which contain both a CP/M-80 and a DOS program, preceded by initial code which is interpreted correctly on both platforms. The methods either combine two fully functional programs each built for their corresponding environment, or add stubs which cause the program to exit gracefully if started on the wrong processor. For this to work, the first few instructions in the .COM file have to be valid code for both 8086 and 8080 processors, which would cause the processors to branch into different locations within the code. For example, the utilities in Simeon Cran's emulator MyZ80 start with the opcode sequence EBh, 52h, EBh. An 8086 sees this as a jump and reads its next instruction from offset +154h whereas an 8080 or compatible processor goes straight through and reads its next instruction from +103h. A similar sequence used for this purpose is EBh, 03h, C3h.[6][7]

Another method to keep a DOS-compatible operating system from erroneously executing .COM programs for CP/M-80 and MSX-DOS machines is to start the 8080 code with C3h, 03h, 01h, which is decoded as a "RET" instruction by x86 processors, thereby gracefully exiting the program, while it will be decoded as "JP 103h" instruction by 8080 processors and simply jump to the next instruction in the program.

Some CP/M-80 3.0 .COM files may have one or more RSX overlays attached to them by GENCOM.[8] If so, they start with an extra 256-byte header (one page). In order to indicate this, the first byte in the header is set to C9h, which works both as a signature identifying this type of COM file to the CP/M 3.0 executable loader, as well as a "RET" instruction for 8080-compatible processors which leads to a graceful exit if the file is executed under older versions of CP/M-80.

C9h is never appropriate as the first byte of a program for any x86 processor (it has different meanings for different generations,[nb 2] but is never a meaningful first byte); the executable loader in some versions of DOS rejects COM files that start with C9h, avoiding incorrect operation.

CP/M-86 and DOS do not share a common file extension for executables.[nb 1] Thus, it is not normally possible to confuse executables. However, early versions of DOS had so much in common with CP/M in terms of its architecture that some early DOS programs were developed to share binaries containing executable code. One program known to do this was WordStar 3.2x, which used identical overlay files in their ports for CP/M-86 and MS-DOS,[9] and used dynamically fixed-up code to adapt to the differing calling conventions of these operating systems at runtime.[9]

Digital Research's GSX for CP/M-86 and DOS also shares binary identical 16-bit drivers.[10]

DOS device drivers start with a file header whose first four bytes are FFFFFFFFh by convention, although this is not a requirement.[11] This is fixed up dynamically by the operating system when the driver loads (typically in the DOS BIOS when it executes DEVICE statements in CONFIG.SYS). Since DOS does not reject files with a .COM extension to be loaded per DEVICE and does not test for FFFFFFFFh, it is possible to combine a COM program and a device driver into the same file[12][11] by placing a jump instruction to the entry point of the embedded COM program within the first four bytes of the file (three bytes are usually sufficient).[11] If the embedded program and the device driver sections share a common portion of code, or data, it is necessary for the code to deal with being loaded at offset +0100h as a .COM style program, and at +0000h as a device driver.[12] For shared code loaded at the "wrong" offset but not designed to be position-independent, this requires an internal address fix-up[12] similar to what would otherwise already have been carried out by a relocating loader, except for that in this case it has to be done by the loaded program itself; this is similar to the situation with self-relocating drivers but with the program already loaded at the target location by the operating system's loader.

Under DOS, some files, by convention, have file extensions which do not reflect their actual file type.[nb 3] For example, COUNTRY.SYS[13] is not a DOS device driver,[nb 4] but a binary NLS database file for use with the CONFIG.SYS COUNTRY directive and the NLSFUNC driver.[13] The PC DOS and DR-DOS system files IBMBIO.COM and IBMDOS.COM are special binary images, not COM-style programs.[nb 4] Trying to load COUNTRY.SYS with a DEVICE statement or executing IBMBIO.COM or IBMDOS.COM at the command prompt will cause unpredictable results.[nb 3][nb 5]

It is sometimes possible to avoid this by utilizing techniques similar to those described above. For example, DR-DOS 7.02 and higher incorporate a safety feature developed by Matthias R. Paul:[14] If these files are called inappropriately, tiny embedded stubs will just display some file version information and exit gracefully.[15][14][16][13]

A similar protection feature was the 8080 instruction C7h ("RST 0") at the very start of Z-System language overlay files, which would result in a warm start (instead of a crash) under CP/M-80 if loaded inappropriately.[17]

In a remotely similar fashion, many (binary) file formats by convention include a 1Ah byte (ASCII ^Z) near the beginning of the file. This control character will be interpreted as "soft" end-of-file (EOF) marker when a file is opened in non-binary mode, and thus, under many operating systems (including RT-11, VMS, CP/M,[18][19] DOS,[20] and Windows[21]), it prevents "binary garbage" from being displayed when a file is accidentally typed at the console.

FatELF logo

FatELF[22] is a fat binary implementation for Linux and other Unix-like operating systems. Technically, a FatELF binary is a concatenation of ELF binaries with some meta data indicating which binary to use on what architecture.[23] Additionally to the CPU architecture abstraction (byte order, word size, CPU instruction set, etc.), there is the advantage of binaries with support for multiple kernel ABIs and versions.

FatELF has several use-cases, according to developers:[22]

• Distributions no longer need to have separate downloads for various platforms.
• Separated /lib, /lib32 and /lib64 trees are not required anymore in OS directory structure.
• The correct binary and libraries are centrally chosen by the system instead of shell scripts.
• If the ELF ABI changes someday, legacy users can be still supported.
• Distribution of web browser plug ins that work out of the box with multiple platforms.
• Distribution of one application file that works across Linux and BSD OS variants, without a platform compatibility layer on them.
• One hard drive partition can be booted on different machines with different CPU architectures, for development and experimentation. Same root file system, different kernel and CPU architecture.
• Applications provided by network share or USB sticks, will work on multiple systems. This is also helpful for creating portable applications and also cloud computing images for heterogeneous systems.[24]

A proof-of-concept Ubuntu 9.04 image is available.[25] As of 25 April 2020, FatELF has not been integrated into the mainline Linux kernel.[26][27]

Although the Portable Executable format used by Windows does not allow assigning code to platforms, it is still possible to make a loader program that dispatches based on architecture. This is because desktop versions of Windows on ARM has support for 32-bit x86 emulation, making it a useful "universal" machine code target. Fatpack is a loader that demonstrates the concept: it includes a 32-bit x86 program that tries to run the executables packed into its resource sections one by one.[28]

GCC and LLVM do not have a fat binary format, but they do have fat object files for link-time optimization (LTO). Since LTO involves delaying the compilation to link-time, the object files must store the intermediate representation, but on the other hand machine code may need to be stored too (for speed or compatibility). An LTO object containing both IR and machine code is known as a fat object.[29]

Even in a program or library intended for the same instruction set architecture, a programmer may wish to make use of some newer instruction set extensions while keeping compatibility with an older CPU. This can be achieved with function multi-versioning (FMV): versions of the same function are written into the program, and a piece of code decides which one to use by detecting the CPU's capabilities (such as through CPUID). Intel C++ Compiler, GNU Compiler Collection, and LLVM all have the ability to automatically generate multi-versioned functions.[30] This is a form of dynamic dispatch without any semantic effects.

Many math libraries feature hand-written assembly routines that are automatically chosen according to CPU capability. Examples include glibc, Intel MKL, and OpenBLAS. In addition, the library loader in glibc supports loading from alternative paths for specific CPU features.[31]