tar (computing)

The file header record contains metadata about a file. To ensure portability across different architectures with different byte orderings, the information in the header record is encoded in ASCII. Thus if all the files in an archive are ASCII text files, and have ASCII names, then the archive is essentially an ASCII text file (containing many NUL characters).

The fields defined by the original Unix tar format are listed in the table below. The link indicator/file type table includes some modern extensions. When a field is unused it is filled with NUL bytes. The header uses 257 bytes, then is padded with NUL bytes to make it fill a 512 byte record. There is no "magic number" in the header, for file identification.

Pre-POSIX.1-1988 (i.e. v7) tar header:

The pre-POSIX.1-1988 Link indicator field can have the following values:

Some pre-POSIX.1-1988 tar implementations indicated a directory by having a trailing slash (/) in the name.

Numeric values are encoded in octal numbers using ASCII digits, with leading zeroes. For historical reasons, a final NUL or space character should also be used. Thus although there are 12 bytes reserved for storing the file size, only 11 octal digits can be stored. This gives a maximum file size of 8 gigabytes on archived files. To overcome this limitation, in 2001 star introduced a base-256 coding that is indicated by setting the high-order bit of the leftmost byte of a numeric field. GNU-tar and BSD-tar followed this idea. Additionally, versions of tar from before the first POSIX standard from 1988 pad the values with spaces instead of zeroes.

The checksum is calculated by taking the sum of the unsigned byte values of the header record with the eight checksum bytes taken to be ASCII spaces (decimal value 32). It is stored as a six digit octal number with leading zeroes followed by a NUL and then a space. Various implementations do not adhere to this format. In addition, some historic tar implementations treated bytes as signed. Implementations typically calculate the checksum both ways, and treat it as good if either the signed or unsigned sum matches the included checksum.

Unix filesystems support multiple links (names) for the same file. If several such files appear in a tar archive, only the first one is archived as a normal file; the rest are archived as hard links, with the "name of linked file" field set to the first one's name. On extraction, such hard links should be recreated in the file system.

Most modern tar programs read and write archives in the UStar (Unix Standard TAR[2][10]) format, introduced by the POSIX IEEE P1003.1 standard from 1988. It introduced additional header fields. Older tar programs will ignore the extra information (possibly extracting partially named files), while newer programs will test for the presence of the "ustar" string to determine if the new format is in use. The UStar format allows for longer file names and stores additional information about each file. The maximum filename size is 256, but it is split among a preceding path "filename prefix" and the filename itself, so can be much less.[11]

The type flag field can have the following values:

POSIX.1-1988 vendor specific extensions using link flag values 'A'..'Z' partially have a different meaning with different vendors and thus are seen as outdated and replaced by the POSIX.1-2001 extensions that also include a vendor tag.

Type '7' (Contiguous file) is formally marked as reserved in the POSIX standard, but was meant to indicate files which ought to be contiguously allocated on disk. Few operating systems support creating such files explicitly, and hence most TAR programs do not support them, and will treat type 7 files as if they were type 0 (regular). An exception is older versions of GNU tar, when running on the MASSCOMP RTU (Real Time Unix) operating system, which supported an O_CTG flag to the open() function to request a contiguous file; however, that support was removed from GNU tar version 1.24 onwards.

In 1997, Sun proposed a method for adding extensions to the tar format. This method was later accepted for the POSIX.1-2001 standard. This format is known as extended tar format or pax format. The new tar format allows users to add any type of vendor-tagged vendor-specific enhancements. The following tags are defined by the POSIX standard:

• atime, mtime: all timestamps of a file in arbitrary resolution (most implementations use nanosecond granularity)
• path: path names of unlimited length and character set coding
• linkpath: symlink target names of unlimited length and character set coding
• uname, gname: user and group names of unlimited length and character set coding
• size: files with unlimited size (the historic tar format is 8 GB)
• uid, gid: userid and groupid without size limitation (this historic tar format was is limited to a max. id of 2097151)
• a character set definition for path names and user/group names (UTF-8)

In 2001, the Star program became the first tar to support the new format. In 2004, GNU tar supported the new format, though it does not write them as its default output from the tar program yet.[12]

It's designed so that all implementations able to read the UStar format will be able to read POSIX.1-2001 as well. The only exceptions are files that make use of extended features, such as longer file names. For compatibility, these are encoded in the tar files as special x or g type files. typically under a PaxHeaders.XXXX directory.[13]^:exthdr.name A Pax-supporting implementation would make use of the info, while non supporting ones like 7-Zip would process them as extra files.[14]

A tarbomb, in hacker slang, is a tar file that contains many files that extract into the working directory. Such a tar file can create problems by overwriting files of the same name in the working directory, or mixing one project's files into another. It is at best an inconvenience to the user, who is obliged to identify and delete a number of files interspersed with the directory's other contents. Such behavior is considered bad etiquette on the part of the archive's creator.

A related problem is the use of absolute paths or parent directory references when creating tar files. Files extracted from such archives will often be created in unusual locations outside the working directory and, like a tarbomb, have the potential to overwrite existing files. However, modern versions of FreeBSD and GNU tar do not create or extract absolute paths and parent-directory references by default, unless it is explicitly allowed with the flag -P or the option --absolute-names. The bsdtar program, which is also available on many operating systems and is the default tar utility on Mac OS X v10.6, also does not follow parent-directory references or symbolic links.[18]

If a user has only a very old tar available, which does not feature those security measures, these problems can be mitigated by first examining a tar file using the command tar tf archive.tar, which lists the contents and allows to exclude problematic files afterwards.

These commands do not extract any files, but display the names of all files in the archive. If any are problematic, the user can create a new empty directory and extract the archive into it—or avoid the tar file entirely. Most graphical tools can display the contents of the archive before extracting them. Vim can open tar archives and display their contents. GNU Emacs is also able to open a tar archive and display its contents in a dired buffer.

The tar format was designed without a centralized index or table of content for files and their properties for streaming to tape backup devices. The archive must be read sequentially to list or extract files. For large tar archives, this causes a performance penalty, making tar archives unsuitable for situations that often require random access to individual files.

Another issue with tar format is that it allows several (possibly different) files in archive to have identical path and filename. When extracting such archive, usually the latter version of a file overwrites the former.

This can create a non-explicit (unobvious) tarbomb, which technically does not contain files with absolute paths or referring parent directories, but still causes overwriting files outside current directory (for example, archive may contain two files with the same path and filename, first of which is a symlink to some location outside current directory, and second of which is a regular file; then extracting such archive on some tar implementations may cause writing to the location pointed to by the symlink).