Count key data

Block diagram of count key data track format used on IBM mainframe computers beginning with S/360 shipment in 1965

"The beginning of a track is signalled when the index marker (index point) is detected.… The marker is automatically recognized by a special sensing device."[2]^:5 Following the index marker is the home address, which indicates the location of this track on the disk, and contains other control information internal to the control unit. A fixed-length gap follows the home address. Next, each track contains a Record 0 (R0), the track descriptor record, which is "designed to enable the entire content of a track to be moved to alternate tracks if a portion of the primary track becomes defective."[2]^:7 Following R0 are the data blocks, separated by gaps.[2]^:9

The principle of CKD records is that since data block lengths can vary, each block has an associated count field which identifies the block and indicates the size of the key, if used (user-defined up to 255 bytes), and the size of the data area, if used.[3][lower-alpha 2] The count field has the identification of the record[lower-alpha 3] in cylinder-head-record format, the length of the key, and the length of the data. The key may be omitted or consist of a string of characters.

Each CKD record consists of a count field, an optional key field, and an optional[lower-alpha 4] "user" data field with error correction/detection information appended to each field and gaps separating each field.[4][5] Because of the gaps and other information, the recorded space is larger than that required for just the count data, key data, or user data. IBM provides a "reference card" for each device, which can be used to compute the number of blocks per track for various block sizes, and to optimize the block size for the device.[6] Later, programs were written to do these calculations. Because blocks are normally not split between tracks, specification of an incorrect block size can waste up to half of each track.

Most often, the key is omitted and the record is located sequentially or by direct cylinder-head-record addressing. If it is present, the key is typically a copy of the first n bytes of the data record (for "unblocked" records, or a copy of the highest key in the block, for "blocked" records), but can be any data which will be used to find the record, usually using the Search Key Equal or Search Key High or Equal CCW. The key (and hence the record) is locatable via hardware commands.[7] Since the introduction of IBM's System/360 in 1964, nearly all IBM large and intermediate system DASDs have used the count key data record format.[8]

The advantages of count key data record format are:

• The record size can be exactly matched to the application block size
• CPU and memory requirements can be reduced by exploiting search-key commands.
• IBM CKD subsystems initially operated synchronously with the system channel and can process information in the gaps between the various fields, thereby achieving higher performance by avoiding the redundant transfer of information to the host.[8] Both synchronous and asynchronous operations are supported on later subsystems.[8]

Reduced CPU and memory prices and higher device and interface speeds have somewhat nullified the advantages of CKD, and it is retained only because IBM's flagship operating system z/OS does not support sector-oriented interfaces.

Originally CKD records had a one-to-one correspondence to a physical track of a DASD device; however over time the records have become more and more virtualized such that in modern IBM mainframes there is no longer a direct correspondence between the a CKD record ID and a physical layout of a track.

The initial feature set provided by IBM with its 1964 introduction of the CKD track format and associated CCWs included: .

• Defective/Alternative Track – enables an alternate track to replace a defective track transparent to the access method in use.
• Record overflow – records can exceed the maximum track length of a DASD[4]
• Multitrack operations – specific CCWs can continue onto the next sequential head[4]
• Command chaining – CCWs could be chained together to construct complex channel programs. The gaps in a CKD track format provided sufficient time between the commands so that all channel and SCU activity necessary to complete a command can be performed in the a gap between appropriate fields.[14] Such programs can search a large amount of information stored on a DASD, upon successful completion returning only the desired data and thereby freeing CPU resources for other activity.[15] This mode of operating synchronous to the gap was later enhanced by additional CCWs enabling a nonsychronous mode of operation.
• Channel switching – an SCU can be shared between channels – initially two channel switching was provided and it was expanded to up to eight channels in later SCUs. The channels can be on the same or different CPUS.

A Scan feature set was also provided but not continued into future CKD subsystems beyond the 2314.

Forty one CCWs implemented the feature set:

Notes:

O = optional feature

S = standard feature

MT = multitrack: when supported CCW will continue to operate on next heads in sequence to end of cylinder

‡ = TIC (Transfer In Channel) and other standard commands not shown.

† = code same as MT Off except as listed

1. File Scan Feature (9 CCWs) only available on 2841 for 2302, 2311 and 2321; they were not available on subsequent DASD controllers for DASD later than 2314.

2. Count is number of bytes in search argument, including mask bytes

The CCWs were initially were executed by two types of SCU attached to the system's high speed Selector Channels. The 2820 SCU[17] controlled the 2301 Drum while the 2841 SCU[4] controlled combinations of the 2302 Disk Storage, 2311 Disk Drive, 2321 Data Cell and/or 7320 Drum Storage. IBM quickly replaced the 7320 with the faster and larger 2303.

Subsequently, the feature set was implemented on the 2314 family of storage controls and an integrated attachment of the System 370 Model 25.

The following example of a channel program[19] reads a disk record identified by a Key field. The track containing the record and the desired value of the key is known. The SCU will search the track to find the requested record. In this example <> indicate that the channel program contains the storage address of the specified field.

  SEEK             <cylinder/head number>
  SEARCH KEY EQUAL <key value>
  TIC              *-8 Back to search if not equal
  READ DATA        <buffer> 

The TIC (transfer in channel) will cause the channel program to branch to the SEARCH command until a record with a matching key (or the end of the track) is encountered. When a record with a matching key is found the SCU will include Status Modifier in the channel status, causing the channel to skip the TIC CCW; thus the channel program will not branch and the channel will execute the READ command.

The block multiplexor channel was introduced beginning in 1971 on some high end System/360 systems along with the 2835 Control Unit and associated 2305 DASD,[20] This channel was then standard on IBM System/370 and subsequent mainframes; when contrasted to the prior Selector channel it offered performance improvements for high speed devices such as DASD, including:

  SEEK             <cylinder/head number>
  SET FILE MASK    <allow write operations>
  SET SECTOR       <sector number=0>
  WRITE R0         <cylinder/head/R0, key length=0, data length=6>
  WRITE CKD        <cylinder/head/R1, key length, data length>
  WRITE CKD        <cylinder/head/R2, key length, data length>
  WRITE CKD        <cylinder/head/R3, key length, data length>  

Defect skipping allows data to be written before and after one of more surface defects[lower-alpha 12] allowing all of a track to be used except for that portion that has the defect. This also eliminates the time that was formerly required to seek to an alternate track.[22] Only a limited number of defects could be skipped so alternate tracks remained supported for those tracks with excess defects.

Defect skipping was introduced in 1974 with the 3340 attached via the 3830 Model 2 Storage Control Unit[22] or integrated attachments on small systems. Defect skipping was essentially a factory only feature until 1981 when CCWs for management along with associated utilities were released.[23]

First introduced with the 3380 DASD on the 3880 Storage Control Unit[24] in 1981 the feature was included with the later CKD DASD subsystems. The dynamic path selection function controls operation of the two controllers, including simultaneous data transfer over the two paths. When supported by the operating system, each controller can serve as an alternate path in the event the other controller is unavailable.[25]

Three additional commands, Set Path Group ID, Sense Path Group ID, and Suspend Multipath Reconnection, are used to support attachment of the 3380 Models having two controllers at the head of a string.[24]

The Set Path Group ID command, with the dynamic path selection (DPS) function, provides greater flexibility in operations on reserved devices. Once a path group for a device has been established, it may be accessed over any path which is a member of the group to which it is reserved. In addition, on 370-XA systems which set the multipath mode bit in the function control byte (byte 0) to a 1, block multiplex reconnections will occur on the first available path which is a member of the group over which the channel program was initiated (regardless of the reservation state of the device).[24]

If the controller designated in the I/O address is busy or disabled, the dynamic path selection allows an alternate path to the device to be established via another storage director and the other controller in the model AA.[25]

Prior to the 1981 introduction of the 3880 director, CKD records were synchronously accessed, all activities required that one CCW be ended and the next initiated in the gaps between the CKD fields.[14] The gap size placed limitations on cable length but did provide for very high performance since complex chains of CCWs could be performed by the subsystem in real time without use of CPU memory or cycles.

Nonsynchronous operation provided by the Extended CKD ("ECKD") set of CCWs removed the gap timing constraint.[14] The five additional ECKD CCWs are Define Extent, Locate Record, Write Update Data, Write Update Key and Data, and Write CKD Next Track.[24]

In nonsynchronous operation, the transfer of data between the channel and the storage control is not synchronized with the transfer of data between the storage control and the device. Channel programs can be executed such that channel and storage control activities required to end execution of one command and advance to the next do not have to occur during the inter-record gap between two adjacent fields. An intermediate buffer in the storage control allows independent operations between the channel and the device. A major advantage of ECKDs is far longer cables; depending upon application it may improve performance.[14]

ECKD CCWs are supported on all subsequent CKD subsystems.

This example nonsynchronous channel program reads records R1 and R2 from track X'0E' in cylinder X'007F'. Both records have a key length of 8 and a data length of X'64' (100₁₀) bytes.[24]

  Define Extent       <extent= X'007F 0000' through track X'0081 000E'>
  Locate Record       <cylinder = X'007F', head = X'000E'
  Read Key and Data   <key record = X'001038'>
  Read Data           <record = X'001108'>

Caching first introduced in DASD CKD subsystems by Memorex[26] (1978) and StorageTek[lower-alpha 13] (1981) was subsequently introduced in late 1981 by IBM on the 3880 Model 13 for models of the 3380 with dynamic pathing.[lower-alpha 14]

The cache is dynamically managed by an algorithm; high activity data is accessed from the high-performance cache and low activity data is accessed from less-expensive DASD storage. A large memory in the Director, the cache, is divided into track slots that store data from the 3380 tracks. A smaller area is a directory that contains entries that allow data to be located in the cache.[27]

Caches were also provided on subsequently introduced storage controls.

Over time a number of path control, diagnostic and/or error recovery CCWs were implemented on one or more storage controls. For example:

• Unconditional Reserve allowed the releasing a device reserved to another channel and reserving the device to the channel issuing the command.
• Read Multiple Count Key Data could more efficiently read full tracks allowing for more efficient backups.

Reduced CPU and memory prices and higher device and interface speeds have somewhat nullified the advantages of CKD, and support continues by IBM to this date because its flagship operating system z/OS continues to use CKD CCWs for many functions.

Originally CKD records had a one-to-one correspondence to a physical track of a DASD device; however over time the records have become more and more virtualized such that in a modern IBM mainframe there is no longer a direct correspondence between the a CKD record ID and a physical layout of a track. An IBM mainframe constructs CKD track images in memory and executes the ECKD and CKD channel programs against the image. To bridge between the native fixed block sized disks and the variable length ECKD/CKD record format, the CKD track images in memory are mapped onto a series of fixed blocks suitable for transfer to and from an FBA disk subsystem.[28]

Of the 83 CKD CCWs implemented for System/360 and System/370 channels 56 are emulated on System/390 and later systems.[28]

• Block (data storage)
• Data set (IBM mainframe)
• Fixed-block architecture (FBA)
• Record (computer science)
• Track (disk drive)
• Volume Table of Contents (VTOC)

• IBM Data Processing Division (February 1974). Introduction to IBM Direct-Access Storage Devices and Organization Methods (PDF) (Tenth ed.). White Plains: International Business Machines. OCLC 8063006. GC20-1649-9. Retrieved 6 August 2014.
• Development of 360/370 Architecture - A Plain Man's View P.J. Gribbin, February 10, 1989, Chapters 8–10.