History of computer animation

John Whitney, Sr (1917-1995) was an American animator, composer and inventor, widely considered to be one of the fathers of computer animation.[1] In the 1940s and 1950s, he and his brother James created a series of experimental films made with a custom-built device based on old anti-aircraft analog computers (Kerrison Predictors) connected by servos to control the motion of lights and lit objects — the first example of motion control photography. One of Whitney's best known works from this early period was the animated title sequence from Alfred Hitchcock's 1958 film Vertigo,[2] which he collaborated on with graphic designer Saul Bass. In 1960, Whitney established his company Motion Graphics Inc, which largely focused on producing titles for film and television, while continuing further experimental works. In 1968, his pioneering motion control model photography was used on Stanley Kubrick's movie 2001: A Space Odyssey, and also for the slit-scan photography technique used in the film's "Star Gate" finale.

One of the first programmable digital computers was SEAC (the Standards Eastern Automatic Computer), which entered service in 1950 at the National Bureau of Standards (NBS) in Maryland, USA.[3][4] In 1957, computer pioneer Russell Kirsch and his team unveiled a drum scanner for SEAC, to "trace variations of intensity over the surfaces of photographs", and so doing made the first digital image by scanning a photograph. The image, picturing Kirsch's three-month-old son, consisted of just 176×176 pixels. They used the computer to extract line drawings, count objects, recognize types of characters and display digital images on an oscilloscope screen. This breakthrough can be seen as the forerunner of all subsequent computer imaging, and recognising the importance of this first digital photograph, Life magazine in 2003 credited this image as one of the "100 Photographs That Changed the World".[5][6]

From the late 1950s and early '60s, mainframe digital computers were becoming commonplace within large organisations and universities, and increasingly these would be equipped with graphic plotting and graphics screen devices. Consequently, a new field of experimentation began to open up.

In 1960, a 49-second vector animation of a car traveling down a planned highway was created at the Swedish Royal Institute of Technology on the BESK computer. The consulting firm Nordisk ADB, which was a provider of software for the Royal Swedish Road and Water Construction Agency realized that they had all the coordinates to be able to draw perspective from the driver's seat for a motorway from Stockholm towards Nacka. In front of a specially designed digital oscilloscope with a resolution of about 1 megapixel a 35 mm camera with an extended magazine was mounted on a specially made stand. The camera was automatically controlled by the computer, which sent a signal to the camera when a new image was fed on the oscilloscope. It took an image every twenty meters (yards) of the virtual path. The result of this was a fictional journey on the virtual highway at a speed of 110 km/h (70 mph). The short animation was broadcast on November 9, 1961 at primetime in the national television newscast Aktuellt.[7][8]

Bell Labs in Murray Hill, New Jersey, was a leading research contributor in computer graphics, computer animation and electronic music from its beginnings in the early 1960s. Initially, researchers were interested in what the computer could be made to do, but the results of the visual work produced by the computer during this period established people like Edward Zajac, Michael Noll and Ken Knowlton as pioneering computer artists.

Edward Zajac produced one of the first computer generated films at Bell Labs in 1963, titled A Two Gyro Gravity Gradient Attitude Control System, which demonstrated that a satellite could be stabilized to always have a side facing the Earth as it orbited.[9]

Ken Knowlton developed the Beflix (Bell Flicks) animation system in 1963, which was used to produce dozens of artistic films by artists Stan VanDerBeek, Knowlton and Lillian Schwartz.[10] Instead of raw programming, Beflix worked using simple "graphic primitives", like draw a line, copy a region, fill an area, zoom an area, and the like.

In 1965, Michael Noll created computer-generated stereographic 3D movies, including a ballet of stick figures moving on a stage.[11] Some movies also showed four-dimensional hyper-objects projected to three dimensions.[12] Around 1967, Noll used the 4D animation technique to produce computer animated title sequences for the commercial film short Incredible Machine (produced by Bell Labs) and the TV special The Unexplained (produced by Walt DeFaria).[13] Many projects in other fields were also undertaken at this time.

In the 1960s, William Fetter was a graphic designer for Boeing at Wichita, and was credited with coining the phrase "Computer Graphics" to describe what he was doing at Boeing at the time (though Fetter himself credited this to colleague Verne Hudson).[14] [15] Fetter's work included the 1964 development of ergonomic descriptions of the human body that are both accurate and adaptable to different environments, and this resulted in the first 3D animated "wire-frame" figures.[16][17] Such human figures became one of the most iconic images of the early history of computer graphics, and often were referred to as the "Boeing Man". Fetter died in 2002.

Ivan Sutherland is considered by many to be the creator of Interactive Computer Graphics, and an internet pioneer. He worked at the Lincoln Laboratory at MIT (Massachusetts Institute of Technology) in 1962, where he developed a program called Sketchpad I, which allowed the user to interact directly with the image on the screen. This was the first Graphical User Interface, and is considered one of the most influential computer programs ever written by an individual.[18]

Utah was a major center for computer animation in this period. The computer science faculty was founded by David Evans in 1965, and many of the basic techniques of 3D computer graphics were developed here in the early 1970s with ARPA funding (Advanced Research Projects Agency). Research results included Gouraud, Phong, and Blinn shading, texture mapping, hidden surface algorithms, curved surface subdivision, real-time line-drawing and raster image display hardware, and early virtual reality work.[19] In the words of Robert Rivlin in his 1986 book The Algorithmic Image: Graphic Visions of the Computer Age, "almost every influential person in the modern computer-graphics community either passed through the University of Utah or came into contact with it in some way".[20]

In 1968, Ivan Sutherland teamed up with David Evans to found the company Evans & Sutherland—both were professors in the Computer Science Department at the University of Utah, and the company was formed to produce new hardware designed to run the systems being developed in the University. Many such algorithms have later resulted in the generation of significant hardware implementation, including the Geometry Engine, the Head-mounted display, the Frame buffer, and Flight simulators.[21] Most of the employees were active or former students, and included Jim Clark, who started Silicon Graphics in 1981, Ed Catmull, co-founder of Pixar in 1979, and John Warnock of Adobe Systems in 1982.

In 1968 a group of Soviet physicists and mathematicians with N.Konstantinov as its head created a mathematical model for the motion of a cat. On a BESM-4 computer they devised a programme for solving the ordinary differential equations for this model. The Computer printed hundreds of frames on paper using alphabet symbols that were later filmed in sequence thus creating the first computer animation of a character, a walking cat.[22][23]

Charles Csuri, an artist at The Ohio State University (OSU), started experimenting with the application of computer graphics to art in 1963. His efforts resulted in a prominent CG research laboratory that received funding from the National Science Foundation and other government and private agencies. The work at OSU revolved around animation languages, complex modeling environments, user-centric interfaces, human and creature motion descriptions, and other areas of interest to the discipline.[24][25][26]

In July 1968, the arts journal Studio International published a special issue titled Cybernetic Serendipity - The Computer and the Arts, which catalogued a comprehensive collection of items and examples of work being done in the field of computer art in organisations all over the world, and shown in exhibitions in London, UK, San Francisco, CA. and Washington, DC.[27][28] This marked a milestone in the development of the medium, and was considered by many to be of widespread influence and inspiration. Apart from all the examples mentioned above, two other particularly well known iconic images from this include Chaos to Order[29] by Charles Csuri (often referred to as the Hummingbird), created at Ohio State University in 1967,[30] and Running Cola is Africa[31] by Masao Komura and Koji Fujino created at the Computer Technique Group, Japan, also in 1967.[32]

The first machine to achieve widespread public attention in the media was Scanimate, an analog computer animation system designed and built by Lee Harrison of the Computer Image Corporation in Denver. From around 1969 onward, Scanimate systems were used to produce much of the video-based animation seen on television in commercials, show titles, and other graphics. It could create animations in real time, a great advantage over digital systems at the time.[33]

The National Film Board of Canada, already a world center for animation art, also began experimentation with computer techniques in 1969.[34] Most well-known of the early pioneers with this was artist Peter Foldes, who completed Metadata in 1971. This film comprised drawings animated by gradually changing from one image to the next, a technique known as "interpolating" (also known as "inbetweening" or "morphing"), which also featured in a number of earlier art examples during the 1960s.[35] In 1974, Foldes completed Hunger / La Faim, which was one of the first films to show solid filled (raster scanned) rendering, and was awarded the Jury Prize in the short film category at 1974 Cannes Film Festival, as well as an Academy Award nomination.

The Atlas Computer Laboratory near Oxford was for many years a major facility for computer animation in Britain.[36] The first entertainment cartoon made was The Flexipede, by Tony Pritchett, which was first shown publicly at the Cybernetic Serendipity exhibition in 1968.[37] Artist Colin Emmett and animator Alan Kitching first developed solid filled colour rendering in 1972, notably for the title animation for the BBC's The Burke Special TV program.

In 1973, Kitching went on to develop a software called "Antics", which allowed users to create animation without needing any programming.[38][39] The package was broadly based on conventional "cel" (celluloid) techniques, but with a wide range of tools including camera and graphics effects, interpolation ("inbetweening"/"morphing"), use of skeleton figures and grid overlays. Any number of drawings or cels could be animated at once by "choreographing" them in limitless ways using various types of "movements". At the time, only black & white plotter output was available, but Antics was able to produce full-color output by using the Technicolor Three-strip Process. Hence the name Antics was coined as an acronym for ANimated Technicolor-Image Computer System.[40] Antics was used for many animation works, including the first complete documentary movie Finite Elements, made for the Atlas Lab itself in 1975.[41]

From around the early 1970s, much of the emphasis in computer animation development was towards ever increasing realism in 3D imagery, and on effects designed for use in feature movies.

The first feature film to use digital image processing was the 1973 movie Westworld, a science-fiction film written and directed by novelist Michael Crichton, in which humanoid robots live amongst the humans.[42] John Whitney, Jr, and Gary Demos at Information International, Inc. digitally processed motion picture photography to appear pixelized in order to portray the Gunslinger android's point of view. The cinegraphic block portraiture was accomplished using the Technicolor Three-strip Process to color-separate each frame of the source images, then scanning them to convert into rectangular blocks according to its tone values, and finally outputting the result back to film. The process was covered in the American Cinematographer article "Behind the scenes of Westworld".[43]

Sam Matsa whose background in graphics started with the APT project at MIT with Doug Ross and Andy Van Dam petitioned Association for Computing Machinery (ACM) to form SICGRAPH (Special Interest Committee on Computer Graphics), the forerunner of ACM SIGGRAPH in 1967.[44] In 1974, the first SIGGRAPH conference on computer graphics opened. This annual conference soon became the dominant venue for presenting innovations in the field.[45][46]

The first use of 3D wireframe imagery in mainstream cinema was in the sequel to Westworld, Futureworld (1976), directed by Richard T. Heffron. This featured a computer-generated hand and face created by then University of Utah graduate students Edwin Catmull and Fred Parke which had initially appeared in their 1972 experimental short A Computer Animated Hand.[47] The same film also featured snippets from 1974 experimental short Faces and Body Parts. The Oscar-winning 1975 short animated film Great, about the life of the Victorian engineer Isambard Kingdom Brunel, contains a brief sequence of a rotating wireframe model of Brunel's final project, the iron steam ship SS Great Eastern.The third movie to use this technology was Star Wars (1977), written and directed by George Lucas, with wireframe imagery in the scenes with the Death Star plans, the targeting computers in the X-wing fighters, and the Millennium Falcon spacecraft.

The Walt Disney film The Black Hole (1979, directed by Gary Nelson) used wireframe rendering to depict the titular black hole, using equipment from Disney's engineers. In the same year, the science-fiction horror film Alien, directed by Ridley Scott, also used wireframe model graphics, in this case to render the navigation monitors in the spaceship. The footage was produced by Colin Emmett at the Atlas Computer Laboratory.[48]

Although Lawrence Livermore Labs in California is mainly known as a centre for high-level research in science, it continued producing significant advances in computer animation throughout this period. Notably, Nelson Max, who joined the Lab in 1971, and whose 1976 film Turning a sphere inside out is regarded as one of the classic early films in the medium (International Film Bureau, Chicago, 1976).[49] He also produced a series of "realistic-looking" molecular model animations that served to demonstrate the future role of CGI in scientific visualization ("CGI" = Computer-generated imagery). His research interests focused on realism in nature images, molecular graphics, computer animation, and 3D scientific visualization. He later served as computer graphics director for the Fujitsu pavilions at Expo 85 and 90 in Japan.[50][51]

In 1974, Alex Schure, a wealthy New York entrepreneur, established the Computer Graphics Laboratory (CGL) at the New York Institute of Technology (NYIT). He put together the most sophisticated studio of the time, with state of the art computers, film and graphic equipment, and hired top technology experts and artists to run it -- Ed Catmull, Malcolm Blanchard, Fred Parke and others all from Utah, plus others from around the country including Ralph Guggenheim, Alvy Ray Smith and Ed Emshwiller. During the late 1970s, the staff made numerous innovative contributions to image rendering techniques, and produced many influential software, including the animation program Tween, the paint program Paint, and the animation program SoftCel. Several videos from NYIT become quite famous: Sunstone, by Ed Emshwiller, Inside a Quark, by Ned Greene, and The Works. The latter, written by Lance Williams, was begun in 1978, and was intended to be the first full-length CGI film, but it was never completed, though a trailer for it was shown at SIGGRAPH 1982. In these years, many people regarded NYIT CG Lab as the top computer animation research and development group in the world.[52][53]

The quality of NYIT's work attracted the attention of George Lucas, who was interested in developing a CGI special effects facility at his company Lucasfilm. In 1979, he recruited the top talent from NYIT, including Catmull, Smith and Guggenheim to start his division, which later spun off as Pixar, founded in 1986 with funding by Apple Inc. co-founder Steve Jobs.

The framebuffer or framestore is a graphics screen configured with a memory buffer that contains data for a complete screen image. Typically, it is a rectangular array (raster) of pixels, and the number of pixels in the width and the height is its "resolution". Color values stored in the pixels can be from 1-bit (monochrome), to 24-bit (true color, 8-bits each for RGB—Red, Green, & Blue), or also 32-bit, with an extra 8-bits used as a transparency mask (alpha channel). Before the framebuffer, graphics displays were all vector-based, tracing straight lines from one co-ordinate to another. In 1948, the Manchester Baby computer used a Williams tube, where the 1-bit display was also the memory. An early (perhaps first known) example of a framebuffer was designed in 1969 by A. Michael Noll at Bell Labs,[54] This early system had just 2-bits, giving it 4 levels of gray scale. A later design had color, using more bits.[55][56] Laurie Spiegel implemented a simple paint program at Bell Labs to allow users to "paint" directly on the framebuffer.

The development of MOS memory (metal–oxide–semiconductor memory) integrated-circuit chips, particularly high-density DRAM (dynamic random-access memory) chips with at least 1 kb memory, made it practical to create a digital memory system with framebuffers capable of holding a standard definition (SD) video image.[57][58] This led to the development of the SuperPaint system by Richard Shoup at Xerox PARC during 1972–1973.[57] It used a framebuffer displaying 640×480 pixels (standard NTSC video resolution) with eight-bit depth (256 colors). The SuperPaint software contained all the essential elements of later paint packages, allowing the user to paint and modify pixels, using a palette of tools and effects, and thereby making it the first complete computer hardware and software solution for painting and editing images. Shoup also experimented with modifying the output signal using color tables, to allow the system to produce a wider variety of colors than the limited 8-bit range it contained. This scheme would later become commonplace in computer framebuffers. The SuperPaint framebuffer could also be used to capture input images from video.[59][60]

The first commercial framebuffer was produced in 1974 by Evans & Sutherland. It cost about $15,000, with a resolution of 512 by 512 pixels in 8-bit grayscale color, and sold well to graphics researchers without the resources to build their own framebuffer.[61] A little later, NYIT created the first full-color 24-bit RGB framebuffer by using three of the Evans & Sutherland framebuffers linked together as one device by a minicomputer. Many of the "firsts" that happened at NYIT were based on the development of this first raster graphics system.[52]

In 1975, the UK company Quantel, founded in 1973 by Peter Michael,[62] produced the first commercial full-color broadcast framebuffer, the Quantel DFS 3000. It was first used in TV coverage of the 1976 Montreal Olympics to generate a picture-in-picture inset of the Olympic flaming torch while the rest of the picture featured the runner entering the stadium. Framebuffer technology provided the cornerstone for the future development of digital television products.[63]

By the late 1970s, it became possible for personal computers (such as the Apple II) to contain low-color framebuffers. However, it was not until the 1980s that a real revolution in the field was seen, and framebuffers capable of holding a standard video image were incorporated into standalone workstations. By the 1990s, framebuffers eventually became the standard for all personal computers.

At this time, a major step forward to the goal of increased realism in 3D animation came with the development of "fractals". The term was coined in 1975 by mathematician Benoit Mandelbrot, who used it to extend the theoretical concept of fractional dimensions to geometric patterns in nature, and published in English translation of his book Fractals: Form, Chance and Dimension in 1977.[64][65]

In 1979–80, the first film using fractals to generate the graphics was made by Loren Carpenter of Boeing. Titled Vol Libre, it showed a flight over a fractal landscape, and was presented at SIGGRAPH 1980.[66] Carpenter was subsequently hired by Pixar to create the fractal planet in the Genesis Effect sequence of Star Trek II: The Wrath of Khan in June 1982.[67]

Bob Holzman of NASA's Jet Propulsion Laboratory in California established JPL's Computer Graphics Lab in 1977 as a group with technology expertise in visualizing data being returned from NASA missions. On the advice of Ivan Sutherland, Holzman hired a graduate student from Utah named Jim Blinn.[68][69] Blinn had worked with imaging techniques at Utah, and developed them into a system for NASA's visualization tasks. He produced a series of widely seen "fly-by" simulations, including the Voyager, Pioneer and Galileo spacecraft fly-bys of Jupiter, Saturn and their moons. He also worked with Carl Sagan, creating animations for his Cosmos: A Personal Voyage TV series. Blinn developed many influential new modelling techniques, and wrote papers on them for the IEEE (Institute of Electrical and Electronics Engineers), in their journal Computer Graphics and Applications. Some of these included environment mapping, improved highlight modelling, "blobby" modelling, simulation of wrinkled surfaces, and simulation of butts and dusty surfaces.

Later in the 1980s, Blinn developed CG animations for an Annenberg/CPB TV series, The Mechanical Universe, which consisted of over 500 scenes for 52 half-hour programs describing physics and mathematics concepts for college students. This he followed with production of another series devoted to mathematical concepts, called Project Mathematics!.[70]

Motion control photography is a technique that uses a computer to record (or specify) the exact motion of a film camera during a shot, so that the motion can be precisely duplicated again, or alternatively on another computer, and combined with the movement of other sources, such as CGI elements. Early forms of motion control go back to John Whitney's 1968 work on 2001: A Space Odyssey, and the effects on the 1977 movie Star Wars Episode IV: A New Hope, by George Lucas' newly created company Industrial Light & Magic in California (ILM). ILM created a digitally controlled camera known as the Dykstraflex, which performed complex and repeatable motions around stationary spaceship models, enabling separately filmed elements (spaceships, backgrounds, etc.) to be coordinated more accurately with one another. However, neither of these was actually computer-based—Dykstraflex was essentially a custom-built hard-wired collection of knobs and switches.[71] The first commercial computer-based motion control and CGI system was developed in 1981 in the UK by Moving Picture Company designer Bill Mather.[72]

3D computer graphics software began appearing for home computers in the late 1970s. The earliest known example is 3D Art Graphics, a set of 3D computer graphics effects, written by Kazumasa Mitazawa and released in June 1978 for the Apple II.[73][74]

Silicon Graphics, Inc (SGI) was a manufacturer of high-performance computer hardware and software, founded in 1981 by Jim Clark. His idea, called the Geometry Engine, was to create a series of components in a VLSI processor that would accomplish the main operations required in image synthesis—the matrix transforms, clipping, and the scaling operations that provided the transformation to view space. Clark attempted to shop his design around to computer companies, and finding no takers, he and colleagues at Stanford University, California, started their own company, Silicon Graphics.[75]

SGI's first product (1984) was the IRIS (Integrated Raster Imaging System). It used the 8 MHz M68000 processor with up to 2 MB memory, a custom 1024×1024 frame buffer, and the Geometry Engine to give the workstation its impressive image generation power. Its initial market was 3D graphics display terminals, but SGI's products, strategies and market positions evolved significantly over time, and for many years were a favoured choice for CGI companies in film, TV, and other fields.[76]

In 1981, Quantel released the "Paintbox", the first broadcast-quality turnkey system designed for creation and composition of television video and graphics. Its design emphasized the studio workflow efficiency required for live news production. Essentially, it was a framebuffer packaged with innovative user software, and it rapidly found applications in news, weather, station promos, commercials, and the like. Although it was essentially a design tool for still images, it was also sometimes used for frame-by-frame animations. Following its initial launch, it revolutionised the production of television graphics, and some Paintboxes are still in use today due to their image quality, and versatility.[77]

This was followed in 1982 by the Quantel Mirage, or DVM8000/1 "Digital Video Manipulator", a digital real-time video effects processor. This was based on Quantel's own hardware, plus a Hewlett-Packard computer for custom program effects. It was capable of warping a live video stream by texture mapping it onto an arbitrary three-dimensional shape, around which the viewer could freely rotate or zoom in real-time. It could also interpolate, or morph, between two different shapes. It was considered the first real-time 3D video effects processor, and the progenitor of subsequent DVE (Digital video effect) machines. In 1985, Quantel went on to produce "Harry", the first all-digital non-linear editing and effects compositing system.[78]

In 1982, Japan's Osaka University developed the LINKS-1 Computer Graphics System, a supercomputer that used up to 257 Zilog Z8001 microprocessors, used for rendering realistic 3D computer graphics. According to the Information Processing Society of Japan: "The core of 3D image rendering is calculating the luminance of each pixel making up a rendered surface from the given viewpoint, light source, and object position. The LINKS-1 system was developed to realize an image rendering methodology in which each pixel could be parallel processed independently using ray tracing. By developing a new software methodology specifically for high-speed image rendering, LINKS-1 was able to rapidly render highly realistic images." It was "used to create the world's first 3D planetarium-like video of the entire heavens that was made completely with computer graphics. The video was presented at the Fujitsu pavilion at the 1985 International Exposition in Tsukuba."[79] The LINKS-1 was the world's most powerful computer, as of 1984.[80]

In the '80s, University of Montreal was at the front run of Computer Animation with three successful short 3D animated films with 3D characters.

In 1983, Philippe Bergeron, Nadia Magnenat Thalmann, and Daniel Thalmann directed Dream Flight, considered as the first 3D generated film telling a story. The film was completely programmed using the MIRA graphical language,[81] an extension of the Pascal programming language based on abstract graphical data types.[82] The film got several awards and was shown at the SIGGRAPH '83 Film Show.

In 1985, Pierre Lachapelle, Philippe Bergeron, Pierre Robidoux and Daniel Langlois directed Tony de Peltrie, which shows the first animated human character to express emotion through facial expressions and body movements, which touched the feelings of the audience.[83][84] Tony de Peltrie premiered as the closing film of SIGGRAPH '85.

In 1987, the Engineering Institute of Canada celebrated its 100th anniversary. A major event, sponsored by Bell Canada and Northern Telecom (now Nortel), was planned for the Place des Arts in Montreal. For this event, Nadia Magnenat Thalmann and Daniel Thalmann simulated Marilyn Monroe and Humphrey Bogart meeting in a cafe in the old town section of Montreal. The short movie, called Rendez-vous in Montreal [85] was shown in numerous festivals and TV channels all over the world.

The Sun Microsystems company was founded in 1982 by Andy Bechtolsheim with other fellow graduate students at Stanford University. Bechtolsheim originally designed the SUN computer as a personal CAD workstation for the Stanford University Network (hence the acronym "SUN"). It was designed around the Motorola 68000 processor with the Unix operating system and virtual memory, and, like SGI, had an embedded frame buffer.[86] Later developments included computer servers and workstations built on its own RISC-based processor architecture and a suite of software products such as the Solaris operating system, and the Java platform. By the '90s, Sun workstations were popular for rendering in 3D CGI filmmaking—for example, Disney-Pixar's 1995 movie Toy Story used a render farm of 117 Sun workstations.[87] Sun was a proponent of open systems in general and Unix in particular, and a major contributor to open source software.[88]

The NFB's French-language animation studio founded its Centre d'animatique in 1980, at a cost of $1 million CAD, with a team of six computer graphics specialists. The unit was initially tasked with creating stereoscopic CGI sequences for the NFB's 3-D IMAX film Transitions for Expo 86. Staff at the Centre d'animatique included Daniel Langlois, who left in 1986 to form Softimage.[89][90]

Also in 1982, the first complete turnkey system designed specifically for creating broadcast-standard animation was produced by the Japanese company Nippon Univac Kaisha ("NUK", later merged with Burroughs), and incorporated the Antics 2-D computer animation software developed by Alan Kitching from his earlier versions. The configuration was based on the VAX 11/780 computer, linked to a Bosch 1-inch VTR, via NUK's own framebuffer. This framebuffer also showed realtime instant replays of animated vector sequences ("line test"), though finished full-color recording would take many seconds per frame.[91][92][93] The full system was successfully sold to broadcasters and animation production companies across Japan. Later in the '80s, Kitching developed versions of Antics for SGI and Apple Mac platforms, and these achieved a wider global distribution.[94]

The first cinema feature movie to make extensive use of solid 3D CGI was Walt Disney's Tron, directed by Steven Lisberger, in 1982. The film is celebrated as a milestone in the industry, though less than twenty minutes of this animation were actually used—mainly the scenes that show digital "terrain", or include vehicles such as Light Cycles, tanks and ships. To create the CGI scenes, Disney turned to the four leading computer graphics firms of the day: Information International Inc, Robert Abel and Associates (both in California), MAGI, and Digital Effects (both in New York). Each worked on a separate aspect of the movie, without any particular collaboration.[95] Tron was a box office success, grossing $33 million on a budget of $17 million.[96]

In 1984, Tron was followed by The Last Starfighter, a Universal Pictures / Lorimar production, directed by Nick Castle, and was one of cinema's earliest films to use extensive CGI to depict its many starships, environments and battle scenes. This was a great step forward compared with other films of the day, such as Return of the Jedi, which still used conventional physical models.[97] The computer graphics for the film were designed by artist Ron Cobb, and rendered by Digital Productions on a Cray X-MP supercomputer. A total of 27 minutes of finished CGI footage was produced—considered an enormous quantity at the time. The company estimated that using computer animation required only half the time, and one half to one third the cost of traditional special effects.[98] The movie was a financial success, earning over $28 million on an estimated budget of $15 million.[99]

The terms inbetweening and morphing are often used interchangeably, and signify the creating of a sequence of images where one image transforms gradually into another image smoothly by small steps. Graphically, an early example would be Charles Philipon's famous 1831 caricature of French King Louis Philippe turning into a pear (metamorphosis).[100] "Inbetweening" (AKA "tweening") is a term specifically coined for traditional animation technique, an early example being in E.G.Lutz's 1920 book Animated Cartoons.[101] In computer animation, inbetweening was used from the beginning (e.g., John Whitney in the '50s, Charles Csuri and Masao Komura in the '60s).[27] These pioneering examples were vector-based, comprising only outline drawings (as was also usual in conventional animation technique), and would often be described mathematically as "interpolation". Inbetweening with solid-filled colors appeared in the early '70s, (e.g., Alan Kitching's Antics at Atlas Lab, 1973,[40] and Peter Foldes' La Faim at NFBC, 1974[35]), but these were still entirely vector-based.

The term "morphing" did not become current until the late '80s, when it specifically applied to computer inbetweening with photographic images—for example, to make one face transform smoothly into another. The technique uses grids (or "meshes") overlaid on the images, to delineate the shape of key features (eyes, nose, mouth, etc.). Morphing then inbetweens one mesh to the next, and uses the resulting mesh to distort the image and simultaneously dissolve one to another, thereby preserving a coherent internal structure throughout. Thus, several different digital techniques come together in morphing.[102] Computer distortion of photographic images was first done by NASA, in the mid-1960s, to align Landsat and Skylab satellite images with each other. Texture mapping, which applies a photographic image to a 3D surface in another image, was first defined by Jim Blinn and Martin Newell in 1976. A 1980 paper by Ed Catmull and Alvy Ray Smith on geometric transformations, introduced a mesh-warping algorithm.[103] The earliest full demonstration of morphing was at the 1982 SIGGRAPH conference, where Tom Brigham of NYIT presented a short film sequence in which a woman transformed, or "morphed", into a lynx.

The first cinema movie to use morphing was Ron Howard's 1988 fantasy film Willow, where the main character, Willow, uses a magic wand to transform animal to animal to animal and finally, to a sorceress.

With 3D CGI, the inbetweening of photo-realistic computer models can also produce results similar to morphing, though technically, it is an entirely different process (but is nevertheless often also referred to as "morphing"). An early example is Nelson Max's 1977 film Turning a sphere inside out.[50] The first cinema feature film to use this technique was the 1986 Star Trek IV: The Voyage Home, directed by Leonard Nimoy, with visual effects by George Lucas's company Industrial Light & Magic (ILM). The movie includes a dream sequence where the crew travel back in time, and images of their faces transform into one another. To create it, ILM employed a new 3D scanning technology developed by Cyberware to digitize the cast members' heads, and used the resulting data for the computer models. Because each head model had the same number of key points, transforming one character into another was a relatively simple inbetweening.[104]

In 1989 James Cameron's underwater action movie The Abyss was released. This was the first cinema movie to include photo-realistic CGI integrated seamlessly into live-action scenes. A five-minute sequence featuring an animated tentacle or "pseudopod" was created by ILM, who designed a program to produce surface waves of differing sizes and kinetic properties for the pseudopod, including reflection, refraction and a morphing sequence. Although short, this successful blend of CGI and live action is widely considered a milestone in setting the direction for further future development in the field.[105]

The Great Mouse Detective (1986) was the first Disney film to extensively use computer animation, a fact that Disney used to promote the film during marketing. CGI was used during a two-minute climax scene on the Big Ben, inspired by a similar climax scene in Hayao Miyazaki's The Castle of Cagliostro (1979). The Great Mouse Detective, in turn, paved the way for the Disney Renaissance.[106][107]

The late 1980s saw another milestone in computer animation, this time in 2D: the development of Disney's "Computer Animation Production System", known as "CAPS/ink & paint". This was a custom collection of software, scanners and networked workstations developed by The Walt Disney Company in collaboration with Pixar. Its purpose was to computerize the ink-and-paint and post-production processes of traditionally animated films, to allow more efficient and sophisticated post-production by making the practice of hand-painting cels obsolete. The animators' drawings and background paintings are scanned into the computer, and animation drawings are inked and painted by digital artists. The drawings and backgrounds are then combined, using software that allows for camera movements, multiplane effects, and other techniques—including compositing with 3D image material. The system's first feature film use was in The Little Mermaid (1989), for the "farewell rainbow" scene near the end, but the first full-scale use was for The Rescuers Down Under (1990), which therefore became the first traditionally animated film to be entirely produced on computer—or indeed, the first 100% digital feature film of any kind ever produced.[108][109]

The 1990s began with much of CGI technology now sufficiently developed to allow a major expansion into film and TV production. 1991 is widely considered the "breakout year", with two major box-office successes, both making heavy use of CGI.

First of these was James Cameron's movie Terminator 2: Judgment Day,[118] and was the one that first brought CGI to widespread public attention. The technique was used to animate the two "Terminator" robots. The "T-1000" robot was given a "mimetic poly-alloy" (liquid metal) structure, which enabled this shapeshifting character to morph into almost anything it touched. Most of the key Terminator effects were provided by Industrial Light & Magic, and this film was the most ambitious CGI project since the 1982 film Tron.[119]

The other was Disney's Beauty and the Beast,[120] the second traditional 2D animated film to be entirely made using CAPS. The system also allowed easier combination of hand-drawn art with 3D CGI material, notably in the "waltz sequence", where Belle and Beast dance through a computer-generated ballroom as the camera "dollies" around them in simulated 3D space.[121] Notably, Beauty and the Beast was the first animated film ever to be nominated for a Best Picture Academy Award.[122]

Another significant step came in 1993, with Steven Spielberg's Jurassic Park,[123] where 3D CGI dinosaurs were integrated with life-sized animatronic counterparts. The CGI animals were created by ILM, and in a test scene to make a direct comparison of both techniques, Spielberg chose the CGI. Also watching was George Lucas who remarked "a major gap had been crossed, and things were never going to be the same."[124][125][126]

Flocking is the behavior exhibited when a group of birds (or other animals) move together in a flock. A mathematical model of flocking behavior was first simulated on a computer in 1986 by Craig Reynolds, and soon found its use in animation. Jurassic Park notably featured flocking, and brought it to widespread attention by mentioning it in the actual script. Other early uses were the flocking bats in Tim Burton's Batman Returns (1992), and the wildebeest stampede in Disney's The Lion King (1994).[127]

With improving hardware, lower costs, and an ever-increasing range of software tools, CGI techniques were soon rapidly taken up in both film and television production.

In 1993, J. Michael Straczynski's Babylon 5 became the first major television series to use CGI as the primary method for their visual effects (rather than using hand-built models), followed later the same year by Rockne S. O'Bannon's SeaQuest DSV.

Also the same year, the French company Studio Fantome produced the first full-length completely computer animated TV series, Insektors (26×13'),[128][129] though they also produced an even earlier all 3D short series, Geometric Fables (50 x 5') in 1991.[130] A little later, in 1994, the Canadian TV CGI series ReBoot (48×23') was aired, produced by Mainframe Entertainment.[131]

In 1995, there came the first fully computer-animation feature film, Disney-Pixar's Toy Story, which was a huge commercial success.[132] This film was directed by John Lasseter, a co-founder of Pixar, and former Disney animator, who started at Pixar with short movies such as Luxo Jr. (1986), Red's Dream (1987), and Tin Toy (1988), which was also the first computer-generated animated short film to win an Academy Award. Then, after some long negotiations between Disney and Pixar, a partnership deal was agreed in 1991 with the aim of producing a full feature movie, and Toy Story was the result.[133]

The following years saw a greatly increased uptake of digital animation techniques, with many new studios going into production, and existing companies making a transition from traditional techniques to CGI. Between 1995 and 2005 in the US, the average effects budget for a wide-release feature film leapt from $5 million to $40 million. According to Hutch Parker, President of Production at 20th Century Fox, as of 2005, "50 percent of feature films have significant effects. They're a character in the movie." However, CGI has made up for the expenditures by grossing over 20% more than their real-life counterparts, and by the early 2000s, computer-generated imagery had become the dominant form of special effects.[134]

Motion capture, or "Mocap", records the movement of external objects or people, and has applications for medicine, sports, robotics, and the military, as well as for animation in film, TV and games. The earliest example would be in 1878, with the pioneering photographic work of Eadweard Muybridge on human and animal locomotion, which is still a source for animators today.[135] Before computer graphics, capturing movements to use in animation would be done using Rotoscoping, where the motion of an actor was filmed, then the film used as a guide for the frame-by-frame motion of a hand-drawn animated character. The first example of this was Max Fleischer's Out of the Inkwell series in 1915, and a more recent notable example is the 1978 Ralph Bakshi 2D animated movie The Lord of the Rings.

Computer-based motion capture started as a photogrammetric analysis tool in biomechanics research in the 1970s and 1980s.[136] A performer wears markers near each joint to identify the motion by the positions or angles between the markers. Many different types of markers can be used—lights, reflective markers, LEDs, infra-red, inertial, mechanical, or wireless RF—and may be worn as a form of suit, or attached direct to a performer's body. Some systems include details of face and fingers to capture subtle expressions, and such is often referred to as "performance capture". The computer records the data from the markers, and uses it to animate digital character models in 2D or 3D computer animation, and in some cases this can include camera movement as well. In the 1990s, these techniques became widely used for visual effects. Video games also began to use motion capture to animate in-game characters in 1995, the earliest examples of this being the Atari Jaguar CD-based game Highlander: The Last of the MacLeods,[137][138] and the arcade fighting game Soul Edge, which was the first video game to use passive optical motion-capture technology.[139]

Another breakthrough where a cinema film used motion capture was creating hundreds of digital characters for the film Titanic in 1997. The technique was used extensively in 1999 to create Jar-Jar Binks and other digital characters in Star Wars: Episode I – The Phantom Menace.

Match moving (also known as motion tracking or camera tracking), although related to motion capture, is a completely different technique. Instead of using special cameras and sensors to record the motion of subjects, match moving works with pre-existing live-action footage, and uses computer software alone to track specific points in the scene through multiple frames, and thereby allow the insertion of CGI elements into the shot with correct position, scale, orientation, and motion relative to the existing material. The terms are used loosely to describe several different methods of extracting subject or camera motion information from a motion picture. The technique can be 2D or 3D, and can also include matching for camera movements. The earliest commercial software examples being 3D-Equalizer from Science.D.Visions[140] and rastrack from Hammerhead Productions,[141] both starting mid-90s.

The first step is identifying suitable features that the software tracking algorithm can lock onto and follow. Typically, features are chosen because they are bright or dark spots, edges or corners, or a facial feature—depending on the particular tracking algorithm being used. When a feature is tracked it becomes a series of 2D coordinates that represent the position of the feature across the series of frames. Such tracks can be used immediately for 2D motion tracking, or then be used to calculate 3D information. In 3D tracking, a process known as "calibration" derives the motion of the camera from the inverse-projection of the 2D paths, and from this a "reconstruction" process is used to recreate the photographed subject from the tracked data, and also any camera movement. This then allows an identical virtual camera to be moved in a 3D animation program, so that new animated elements can be composited back into the original live-action shot in perfectly matched perspective.[142]

In the 1990s, the technology progressed to the point that it became possible to include virtual stunt doubles. Camera tracking software was refined to allow increasingly complex visual effects developments that were previously impossible. Computer-generated extras also became used extensively in crowd scenes with advanced flocking and crowd simulation software. Being mainly software-based, match moving has become increasingly affordable as computers become cheaper and more powerful. It has become an essential visual effects tool and is even used providing effects in live television broadcasts.[143]

In television, a virtual studio, or virtual set, is a studio that allows the real-time combination of people or other real objects and computer generated environments and objects in a seamless manner. It requires that the 3D CGI environment is automatically locked to follow any movements of the live camera and lens precisely. The essence of such system is that it uses some form of camera tracking to create a live stream of data describing the exact camera movement, plus some realtime CGI rendering software that uses the camera tracking data and generates a synthetic image of the virtual set exactly linked to the camera motion. Both streams are then combined with a video mixer, typically using chroma key. Such virtual sets became common in TV programs in the 1990s, with the first practical system of this kind being the Synthevision virtual studio developed by the Japanese broadcasting corporation NHK (Nippon Hoso Kyokai) in 1991, and first used in their science special, Nano-space.[144] [145] Virtual studio techniques are also used in filmmaking, but this medium does not have the same requirement to operate entirely in realtime. Motion control or camera tracking can be used separately to generate the CGI elements later, and then combine with the live-action as a post-production process. However, by the 2000s, computer power had improved sufficiently to allow many virtual film sets to be generated in realtime, as in TV, so it was unnecessary to composite anything in post-production.

Machinima uses realtime 3D computer graphics rendering engines to create a cinematic production. Most often, video games machines are used for this. The Academy of Machinima Arts & Sciences (AMAS), a non-profit organization formed 2002, and dedicated to promoting machinima, defines machinima as "animated filmmaking within a real-time virtual 3-D environment". AMAS recognizes exemplary productions through awards given at its annual[146][147] The practice of using graphics engines from video games arose from the animated software introductions of the '80s "demoscene", Disney Interactive Studios' 1992 video game Stunt Island, and '90s recordings of gameplay in first-person shooter video games, such as id Software's Doom and Quake. Machinima Film Festival. Machinima-based artists are sometimes called machinimists or machinimators.

In 2000, a team led by Paul Debevec managed to adequately capture (and simulate) the reflectance field over the human face using the simplest of light stages.[162] which was the last missing piece of the puzzle to make digital look-alikes of known actors.

The first mainstream cinema film fully made with motion capture was the 2001 Japanese-American Final Fantasy: The Spirits Within directed by Hironobu Sakaguchi, which was also the first to use photorealistic CGI characters.[163] The film was not a box-office success.[164] Some commentators have suggested this may be partly because the lead CGI characters had facial features which fell into the "uncanny valley".[165] In 2002, Peter Jackson's The Lord of the Rings: The Two Towers was the first feature film to use a real-time motion capture system, which allowed the actions of actor Andy Serkis to be fed direct into the 3D CGI model of Gollum as it was being performed.[166]

Motion capture is seen by many as replacing the skills of the animator, and lacking the animator's ability to create exaggerated movements that are impossible to perform live. The end credits of Pixar's film Ratatouille (2007) carry a stamp certifying it as "100% Pure Animation — No Motion Capture!" However, proponents point out that the technique usually includes a good deal of adjustment work by animators as well. Nevertheless, in 2010, the US Film Academy (AMPAS) announced that motion-capture films will no longer be considered eligible for "Best Animated Feature Film" Oscars, stating "Motion capture by itself is not an animation technique."[167][168]

The early 2000s saw the advent of fully virtual cinematography with its audience debut considered to be in the 2003 films The Matrix Reloaded and The Matrix Revolutions with its digital look-alikes so convincing that it is often impossible to know if some image is a human imaged with a camera or a digital look-alike shot with a simulation of a camera. The scenes built and imaged within virtual cinematography are the "Burly brawl" and the end showdown between Neo and Agent Smith. With conventional cinematographic methods the burly brawl would have been prohibitively time-consuming to make with years of compositing required for a scene of few minutes. Also a human actor could not have been used for the end showdown in Matrix Revolutions: Agent Smith's cheekbone gets punched in by Neo leaving the digital look-alike naturally unhurt.