Telepresence

To provide a telepresence experience, technologies are required that implement the human sensory elements of vision, sound, and manipulation.

A minimum system usually includes visual feedback. Ideally, the entire field of view of the user is filled with a view of the remote location, and the viewpoint corresponds to the movement and orientation of the user's head. In this way, it differs from television or cinema, where the viewpoint is out of the control of the viewer.

In order to achieve this, the user may be provided with either a very large (or wraparound) screen, or small displays mounted directly in front of the eyes. The latter provides a particularly convincing 3D sensation. The movements of the user's head must be sensed, and the camera must mimic those movements accurately and in real time. This is important to prevent unintended motion sickness.

Another source of future improvement to telepresence displays, compared by some to holograms, is a projected display technology featuring life-sized imagery.[13]

Sound is generally the easiest sensation to implement with high fidelity, based on the foundational telephone technology dating back more than 130 years. Very high-fidelity sound equipment has also been available for a considerable period of time, with stereophonic sound being more convincing than monaural sound.

Monty, a telemanipulation prototype from Anybots

The ability to manipulate a remote object or environment is an important aspect for some telepresence users and can be implemented in a large number of ways depending on the needs of the user. Typically, the movements of the user's hands (position in space, and posture of the fingers) are sensed by wired gloves, inertial sensors, or absolute spatial position sensors. A robot in the remote location then copies those movements as closely as possible. This ability is also known as teleoperation.

The more closely the robot re-creates the form factor of the human hand, the greater the sense of telepresence. The complexity of robotic effectors varies greatly, from simple one axis grippers, to fully anthropomorphic robot hands.

Haptic teleoperation refers to a system that provides some sort of tactile force feedback to the user, so the user feels some approximation of the weight, firmness, size, and/or texture of the remote objects manipulated by the robot. A new form of technology, called collaborative telepresence, is currently being developed which will eventually be used to collaborate with others while seeming like you are in the same room as the other person, keeping a normal social distance. Collaborative telepresence uses haptic sensors like these to allow a sense of touch.

iRobot Ava 500, an autonomous roaming telepresence robot.

The prevalence of high quality video conferencing using mobile devices, tablets and portable computers has enabled considerable growth in telepresence robots to help give a better sense of remote physical presence for communication and collaboration in the office, home or school when one cannot be there in person. The robot avatar can move or look around at the command of the remote person. Drivable telepresence robots – typically contain a display (integrated or separate phone or tablet) mounted on a roaming base. Some examples of roaming telepresence robots include Beam by Suitable Technologies, Double by Double Robotics, RP-Vita by iRobot, Anybots, Vgo, TeleMe by Mantarobot, and Romo by Romotive.[14]

More modern roaming telepresence robots may include an ability to operate autonomously. The robots can map out the space and be able to avoid obstacles while driving themselves between rooms and their docking stations.[15]

Telepresence's effectiveness varies by degree of fidelity. Research has noted that telepresence solutions differ in degree of implementation, from "immersive" through "adaptive" to "lite" solutions.[16] At the top are immersive solutions where the environments at both ends are highly controlled (and often the same) with respect to lighting, acoustics, decor and furniture, thereby giving all the participants the impression they are together at the same table in the same room, thus engendering the "immersive" label.

Adaptive telepresence solutions may use the same technology, but the environments at both ends are not highly controlled and hence often differ. Adaptive solutions differ from telepresence lite solutions not in terms of control of environments, but in terms of integration of technology. Adaptive solutions use a managed service, whereas telepresence lite solutions use components that someone must integrate.

A telepresence conference between participants in Ghana and Newark, New Jersey in 2012

A good telepresence strategy puts the human factors first, focusing on visual collaboration configurations that closely replicate the brain's innate preferences for interpersonal communications, separating from the unnatural "talking heads" experience of traditional videoconferencing. These cues include life–size participants, fluid motion, accurate flesh tones and the appearance of true eye contact.[17] This is already a well-established technology, used by many businesses today. The chief executive officer of Cisco Systems, John Chambers in June 2006 at the Networkers Conference compared telepresence to teleporting from Star Trek, and said that he saw the technology as a potential billion dollar market for Cisco.[18]

Rarely will a telepresence system provide such a transparent implementation with such comprehensive and convincing stimuli that the user perceives no differences from actual presence. But the user may set aside such differences, depending on the application.

The fairly simple telephone achieves a limited form of telepresence using just the human sensory element of hearing, in that users consider themselves to be talking to each other rather than talking to the telephone itself.

Watching television, for example, although it stimulates our primary senses of vision and hearing, rarely gives the impression that the watcher is no longer at home. However, television sometimes engages the senses sufficiently to trigger emotional responses from viewers somewhat like those experienced by people who directly witness or experience events. Televised depictions of sports events as an example can elicit strong emotions from viewers.

As the screen size increases, so does the sense of immersion, as well as the range of subjective mental experiences available to viewers. Some viewers have reported a sensation of genuine vertigo or motion sickness while watching IMAX movies of flying or outdoor sequences.

Because most currently feasible telepresence gear leaves something to be desired; the user must suspend disbelief to some degree, and choose to act in a natural way, appropriate to the remote location, perhaps using some skill to operate the equipment. In contrast, a telephone user does not see herself as "operating" the telephone but merely talking to another person with it.

An Online Video Web Conference in an office, December 2010.

Telepresence refers to a user interacting with another live, real place, and is distinct from virtual presence, where the user is given the impression of being in a simulated environment. Telepresence and virtual presence rely on similar user-interface equipment, and they share the common feature that the relevant portions of the user's experience at some point in the process will be transmitted in an abstract (usually digital) representation. The main functional difference is the entity on the other end: a real environment in the case of telepresence, vs. a computer in the case of immersive virtual reality.

Presence is very similar to distal attribution or externalization which is like projecting one's presence and mind beyond the limits of our sensory organs and perceiving the environment in such a way. A distinction is made between two separate perceptions. The first being the unmediated perceptions in which we are unable to feel anything beyond our physical surroundings. The second being the mediated presence through technology which forces us to suddenly perceive two different environments at the same time: The one immediately around us and the one projected for us through technology. Mediated experiences are not limited to virtual technology and can also be experienced with spatially distant places such as space with a telescope or a camera.[19]

Telepresence can be used to establish a sense of shared presence or shared space among geographically separated members of a group.

Robonaut 2 aboard the International Space Station

Many other applications in situations where humans are exposed to hazardous situations are readily recognised as suitable candidates for telepresence. Mining, bomb disposal, military operations, rescue of victims from fire, toxic atmospheres, deep sea exploration, or even hostage situations, are some examples. Telepresence also plays a critical role in the exploration of other worlds, such as with the Mars Exploration Rovers, which are teleoperated from Earth.

Small diameter pipes otherwise inaccessible for the examination can now be viewed using pipeline video inspection.

The possibility of being able to project the knowledge and the physical skill of a surgeon over long distances has many attractions. Thus, again there is considerable research underway in the subject. (Locally controlled robots are currently being used for joint replacement surgery as they are more precise in milling bone to receive the joints.) The armed forces have an obvious interest since the combination of telepresence, teleoperation, and telerobotics can potentially save the lives of battle casualties by allowing them prompt attention in mobile operating theatres by remote surgeons.

Recently, teleconferencing has been used in medicine (telemedicine or telematics), mainly employing audio-visual exchange, for the performance of real time remote surgical operations – as demonstrated in Regensburg, Germany in 2002.[20] In addition to audio-visual data, the transfer of haptic (tactile) information has also been demonstrated in telemedicine.[21]

A professional development expert in Denver uses telepresence to coach a teacher in Utah during the initial research of Project ThereNow.

Research has been conducted on the use of telepresence to provide professional development to teachers. Research has shown that one of the most effective forms of teacher professional development is coaching, or cognitive apprenticeship. The application of telepresence shows promise for making this approach to teacher professional development practical.[22]

The benefits of enabling schoolchildren to take an active part in exploration have also been shown by the JASON and the NASA Ames Research Center programs. The ability of a pupil, student, or researcher to explore an otherwise inaccessible location is a very attractive proposition; For example, locations where the passage of too many people is harming the immediate environment or the artifacts themselves, e.g. undersea exploration of coral reefs, ancient Egyptian tombs, and more recent works of art.

Another application is for the remote classroom which allows a professor to teach students in different campuses at the same time. An example of this application is in classrooms of the law schools of Rutgers University. Two identical rooms are located in two metropolitan areas. Each classroom is equipped with studio lighting, audio, and video conference equipment connected to a 200-inch monitor on the wall that students face to give an impression that they are all in the same classroom. This allows professors to be on either campus and facilitates the interaction among students in both campuses during the classes.[23]

True telepresence is a multidisciplinary art and science that foundationally integrates engineering, psychology, and the art of television broadcast.

In 1998, Diller and Scofidio created the "Refresh", an Internet-based art installation that juxtaposed a live web camera with recorded videos staged by professional actors. Each image was accompanied with a fictional narrative which made it difficult to distinguish which was the live web camera.

A soap opera for iMacs

In 1993, Eduardo Kac and Ed Bennett created a telepresence installation "Ornitorrinco on the Moon", for the international telecommunication arts festival "Blurred Boundaries" (Entgrenzte Grenzen II). It was coordinated by Kulturdata, in Graz, Austria, and was connected around the world.

From 1997 to the present Ghislaine Boddington of shinkansen and body>data>space has explored, in a collaboration process she has called The Weave[24] using performing arts techniques, the extended use of telepresence into festivals, arts centres and clubs and has directed numerous workshops leading to exploration of telepresence by many artists worldwide. This methodology has been used extensively to develop skills in tele-intuition for young people in preparation for the future world of work through the body>data>space / NESTA project "Robots and Avatars" an innovative project explores how young people will work and play with new representational forms of themselves and others in virtual and physical life in the next 10–15 years.

An overview of telepresence in dance and theatre through the last 20 years is given in Excited Atoms[25] a research document by Judith Staines (2009) which one can download from the On The Move website

• The Naked Sun (1957) – a novel mostly occurring on Solaria, a planet with an extremely low population where all personal contact is considered obscene, and all communication occurs through holographic telepresence.
• Neuromancer (1984) – a novel that not only uses telepresence extensively but introduces cyberspace and the Internet (aka the Matrix) as the underlying vehicle to make telepresence possible.

• Space: Above and Beyond (Episode 20, "Star Dust" - April 19, 1996)
• 24 (2001–2010)
• The Big Bang Theory (Episode 65, September 2010)
• Modern Family (Episode 144, "American Skyper" – May 2015)
• Bob's Burgers (Episode 115, "Ex Mach Tina" – January 2017)
• Community (Episode 102, "Laws of Robotics and Party Rights" – April 2015)

• Iron Man 3 (2013) – Tony Stark uses the Mark 42 suit using telepresence technology.
• Ra.One (2011)
• Darkdrive (1996)
• Avatar (2009)
• G.I. Joe: The Rise of Cobra (2009)
• Gamer (2009)
• Surrogates (2009)
• Sleep Dealer (2008)
• Avalon (2001)
• The Matrix (1999)
• The Star Wars films (1977–present) with holographic technology used for communication
• A Hologram for the King (2016) uses telepresence as a main theme

• Splinter Cell: Conviction (2010)
• Half-Life 2: Episode 2 (2007)
• Mass Effect 2 - Commander Shepard communicates with the Illusive Man using a quantum entanglement based telepresence system.
• Elite: Dangerous (2014) - players and non-player characters can utilize telepresence to perform as crew members without risk of death.
• Death Stranding (2019) - Telepresence plays a prominent and important role in communication for the denizens of the game's envisioned future

• Lamar Waldron's "M.I.C.R.A. Mind Controlled Remote Automaton" (1987), related the story of a college female paralyzed by a neck injury, who volunteered to be the remote pilot of an android body created by one of her professors.
• The Surrogates, a 2005–2006 comic book series, inspired the film Surrogates (2009) and explains about a future where humans have a widespread use of mind-controlled androids.

• Active Worlds and Second Life allow voice chat with 3D avatars