Philip Rubin

Philip Rubin received his BA in psychology and linguistics in 1971 from Brandeis University and subsequently attended the University of Connecticut where he received his PhD in experimental psychology in 1975 under the tutelage of Michael Turvey, Ignatius Mattingly, Philip Lieberman, and Alvin Liberman.

Philip Rubin's research spans a number of disciplines, combining computational, engineering, linguistic, physiological, and psychological approaches to study embodied cognition, most particularly the biological bases of speech and language. He is best known for his work on articulatory synthesis (computational modeling of the physiology and acoustics of speech production), speech perception, sinewave synthesis, signal processing, perceptual organization, and theoretical approaches and modeling of complex temporal events. At the same time, he has been involved in leadership roles related to science administration, policy, and advocacy.

Starting in the early 1970s, Rubin worked on foundational issues in speech technology. These include: participating with Rod McGuire on Haskins aspects of the ARPANET Network Voice Protocol, a predecessor of Voice over IP; collaborating with Leonard Szubowicz, Douglas Whalen, and others on digitized speech, particularly extensions of the Haskins Pulse-code modulation (PCM) implementation,[7] focusing on expanding temporal markers and event labels; and working with Patrick Nye on the Digital Pattern Playback, which was eventually replaced by Rubin's HADES system.

During his time at Haskins Laboratories, Rubin was responsible for the design of many computational models and other software systems. Most prominent are ASY,[8] the Haskins articulatory synthesis program, and SWS,[9] the Haskins sinewave synthesis program, both developed in the 1970s. [10] ASY expanded the Mermelstein vocal-tract model developed at Bell Laboratories, adding additional articulatory control, simulation of nasal sounds, sound generation, and digital sound production. Most importantly, Rubin designed and implemented an approach for describing and controlling articulatory events, now known as speech gestures. In addition to use in standard articulatory synthesis, the ASY program has been used as part of a gestural-computational model [11] that combines articulatory phonology, task dynamics,[12] and articulatory synthesis. With Louis Goldstein and Mark Tiede,[13] Rubin designed a radical revision of the articulatory synthesis model, known as CASY,[14] the configurable articulatory synthesizer. This 3-dimensional model of the vocal tract permits researchers to replicate MRI images of actual speakers and has been used to study the relation between speech production and perception.

The sinewave synthesis system designed by Rubin, known as SWS, is based on a technique for synthesizing speech by replacing the formants (main bands of energy) with pure tone whistles, and was designed to explore the spatiotemporal aspects of speech signals. It was the first sinewave synthesis system developed for the automatic, large-scale creation of stimuli for perceptual experiments, and has been used by Robert Remez, Rubin, David B. Pisoni,[15] and other colleagues and researchers to study the time-varying characteristics of the speech signal. Rubin is also the designer of the HADES signal processing system and the SPIEL programming language, a predecessor of MATLAB.

From 1992 through 2012, Rubin was the core and administrative leader of Haskins Laboratories' main research activity, the National Institutes of Health/NICHD funded P-01 program project, “The Nature and Acquisition of the Speech Code and Reading.” [16]

In 1998, he was the co-founder and first President of AVISA, the Auditory-Visual Speech Association, now part of the International Speech Communication Association (ISCA).

He is co-creator, with Eric Vatikiotis-Bateson,[17] of the Talking Heads website.[18]

With Carol Fowler, Robert Remez, and Michael Turvey, Rubin introduced the consideration of speech in terms of a dynamical systems / action theory perspective.[19] Rubin's theoretical approach to perception and production, particularly in the case of speech, eschews attention to the momentary and punctate aspects of the signal, focusing not on traditional features and cues, but on spatiotemporal coordination of global aspects of the system, such as spectral coherence over long stretches of time (an approach related to current speech understanding systems, like Siri or Amazon Alexa).

With Robert Remez and various other colleagues, he has used the technique of sinewave synthesis to explore perceptual organization. They have noted that "the criteria for the perceptual organization of speech - visible, audible, and even palpable - are actually specified in a general form, removed from any particular sensory modality ..."[20], but, to Rubin, related to the underlying spectral coherence of signals created by biological actvity.

Rubin's approach stresses the constraints and structure stemming from the realities of embodied systems, again across both time and physical space. He expanded the modeling of speech production to incorporate an event based approach to control movement over time and articulatory space of the vocal tract, by building on the conceptual approach developed by Paul Mermelstein and colleagues at Bell Laboratories.[21] This work was influenced, in part, by the event-based focus of James J. Jenkins. Rubin's articulatory synthesis model, ASY, illustrates how simple physical changes, such as velar opening, directly account for degrees of nasality, avoiding the complexity of attempting to reconcile numerous spectral cues.[22] This event orientation evolved into a gestural computational system[23] developed at Haskins Laboratories that combined ASY with the articulatory phonology of Catherine Browman and Louis Goldstein, and the task dynamic model of Elliot Saltzman. In this system utterances are organized ensembles (or constellations) of units of articulatory action called gestures. Each gesture is modeled as a dynamical system that characterizes the formation (and release) of a local constriction within the vocal tract (the gesture’s functional goal or `task’). Goldstein and Rubin have described the "dances of the vocal tract" that underlie the production of continuous speech.[24] Constraints stemming from such embodiment also can be exploited in the recovery of vocal tract shapes from the acoustic signal as seen in the continuity mapping approach of John Hogden, used by Hogden, Rubin, and colleagues to re-conceptualize how realistic physical constraints affect pattern recognition.[25]

Rubin expanded his approach to understanding the importance of spatiotemporal coordination in his collaborations on audiovisual speech with Eric Vatikiotis-Bateson, Hani Yehia, and other colleagues, focusing on multimodality by exploring the simultaneous combination of speech, facial information, and gesture,[26] leading to innovations in analysis, synthesis, and simulation.

From 2000-2003 Rubin was the Director of the Division of Behavioral and Cognitive Sciences (BCS)[27] at the National Science Foundation (NSF) in Arlington, Virginia, where he helped launch the Cognitive Neuroscience,[28] Human Origins (HOMINID),[29] Documenting Endangered Languages,[30] and other programs, was part of the NBIC convergence (Nanotechnology, Biotechnology, Information technology and Cognitive science) activities, and was the first chair of the Human and Social Dynamics priority area.[31]

Rubin returned to the NSF during the second term of the Obama Administration to serve as a Senior Advisor in the Directorate for Social, Behavioral and Economic Sciences (SBE).

Rubin has been in several leadership roles related to science policy and advocacy. From 2006-2011 he was the Chair of the National Academies Board on Behavioral, Cognitive, and Sensory Sciences;[32] a member-at-large of the Board of the Federation of Associations in Behavioral and Brain Sciences (FABBS);[33] and the co-leader of the Yale-Haskins Teagle Foundation Collegium on Student Learning.[34] He is also the former Chairman of the Board of the Discovery Museum and Planetarium in Bridgeport, Connecticut, where he also worked with museum staff, trustees, city and state representatives, and others to establish the Discovery Magnet School on museum grounds, the first pre-K to 8 interdistrict public magnet school in the region with a science theme.[35] He is currently the President-Elect and a member of the Board of Directors of FABBS.

While at the NSF during the Clinton and Bush administrations, Rubin was the NSF ex officio representative to the National Human Research Protection Advisory Committee (NHRPAC)[36] and the Secretary's Advisory Committee on Human Research Protections (SACHRP),[37] established to provide advice to the Secretary of Health and Human Services on issues related to the protection of human research subjects. He was also the co-chair of the inter-agency National Science and Technology Council (NSTC) Committee on Science (COS) Human Subjects Research Subcommittee (HSRS) under the auspices of the President's Office of Science and Technology Policy (OSTP) and was also formerly the co-chair of the HSRS Behavioral Research Working Group. After leaving the NSF in 2003, he continued to be active on human subjects issues as they relate to public policy, including lecturing, writing, co-authoring an AAUP report,[38] participating in activities of the Yale Bioethics Center,[39] and serving on the advisory board of the Journal of Empirical Research on Human Research Ethics.[40]

Rubin has been a long-time member of the Yale University Technology and Ethics working group [41] and Yale's Science, Technology, and Utopian Visions working group, hosted by the Whitney Humanities Center.

He has also expressed concerns about the ethical and scientific oversight of the use of certain tools and techniques by the intelligence, law enforcement, military, and national security communities, considering some of them to be boondoggles. An example includes his serving as the Chair of the National Research Council (NRC) Committee on Field Evaluation of Behavioral and Cognitive Sciences-Based Methods and Tools for Intelligence and Counter-Intelligence.[42] In a workshop report from that committee he provided an analysis of the use of voice stress technologies in the detection of deception and said "not only is there no evidence that voice stress technologies are effective in detecting stress, but also the hypothesis underlying their use has been shown to be false."[43] He was also a member of the NRC Committee on Developing Metrics for Department of Homeland Security Science and Technology Research;.[44] On April 6, 2011, he provided testimony at a hearing of the House Committee on Investigations and Oversight - Behavioral Science and Security: Evaluating TSA's SPOT Program.[45] In his written and oral testimony, he criticized the TSA's SPOT passenger screening program, including raising concerns about the limitations that the Department of Homeland Security imposed on an outside review and oversight committee for the SPOT program, known as the Technical Advisory Committee (TAC), of which he was a member, saying "TAC has not been asked to evaluate the overall SPOT program, the validity of indicators used in the program, consistency across measurement, field conditions, training issues, scientific foundations of the program and/or behavioral detection methodologies, etc. In order to appropriately scientifically evaluate a program like SPOT, all of these and more would be needed." He went on to say, "Shining a light on the process by making information on methodologies and results as open as possible (such as with devices like the polygraph, ..., voice-stress analysis, and neuroimaging) is necessary for determining if these technologies and devices are performing in a known and reliable manner. Clearly establishing the scientific validity of underlying premises, foundations, primitives, is essential. The larger the base of comparable scientific studies, the easier it is to establish the validity of techniques and approaches. ... In our desire to protect our citizens from those who intend to harm us, we must make sure that our own behavior is not unnecessarily shaped by things like fear, urgency, institutional incentives or pressures, financial considerations, career and personal goals, the selling of snake oil, etc., that lead to the adoption of approaches that have not been sufficiently and appropriately scientifically vetted."

In May 2016 Rubin was a signatory to an Open Letter[46] to the World Health Organization (WHO) calling on them to move or postpone the 2016 Summer Olympics in Rio de Janeiro over the 2016 epidemic of Zika fever.[47] He also appeared on air on ESPN with Hannah Storm to discuss risk assessment and Zika.[48]

In February 2012 Philip Rubin took a position as the Assistant Director for Social, Behavioral, and Economic Sciences at the Office of Science and Technology Policy (OSTP) in the Executive Office of the President of the United States.[49] He also served as a Senior Advisor at the National Science Foundation in the Social, Behavioral and Economic Sciences (SBE) Directorate.[50] At OSTP he led the White House's neuroscience initiative.[51] On April 16, 2012, Congressman Chakah Fattah (D-PA) introduced House Resolution 613, supporting the OSTP interagency working group on neuroscience that Rubin organized. The resolution also "... commends President Barack Obama for the expeditious appointment of Dr. Philip Rubin to lead the working group's efforts."[52] In June 2012 was named by John Holdren, Assistant to the President for Science and Technology and Director of OSTP, to be OSTP's Principal Assistant Director for Science, taking over the duties of Nobel laureate Carl Wieman, who resigned as Associate Director for Science on June 2.[53] In this new role Rubin also became Co-Chair of the National Science and Technology Council's Committee on Science, serving with other Co-Chairs, Francis Collins and Subra Suresh, Directors of the NIH and NSF, respectively.[54] He also co-chaired the interagency Common Rule Modernization Working Group.

During his tenure, key priorities for the OSTP Science Division included:

• Support for fundamental and translational research at NIH, NSF, NIST, and NASA.
• Public Access to research results, Open Data, and Open Science (led by Michael Stebbins).
• Large-scale, scientific infrastructure (led by Gerald Blazey, Altof Carim, and Tamara Dickinson), including such projects as ITER and the James Webb Space Telescope.
• Biomedical innovation (led by Michael Stebbins and Col. Geoffrey Ling).
• Neuroscience (led by Rubin and Carlos Peña, now at the FDA, with contributions by Danielle Carnival, Meredith Drossback, Michael Stebbins, Carl Wieman, and others), including areas such as traumatic brain injury, neurodegenerative disease, the BRAIN Initiative, cognitive science, development, emotion, and learning, and the chartering of the NSTC Interagency Working Group on Neuroscience (IWGN). Rubin represented the White House at the G8 Dementia Summit on December 11, 2013 in London, held to shape an effective international response to dementia.
• Mental health, including areas such as PTSD, stigma, and suicide (Rubin and Stebbins).
• Forensic science (led by Tania Simoncelli, now Director of Science Policy at the Chan Zuckerberg Initiative).
• Behavioral science and social science (including planning meetings on behavioral economics, encouraged by Richard Suzman (then at NIH/NIA)), and supported by Philip Rubin, Alan Krueger (then at the Council of Economic Advisers), and Alan Kraut (then at the Association for Psychological Science), leading to the Behavioral Impacts program (led by Tom Kalil and Maya Shankar).
• Broadening participation in science (led by Joan Frye, now retired, Sean Jones, now at the NSF, and Danielle Carnival, who then went on to become Vice President of the Biden Cancer Initiative).
• Human subjects research issues (led by Rubin and Tania Simoncelli), including modernization of the Common Rule.
• Uniform Guidance for research and related regulations (led by Joan Frye, and Kelsey Cook and Sean Jones, both now at the NSF).
• Language and communication, (led by Philip Rubin), including the chartering of an interagency NSTC working group.
• Aging, including interfacing with the White House Conference on Aging and with PCAST activities on technology and aging[55] (Rubin).
• Space science (led by Tamara Dickinson, then principal assistant director for Energy and Environment at OSTP).

In February 2015, Rubin retired from OSTP and the NSF.[56]

Rubin was the founder, in 1984, and first president of YMUG (now known as YaleMUG, the Yale Macintosh Users Group)[57] and the publisher of The Desktop Journal. Other co-founders and early members include Tony Cecala,[58] Eric Celeste,[59] Richard Crane,[60] Ward J. McFarland, Jr.,[61] David Pogue, Michael D. Rabin,[62] Tom Rielly,[63] Elliot Schlessel, Ed Seidel, and Sharon Steuer.[64]

Rubin is the co-founder, with Elliot Saltzman, of the IS Group, an informal, collaborative group founded in the 1980s of scientists and technologists from around the country exploring cutting-edge issues such as dynamical systems, evolution, artificial intelligence, linguistics, robotics, network science, neuroscience, and other topics.[65]

Between 1999 and some time in the early 2000s, Rubin was technical advisor at ZeniMax Media Parent company of video game publisher Bethesda Softworks.[66]

Philip Rubin is a member of the American Academy of Arts and Sciences Challenges for International Scientific Partnerships (CISP) [67] Large-Scale Science Working Group. He is also a member of the American Academy's Commission on Language Learning, created to examine the current state of language education in response to a bipartisan request from members of the United States Senate and House of Representatives.[68]

Rubin is a member of the Beyond Conflict Neuroscience and Social Conflict Initiative steering committee.[69] This organization assists leaders in divided societies who are struggling with conflict, reconciliation, and societal change. The initiative explores how insights from cognitive science and neuroscience can inform the practice of conflict resolution and diplomacy.

In May 2015 Rubin served as a judge for the DARPA Robotics Challenge (DRC) Robots4Us[70] Student Video Contest and was an invited participant in the activities at the DRC Finals in June 2015.[71]

On May 26–27, 2016 Rubin was a participant in the first ever White House Foster Care and Technology Hackathon.[72]

From 2016 through 2019, Rubin served as Director and Treasurer of iGIANT (impact of Gender/Sex on Innovation and Novel Technologies), founded and led by Dr. Saralyn Mark. [73] In Jan. 2020 Rubin was named as a Board Member Emeritus of iGIANT.

In 2017 and 2018, Rubin served as a member of the National Academy of Public Administration's NASA Advisory Council: Organizational Assessment panel. The NASA Authorization Act of 2017 directed the Academy to conduct a review "to assess the effectiveness of the NASA Advisory Council and to make recommendations to Congress." [74]

In December, 2017, Governor of Connecticut, Dannel P. Malloy, appointed Philip Rubin to serve as a member of the UConn Board of Trustees, the governing body for the University of Connecticut. [75]

In October, 2018, Rubin was elected as a member of the Board of Directors of Haskins Laboratories. [76]

In December, 2019, he was appointed by the UConn Board as Vice-Chair, leading their new, legislatively encouraged standing committee on Research, Entrepreneurship and Innovation.[77]

• 1999 – Elected to Fellow of the Acoustical Society of America
• 2002 – Elected to Fellow of the American Association for the Advancement of Science, for “ … major contributions to the understanding of human speech processing and the technology of speech analysis.”
• 2003 – Elected to Fellow of the Association for Psychological Science, for “… sustained outstanding achievements in psychological science.”
• 2003 – Award for “Commendable Performance ... for his superior leadership of all federal government departments and agencies involved in the protection of human subjects,” from the Human Subjects Research Subcommittee, Committee on Science, National Science and Technology Council, Office of Science and Technology Policy, Executive Office of the President of the United States
• 2006 – Elected to Fellow of the American Psychological Association, “… in recognition of outstanding and unusual contributions to the science and profession of psychology.”
• 2007 – Elected to member of the Philosophical Society of Washington
• 2007 – Elected to Fellow of Trumbull College at Yale University
• 2008 – Elected to member of Sigma Xi
• 2010 – American Psychological Association's Meritorious Research Service Commendation, “In recognition of your outstanding contributions to psychological science through your service as a leader in research management and policy development at the national level.”[78]
• 2012 – Elected to Senior Member of the IEEE
• 2013 – Elected to Fellow of the National Academy of Public Administration
• 2014 – OSTP Award for Excellence, White House Office of Science and Technology Policy
• 2015 – Distinguished Service Award, COSSA (Consortium of Social Science Associations), "in recognition of his career of service to the social and behavioral science community."[79]
• 2015 – Added to the FABBS (Federation of Associations in Behavioral and Brain Sciences) "In Honor Of ..." gallery of scientists, "recognizing eminent, senior scientists who have made important and lasting contributions to the sciences of mind, brain, and behavior."[80]
• 2015 – Elected to Fellow of the Psychonomic Society
• 2016 – Granted lifetime membership in Nu Rho Psi, The National Honor Society in Neuroscience, "In recognition of outstanding achievement in the areas of neuroscience scholarship and research …".[81]
• 2017 – Elected to member of the Connecticut Academy of Science and Engineering (CASE).[82]
• 2020 – iGIANT Pioneer Award 2020, for "contributions to accelerating the translation of research into gender/sex-specific design elements."[83]
• 2020 – Elected as the President-Elect of the Federation of Associations in Behavioral and Brain Sciences (FABBS)

Philip Rubin was born on May 22, 1949, in Newark, New Jersey. His maternal grandmother was Fannie Auerbach (née Rothschild) (1891-1976).[84] He spent most of his childhood in Newark and graduated in 1967 from Union High School in Union, New Jersey,[85] where his interest in science, along with classmates such as Marty Kaplan, was nurtured by the inspirational advanced biology teacher Irwin Jaeger.[86] In the 1960s he was a co-founder and guitar player in the seminal New Jersey garage band, "The Institution."[87] Rubin is a photographer who, since the 1970s, has concentrated on pictures of wall art, including murals, graffiti, and painted buildings, in the urban centers of the cities that he has visited. Speaking of the transient nature of wall art, he has said, "The artist is often unknown; the passing of time and the public venues invite unanticipated collaboration."[88] His work has been exhibited and sold at numerous venues.[89][90][91] He is married to Joette Katz, retired Associate Justice of the Connecticut Supreme Court, and currently a partner at the law firm, Shipman & Goodwin LLP. They have two children, Dr. Jason Wilder Katz Rubin, a pediatrician at Seattle Children's Hospital and Assistant Professor at the University of Washington School of Medicine in Seattle,[92][93] and Samantha Katz,[94] a creative director and curator who is the founder of Created Here.[95]

In March 2010, Audiobulb Records released a CD by artist Autistici, titled Detached Metal Voice - Early Works (Vol. I). This album has been described as a collection of tracks that "explores the raw extrusion of the human condition."[96] There is an homage to voice synthesis that includes excerpts from many of the early laboratory attempts to produce the human voice via articulatory synthesis, including work pioneered by Philip Rubin and colleagues at Haskins Laboratories, based on earlier work at Bell Laboratories.[97]

Rorschach Audio - Art and Illusion for Sound discusses "Sine-Wave Speech, The Clangers" and other topics, influenced by the sinewave synthesis work of Rubin and colleagues.[98]

The sinewave speech projection "Poulomi's Ode to Young-Hae Chang Heavy Industries" was presented at Rich Mix London on Dec. 3, 2010.[99]

• Articulatory Synthesis
• SineWave Synthesis
• Discovery Museum & Planetarium
• Federation of Associations in Behavioral & Brain Sciences (FABBS)
• Haskins Laboratories
• Haskins profile
• National Academies Board on Behavioral, Cognitive, and Sensory Sciences
• National Science Foundation
• NSF Reflections. APS Observer, V. 17, #4
• University of Connecticut
• White House Office of Science and Technology Policy
• Yale School of Medicine
• Yale University