Chris J. Harris

The early part of Harris' academic career followed the seminal work of the then international research leaders in control theory such as Professors R Bellman, R W Brockett, Sir Alistair MacFarlane, C A Desoer,and V M Popov. Harris's research was devoted to initially to the stability linear time varying systems and then to multi-variable nonlinear systems using only input/output data. These two areas of study culminated in 2 monographs in the famous R Bellman `Mathematics in Science & Engineering` series, volumes 153 & 168 respectively ( publications 6 & 7 below). This early research raised fundamental questions on the computationally efficient identification of nonlinear dynamical process from observations. An accurate and sparse model leads potentially to greater physical insight and consequent simpler state estimators, better probability classifiers for fault diagnostics and more accurate controllers. Consequent research over a research career of Harris over 50 years have led to a series of algorithms that provide the parsimonious adaptive identification, data fusion, classification and control of a priori unknown nonlinear dynamical processes using only observed data, this work has independently had significant take up with a wide variety of industries with well over 200 applications reported in the literature.

1. Adaptive/Intelligent Data Modelling. Harris with Dr M Brown pioneered the theory of Adaptive or Intelligent Neurofuzzy Modelling and Control, summarised in the highly cited research monograph (pubs 5) whereby the inherent transparency of fuzzy logic is integrated with the analytical self learning capability of neural networks. This monograph collects together a series of his papers on linear in the parameters associative memory neural networks (AMN), together with the general theory of instantaneous learning laws - an extension of the generalised least squares estimation of DW Clarke. Paper(ref 3) starts with the Cerebellum Model Articulation Controller (CMAC) algorithm (ref 4) which is generalised and extended to metric basis functions, greatly increasing its utility to continuous processes making it ideal for real time robotic applications; CMAC has been developed by Harris for many real time control problems such as intelligent car driving in lanes (ref 5), in the pan-European Prometheus research programme, which when integrated with the vision based tracking system (4) produced the world's first road worthy driverless car demonstrator. Paper (ref 3) then develops AMN's via adaptive learning schemas that solve the inherent problem of the curse of dimensionality of AMN's by exploiting the inherent transparency and parsimony of local basis functions providing transparency (ie linguistic interpretation), good generalisation, and rejection of noisy data via parametric regularisation. Here model parsimony (or minimal model structure) is achieved by expressing the network as a sum of univariate and multvariate sub-models to form new tensor product multivariate sub-models (the so-called analysis of variance - ANOVA representation), which are iteratively pruned to optimise a composite performance index based upon model size, mean squared error and input data size - generating the renown and highly used adaptive spline modelling algorithm (ASMOD).These Neurofuzzy modelling algorithms have been commercially implemented in software by NeuFrame & Matlab (ref 17) and applied by Harris to a variety of real demonstrators including underwater vehicles with FAU (USA), car driver support with Jaguar (with Sir Mike J Brady at Oxford), robotic grippers, structural properties of materials with Sir Peter E Gregson (CIT), gas turbines with Rolls Royce and cancer diagnosis as well over a 100 applications by others in medicine, robotics, defence & space, materials processing, manufacturing, engine & power control, transportation, commerce and macroeconomics.

For this work Harris was elected Fellow of the Royal Academy of Engineering in 1996. Fundamental neurofuzzy modelling research continued via a new robust extended Gram-Schmidt orthogonal decomposition and regularisation algorithm (ref 7), coupled with local orthogonal least squares (LOLS) and D-optimal experimental design to automatically determine model internal structure and associated optimal parameter estimates; this new approach to adaptive neurofuzzy models generates significant model transparency with each derived rule having a belief measure. This computationally simple approach to automatic sparse rule construction is the most efficient and accurate neurofuzzy network available and has attracted considerable attention, particularly in life sciences.

The grand challenge in nonlinear modelling is “white box” modelling ie, from data alone can we discover the underlying causal phenomenology? This is highly critical in understanding complex phenomena such as gene evolution, as well as in generating optimal controllers, fault detectors/classifiers and knowledge integration/fusion for a priori unknown processes. To this end Harris and his colleagues has generated a pioneering series of increasingly efficient and parsimonious nonlinear modelling algorithms to overcome the curse of dimensionality. All AMN's (e.g. RBF's, Gaussian B-splines, neurofuzzy) regression models can be made highly parsimonious by using a composite D-optimal design of experiments criteria, further by incorporating our famous local Orthogonal Least Squares (LOLS) algorithm (ref 8), the resultant model construction process (ref 9) is fully automated and has proven very popular in applications ranging from mobile communications, speech and text processing to thin film deposition modelling. The recent paper (ref 8) provides a unified theory of parsimonious orthogonal least squares modelling for regression, classification and probability density estimation with illustrative bench mark examples that illustrate the approach effacy over `best in class` algorithms. Recently, by using model generalization itself as the model selector via cross validation and the information theoretic metric of predicted radical sum of squares (PRESS) statistics; which combined with their (ref 9) OLS produces the ‘state of the art’ automatic sparse modelling algorithm (ref 10) with diverse applications by others, including plant remote sensing, CO2 spectral monitoring, speech ultra sound, antenna design and tool machining.

In 2001, for the above research, Harris was awarded the 79th IEE International Faraday medal for “International acclaim in Intelligent Control and Neurofuzzy Modelling”. Repeated weighted boosting search (RWBS) optimization is a new guided global stochastic optimization algorithm (ref 11) that can handle non-smooth and/or multi-mode cost functions. Compared with other global optimizers such as the popular genetic algorithm (GA) and adaptive simulated annealing (ASA) algorithm, RWBS is considerably easier to implement, has significantly fewer parameters to tune or pre-select by the user, and performs at least as well and usually considerably better than GA and ASA algorithms. This RWBS algorithm can also be utilized for RBF networks, generalized kernel models, density estimators as well as kernel classifier design, by optimizing the kernels (see ref 12) one by one in an orthogonal forward selection (OFS) procedure through maximization the Fisher ratio in the RWBS. For bench mark examples the OFS-RWBS classifier (ref 11) is up to 10 times smaller than state of the art methods such as support vector machines (SVM). Rather than tune kernels or radial basis network centre vectors/covariance matrices via RWBP, Harris has recently (ref 13) generated a novel Particle Swarm optimizer (PSO) which coupled with minimizing the modelling leave one out (LOO) mean square error produces a population based stochastic global optimizer. The PSO algorithm is inspired by biological behavior, is simple, converges quickly and is insensitive to local minima. The PSO aided orthogonal forward regression algorithm (ref 13) for individual tunable radial basis function models, offers even better generalization performance, parsimony and computational advantages over the ‘state of the art’ fixed mode RBF identification algorithm of (ref 10), with even more significant advantages over SVMs. Harris's current work is on data based modelling for nonlinear and non-stationary dynamical systems (ref 23) with his long-term collaborator Prof Sheng Chen.

2. Data Fusion, Diagnostics and Control for nonlinear Autonomous Systems using data alone. As the Lucas Professor (1987-1997) Harris had a strong research interest in autonomous guided vehicles and derived a series of intelligent data driven methodologies for auto-car driving, including fuzzy logic based self parking (ref 3), auto-motorway driving and collision avoidance via vehicle detection. Whilst radar and acoustics provided useful range/range rate information, vision based vehicle detection (ref 6) is far more robust in avoiding false alarms as well as in auto tracking since it can be coupled with a vision based lane marking detector to place potential obstacles relative to its own vehicle. In paper (ref 6) Prof Harris et al. developed a highly successful real time road vehicle detection and recognition system that utilises principal component analysis (PCA) to compress the inherent high vision generated data to determine prime ‘target’ features or dominant eigenvectors (now the norm in subsequent vehicle detection & tracking applications), then a AMN to enable multiple potential vehicles in a single image to be classified . Later an obstacle tracker was derived by temporal integration, which allowed autonomous motorway driving to be achieved for the first time on a Jaguar test bed in 1999. The use of PCA or eigen-space transformation (EST) in vision has been further extended by Profs M N Nixon (Southampton Uni) & Harris as a new bio-metric (ref 14) for the automatic recognition of human gait for security systems, that segments body parts, via a statistical method that combines EST (to optimally reduce the dimension of the input space spatial templates) and then a canonical space transformation for feature extraction of spatial templates to recognize/classify individual people. Target tracking is greatly improved by using Multi-sensory Data Fusion (MSDF). MSDF aim is to produce a probabilistic model of an entity from a set of independent data/knowledge sources, with reduced uncertainty, for application in the areas of process understanding, classification, tracking and guidance and control. Kalman filtering is amongst the most popular of recursive algorithms in MSDF (see eg.Prof H Durrant-Whyte (Sydney Uni) seminal work) as it gives an optimal linear, unbiased, minimum variance estimates of a system state from observed data. In the highly cited and used paper (ref 15), two basic methods of measurement fusion for Kalman-Filter based MSDF are developed, the first integrates sensor measurement information by augmenting the observation vector, the second by replacing the observation vectors by a single vector of individual observation weighted by their inverse covariance matrices. Harris has shown that these two methods are both optimal and informationally identical under certain conditions. Measurement fusion methods (ref 15) are extended in (ref 16) to state vector fusion (SVF) which utilise state estimate covariance matrices instead of measurement noise covariance. SVF methods are very robust, highly flexible and work well in decentralized MSDF architectures and have found significant application. Harris has developed a variety of SVF algorithms, including track-to-track SVF in which the overall estimate is fedback to the final predictor and a SVF track fusion model with fused prediction SVF, in which all individually predicted fused estimates are fedback (ref 16). Not surprisingly this approach is superior to track-to-track SVF for dissimilar sensors and operates well in high noise environments. For many practical processes, the underlying systems are unknown or highly nonlinear; a powerful approach (ref 17) to this problem (akin to the extended Kalman filter(EKF) but without its stability issues) is feedback linearization via neurofuzzy networks with an analysis of variance (ANOVA) model decomposition( see paper1). Feedback linearization forces the underlying nonlinear process to behave linearly by a state coordinate transformation and a feedback control law, the resultant model together with linear observations is directly applicable in MSDF applications via a modified Kalman filter approach (ref 17) :in practise this data driven approach outperforms the EKF even with perfect process knowledge ! These MSDF algorithms have been used with great success by others in adaptive robotic manipulators, robotic vehicles, medicine, secure communications, motor control and diagnostics, metrology, navigation, and air-ground–sea target tracking.

Following the fatal RAF Chinook crash in 1994, Harris was funded by Westlands and MOD to produce an all weather/flight conditions Helicopter MSDF neurofuzzy guidance system, the result was successfully flight trialed; the resulting publication (ref 18) won the 1997 Royal Aeronautical Society Simms paper award (followed again in 1998). Following this success Racal Marine funded Harris to generate a marine collision avoidance system (MANTIS) also based on a neurofuzzy guidance and control system, that was able to deal with up to six imminent ship collisions in confined waters, the individual ship line of sight guidance system paper (ref 19) was awarded the 1999 Institute of Mechanical Engineers Donald Groen prize.

Fundamental to fault diagnostics, classification evaluation and multisensory data fusion is the construction of the probability density function (pdf) of the underlying process from data samples. The conventional approach is to use the non-parametric Parzen Window (PW) estimate, whilst it is simple and accurate it scales directly with sample size, and thence inappropriate for data rich sources such as sensors. By extending previous work on sparse modelling (refs 7&10) Harris derived a highly efficient very sparse density estimator (ref 20), which reposes pdf estimation as a regression problem, then uses orthogonal forward regression to automatically produce sparse pdf estimates by incrementally minimizing a cost criterion based upon leave one out test coupled (for minimum generalization error) with local regularization to find the lease squares solution for the pdf parameters. The resultant algorithm is highly efficient (typically 50 times better than PW estimators), simple to implement requiring no user parameters. It has been applied by others to prostate cancer detection and to automatic speech recognition.. There are many signal processing techniques (Kernel based algorithms, Support vector machines, Relevance vector machines and OFR) for classifier construction, most fail when the data is imbalanced since the inherent least squares estimators treat all data equally, producing unfavourable predictors for the minority class – which may be safety critical (e.g. a metal fatigue or cancer detection). Instead in paper (ref 20), Harris produced an imbalanced data classifier that is sensitive to data importance by utilizing a new forward regularized orthogonal weighted least squares algorithm whose parametric selection is sensitive to class labels. Model selection that optimizes model generalisation capability for imbalanced data sets, is via a maximal leave one out area under the curve criterion akin to that used in communications. The resultant algorithm is the very ‘best in class’ for prediction error (including the best support vector machine classifier - SUPANOVA (2000) – see (ref 3)), also highly computationally efficient and very robust to data noise. Current applications include prostate cancer diagnostics via multi-sensory data fusion of PSA and PCA3 (gene) data, via individual pdf estimation (ref 20)

Data based Intelligent Control (i) Fuzzy control. Fuzzy logic has been developed extensively for a wide range of domestic products since Prof Lofti Zadeh's seminal work in 1965, for fixed or static fuzzy rule basis. As with all expert systems the fundamental weakness of static fuzzy logic is knowledge elicitation about the underlying process. Following a series of papers in 1993 (ref 21), Harris produced the first form of self organising fuzzy controller (SOFLIC) to automatically produce a complete set of fuzzy rules from observations alone. Two forms of SOFLIC were derived; the direct adaptive fuzzy controller which manipulates controller parameters without recourse to plant identification/modelling and the more applicable indirect SOFLIC (cf. model reference) which is characterized by an iterative online identification algorithm to formulate an intermediate rule base plant model, which is then utilized to formulate the controller. This separation of modelling and control enables the performance specification to be changed by the designer. Both approaches of SOFLIC can control processes with little apriori knowledge, have fast adaptation to plant variations, can deal with non-stationary non-linearities and have good noise and disturbance rejection. As with all rule based algorithms the controller complexity grows exponentially with input space dimension. The SOFLIC is now readily extended to high dimensional problems by utilizing a sparse modelling neurofuzzy algorithm such as (ref 7). The indirect SOFLIC has been demonstrated very effectively (ref 21) against all the 1990 IFAC world bench mark problems of ship heading, car tracking and auto car control and guidance.

(ii) Learning or Adaptive Robotic control. Modern robotic multi manipulators are highly nonlinear coupled multi-axis dynamic systems operating under varying size and geometry mass loads. High speed trajectory tracking control is fundamental, requiring adaptive or neural network based system identifiers (models) and controllers. In the pioneering and subject leading research monograph (ref 22), general Lagranian equations of motion are formulated as nonlinear functions of measurable/observable vector position, velocity and acceleration. Using the latter two measurements, process models are identified (ref 22) in both Cartesian and task spaces by linear in the parameters network (see RBF's in (ref 3)) Fixed structure three term controllers, but with learning RBF parameters, (akin to sliding mode control - via Lyapunov's Stability theory to ensure asymptotic stability) are derived and evaluated on real world demonstrators – a line of sight stabilized platform and a free bar linkage system – to illustrate controller stability, robustness and adaptivity and insensitivity to parametric uncertainty, faults and noise. Alternatively utilizing neural network based feedback linearization modelling (ref 17) Harris et al. show (ref 22) that conventional model reference controller design is directly applicable to rigid body and flexible body robots as well as to multiple interacting robotic manipulators. Whilst being highly cited paper (ref 22) has been extensively utilized by others in fabrication and process manufacturing, wheeled robots, auto swing cranes, artificial fingers/hands, hybrid joint and cooperating robots.

For Harris’ research in intelligent control he was awarded the 1998 IEE Senior Achievement medal for “outstanding contributions to Electrical Engineering”.