Central to any discussion on parameter adaptation is the notion of regressor excitation, among which the most important notion is persistent excitation (PE). There is a general consensus that the PE condition is too strong a requirement to ensure parameter convergence and robustness of adaptive laws. While one may forego parameter convergence but still achieve regulation, robustness is paramount to the applicability of adaptive schemes in real world settings. 

The problem of robust adaptation has remained a longstanding open problem in adaptive control. The classical literature focused on providing robustness without the PE condition using modifications of adaptive control laws. These modifications either trade off regulation for robustness or require a priori known bounds on the unknown parameters. The modern line of work seeks to weaken the PE condition. Inspired by the Use it or Lose it Principle from systems neuroscience, we propose the $\mu$-modification, based on the observation that robustness is only needed along the subspace that lacks persistent excitation. 

In this talk, we first present the main ideas of the $\mu$-modification. Then we give an overview of persistently exciting subspaces and introduce subspace estimation techniques. Simulations will show the efficacy of our new methods.

Poster title

With the recent advancements in robotics, the demand for adopting robotic solutions in many aspects
of our lives has experienced an exponential rise. Nowadays, such systems must be capable of reliably
performing delicate tasks in unknown and hostile environments without human intervention for a long
period of time. This renders robotic systems susceptible to experiencing performance degradation and
unexpected damage that cannot be immediately compensated for. The complete rigid dynamics of legged
robots is characterized by highly nonlinear multibody models that may be exposed to singularities.
Existing multi-legged robot models typically use simplifying assumptions due to considering ideal
robot movements, with limitations on capturing off-minimal motion scenarios. As an example, in a
mechanically damaged robot, with a nonfunctional leg, the simplifying assumption such as the exclusive
2D motion of the system is violated. In such a scenario, a whole-body and singularity-free model of
the robot is necessary to conduct damage identification and offer mitigation strategies.
In this talk, I will show how to employ the Boltzmann-Hamel equations and the Screw theory to
develop solutions to singularity-free whole-body modeling that include coupling between legs/main
body dynamics and extend the theory to multi-branch systems. I will present a brief reformulation of
the Boltzmann�Hamel equations in terms of quasi-velocity coordinates to describe the dynamics of a
legged robotic system. These velocity coordinates are not the time derivative of position coordinates,
but form a local diffeomorphism between a neighborhood of 0 _ R^k and an open neighborhood at
every global coordinate. The method�s strength is apparent when the local parameters are set to zero at
a global coordinate, after calculating the derivatives in the Lagrangian equations of motion that results
in a parameterization-independent final equation. Finally, I will present the explicit inertia and Coriolis
matrices for a general legged-robot needed for the implementation, together with several relations that
can be used to speed up the algorithms. The derived equations are critical in the simulation and control
multi-legged robots. The closed-form formulation makes the proposed approach well-suited for analyzing
damaged systems and designing model-based controllers. The framework is validated with a Simscape
model of a six-legged robot and is then used to simulate the unstable motion of a mechanically damaged
robot. In this simulation I have a damaged 6-legged robot with a nonfunctional support leg. This
damage scenario inevitably led to a large limping that needed a global, whole-body dynamical model
for effective capture of the motion. Simulation results demonstrate the effectiveness of the proposed
method in modeling off-nominal situations. In our future work, I intend to (i) extend our proposed
model to include multi-degree-of-freedom joints, (ii) consider more realistic contact models, and (iii)
improve robot control to mitigate damage.

Singularity-Free Whole-Body Dynamical Equations of Legged Robots for Damage Identification

With the recent advancements in robotics, the demand for adopting robotic solutions in many aspects of our lives has experienced an exponential rise. Nowadays, such systems must be capable of reliably performing delicate tasks in unknown and hostile environments without human intervention for a long period of time. This renders robotic systems susceptible to experiencing performance degradation and unexpected damage that cannot be immediately compensated for. The complete rigid dynamics of legged robots is characterized by highly nonlinear multibody models that may be exposed to singularities. Existing multi-legged robot models typically use simplifying assumptions due to considering ideal robot movements, with limitations on capturing off-minimal motion scenarios. As an example, in a mechanically damaged robot, with a nonfunctional leg, the simplifying assumption such as the exclusive 2D motion of the system is violated. In such a scenario, a whole-body and singularity-free model of the robot is necessary to conduct damage identification and offer mitigation strategies. In this talk, I will show how to employ the Boltzmann-Hamel equations and the Screw theory to develop solutions to singularity-free whole-body modeling that include coupling between legs/main body dynamics and extend the theory to multi-branch systems. I will present a brief reformulation of the Boltzmann�Hamel equations in terms of quasi-velocity coordinates to describe the dynamics of a legged robotic system. These velocity coordinates are not the time derivative of position coordinates, but form a local diffeomorphism between a neighborhood of 0 _ R^k and an open neighborhood at every global coordinate. The method�s strength is apparent when the local parameters are set to zero at a global coordinate, after calculating the derivatives in the Lagrangian equations of motion that results in a parameterization-independent final equation. Finally, I will present the explicit inertia and Coriolis matrices for a general legged-robot needed for the implementation, together with several relations that can be used to speed up the algorithms. The derived equations are critical in the simulation and control multi-legged robots. The closed-form formulation makes the proposed approach well-suited for analyzing damaged systems and designing model-based controllers. The framework is validated with a Simscape model of a six-legged robot and is then used to simulate the unstable motion of a mechanically damaged robot. In this simulation I have a damaged 6-legged robot with a nonfunctional support leg. This damage scenario inevitably led to a large limping that needed a global, whole-body dynamical model for effective capture of the motion. Simulation results demonstrate the effectiveness of the proposed method in modeling off-nominal situations. In our future work, I intend to (i) extend our proposed model to include multi-degree-of-freedom joints, (ii) consider more realistic contact models, and (iii) improve robot control to mitigate damage.

Full-Pose Target Tracking Taskspace Control of Space-Manipulators on Lie Groups

Decentralized stochastic control problems with local information involve problems where multiple agents and subsystems which are coupled via dynamics and/or cost are present. Typically, however, the dynamics of such couplings is difficult to precisely model, leading to questions on robustness. Additionally, when such a coupling can be modelled, the problem arrived at is typically non-convex. 

In this talk, we develop a framework for risk-sensitive control for the study of decentralized stochastic control problems with uncertain interactions among a finite or infinite collection of interacting particles. We will generalize the former method, via tools from Gaussian measure theory and an associated variational equality.

A particular application includes mean-field models consisting of (a generally large number of) interacting agents. The key idea of mean field control is that for large enough systems of agents, one can approximate all interactions through a mean field coupling term - that is, each agent interacts with the average behaviour of all agents. However, due to this coupling, mean field control problems are often very hard to solve for the case with small or moderate number of agents, leading to an interest in effective approximations.  

We thus present a robustness framework for optimal decentralized control of interacting agents, where we show that a decentralized control problem with interacting agents can be approximated by considering a risk-sensitive version of non-interacting agents/particles. This leads to a tractable robust formulation where we give a bound on the value of the cost function in terms of the risk-sensitive cost function for the non-interacting case plus a term involving the ``strength" of the interaction as measured by relative entropy.

Robustness Properties of Risk Sensitive Control of Interacting Particles with Local Information

In the classical Shannon's information-theoretic lossy coding problem, one is allowed to encode long sequences of source symbols at once in order to achieve a lower distortion, which is optimal in the limit of unbounded block lengths. Such a block-coding approach is undesirable in many delay-sensitive applications, such as in networked control, sensor networks, live-streaming, etc.

We are interested in a variant of Shannon's lossy coding problem, where one wishes to send an information source at a fixed rate causally with no delay over a channel, while minimizing the average distortion at the receiver. Accordingly, we consider the zero-delay case where one wishes to encode source symbols as soon as they are available. 

It has been shown that this problem lends itself to stochastic control techniques which lead to existence, structural, and general approximation results. However, these techniques do not provide closed-form solutions for either optimal performance or codes, and they lead to algorithmic implementations that are computationally difficult. To address this problem, we propose a reinforcement learning approach. After developing some supporting technical results on regularity and stability properties (including unique ergodicity of an associated Markov process), and building on recent results on quantized Q-learning, we rigorously justify convergence of a quantized Q-learning algorithm to a near-optimal policy for this problem. These theoretical results are supported by simulations. 

We consider both the case of a noiseless channel (often called the optimal quantization problem) and the case of a noisy channel with feedback.

Reinforcement Learning for Zero-Delay Coding of Markov Sources

Accurate pose estimation is crucial to fast, safe, and autonomous operation of planetary Wheeled Mobile Robots (WMRs). To achieve that, robotics and control communities have been attracted to adopting modern tools such as Lie groups to model the robot pose. The reason for such an acceptance is that propagation of pose on Lie groups is singularity-free, representation is unique, and also provides a geometric perspective. The pose of a robot in 3-D world can be modeled by the elements of Special Euclidean SE(3) Lie group which consists of a rotation matrix and a translation vector where the rotation matrix itself belongs to Special Orthogonal SO(3) Lie group. Pose estimation is conducted by a filter that uses noise corrupted measurements of some sensors such as IMU, encoders, and GPS to update the pose in terms of a mean and a covariance matrix. In the recent years there has been numerous attempts to adapt classical estimation tools on Lie groups which led to the development of the Extended Kalman Filter (EKF) on Lie groups, Invariant EKF, Unscented Kalman Filter (UKF) on Lie groups, and stochastic filters. When the estimation problem relies on multiple sensor stations, sensor-level or track-level approaches can be used to fuse and update the estimates based on the measurements of all the sensors.  The former approach augments the measurements from the sensor stations and the latter one lets any sensor station to generate a local estimate of the pose and then combines the local tracks according to a fusion rule. In this talk the focus is on the track-level fusion on Lie groups with application to pose estimation of WMRs. I will first go over some preliminary materials such as representation of random pose variables on Lie groups using elements of SE(3) and Lie algebra and use of Baker�Campbell�Hausdorff formula to approximate the fusion error. Then the available track-level fusion methods in the literature will be reviewed and the main challenges of fusion on Lie groups will be discussed. Since all sensor stations are observing and estimating a common process, their estimates are correlated and the main dilemma in track-level fusion is how to account for these correlation terms. The available methods in the literature assume the local tracks are independent and either ignore the correlations or use time-consuming optimization problems to compensate for lack of knowledge of them by ensuring the fused covariance is conservative with respective to the true covariance.  In my proposed method I first solve the problem assuming that the tracks are independent and then use a finite number of random samples from a zero-mean Gaussian process to adjust the estimated covariance matrix with respect to when the correlations are ignored. In the proposed method, the fused estimate is obtained using an optimization problem that minimizes the error between the unknown fused estimates and known local estimates. I have used maximum probability and least square approaches for the optimization problem and also developed a methodology for constrained track-level fusion on Lie groups. In the numerical examples I will show that the proposed method is simple, fast, and delivers more consistent estimates with respect to its rival algorithms, which means the generated estimates are neither pessimistic nor optimistic. I will also investigate the effect of several parameters (e.g., degree of correlation, the order of the linearization error, the number of sample points, etc.) on the performance of the fusion filter in terms of average root mean squared error and average normalized estimation error squared.

Track-Level Fusion on Lie Groups for Pose Estimation

In recent years, fast frequency and voltage regulation using Distributed Energy Resources (DERs) has gained significant attention from the research community. With the increasing deployment of DERs at the Low-Voltage and Medium-Voltage distribution network, effective coordination between Transmission System Operator (TSO) and Distribution System Operator (DSO) is crucial for the stability of the power network.
In this research, we propose a novel, real-time, distributed, scalable, and multi-layered control structure that consists of hierarchical control areas that work in coordination to respond to power set-point requests from the TSO. The controllers leverage the available DERs to quickly respond and track the set-points by solving an optimization problem. The key innovation behind our proposed structure is the decomposition of the optimization problem of distribution network DERs into smaller, independent, and local problems. Such an approach allows for maintaining data privacy to local controllers only, minimizing communication data between the controllers, and ensuring network stability when responding to TSO requests. To enhance the communication between the controllers of different control areas, we introduce the notion of Virtual-DER (VDER). The VDERs act as regular DERs when communicating with higher-order controllers and pass the power set-point requests to lower-level controllers for tracking. This approach allows for easy scalability by providing updated sensitivity matrices for each controller, while maintaining areas and DERs privacy by keeping their measurements locally.
We provide a stability analysis of the proposed structure and analyze single controller stability using Fenchel duality. We develop the analysis further in the dual problem to analyze the overall network stability. We provide the sufficient conditions for having linear convergence using the Projection Theorem. Additionally, we provide a detailed tuning guideline of the controllers' design parameters to ensure system stability and better utilize the DERs in the distribution network.
To validate the effectiveness of our proposed structure, we conduct tests on multiple test circuits, including IEEE-123 and IEEE-8500, with different numbers of DERs. We use our developed MATLAB�-based application, MATDSS, which integrates OpenDSS� Engine with the proposed structure, to run the tests. The application provides an easy environment to define the controllers for any circuit that is developed in OpenDSS�. In addition, the program allows for running different dynamical test cases, using custom DERs dynamical models, a user-friendly configuration for DERs, circuit zoning, and layering of control areas, providing real-time monitoring while running the simulation, and interactive data visualization.
In conclusion, our proposed structure offers a promising technique for fast frequency and voltage regulation using DERs in the distribution network. The distributed approach of our proposed structure allows for effective coordination between TSO and DSO, while maintaining data privacy and network stability. Our proposed structure is versatile, scalable, and has the potential to improve the utilization of DERs in the distribution network.

Transmission and Distribution Networks Coordination: A Novel Multi-Layered Control Structure

Control systems with `human-in-the-loop' or `cyberphysical human systems' (CPHS) arise in a variety of applications ranging from manufacturing and autonomous driving to disaster management and healthcare. In such systems it is important to take human factors such as trust and workload while designing the system. Over or under trust leads to misuse or disuse of automation while increased workload degrades performance due to fatigue. An important aspect in such systems is to estimate the internal cognitive state of the human. In this work, we present a framework for estimating the human's internal state based on measurement data. In particular, we consider a task allocation problem where a human and an automation jointly perform a series of binary classification tasks based on a pre-defined set of rules. We collect subjective responses to standard questionnaires of trust and workload along with the interaction data such as classification accuracy, average classification time. We then design estimation algorithms to estimate the subjective responses based on the interaction data. The estimation of these subjective responses facilitates the design of an adaptive task allocation strategy at the automation which allocates tasks based on the estimated cognitive states of the human.

While it is well-known that a Kalman filter is an optimal state estimator of linear systems(given Gaussian noises), optimal estimation of non-linear systems of ordinary differential equations is an open problem. In this talk, we will discuss some first steps on a pathway to optimal state estimation using neural networks. We propose to use a recurrent neural network(RNN) to estimate the state of a system. Our codes will be tested on a number of examples, including the Lorenz attractor. The performance of the neural network based estimator will be compared to the popular extended Kalman filter(EKF).

RNN vs EKF for state estimation

Multistability, which refers to the coexistence of multiple stable equilibria or attractors in nonlinear dynamical systems, is a widespread phenomenon in natural sciences, including optics, mechanics, chemistry, and biology. In particular, it plays a crucial role in gene regulatory networks, where stable equilibria correspond to different cell phenotypes. 

In this talk, we study networks composed of a class of identical bistable systems, with a focus on the analysis of clustering and synchronization. We show that, in general, the network exhibits multistable behaviour, with new stable equilibria arising as a consequence of the coupling. Under mild assumptions, we show that the solutions converge globally to the set of equilibria and rule out the presence of non-trivial chain recurrent sets, which include periodic orbits and other returning trajectories. 

Using a piecewise linear approximation of the vector field, we determine the location of equilibria as a function of the coupling gain and we study their local stability. We finally provide sufficient conditions on the coupling graph to retain bistability, showing that almost all the solutions synchronize to either one of the two stable equilibria of the isolated systems. Our analysis employs tools from piece-wise smooth analysis and asymptotically autonomous systems, in combination with more classical approaches such as the Krasovski-La Salle invariance principle and Lyapunov theory. To the best of our knowledge, this is the first result that completely characterizes the positive limit set of a network of bistable systems.  

We consider the boundary stabilization of a parabolic partial differential equation coupled with an elliptic partial differential
equation. Even in the situation when these equations are exponentially stable when uncoupled, the coupled system may be
unstable. In fact, increasing the coupling factor tends to destabilize the dynamics of the system. A backstepping approach
is used to design a boundary control that will stabilize the system, or more generally, improve the decay rate in the situation
when the original system is stable. The result is illustrated with simulations.

This talk addresses the problem of designing output feedback controllers for constrained linear systems subject to bounded process and measurement disturbances. The presented solution extends the set-theoretic model predictive control framework to deal with constrained output regulation problems. In particular, this is achieved by adequately exploiting the extended Farka's lemma to offline compute through bilinear optimization problems, a family of robust one-step controllable sets, and associated output feedback control gains. It is then shown that such computations can be online leveraged to design a simple switching controller capable of ensuring, by construction, that the state-trajectory of the system is always uniformly ultimately bounded, in a finite number of steps, into a small, robust control invariant region. Finally, the effectiveness and benefits of the proposed solution are shown through a numerical example and comparisons with three alternative schemes.

RELEVANCE OF THE TALK:
This new control framework can deal with safety-critical systems in which the plant's safety constraints must be fulfilled regardless of disturbances and measurement noise. Although this paper introduces some fundamental theoretical results, the obtained strategy can find applications in different application areas ranging from smart grids to intelligent transportation systems, medical technologies, and industry 4.0.

PUBLICATION RELATED TO THE TALK:
[J1] - W. Lucia, J. G. Ernesto and E. Castelan, "Set-Theoretic Output-Feedback control: a bilinear programming approach," Automatica, 2023 (Accepted) 

Set-Theoretic Output-Feedback control: a bilinear programming approach

Markov jump linear systems (MJLS) are a good approximation of  time-varying systems arising in various applications including networked control systems and  cyber-physical systems. There is a rich literature on the stability analysis and optimal control  of MJLS. The question of system identification, i.e., identifying the dynamics from data, has not received much attention in this setup. In this presentation, we investigate the problem of  system identification for autonomous Markov jump linear systems  with complete state observations.

We propose switched least squares method for  identification of MJLS, show that this method is consistent, and derive  data-dependent and data-independent rates of convergence. We prove strong consistency of the switched least squares method and establish a  $O(\sqrt{\log(T)/T})$ rate of convergence, which matches with the rate of convergence of non-switched linear systems. In contrast to the existing high-probability convergence guarantees in the literature, our results show that the estimates converge to the true parameters almost surely. Moreover, we derive our results under weaker assumptions compared to the existing literature. We present numerical examples to illustrate the performance of the proposed method.

Strong Consistency and Convergence Rate of  Switched Least Squares System Identification for Markov Jump Linear Systems

Rolling without slipping is a constraint that appears in many robotic systems,
such as autonomous rovers, and is an example of an ideal nonholonomic
constraint. In the context of mechanical systems, nonholonomic constraints
normally refer to point-wise linear constraints on the space of velocities at
each configuration, which cannot be described by a derivative of a set of
constraints on the configuration space. In a recent Lagrangian slow-fast realization,
the constraint forces are modelled as dissipative viscous friction
forces whose limit at large coefficients converges to the ideal nonholonomic
model. This approach decomposes the system into two time-scales, where
the slow variables are in the nonholonomic distribution and the fast variables
represent small slip and is described by an invariant manifold. In
contrast to ideal nonholonomic constraints, such a slow-fast realization of
constraint forces captures energy dissipation and slip-dynamics in vehicle
systems before losing traction, and also captures the steep linear regime of
the Bakker-Pacejka magic tire formula. In this talk, I propose a slow-fast realization
of nonholonomic Hamiltonian control systems mediated by strong
friction forces which is viewed as a singular perturbation of the nonholonomic
system. I propose a systematic decomposition of the dynamics into slow and
fast directions using the kinetic energy metric and the geometry of fast friction
reaction forces. The invariant manifold associated with the singularly
perturbed system is identified with a power series. I develop a recursive
approximation of the invariant manifold based on the order of the control
input�s time derivative appearing in the higher-order approximations of the
invariant manifold. Accordingly, I introduce a recursive procedure to perform
input-output linearization of the slow system, subject to the recursive
approximation of the invariant manifold. I introduce a PD trajectory tracking
control problem for the approxmated perturbed dynamics. Closed-loop
stability analysis is performed on both the approximated perturbed dynamics
and on the unrestricted dynamics. I prove that if the internal dynamics
of the nonholonomic system is exponentially stable, then the stability of the
perturbed system remains exponentially stable. I prove that the output dynamics
for the unrestricted system is asymptotically stable when the full
power series representing the invariant manifold is known. Moreover, I prove
that the output dynamics are uniformly bounded for closed-loop dynamics by
applying the approximate recursive control law. My approach is illustrated
through a numerical case study on a differential robot.

Recursive Input-Output Linearization for Slow-Fast Realization of Nonholonomic Hamiltonian Control Systems

This talk presents a recently proposed control paradigm to solve trajectory tracking problems for input-constrained wheeled mobile vehicles. The proposed solution is applicable to both differential-drive and car-like robots, and it is the result of the combination of Model Predictive Control (MPC) strategies and feedback linearization techniques. In the talk, first, it is shown that if a feedback linearized model of the robot is exploited for predictions in MPC, then the set of admissible inputs for the linearized model is a nonconvex and state-dependant polyhedron, leading to non-convex and computationally expensive optimization problems with local minima issues. Then, a novel worst-case circular approximation of the state-dependent input constraints set is analytically derived and used to design reference tracking controllers that are, by design, recursively feasible and non-conservative. Finally, the effectiveness and benefits of the proposed control solutions are shown with simulations and laboratory experiments involving the Khepera IV differential-drive robot. The obtained results have been contrasted with different popular competitor schemes.

RELEVANCE OF THE TALK:
Mobile robots are becoming increasingly widespread in various real-world applications, ranging from manufacturing and transportation to search and rescue operations. To perform these tasks effectively, it is crucial for the robots to accurately follow a desired trajectory. This is a challenging problem due to the presence of nonholonomic constraints, model inaccuracies, sensor noise, and input-saturation constraints. To improve the accuracy of the performed trajectory, the input constraints acting on the robot’s model should be directly included in the control design. Unfortunately, many of the available control algorithms are unable to do so. To this end, one of the main objectives of this talk is to show how to directly handle the input constraints in feedback linearization-based model predictive control schemes, providing a guaranteed trajectory tracking control solution for wheeled mobile robots.
 
PUBLICATIONS RELATED TO THE TALK:
[J1] C. Tiriolo, G. Franzé, and W. Lucia, “A receding horizon trajectory tracking strategy for input constrained differential-drive robots via feedback linearization,” IEEE Transactions on Control Systems Technology, DOI 10.1109/TCST.2022.3219298, 2022. 

[Video of the performed experiments] - https://www.youtube.com/playlist?list=PLh-6B_s-jPuT8RTDOJM96GXu4y1IBIeoC

[J2] C. Tiriolo and W. Lucia, “On the design of control invariant regions for feedback linearized car-like vehicles,” IEEE Control Systems Letters, vol. 7, pp. 739–744, 2022."

Feedback-Linearization-based Predictive Control Strategies for Input Constrained Wheeled Mobile Robots.

Many systems in the modern world are composed of very large networks of interlinked nodes. The direct analysis and synthesis of such systems is diffi- cult analytically, and often intractable computationally or impossible due to concerns over privacy or costs associated with data collection. One approach to address the analytical difficulty for linear systems on large graphs is to use graphon theory which approximates the network as a linear operator given by an integral kernel in function space. This formulation for the control of large deterministic systems on networks was initiated by Gao and Caines (2020, 2021); in Dunyak and Caines (2022), Gaussian noise in the space of nodes, termed Q-noise, was included in these systems, analogous to the addition of correlated Gaussian noise in finite dimensional systems. This approach recovered many standard linear control theory results.
In the current work, we prove that a linear-quadratic Gaussian optimal con- trol problem defined on a large graph can be approximated by the solution to a corresponding infinite dimensional stochastic linear quadratic optimal control problem. Then, we demonstrate that the behavior of the finite dimen- sional system can be simulated by a numerical approximation of the infinite dimensional system in a scale-free manner, allowing a large random graph to be simulated by a smoothed linear system of lower order under the right conditions.
Work with Peter E. Caines

Stochastic Optimal Control Problems on Large Networks

A partially observed Markov process is a Markov chain in which the underlying
state space is hidden from a controller or decision maker, but can be indirectly observed
through a measurement channel. We consider a time-homogeneous hidden
Markov model with a time-homogeneous measurement channel. The model�s
initial state is selected according to an initial distribution, and the probability
measure over the state space is updated after each observation, leading to a
nonlinear filter process. 

Our research question is to determine the conditions under which this probability-measure valued filter process (which itself is a Markov process) is uniquely ergodic, i.e., possesses a unique invariant probability measure. 

Such a problem is important in understanding long-term behaviour of empirical processes, and has implications in robust stochastic control as well as reinforcement learning for partially observed models.

While the unique ergodicity problem has been extensively studied
in the literature for countable models, the case of a continuous state space has not been thoroughly investigated. We study the case where the state space is compact
and the observation space is countable, and establish conditions
under which the filter process is uniquely ergodic. 

To arrive at unique ergodicity, we require two assumptions: the first assumption guarantees that the filter process is an $L$-chain, which means that for any bounded and Lipschitz continuous function with compact support, the sequence of functions obtained by integrating the transition kernel of the filter process over any number of n steps is equi-continuous.
The second assumption we use is the existence of a reachable state, whose any (open) neighborhood can be reached from any given belief state with positive probability in some finite time. We show that the filter Markov process is uniquely invariant under these assumptions.

Unique Ergodicity of Non-Linear Filters

In recent years control centrality has gained increasing attention in different fields of control theory and biological systems. In particular, there has been a growing interest in control centrality measures for weighted, directed and signed networks, which can be applied to genetic regulatory networks, in which the node are genes, and the node's states are the concentration of mRNA and protein product. The strength and orientation of regulatory relationships are represented by the weighted and directed links. Here, we introduce a new control centrality measure based on optimal control theory. The proposed measure quantifies the ability of a node to control a linear time-invariant network. Most methods for control of genetic networks have focused on stable and controllable networks. Here, in contrast, we decompose the network into stable and unstable parts. (Unstable parts arise in, e.g. oscillations and bifurcations; stabilization of such unstable states in tumor types plays a vital role in driving them to desired steady states using control inputs). The measure is extended for unstable genetic networks by introducing some new controllability and reachability Gramians. These Gramians can be related to the performance metric measured by input effort implemented to the network through external environment. This is done via introduction of a proper cost function. Optimal value of effort consumed by each input node, e.g., a controlling gene, is utilized to transfer the entire network's initial states, e.g., gene concentrations, to their desired final values in steady state. To validate the effectiveness of the proposed method, it is compared in simulation with other standard centralities for a genetic network. 

Learning to control dynamical systems while ensuring formal guarantees remains a challenging task. Recent advancements in machine learning, combined with classical control theory, present new opportunities for addressing this challenge through data-driven approaches. This talk presents two learning-based approaches for learning unknown dynamics and obtaining stability guarantees for nonlinear dynamical systems: neural networks and the Koopman operator. The first approach involves a learning framework that simultaneously stabilizes an unknown nonlinear system with a neural controller and learns a neural Lyapunov function to certify a region of attraction (ROA) for the closed-loop system. The algorithmic structure consists of two neural networks and a satisfiability modulo theories (SMT) solver. The neural networks learn the unknown dynamics and identify a valid Lyapunov function with a provably stabilizing nonlinear controller. Meanwhile, the SMT solver verifies that the candidate Lyapunov function satisfies the Lyapunov conditions. The second approach involves an algorithm to learn the dynamics and Lyapunov functions for unknown systems by learning the Koopman operator and corresponding eigenfunctions in a discrete-time setting. The obtained eigenfunctions can construct candidate Lyapunov functions for the continuous-time system with theoretical guarantees. An SMT solver can also certify an estimate of the ROA for the unknown nonlinear system. The effectiveness of both approaches is illustrated with numerical experiments on typical nonlinear control problems.

Data-Driven Control Meets Formal Verification: Neural Lyapunov Control with Stability Guarantees

In this talk, we present a novel methodology for guaranteed collision avoidance when a set of ground autonomous vehicles move in a shared planar environment. We consider a scenario where each vehicle follows a trajectory imposed by a local planner and where each unmanned vehicle is subject to a different set of constraints and disturbances. Inspired by how traffic lights work in our cities, we present a novel control architecture where a centralized traffic manager, in conjunction with ad-hoc designed local vehicle controllers, can ensure the absence of collisions. The proposed solution is obtained by exploiting, for the local vehicles controllers, a dual-mode Model Predictive Controller (MPC) and, for the traffic manager, set-theoretic and controllability properties. Moreover, after modeling the potential vehicle collisions with a graph, connectivity arguments are used to obtain an optimal collision resolution which minimizes the number of vehicles that need to be stopped at the current time. In addition, if a finite preview of the future waypoints for each vehicle is available, then the traffic manager is also able, in accordance with the local controller, to speed up or speed down the vehicles in order to minimize the chances of future collisions. The resulting control scheme ensures collision-free reference tracking. Simulation results will be shown to provide tangible evidence of the features of the proposed framework. Finally, an ongoing work aiming to apply the presented solution to robots modelled by differential-drive robots will be briefly illustrated.

RELEVANCE OF THE TALK: A Multi-Agent System (MAS) traversing on a shared planar environment requires safety critical strategies to ensure the absence of collisions. Most of the existing solutions typically do not take into account the unavoidable presence of vehicle constraints (e.g., velocity and/or acceleration constraints) and disturbances. On the other hand, the proposed framework defines a first step towards a robust solution to the problem. The methodology presented here is particularly relevant in smart factories or smart cities where a large number of ground vehicles move in a shared and cluttered environment.

PUBLICATIONS RELATED TO THE TALK:
[1] - M. Bagherzadeh, S. Savehshemshaki and W. Lucia, ""Guaranteed Collision-Free Reference Tracking in Constrained Multi Unmanned Vehicle Systems,"" in IEEE Transactions on Automatic Control, vol. 67, no. 6, pp. 3083-3089, 2022. 
[2] - S. Savehshemshaki and W. Lucia, ""A Robust Receding-Horizon Collision Avoidance Strategy for Constrained Unmanned Ground Vehicles Moving in Shared Planar Environments,"" 2022 IEEE 61st Conference on Decision and Control (CDC), Cancun, Mexico, 2022, pp. 5253-5258, 2022.
[3] - C. Tiriolo, G. Franze', W. Lucia, ""An Obstacle-Avoidance Receding Horizon Control Scheme for Constrained Differential-Drive Robot via Dynamic Feedback Linearization,"" American Control Conference, 2023 (Accepted)."

Robust Set Theoretic MPC-based Guidance for Guaranteed Collision Avoidance in Multi-Agent Robotic System

This talk presents a novel approach for computing robust backward reachable sets from noisy data for unknown linear systems subject to bounded disturbances and state and input constraints. In particular, an algorithm for obtaining zonotopic inner approximations that can be used for control purposes is presented. It is shown that such sets, if built on an extended space (including the state and input vectors), can be used to embed the system's one-step evolution in the computed extended regions. Such a result is then exploited to build a set-theoretic model predictive controller that, offline, builds a recursive family of robust data-driven backward reachable sets and, online, computes recursively admissible control actions without explicitly resorting to either a model of the system or the available data. 

RELEVANCE OF THE TALK:
To the best of our knowledge, the idea presented here is the first one capable of computing data-driven robust backward reachable sets from a collection of input-state trajectories. One of the peculiar features of the proposed solution relies on the fact that the proposed extended computation of the robust backward reachable set can be directly used to obtain a data-driven implementation of the dual-mode receding horizon controller developed in [2].

PUBLICATION RELATED TO THE TALK:
[1] - M. Attar and W. Lucia, ""Data-Driven Robust Backward Reachable Sets for Set-Theoretic Model Predictive Control,"" submitted for potential publication in L-CSS/CDC 2023 (Under Review). link[https://arxiv.org/abs/2303.04749]

REFERENCES:
[2] - D. Angeli, A. Casavola, G. Franze and E. Mosca, ""An ellipsoidal off-line MPC scheme for uncertain polytopic discrete-time systems,"" Automatica, 2008."

Data-Driven Robust Backward Reachable Sets for Set-Theoretic Model Predictive Control 

We consider the problem of data-driven predictive control for an unknown discrete-time linear time-periodic (LTP) system of known period. Our proposed strategy generalizes both Data-enabled Predictive Control (DeePC) and Subspace Predictive Control (SPC), which are established data-driven control techniques for linear time-invariant (LTI) systems. The approach is supported by an extensive theoretical development of behavioral systems theory for LTP systems, culminating in a generalization of Willems' fundamental lemma. Our algorithm produces results identical to standard Model Predictive Control (MPC) for deterministic LTP systems. Robustness of the algorithm to noisy data is illustrated via simulation of a regularized version of the algorithm applied to a stochastic multi-input multi-output LTP system.

Data-Driven Model Predictive Control for Linear Time-Periodic System

he multiplier method is used  to obtain a  lower bound on the exponential decay rate  for port-Hamiltonian systems in one space dimension with boundary dissipation. Unlike previous results, the physical parameters of the system may be spatially varying, and an explicit bound on the decay rate is in terms of the physical parameters is obtained. 
The dissipation can occur on the  boundary or internally or both.  The result is illustrated by a vibrating beam with variable parameters and cross-section.

Welcome

Mean Field Game (MFG) theory treats the existence of Nash equilibria for large population dynamical games by approximating them with infinite population games of negligible agents. The MFG equations consist of the Hamilton-Jacobi-Bellman  equation for the control of a generic agent and  the Fokker-Planck-Kolmogorov equation describing its state evolution, and these  are linked by the system's mean field.


Graphons may be viewed as limits of node adjacency matrices in an appropriate function space. Graphon MFG (GMFG) systems (Caines and Huang, CDC 2018-19, SICON 2021) consist of MFGs where large sub-populations are distributed over the nodes of large dense graphs modelled by graphons. The solution of the GMFG equations gives a system's Nash equilibria and this permits an analysis of the optimality of the node dependent Nash values with respect to graphon index (Huang, Gao, Foguen-Tchuendom, Caines, CDC 2021,2022, MTNS 2022). This presentation will introduce metric space embedded graph limits in the form of vertexon-graphexons (Caines, CDC 2022). This formulation  enables the modelling of networks situated in space and has the advantage of including both large sparse and dense networks; furthermore, the resulting GXMFG set-up permits meaningful differentiation of variables with respect to network location which is not the case for graphon formulations.  

Mean Field Games on Large Dense and Sparse Networks

In decentralized systems with partial state observations and partial action monitoring, reinforcement learning agents may be oblivious to the presence of other agents. Such independent learning agents may therefore employ single-agent reinforcement learning algorithms, treating their partial state observation as though it were a state variable. We study N-player mean-field games from such a decentralized learning perspective with two primary objectives: (i) to study the convergence properties of independent learners, and (ii) to identify structural properties of N-player mean-field games that can guide algorithm design. Toward the first objective, we study the learning iterates obtained by independent learners, and we use recent results from partially observed stochastic control theory to show that these iterates converge under mild conditions. In particular, we consider four information structures corresponding to information at each agent: (1) global state + local action; (2) local state, mean-field state + local action; (3) local state, compressed mean-field state + local action; (4) local state with local action. We present a notion of subjective equilibrium suitable for the analysis of independent learners. Toward the second objective, we study a family of dynamical systems on the set of joint policies. The dynamical systems under consideration are subject to a so-called _-satisficing condition: agents who are subjectively _-best-responding at a given joint policy do not change their policy. We establish a useful structural property relating to such dynamical systems. Finally, we develop an independent learning algorithm for N-player mean-field games that drives play to subjective _-equilibrium under self-play, exploiting the aforementioned structural properties to guarantee convergence of policies. Notably, we avoid requiring agents to follow the same policy (via a representative agent) during the learning process, which has been the typical approach in the existing literature on learning for mean-field games.

Independent Learning in Stochastic and Mean-Field Games: Satisficing Paths and Convergence to Subjective Equilibria

Coffee break

Many iterative algorithms in optimization, games, and learning can be viewed as dynamical systems with inputs (measurements, historical data, user feedback), internal states (decision variables, state estimates, Lagrange multipliers), outputs (residuals, actuator commands), and uncertainties (noise, unknown parameters). The last few years have witnessed a growing interest in studying how learning, optimization and game-theoretic algorithms behave when placed in closed loop with noisy, uncertain, and dynamic physical systems and conversely how systems theory can be leveraged to both analyze existing algorithms and synthesize new ones. Notable recent examples include applying robust control tools (such as integral quadratic constraints) to analyze and synthesize gradient methods, repurposing optimization algorithms as feedback controllers for physical systems, and studying reinforcement learning algorithms using hybrid systems theory.

This dynamical systems perspective on algorithms is crucial for tackling some of the challenges arising when analyzing and synthesizing the algorithms that underlie modern autonomous, AI-driven, and socio-technical systems. These include algorithms that: (i) operate online (e.g., running algorithms with streaming data), (ii) include humans in the loop (e.g., autonomous driving, recommender systems), (iii) are interconnected with physical systems (e.g., optimization-based feedback controllers, real-time MPC), (iv) involve self-interested decision makers with shared resources (e.g., traffic/logistic networks), or (v) need to be robust to severe uncertainty (e.g., exogenous disturbances, intrinsic randomness, or non-stationarity). These new and timely applications are driving the development of new theoretical tools that synergistically build on control, optimization, game, and learning theory as well as novel algorithms and computational techniques that are specialized for noisy, time-varying, and dynamic environments.

In this talk, I discuss two scenarios where optimization algorithms are used directly as controllers for physical systems. First, I present Time-distributed Optimization (TDO), a unifying framework for studying the system theoretic consequences of computational limits in the context of Model Predictive Control (MPC). I show that it is possible to recover the stability and robustness properties of optimal MPC despite limited computational resources and illustrate how a system-theoretic view of algorithms can be exploited to certify the closed-loop system. Further, I illustrate the applicability of the these methods in the real-world through diesel engine, and autonomous driving examples. Second, I discuss Feedback Equilibrium Seeking (FES), a design framework for dynamic feedback controllers that track solution trajectories of time-varying generalized equations, such as local minimizers of nonlinear programs or competitive equilibria (e.g., Nash) of non-cooperative games. I present tracking error, stability, and robustness results for the sampled-data cased and provide illustrative examples in DC power grids and supply chains. 

A Systems-theoretic Viewpoint on Real-time Optimization and Feedback Games

Lunch

We consider a set of agents playing a game whose goal is to achieve a collective configuration described by an unknown Nash equilibrium. We call this - the problem of seeking a Nash equilibrium of the game. The game is non-cooperative in the way actions are taken (each agent aims to minimize its own individual cost), but the setting is collaborative (agents may share some information with neighbours to compensate the possible lack of global information). We show how dissipativity and passivity theory can be used in the analysis and design of distributed dynamics. We review some popular game-theoretic algorithms and show that they can be cast as instances of a feedback interconnection between a dissipative/passive dynamical system and some game mapping. Once this is done, convergence to a Nash equilibrium follows from standard passivity theory, based on simple and concise arguments. We also discuss how passivity-inspired ideas can be used to design novel Nash equilibrium-seeking algorithms and learning dynamics.

On the role of dissipativity/passivity theory in game theory

Game theory, wherein competitive agents seek to optimize coupled cost functions, has received increased interest in various engineering fields over the past decade. In many engineering applications, the agents involved are real-world dynamical systems, such as mobile robots or power generators. These dynamics are ignored by conventional Nash equilibrium (NE) seeking algorithms. Over the past few years, much attention has been given to NE seeking algorithms that incorporate these dynamics. Current results focus on specific designs for specific types of agents, lacking a general framework. By extending results from the field of steady-state optimization, we introduce a novel new framework to solve the problem. Using our methodology, we show that the problem can be reduced to the design of a set of decentralized or distributed stabilizing controllers. Various controller designs are presented and trade-offs are discussed. 

The game theoretic steady state control problem

In the talk we investigate the connection between turnpike and strict dissipativity properties for finite dimensional linear quadratic (LQ) optimal control problems.
Turnpike phenomena have received increasing interest from the control community, owing to their connection with optimal synthesis and stability of Model-Predictive Control schemes.
The focus of the talk is to provide characterizations of strict dissipativity and the newly introduced property of strict pre-dissipativity in terms of the system matrices related to the LQ problem. These characterizations then lead to new necessary and sufficient conditions for turnpike in terms of spectral criteria and Lyapunov matrix inequalities. 
In particular, the techniques allows to encompass the presence of state and input constraints, and the resulting analysis emphasizes the role of state constraints in inducing turnpike phenomena.
Several examples illustrate the different turnpike behaviors of the system in the presence or absence of constraints.

Turnpike properties and strict dissipativity for LQ optimal control problems

Globally, there has always been an interest in pursing cyber-physical applications that accomplish tasks with varying levels of autonomy and coordination. With the advancements in technology, digital implementation has been made viable and, as a result, most of these applications are now seeing the light of the day. One of the key aspects for digital implementation, especially, in the case of multi-vehicle/multi-agent applications, is designing protocols for communication over networks. Naturally, the immediate question one would ask is: (Q1) how often should the systems broadcast their information? 

	For multi-agent systems (MASs), researchers have addressed this question under various scenarios involving a) linear or nonlinear agent dynamics over b) undirected or directed communication networks with c) synchronous or asynchronous broadcast protocols accomplishing d) various coordinated tasks such as consensus, formation, tracking, and optimization. However, most of these studies addressed Q1 by constructing linear matrix inequalities (LMIs) which are numerically solved to obtain bounds on the sampling periods (which refer to the intervals between consecutive broadcast instants in the case of time-triggered/sampled-data protocols). Often, the methodologies employed to study a specific scenario may not directly extend to the next one; for instance, those investigating synchronous protocols may not be able to admit asynchronous behaviors stemming from drifting away of local timers. Therefore, individually addressing Q1 for each of the aforementioned scenarios may often result in solving multiple LMIs making it inconvenient for a control engineer to decide on a sampling period. 

	In our work, we present a unified framework to study digital implementation of continuous-time distributed algorithms associated with the domain of both control and optimization. The framework models a MAS, with continuous-time dynamics and discrete-time communications, as a hybrid dynamical system and designs two different event-triggering conditions (ETCs) which make decisions on broadcasting, thereby, addressing the question Q1. If the ETCs are rendered ineffective, then the broadcast protocols reduce to time-triggered ones; in such situations, one has the freedom (to a certain extent) to assign the broadcast instants such that the resultant protocol is potentially both asynchronous and aperiodic. In order to facilitate digital implementation, the work allows for explicit computation of the bounds on sampling periods. The effectiveness of the framework is demonstrated through case studies involving MASs such as: i) achieving consensus among nonlinear agents (with Lipschitz and one-sided Lipschitz dynamics) over both undirected and directed networks, and ii) solving for global minima using accelerated gradient algorithm over weight-balanced digraphs. The developed framework can be further extended to include network-induced imperfections.

A Unified Framework for Distributed Control: A Model-Based Hybrid Systems Approach

Event-triggered control provides an effective way to update the control signals at a sequence of discrete-time moments determined by certain execution rules, often referred to as an event. The main advantage is to improve the efficiency of control implementations while still guaranteeing the desired performance levels of closed-loop systems. This talk focuses on event-triggered control for the stabilization of time-delay systems. We present a novel event-triggering algorithm with two tunable parameters, based on a Lyapunov-Krasovskii functional result for input-to-state stability. The proposed algorithm ensures the resulting closed-loop systems are asymptotically stable. Sufficient conditions on these tunable parameters are also derived to exclude Zeno behavior. This is joint work with Bahman Gharesifard (UCLA) and Elena Braverman (University of Calgary).

Event-Triggered Stabilization of Time-Delay Systems

The essential step in abstraction-based control synthesis for nonlinear systems to satisfy a given specification is to obtain a finite-state abstraction of the original system. The complexity of the abstraction often determines the efficiency of the algorithm. Recent literature has demonstrated the soundness of abstractions in preserving robust probabilistic satisfaction of omega-regular linear-time properties for the control synthesis of discrete-time nonlinear stochastic systems modeled by nonlinear stochastic difference equations. However, unnecessary transitions within the abstractions are difficult to quantify, and the completeness of abstraction-based control synthesis in the stochastic setting remains an open theoretical question.

In this research, we address this fundamental question from a topological view of the metrizable space of probability measures and propose constructive finite-state abstractions for control synthesis of probabilistic linear temporal specifications. Such abstractions are both sound and approximately complete. This means that given a concrete discrete-time fully-observed stochastic system and an arbitrarily small L1-perturbation of this system, there exists a family of finite-state controlled Markov chains that abstracts the concrete system and is also abstracted by the slightly perturbed system. In other words, given an arbitrarily small prescribed precision, an abstraction always exists to decide whether a control strategy exists for the concrete system to satisfy the probabilistic specification. The proposed approach also applies to automated verification of stochastic systems, as well as control synthesis for stochastic systems with partial observations.

Robustly Complete Finite-State Abstractions for Stochastic Systems

While stochastic and robust control are the dominant paradigms in our field, there is a fascinating paradigm at their intersection called risk-aware control theory. This area is about the analysis and control of dynamical systems with respect to a vast spectrum of possibilities between the average case and the worst case. The flexibility and generality offered by risk-aware control theory has broad significance because control systems often require an awareness of rare harmful possibilities without being overly cautious and different applications have different needs and preferences about managing uncertainty. Major early players in this area include David Jacobson (1970�s) and Peter Whittle (1980�s-1990�s). There have been many exciting developments since those times, motivated by advances in finance and operations research in the early 2000�s. In this talk, I will present an overview of risk-aware control theory, including a brief history, along with the recent development of risk-aware safety analysis, with focus on the governing principles. 
We have developed a safety analysis framework for stochastic discrete-time systems, which minimizes the severity of harmful outcomes in a fraction of the worst cases. We use a mathematical tool from finance and operations research called conditional value-at-risk in combination with stochastic dynamic programming from stochastic optimal control. In contrast, classical safety analysis frameworks characterize safety in probability or in the worst-case (which requires the assumption of bounded disturbances), which can obscure the variability of stochastic system behaviour. We will showcase our theoretical contributions in this talk, which include a dynamic programming-based methodology to compute risk-aware safe sets exactly in principle under appropriate assumptions. Moreover, we will present a numerical illustration inspired by a stormwater infrastructure design problem as well as an approach to alleviate the curse of dimensionality.

A Brief Overview of Risk-Aware Control Theory

Complex systems consist of a collection of devices with local dynamics and control that are coupled by physical or cyber networks. Examples include the electrical power grid, communication systems, and networks of mobile robots and autonomous vehicles. Such systems provide essential services to society but are notoriously challenging to analyze or control due to their large scale and complexity. The talk will discuss new methods to enable safe, reliable, and desired operation of complex systems.

The first part of the talk focuses on optimal constrained control in complex systems. In many systems it is important for linear feedback controllers to provide desired performance while simultaneously satisfying saturation and resource limits non_-conservatively. The design is formulated as an optimal control problem where the objective is to minimize the mismatch between actual and desired behavior subject to state, input, and output constraints. To solve this challenging, nonconvex optimization problem, a novel control design technique is developed which transforms the original problem into a convex optimization that can be solved efficiently. The key idea is to combine system level synthesis, a recent technique which renders the problem convex but infinite dimensional, with a novel finite dimensional approximation based on simple poles. The approximation is shown to converge to the Hardy space of stable transfer functions, resulting in a design method with bounded suboptimality. The method is applied for control design of a dynamic virtual power plant of distributed energy resources.

The second part of the talk focuses on stability analysis in complex systems. Complex systems naturally experience disturbances that disrupt normal operation and can lead to instability. Whether a system is able to recover from a particular nonlinear disturbance depends on the values of system parameters, which are typically uncertain and time-varying. Therefore, it is valuable to compute the margins for safe operation: the smallest change in system parameter values from an initial operating point that would make the system vulnerable to a particular disturbance. A theoretical framework based on modern dynamical systems theory is developed which leads to the transformation of this abstract problem into a class of concrete numerical optimization algorithms which can be solved efficiently. These algorithms exploit properties of the region of attraction boundary of a stable equilibrium point in order to numerically compute the margins for safe operation. The resulting methods are applied to assess fault vulnerability in power systems where they compute safety margins in high dimensional parameter space.

Optimal Constrained Control and Nonlinear Stability Analysis for Complex Systems

A ubiquitous operation in control theory is system composition: breaking down a complex system into simpler constituents involves parallel and serial compositions, and the stabilization of a plant using a feedback controller involves a closed-loop composition. It is a fundamental tool for understanding complex systems and is the basis for design strategies such as back-stepping or decentralized control. However, with the usual model of systems as differential equations or transfer functions, compositions are not always well-defined. This problem is especially manifest when dealing with systems defined by partial differential equations (PDEs) with boundary inputs or outputs, i.e., systems where the inputs and/or the outputs enter in the system through the boundary conditions. For these systems, even the simplest operation of serial composition may fail to be well-defined and this represents an obstacle to both analysis and control. 

For these reasons, in this talk, we choose input-state-output relations as systems representations. In this framework, compositions are always defined and control-theoretic notions such as stability, stabilizability, detectability and input-output gains, can be defined as natural generalizations of the differential equation model counterpart. It is then possible to unambiguously ask and answer questions about properties of compositions, such as well-posedness and stability. We will show that PDEs with boundary inputs and outputs constitute a subclass of well-posed systems, which can be described as compositions of abstract differential equations. Under this framework,  we provide necessary and (separately) sufficient conditions for feedback stabilization by finite-dimensional controllers thus generalizing existing control strategies for specific parabolic PDEs. In particular, we show that a PDE (with boundary inputs and outputs) that is feedback stabilizable is the parallel composition of a stable system and a_finite-dimensional, stabilizable, detectable system. Conversely, the system is feedback stabilizable if, additionally, it decomposes into the parallel composition of a finite-dimensional system, and a system with small input-output gain. Finally, we show that the proposed framework can also be used to address the problem of analysis and control of synchronization of infinite-dimensional linear systems with boundary inputs and outputs.

Modelling of infinite-dimensional systems for network analysis

We consider a controlled reaction-diffusion equation, modeling the spreading of an invasive population. Our goal is to derive a simpler model, describing the controlled evolution of a contaminated set. We first analyze the optimal control of 1-dimensional traveling wave profiles. Using Stokes’ formula, explicit solutions are obtained, which in some cases require measure-valued optimal controls. Then we introduce a family of optimization problems for a moving set and show how these can be derived from the original parabolic problems, by taking a sharp interface limit. Assuming that the initial contaminated set  is convex, we prove that an eradication strategy is optimal  if an only if at each given time the  control is active along the portion of the boundary where the curvature is maximal. We then consider the eradication problem with geographical constraints, and  derive necessary and sufficient conditions for the existence of a solution.

Optimal Control of Moving Sets

In this talk, I will provide an overview of the virtual constraints approach to control underactuated mechanical systems. This technique has gained prominence in the field of biped robot locomotion, but is applicable to a wider range of problems.

In the virtual constraint approach, motion specifications (e.g., making a biped robot walk) are encoded as constraints to be enforced via feedback control. Geometrically, a virtual constraint is a closed embedded submanifold of the state space that can be rendered invariant through feedback. We call this surface the constraint manifold. Enforcing the constraint involves asymptotically stabilizing the constraint manifold, and the closed-loop dynamics of the mechanical system on the constraint manifold represent the system's steady-state behaviour. By designing the geometry of the constraint manifold appropriately, we can shape the closed-loop dynamics. For instance, we can ensure that almost all closed-loop orbits on the constraint manifold are closed, or that there is a stable limit cycle, or that solutions traverse a certain region with bounded speed, and so on.

The virtual constraint approach requires a shift in how we think about motion control for mechanical systems. Traditionally, one first designs a reference signal via motion planning, and then designs a feedback controller that asymptotically tracks the reference signal. In the virtual constraint approach, there are no reference signals in the feedback loop, and motion planning is replaced by ""constraint planning"", where we shape the geometry of the constraint manifold to induce desired steady-state behaviours. The absence of timed reference signals in the control loop offers significant advantages that I will discuss during the talk.

Developing a comprehensive framework for controlling mechanical systems through virtual constraints involves addressing several theoretical questions. Some of these questions have been answered in the literature, while others remain open problems. During the talk, I will discuss what we know about virtual constraints and the open problems in this area.

Virtual Holonomic Constraints: An Overview

The most basic task for a classical, fully actuated manipulator is a
point-to-point motion: moving from one configuration to another along some
path, typically designed to avoid obstacles and perhaps minimize some cost
criterion, such as total length. Broadly speaking, there are two approaches to
this problem: reference tracking, where the motion is specified as a timed
trajectory and a controller is designed to achieve asymptotic tracking, and
maneuver regulation, where the primary task is to render invariant and
stabilize the path (the geometric problem), and secondarily to stabilize a
speed profile along the path (the dynamic problem). The maneuver regulation
approach is attractive in situations where a disturbance could cause the timed
reference signal of a reference tracking approach to diverge significantly from
the actual state of the robot, causing the robot to make a potentially
disastrous attempt to ""catch up"" with the signal.

Applying the maneuver regulation approach to underactuated mechanical systems
runs into several difficulties. Rendering the desired path invariant can
exhaust all of the available controls, and so moving from one controlled
equilibrium to another while maintaining path invariance is impossible. In this
work, we show how to appropriately relax the maneuver regulation problem for a
mechanical system with degree of underactuation one. Given a nominal path
connecting two controlled equilibria of the system which satisfies certain
properties, we show how to generate a surface containing this path such that
the surface can be rendered controlled invariant. Rendering the surface
invariant uses all but one of the available control degrees of freedom. The
remaining control can then be used to achieve the required motion from the
initial to final configuration while maintaining invariance of the surface. We
demonstrate our method for a simple underactuated system, the cart-acrobot.

Point-to-Point Maneuver Regulation for Underactuated Mechanical Systems

Soft robotic systems use light-weight visco-elastic materials to take advantage of inherently safe responses and continuous deformations to achieve complex motions, otherwise impossible with rigid robots. Rigid-flexible multi-body systems have been introduced to commercial applications in the form of rigid manipulators with soft grippers. Models for such systems are limited by slow finite element or inaccurate quasi-static Constant Curvature (CC) solutions. Fast and accurate models are yet to be developed for effective model-based control design and simulation of complex flexible systems. 
We develop a fast, recursive and parameterization-free formulation for forward and inverse dynamics of soft robots modelled as multi-body systems with rigid and flexible bodies. The system is composed of bodies serially connected with discrete single-degree-of-freedom joints from a fixed base. The joint kinematics is captured through the exponential map of the Special Euclidean group SE(3). We couple the Newton-Euler equation for rigid bodies and a set of Partial Differential Equations (PDEs) on the SE(3) for dynamic Cosserat rods to formulate the dynamics of multi-body systems. Our proposed inverse dynamics recursively determines the system's response as well as the required joint torques to follow a prescribed joint-space trajectory; while the forward dynamics inherits the same structure and determines the system's motion for given joint torques. Unlike rigid multi-body systems, where the kinematic and dynamic recursions are decoupled, the inclusion of flexible bodies necessitates solving a coupled set of PDEs with applied separated Boundary Conditions (BCs). We develop a shooting-method-based BC solver to move BCs to one point and a numerical framework to integrate these equations. 
The framework adopts a fast finite difference method to semi-discretize the PDEs and transform them into ordinary differential equations, which are integrated using a geometrically exact integrator on the SE(3).
Implementing the proposed algorithms, we study the dynamic response of an example. A software library to simulate generic rigid-flexible multi-body systems, including the presented examples, is distributed.
A numerical study is conducted into various integration and discretization schemes, noting their accuracy and computational efficiency in the context of published solutions. The proposed geometric integrators are demonstrated to improve solution accuracy compared to the published finite difference models. The recursive model is computationally efficient and is used for model-based control of rigid-flexible multi-body systems.

Recursive Dynamics of Soft Manipulators with Discrete Joints on SE(3)

Achieving reference tracking and disturbance rejection in the presence of model uncertainty are two of the fundamental goals of feedback design. In many applications of interest, the system to be controlled is complex and no accurate dynamic model is available, but the system is known to be stable, either inherently by physics or through an initial compensator design. Motivated by the tuning simplicity of integral controllers for such applications, in 1976 E. J. Davison put forward the idea of the tuning regulator, a low-order output regulator design for stable MIMO LTI systems. Crucially, the design was data-driven -- requiring only selected frequency response data from the plant -- and could in principle be tuned online during commissioning. 

This talk will outline some of our recent advances in tuning regulator design for both linear and nonlinear systems. In the nonlinear case we focus on integral control in both continuous and discrete-time, and provide generalized conditions based on contraction theory for closed-loop stability. Applications of the ideas to feedback-based online optimization of dynamic systems and to control of renewable-dominated power transmission systems will be highlighted. For LTI systems, we revisit the original tuning regulator in continuous-time, and improve upon it by (i) providing more systematic pole-placement and LMI-based gain design procedures, and (ii) reducing the large the number of online tuning parameters to a single scalar. We conclude with an outlook on the fusion of such traditional control ideas with modern data-driven design methods.

The Return of the Tuning Regulator

This talk addresses the problem of designing a novel PI-like output feedback tracking controller for linear systems subject to state and input constraints. The proposed solution exploits the internal model principle to design via robust positive invariance and the Extended Farkas' Lemma, a constrained set-point tracking controller. The considered polyhedral setup results in a bilinear optimization design problem that can be solved offline, allowing to simultaneously determine the sets of admissible set-points and plant's initial conditions. The proposed solution guarantees asymptotic offset-free set-point tracking, local stability, and constraints fulfillment. Simulation results will be presented to show the effectiveness of the proposed design and contrast the proposed strategy with an alternative scheme.

RELEVANCE OF THE TALK:
The peculiar feature of the proposed design consists in the definition of a single bilinear programming problem capable of simultaneously computing the controller parameters, the set of admissible reference signals, and the controller's domain of attraction. The proposed solution leveraged robust positive invariance arguments to define the algebraic conditions under which the proposed controller ensures set-point tracking and constraint fulfillment. Differently from the existing literature, the proposed solution is not limited to plants having a single input and single output structure; the state, input and reference constraints can be asymmetric; the magnitude of the integral error can be taken into account in the design and treated as a design parameter. In addition, although the proposed solution defines a standalone tracking controller, its design can also be used to build a terminal controller for dual-mode model predictive strategies. Consequently, the developed strategy can find applications in the chemical and robotic industries and in the development of intelligent transportation systems and self-driving cars.

PUBLICATION RELATED TO THE TALK:
[J1] - Santos, G. F., Castelan, E. B., Lucia, W., "" On the design of constrained PI-like output feedback tracking controllers via robust positive invariance and bilinear programming."" IEEE Control Systems Letters, DOI: 10.1109/LCSYS.2023.3251312, 2023."

On the design of constrained PI-like output feedback tracking controllers via robust positive invariance and bilinear programming

This talk gives an overview of our work over the last few years in the area of systems neuroscience. Distinct from the more familiar application of systems biology, systems neuroscience is a relatively new application of control theory that regards the functional behaviour of brain regions and their interactions with each other and the body. We have focused on developing control theoretic models of regulatory brain functions mediated by the cerebellum. First we review our work on the oculomotor system, phylogenetically the oldest motor system, providing a blueprint for all other motor systems. We present a model of the cerebellum based on adaptive internal models capturing its  primary role of disturbance rejection. We briefly discuss a model of visuomotor adaptation, a system that operates in discrete time, yet follows the same design principles as the (continuous time) oculomotor system. Finally, we discuss our recent work on the optimal steady state regulation problem, a problem that captures the secondary role of the cerebellum to mediate long term adaptation of lower level brain processes in the brainstem and spinal cord. Prevailing mathematical themes are: adaptive control, output regulation, adaptive internal models, persistent excitation, and averaging analysis.

Problems of Systems Neuroscience and Control Theory

This work connects models for virus spread on networks with their equivalent neural network representations. Based on this connection, we propose a new neural network architecture, called Transmission Neural Networks (TransNNs) where activation functions are primarily associated with links and are allowed to have different activation levels. This connection also leads to the discovery and the derivation of three new activation functions with tunable or trainable parameters. We show that TransNNs with a single hidden layer and a fixed non-zero bias term are universal function approximators. Moreover, we establish threshold conditions for virus spread on networks where the dynamics are characterized by TransNNs. Finally, we present new derivations of continuous time epidemic models on networks based on TransNNs.
*This is joint work with Peter E. Caines

Transmission Neural Networks: From Virus Spread Model to Neural Networks

The linear threshold model, originally introduced by Granovetter [1], can be used to model the spread of technologies, innovations, and infectious diseases. According to this model, an individual would choose one option from {A,B} if a large enough proportion of the population has already done so. More specifically, each individual has a threshold _ _ [0, 1] and an initial choice and gets activated over a discrete-time sequence, called the activation sequences, to revise their decisions. If the decision of agent i at time t is denoted by di(t), his decision at time (t + 1) under the linear threshold model equals
(A xA(t) � _i
di(t+1)= B xA(t)<_ (1)
i
where xA(t) denotes the proportion of individuals who have chosen option A at t.
Given the initial decisions of the population, the activation sequence determines the dynamics of the population over time under the linear threshold update rule. The dynamics of the linear threshold model were studied in different contexts, but to the best of our knowledge, no study investigated the notion of identifiability for this model. That is, whether the agents� threshold distribution is taken as the model parameters, can be uniquely obtained from observed input and output. The answer facilitates future analysis on the predicting and controlling the decision dy- namics. The problem can be approached by using output distinguishability [2] as follows: a pair of threshold distributions (_1,_2) are said to be indistinguishable if two same-size populations, one with threshold distribution _1 and another with threshold distribution _2, exhibit the same total numbers of A-selected individuals for every activation sequence u and at every time t � 0 and starting from every same initial state x0. Otherwise, the pair is said to be distinguishable. A threshold distribution _1 is identifiable if the pair (_1,_2) is distinguishable for all _2 in the space of parameter where _ 1 _= _ 2.
For any given initial number of A-playing agents A(0), there exists an initial population state and an activation sequence such that the threshold distribution is identifiable under the linear threshold dynamics. We determine those initial states of the population for which the threshold distribution is identifiable. If _1 is the threshold distribution of a population and _2 is any other possible threshold distribution for the same-size population such that _1 _= _2, then there exists some individual i who has different thresholds in two populations. For any initial state that would reach to state xt, in which the individual i violates its threshold exactly in one population or it violates its threshold in both populations but has different choices in two populations, under some activation sequence for some time t � 0, the threshold distribution is identifiable.

Threshold identifiability in the linear threshold model

Using the Koopman operator, nonlinear systems can be
expressed as infinite-dimensional linear systems. Data-driven methods
can then be used to approximate a finite-dimensional Koopman operator,
which is particularly useful for system identification, control, and
state estimation tasks. However, approximating large Koopman operators
is numerically challenging, leading to unstable Koopman operators
being identified for otherwise stable systems. This talk will present
a selection of techniques to regularize the Koopman regression
problem, including a novel H-infinity norm regularizer. In particular,
how to re-pose the system identification problem in order to leverage
numerically efficient optimization tools, such as linear matrix
inequalities, will be presented.

Regularization Techniques in Koopman-based System Identification

The Koopman operator allows any nonlinear system to be rewritten as an infinite-dimensional linear system by viewing it in terms of an infinite set of lifting functions instead of a state vector. The main feature of this representation is its linearity, making it compatible with existing linear systems theory, including optimal control theory. A finite-dimensional approximation of the Koopman operator can be identified from experimental data by choosing a finite subset of lifting functions, applying them to the data, and solving a least squares problem in the lifted space.

Existing Koopman operator approximation methods are designed to identify open-loop systems. However, it is impractical or impossible to run experiments on some systems without a controller. Furthermore, a controller introduces correlations between the system's input and output, making some dynamics difficult to identify if the controller is neglected. The proposed research addresses this limitation in the current state-of-the-art by introducing a method to identify a Koopman model of the closed-loop system, and extract a Koopman model of the open-loop plant given knowledge of the controller. This is accomplished by leveraging the linearity of the Koopman representation of the system. The proposed approach widens the applicability of Koopman operator identification methods to a wider class of systems.

The effectiveness of the proposed closed-loop Koopman operator approximation method is demonstrated experimentally using a Harmonic Drive gearbox exhibiting nonlinear vibrations.

Closed-Loop Approximation of the Koopman Operator

Safe planning and control of multi-robot performing complex tasks has been a challenging problem. Signal Temporal Logic (STL) [1], a class of formal specification languages, offers an expressive and mathematically unambiguous way to represent complex tasks that have requirements in space, time, and even reactive require- ments across agents, e.g., robot 1 should visit a region in the time interval of 2 to 6 seconds from now and then stay there until robot 2 also visits the region. The problem of generating motion plans or controllers such that systems satisfy STL specifications is one that has been studied widely over the last decade [2].
Control and planning methods that offer guarantees on safety and mission satisfaction [3, 4] generally do not scale well [5]. On the other hand, more computationally tractable approaches do not offer much in terms of safety guarantees [6]. In this talk, I will present a family of robust and predictive multi-agent motion planning and control methods that overcome these limitations for a wide variety of task objectives, represented using Signal Temporal Logic (STL). The central idea behind these methods is to translate the problem of STL satisfaction into one of maximizing a smooth robustness measure associated with the specification [5]. For certain dynamical systems, e.g., multi-rotor unmanned aerial vehicles (UAVs), we can also utilize a concise representation of trajectories (e.g., min-jerk splines) and formulate a (possibly non-convex) predictive optimization problem that can be solved efficiently using off-the-shelf gradient based optimizers. Solving this optimization generates trajectories that satisfy the STL specificaiton. I also show how we can get desirable properties such as kinematic feasibility [7] and also synthesize controllers that can track these trajectories with bounded tracking error [8]. The problem can be solved in either a centralized or decentralized manner [9].
The performance and scalability of these methods will be demonstrated through multi-robot simulation studies and experiments on actual quadrotor aerial robots. Finally, I will discuss some challenges and future work in this area.

Predictive Motion Planning for Multi-Agent Systems with Signal Temporal Logic objectives

We show that the discounted cost, cost up to an exit time, and ergodic cost involving controlled non-degenerate diffusions are continuous on the space of stationary control policies when the policies are given a topology introduced by Borkar [V. S. Borkar, A topology for Markov controls, Applied Mathematics and Optimization 20 (1989), 55�62]. The same applies for finite horizon problems when the control policies are Markov and the topology is revised to include time also as a parameter. 

We then establish that finite action/piecewise constant stationary policies are dense in the space of stationary Markov policies under this topology and the same holds for continuous policies. Using the above mentioned continuity and denseness results we establish that finite action/piecewise constant policies approximate optimal stationary policies with arbitrary precision. 

This gives rise to the applicability of many numerical methods such as policy iteration and stochastic learning methods for discounted cost, cost up to an exit time, and ergodic cost optimal control problems in continuous-time. For the finite-horizon setup, we establish additionally near optimality of time-discretized policies by an analogous argument. 

As a further utility of the aforementioned approach, by showing additionally that continuous policies are dense in the space of stationary policies, we show that one can obtain a discrete-time Markov Decision Process whose solution (available via a rich collection of both analytical and simulation based methods) can be interpolated/extended for an original continuous-time problem under each of the criteria presented.

We thus present a unified and concise approach for approximations directly applicable under several commonly adopted cost criteria. 

Continuity of Cost in Borkar Control Topology and Implications on Discrete Space and Time Approximations for Controlled Diffusions under Several Criteria

The future of space exploration missions critically relies on cooperative networked robotic systems. In such systems, each individual robot uses its own states and local information from neighboring robots or environment to collaboratively execute a mission, whether it be manipulation, data collection, consensus, swarming, formation, etc. To ensure their long-term, intelligent autonomy, multi-robot systems must be equipped with global Guidance, Navigation, and Control (GNC) technologies that are robust against uncertainties, disturbances, and faults that could appear anywhere in the chain, from the robot to the network level.
In this talk, we focus on the network of autonomous Wheeled Mobile Robots (WMRs). At the individual robot level, we detail our findings on optimal robust output-tracking control of nonholonomic WMRs. For WMRs traversing rugged and soft terrains on other planets, inevitable violation of the nonholonomic constraints at the wheels (slip) significantly degrades the robots� performance and poses grand challenges in the design of GNC systems. We will review our recently developed mitigation strategies to regain output-tracking performance of WMRs subject to small slip velocities and to prevent the wheels from manifesting large slip (dynamic traction control). For reliable sharing of information among agents within a multi-robot system, we introduce our novel navigation module that corrects the odometry by estimating wheel slip using only proprioceptive sensors. Finally, the network level robustness is guaranteed by an innovative fault-tolerant formation control structure. With the formation robustly controlled, we will conclude the talk by reviewing our proposed methods for the guidance and control of manipulator systems onboard vehicles that could be deployed in ground, aerial or orbital planetary exploration missions.

Resilient Autonomy of Cooperative Multi-Robot Systems for Space Exploration

Applying reinforcement learning (RL) to partially observable Markov decision processes (POMDPs) is a challenging problem because decisions need to be made based on the entire history of observations and actions. However, in several scenarios, we may have state information available during the training phase only. We are interested in exploiting the availability of this state information during the training phase to efficiently learn a history-based policy using RL. Specifically, we consider actor-critic algorithms in RL to incorporate this paradigm. This involves using only history information for the actor, and history and state information for the critic. For this reason, it is referred to in the literature as an asymmetric actor-critic approach. Motivated by the recent success of using representation losses in RL for POMDPs, we derive similar theoretical results for the asymmetric actor-critic case and are interested in evaluating whether adding such auxiliary losses in experiments helps in performance. Towards this goal, we learn a representation for the history---called an approximate information state (AIS)---that we prove is approximately optimal to the degree of approximation in the model by certain specified metrics. Additionally, we provide an explanation on why the policy gradient using a state-history critic has superior performance compared to that of a history-only critic.

9th Meeting on System and Control Theory .css-1ruktgp{font-size:var(--chakra-fontSizes-xl);}@media screen and (min-width: 30em){.css-1ruktgp{font-size:var(--chakra-fontSizes-3xl);}}@media screen and (min-width: 48em){.css-1ruktgp{font-size:var(--chakra-fontSizes-4xl);}}May 3 - 5, 2023 @ Waterloo

9th Meeting on System and Control Theory
May 3 - 5, 2023 @ Waterloo