CEE Seminar Series: Computational sequential decision-making under uncertainty at a human level or above, with applications in engineering

The recording for this seminar is now available.

Abstract: This talk will initiate with a high-level, quick overview of some of the recent research efforts of Papakonstantinou’s group, with an emphasis on developed methodologies to tackle and quantify stochasticity, as this naturally emerges in a diverse range of engineering problems and applications. Some selected topics will include spatial stochastic variability of reinforced concrete deterioration, stochastic simulation of earthquake ground motions, online inference and system assessment and updating based on monitoring data, efficient quantification of probability of rare events in high-dimensional spaces, and fragility assessment based on hazard events data, among others.

The main body of the talk will then focus on a seamless integration of stochastic models and data for computational sequential decision-making under uncertainty, able to directly and autonomously offer optimal actions to decision-makers/agents, comparable to or above human level capabilities. At the core of every engineering problem lies a decision-making quest, either directly or indirectly. Yet, despite significant progress in probabilistic methods and techniques, the actual decision-making process is still largely dependent on the static and rather limited traditional cost-benefit analysis framework, and dedicated rigorous computational methodologies for engineering decisions under uncertainty are practically elusive. As shown in this presentation, challenging sequential decision-making problems in nonstationary dynamic environments can be efficiently formulated along the premises of optimal stochastic control, through Markov Decision Processes (MDPs), Partially Observable Markov Decision Processes (POMDPs), and mixed approaches thereof. However, optimal adaptive planning for large systems with multiple components is computationally hard and severely suffers from the curse of dimensionality. New developments on Deep Reinforcement Learning (DRL) methods and their capacity of addressing this problem are discussed, with emphasis on multi-agent DRL formulations and novel algorithmic schemes developed by Papakonstantinou and co-workers, specifically tailored to the needs of large systems and able to solve otherwise intractable problems with immense state and action spaces. DRL relations to Artificial Intelligence and Machine Learning are also explained through their fundamental mathematical attributes, underlying computational aspects, and connections to engineering. Numerous ongoing efforts along these lines are shown, from quantifying the Value of (monitoring) Information (i.e., is it worth investing in acquiring data?), to centralized/decentralized infrastructure management under risk and budget constraints, emergency response of cooperating agents, automated vehicles and robotics, wildfire prevention, and autonomous exploration, path planning and navigation.

At a practical level, this talk with hence inform the audience about stochastic methods and various forms of probabilistic analysis and inference, as well as about recent progress in relation to the science of autonomy, as this applies in engineering and beyond.

Biography: Kostas Papakonstantinou is an assistant professor of civil engineering at Penn State. He obtained a five-year diploma in civil engineering and a master's degree in structural engineering from the National Technical University of Athens, and master's and doctoral degrees in civil engineering at the University of California, Irvine. Prior to joining Penn State, he was an associate research scientist at the Department of Civil Engineering and Engineering Mechanics at Columbia University. Papakonstantinou’s work focuses on probabilistic analysis and stochastic mechanics, decision-making under uncertainty, machine learning, optimization/inverse methods, and their integration with computational structural mechanics and engineering applications. Papakonstantinou has received various awards for his work in these areas, including a National Science Foundation CAREER award, a fellowship award by the Institute of International Education and the Fulbright Foundation, an editor’s choice paper award from the Journal of Engineering Mechanics, and several student awards for joint work and co-authored papers.

Additional Information:

Registration is required prior to the event.