ºÚÁÏÍø

Three Aalto University researchers have received Starting Grants from the European Research Council. The projects run for five years. 

New mathematical foundations for continual, safe reinforcement learning 

Assistant Professor Dominik Baumann from the Department of Electrical Engineering and Automation will lead a project focusing on reinforcement learning (RL), a subfield of artificial intelligence that has driven several recent breakthroughs. For example, in 2016, an RL-based system (AlphaGo) defeated the world champion in the game Go, a milestone once thought beyond the reach of artificial agents. RL methods also play a key role in training large language models like ChatGPT.  

At a high level, in RL an artificial agent repeatedly interacts with an environment: it observes the current state, selects an action, and receives a reward indicating the quality of that action.  

’Over time, through trial and error, the agent learns policies that produce increasingly effective behavior. It is much like training a dog with consistent feedback and rewards,’ Baumann says.  

In the project, Baumann and his team focus on two things. First, standard RL typically optimizes the expected reward, which is an average over many trials and can be overly influenced by rare, high-reward outliers. This can favor risky actions that occasionally succeed but often fail. In safety-critical settings such as autonomous driving, such risk-seeking behavior is undesirable; a cautious policy that reliably avoids accidents is preferable. To this end, they aim to move beyond expected-value-based objectives and instead optimize the individual agent’s long-term behavior.   

Second, RL agents are typically trained for a fixed amount of time and then deployed, which works well for a game like Go, where the rules don’t change. However, when agents are deployed in the real world, they encounter situations not seen during training.  

’Using the autonomous driving example, the vehicle will face novel scenarios in the real world. We therefore seek to enable agents to continue learning during deployment, doing so efficiently and with explicit safety guarantees,’ Baumann says. 

Project name: Ergodicity-aware continual reinforcement learning (ErgCRL). 

More durable materials starting from micrometers 

Academy Research Fellow Tero Mäkinen studies how damage happens to materials like metals and biomaterials, and how that damage spreads starting from micrometers to larger cracks. 

‘The goal is to find out the physical processes involved in damage and, based on that, develop design principles for more durable materials’, Mäkinen says. 

The results could help with designing materials that are structurally more able to mitigate spread and less prone to break in the first place. 

‘Fatigue, creep and fractures limit the life spans of nearly all structures and materials. A better understanding of the mechanisms that lead to failure will help in designing new materials that can withstand those things for a longer time.’ 

Project name: Design rules for failure-resistant materials (FORTIFY). 
 

New electrolyte design for decoupled ion transport &²Ô²ú²õ±è;

As battery-powered devices become more widespread, more efficient ways of storing energy are needed. Researchers are now seeking new electrolyte materials that could enable the manufacture of high-energy-density batteries. An electrolyte transports ions between electrodes and must be both mechanically robust and compatible with the electrodes. Combining these properties in a single material is challenging. &²Ô²ú²õ±è;
 &²Ô²ú²õ±è;
The ERC project led by postdoctoral researcher Owies Wani aims to develop a new type of structure for polymer electrolytes, in which ion transport and the material’s mechanical properties can be controlled more independently than before. &²Ô²ú²õ±è;

‘The results could contribute to the development of safer, higher-energy-density batteries, particularly lithium metal batteries and other post-lithium-ion technologies,’ Wani says. &²Ô²ú²õ±è;

The project uses hybrid liquid-crystal electrolytes that combine the benefits of molecular and colloidal liquid crystals. &²Ô²ú²õ±è;

‘The electrolyte material will be designed to form nanoscale molecular pathways along which ions can move efficiently. Colloidal components will reinforce the structure and ensure that ions can travel continuously through the material,’ Wani explains. &²Ô²ú²õ±è;

Project name: Decoupling Ionic Transport from Mechanical Properties using Hybrid Liquid Crystal Electrolytes (DECOUPLE). &²Ô²ú²õ±è;

The European Research Council (ERC) funding is awarded to leading researchers for pioneering work at the frontiers of science. ERC Starting Grants are designed to support talented early-career scientists (2–7 years since completion of PhD). Link to the ERC press release: