ISLAB/CAISR - User contributions [en]

“Learning to Preserve and Share Useful Behaviors Across Agents in Asynchronous RL via Meta-Learning”

2025-09-08T14:34:22Z

Alexander: Alexander moved page “Learning to Preserve and Share Useful Behaviors Across Agents in Asynchronous RL via Meta-Learning” to [[A Meta-Learning Approach for Preserving and Transferring Beneficial Behaviors in Asynchronous Multi-Agent Reinforceme...

#REDIRECT [[A Meta-Learning Approach for Preserving and Transferring Beneficial Behaviors in Asynchronous Multi-Agent Reinforcement Learning]]

A Meta-Learning Approach for Preserving and Transferring Beneficial Behaviors in Asynchronous Multi-Agent Reinforcement Learning

2025-09-08T14:34:22Z

{{StudentProjectTemplate
|Summary=Develop a meta-learning system that preserves beneficial behaviors discovered by individual agents and adapts them for transfer across a population in asynchronous reinforcement learning.
|Keywords=Meta-Learning, Asynchronous Reinforcement Learning, Behavior Preservation, Adaptive Aggregation, Multi-Agent Learning
|References=Mnih, V. et al. (2016). Asynchronous Methods for Deep Reinforcement Learning. ICML 2016.

Finn, C., Abbeel, P., & Levine, S. (2017). Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. ICML 2017.

Gupta, A. et al. (2018). Meta-Reinforcement Learning of Structured Exploration Strategies. NeurIPS 2018.

Qi, P. (2024). Model Aggregation Techniques in Federated Learning: A Comprehensive Survey. Future Generation Computer Systems, 139, 1-15

Wu, H. et al. (2024). Adaptive Multi-Agent Reinforcement Learning for Flexible Resource Management. Applied Energy, 374, 121-135

OpenAI Gym. https://www.gymlibrary.ml/
Ray RLLib. https://docs.ray.io/en/latest/rllib.html
PyTorch. https://pytorch.org/
|Prerequisites=Deep learning course. Good programming knowledge in Python. Good knowledge of ML and preferably Meta-learning/RL and OpenAI gym.
|Supervisor=Alexander Galozy,
|Level=Master
|Status=Open
}}
Asynchronous reinforcement learning (AsyncRL) allows multiple agents to explore environments in parallel, but standard aggregation methods risk diluting rare yet beneficial behaviors discovered by individual actors. This project proposes a meta-learning approach in which a meta-model dynamically adapts how each actor’s updates influence the global policy based on their state, novelty, and potential usefulness to other agents. The meta-model aims to quickly preserve advantageous behaviors while evaluating their transferability, improving learning efficiency, stability, and knowledge propagation across the agent population. Experiments will be conducted in benchmark environments such as CartPole, LunarLander, and Pong, comparing the meta-learning approach to standard aggregation baselines. The project investigates how population-level adaptive weighting can balance exploration and exploitation, effectively generalizing classical single-agent meta-RL approaches like First-Explore to multi-agent asynchronous learning. The outcomes include insights into behavior preservation, transferability, and scalable multi-agent learning.

Research Questions:

How can a meta-model preserve beneficial behaviors discovered by individual agents in asynchronous RL?

How can the meta-model evaluate and transfer these behaviors to improve other agents’ learning?

Does meta-learning-based adaptive aggregation improve population-level learning efficiency, stability, and sample efficiency compared to standard methods?

Outcomes:

Functional meta-model for adaptive aggregation of actor updates, preservation of rare but beneficial behaviors in the global policy and transfer of useful behaviors across agents

Benchmark evaluation on standard RL environments (CartPole, LunarLander, Pong)

Metrics: return, sample efficiency, stability, behavior retention, transfer success

Open-source, reproducible codebase

Complete thesis documentation with methodology, experiments, and analysis

Alexander Galozy

2025-09-08T14:29:20Z

Alexander:

{{Person
|Family Name=Galozy
|Given Name=Alexander
|Title=PostDoc
|Position=Ph.D
|Email=alexander.galozy@hh.se
|Image=Alexander_Galozy.jpg
|Country=Schweden
|Office=E503
}}
{{AssignProjects
|project=IMedA
}}
{{AssignSubjectAreas
|SubjectArea=Artificial Intelligence
}}
{{AssignSubjectAreas
|SubjectArea=Data Mining
}}
{{AssignSubjectAreas
|SubjectArea=Deep/Machine Learning
}}
{{AssignSubjectAreas
|SubjectArea=Reinforcement Learning
}}
{{AssignApplicationAreas
|ApplicationArea=Health Technology
}}
{{AssignApplicationAreas
|ApplicationArea=Robotics
}}
{{StaffTemplate
|head_image=Alexander_Galozy.jpg
}}

[[Category:Staff]]


{{ShowPerson}}
{{InsertProjects}}

A Meta-Learning Approach for Preserving and Transferring Beneficial Behaviors in Asynchronous Multi-Agent Reinforcement Learning

2025-09-08T14:22:14Z

Alexander: Created page with "{{StudentProjectTemplate |Summary=Develop a meta-learning system that preserves beneficial behaviors discovered by individual agents and adapts them for transfer across a popu..."

Adaptive Knowledge Aggregation in Asynchronous Reinforcement Learning

2025-09-08T13:26:16Z

Alexander: Created page with "{{StudentProjectTemplate |Summary=Develop an adaptive method to combine knowledge from multiple reinforcement learning agents more efficiently in asynchronous setups. |Keyword..."

{{StudentProjectTemplate
|Summary=Develop an adaptive method to combine knowledge from multiple reinforcement learning agents more efficiently in asynchronous setups.
|Keywords=Asynchronous Reinforcement Learning, Knowledge Aggregation, Adaptive Aggregation, Sample Efficiency, Robustness
|TimeFrame=Fall 2025
|References=Mnih, V. et al., Asynchronous Methods for Deep Reinforcement Learning. (2016)

Shen, H. et al., Towards Understanding Asynchronous Advantage Actor–Critic: Convergence and Linear Speedup. (2020)

Ma, J. et al., FedStaleWeight: Buffered Asynchronous Federated Learning with Fair Aggregation via Staleness Reweighting. (2024)

Wu, Y. et al., Uncertainty Weighted Actor–Critic for Offline Reinforcement Learning. (2021)

Kumar, A. et al., Adaptive aggregation for RL in average reward MDPs. (2012)

Littlestone, N. & Warmuth, M., The Weighted Majority Algorithm. (1994)
|Prerequisites=Deep Reinforcement Learning course. Good knowledge of Machine Learning and preferably Reinforcement learning. Good programming skills - Required for the implementation of investigated methods.
|Supervisor=Alexander Galozy
|Level=Master
|Status=Open
}}
Reinforcement learning (RL) can achieve impressive results but often requires long training times and substantial computational resources. Asynchronous reinforcement learning (AsyncRL) improves efficiency by running many actors in parallel, yet current methods aggregate updates with simple strategies such as parameter averaging or replay sharing, which fail to consider the quality, staleness, or diversity of contributions. This project will investigate how to design better aggregation mechanisms by implementing and evaluating several baselines (parameter averaging, replay sharing, ensemble voting) and comparing them to a novel adaptive aggregator that weights actor updates according to their confidence and freshness. Using standard benchmarks such as CartPole, LunarLander, and Pong, the project aims to determine whether adaptive aggregation improves learning speed, robustness, and communication efficiency in AsyncRL

Research Questions:

1. How do existing aggregation strategies perform in asynchronous reinforcement learning?

2. Can an adaptive aggregation method that accounts for update quality and staleness improve performance over standard methods?

3. What are the trade-offs between efficiency, stability, and communication cost for different aggregation approaches?

Expected Outcomes

1. Working AsyncRL system with multiple aggregation strategies.

2. A novel adaptive aggregator that is systematically evaluated.

3. Thesis report with results, analysis, and discussion of trade-offs.

4. Open-source code for reproducibility.

Alexander Galozy

2019-09-30T13:45:08Z

Alexander:

{{Person
|Family Name=Galozy
|Given Name=Alexander
|Title=M. Sc., Dipl.-Ing(FH)
|Position=Ph.D. Student
|Email=alexander.galozy@hh.se
|Image=Alexander_Galozy.jpg
|Country=Schweden
|Office=E503
}}
{{AssignProjects
|project=IMedA
}}
{{AssignSubjectAreas
|SubjectArea=Artificial Intelligence
}}
{{AssignSubjectAreas
|SubjectArea=Data Mining
}}
{{AssignSubjectAreas
|SubjectArea=Deep/Machine Learning
}}
{{AssignSubjectAreas
|SubjectArea=Reinforcement Learning
}}
{{AssignApplicationAreas
|ApplicationArea=Health Technology
}}
{{AssignApplicationAreas
|ApplicationArea=Robotics
}}
{{StaffTemplate
|head_image=Alexander_Galozy.jpg
}}

[[Category:Staff]]


{{ShowPerson}}
{{InsertProjects}}

Alexander Galozy

2019-09-30T13:43:34Z

Alexander:

{{Person
|Family Name=Galozy
|Given Name=Alexander
|Title=M. Sc., Dipl.-Ing(FH)
|Position=Ph.D. Student
|Email=alexander.galozy@hh.se
|Image=Alexander_Galozy.jpg
|Country=Schweden
|Office=E503
}}
{{AssignProjects
|project=IMedA
}}
{{AssignSubjectAreas
|SubjectArea=AI
}}
{{AssignSubjectAreas
|SubjectArea=Data Mining
}}
{{AssignSubjectAreas
|SubjectArea=Deep/Machine Learning
}}
{{AssignSubjectAreas
|SubjectArea=Reinforcement Learning
}}
{{AssignApplicationAreas
|ApplicationArea=Health Technology
}}
{{AssignApplicationAreas
|ApplicationArea=Robotics
}}
{{StaffTemplate
|head_image=Alexander_Galozy.jpg
}}

[[Category:Staff]]


{{ShowPerson}}
{{InsertProjects}}

Alexander Galozy

2019-09-30T13:40:38Z

Alexander:

{{Person
|Family Name=Galozy
|Given Name=Alexander
|Title=M. Sc., Dipl.-Ing(FH)
|Position=Ph.D. Student
|Email=alexander.galozy@hh.se
|Image=Head-unknown.jpg
|Country=Schweden
|Office=E503
}}
{{AssignProjects
|project=IMedA
}}
{{AssignSubjectAreas
|SubjectArea=AI
}}
{{AssignSubjectAreas
|SubjectArea=Data Mining
}}
{{AssignSubjectAreas
|SubjectArea=Deep/Machine Learning
}}
{{AssignSubjectAreas
|SubjectArea=Reinforcement Learning
}}
{{AssignApplicationAreas
|ApplicationArea=Health Technology
}}
{{AssignApplicationAreas
|ApplicationArea=Robotics
}}
{{StaffTemplate
|head_image=Alexander_Galozy.jpg
}}

[[Category:Staff]]


{{ShowPerson}}
{{InsertProjects}}

File:Alexander Galozy.jpg

2019-09-30T13:39:42Z

Alexander:

File:Alexander Galozy.jpeg

2019-09-30T13:38:36Z

Alexander:

Alexander Galozy

2019-09-30T13:34:29Z

Alexander:

{{Person
|Family Name=Galozy
|Given Name=Alexander
|Title=M. Sc., Dipl.-Ing(FH)
|Position=Ph.D. Student
|Email=alexander.galozy@hh.se
|Image=Head-unknown.jpg
|Country=Schweden
|Office=E503
}}
{{AssignProjects
|project=IMedA
}}
{{AssignSubjectAreas
|SubjectArea=AI
}}
{{AssignSubjectAreas
|SubjectArea=Data Mining
}}
{{AssignSubjectAreas
|SubjectArea=Deep/Machine Learning
}}
{{AssignSubjectAreas
|SubjectArea=Reinforcement Learning
}}
{{AssignApplicationAreas
|ApplicationArea=Health Technology
}}
{{AssignApplicationAreas
|ApplicationArea=Robotics
}}
{{StaffTemplate
|head_image=Head-unknown.jpg
}}

[[Category:Staff]]


{{ShowPerson}}
{{InsertProjects}}

Alexander Galozy

2019-09-30T13:32:38Z

Alexander:

{{Person
|Family Name=Galozy
|Given Name=Alexander
|Title=M. Sc., Dipl.-Ing(FH)
|Position=PhD Student
|Email=alexander.galozy@hh.se
|Image=Head-unknown.jpg
|Street Address=Södra Vägen 9
|Country=Schweden
}}
{{AssignProjects
|project=IMedA
}}
{{AssignSubjectAreas
|SubjectArea=AI
}}
{{AssignSubjectAreas
|SubjectArea=Data Mining
}}
{{AssignSubjectAreas
|SubjectArea=Learning Systems
}}
{{AssignSubjectAreas
|SubjectArea=Mechatronics
}}
{{AssignApplicationAreas
|ApplicationArea=Health Technology
}}
{{AssignApplicationAreas
|ApplicationArea=Robotics
}}
{{StaffTemplate
|head_image=Head-unknown.jpg
}}

[[Category:Staff]]


{{ShowPerson}}
{{InsertProjects}}

Improving MEDication Adherence through Person Centered Care and Adaptive Interventions

2018-10-19T10:33:53Z

Alexander:

{{StudentProjectTemplate
|Summary=Improving MEDication Adherence through Person Centered Care and Adaptive Interventions
|Timeframe Winter2018 /Spring 2019
|Supervisor=Alexander Galozy, Sławomir Nowaczyk,
|References= Definitions, variants, and causes of nonadherence with medication: a challenge for tailored interventions:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3711878/

Predictors of Medication Adherence Using a Multidimensional Adherence Model in Patients with Heart Failure:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2603618/

A machine learning approach for medication adherence monitoring using body-worn sensors:
https://ieeexplore.ieee.org/document/7459425

Machine Learning Classification of Medication Adherence in Patients with Movement Disorders Using Non-Wearable Sensors:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5729888/

|Prerequisites= Courses Artificial Intelligence, learning systems. Good knowledge in applied data science/machine learning. Ability to implement state-of-the-art machine learning algorithms in a programming environment of your choice.
|Level=Master
|Status=Open
}}

Medications nonadherence is major public health concern leading to increased mortality, morbidity and billions in increased costs for out-of-plan treatment. Doctors and health care providers a like, have a keen interest to alleviate this problem by finding out the reasons for medication non-adherence and designing appropriate interventions with a high likelihood of success. Patient behavior is extraordinary complex and reasons for patients to not taking their medications are varied and individual. This thesis will address the problem of classifying the reason for secondary medication non-adherence utilizing smart home sensor data, data from electronic health records and questionnaires.

The student is expected to perform literature review on the subject of secondary medication adherence, smart home environments and machine learning on medical data (especially electronic health records). In the medical domain, it is quite important to explain the reasons for a particular classification. The student should analyze the tradeoffs between better classification performance and interpretability.

Improving MEDication Adherence through Person Centered Care and Adaptive Interventions

2018-10-19T10:33:20Z

Alexander:

{{StudentProjectTemplate
|Summary=Improving MEDication Adherence through Person Centered Care and Adaptive Interventions
|Timeframe Winter2018 /Spring 2019
|Supervisor=Alexander Galozy, Sławomir Nowaczyk,
|References= Definitions, variants, and causes of nonadherence with medication: a challenge for tailored interventions:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3711878/

Predictors of Medication Adherence Using a Multidimensional Adherence Model in Patients with Heart Failure
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2603618/

A machine learning approach for medication adherence monitoring using body-worn sensors
https://ieeexplore.ieee.org/document/7459425

Machine Learning Classification of Medication Adherence in Patients with Movement Disorders Using Non-Wearable Sensors:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5729888/

|Prerequisites= Courses Artificial Intelligence, learning systems. Good knowledge in applied data science/machine learning. Ability to implement state-of-the-art machine learning algorithms in a programming environment of your choice.
|Level=Master
|Status=Open
}}

Medications nonadherence is major public health concern leading to increased mortality, morbidity and billions in increased costs for out-of-plan treatment. Doctors and health care providers a like, have a keen interest to alleviate this problem by finding out the reasons for medication non-adherence and designing appropriate interventions with a high likelihood of success. Patient behavior is extraordinary complex and reasons for patients to not taking their medications are varied and individual. This thesis will address the problem of classifying the reason for secondary medication non-adherence utilizing smart home sensor data, data from electronic health records and questionnaires.

The student is expected to perform literature review on the subject of secondary medication adherence, smart home environments and machine learning on medical data (especially electronic health records). In the medical domain, it is quite important to explain the reasons for a particular classification. The student should analyze the tradeoffs between better classification performance and interpretability.

Improving MEDication Adherence through Person Centered Care and Adaptive Interventions

2018-10-19T10:32:07Z

Alexander:

{{StudentProjectTemplate
|Summary=Improving MEDication Adherence through Person Centered Care and Adaptive Interventions
|Timeframe Winter2018 /Spring 2019
|Supervisor=Alexander Galozy, Sławomir Nowaczyk,
|References= Definitions, variants, and causes of nonadherence with medication: a challenge for tailored interventions:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3711878/
Predictors of Medication Adherence Using a Multidimensional Adherence Model in Patients with Heart Failure:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2603618/
A machine learning approach for medication adherence monitoring using body-worn sensors
https://ieeexplore.ieee.org/document/7459425
Machine Learning Classification of Medication Adherence in Patients with Movement Disorders Using Non-Wearable Sensors:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5729888/
|Prerequisites= Courses Artificial Intelligence, learning systems. Good knowledge in applied data science/machine learning. Ability to implement state-of-the-art machine learning algorithms in a programming environment of your choice.
|Level=Master
|Status=Open
}}

Medications nonadherence is major public health concern leading to increased mortality, morbidity and billions in increased costs for out-of-plan treatment. Doctors and health care providers a like, have a keen interest to alleviate this problem by finding out the reasons for medication non-adherence and designing appropriate interventions with a high likelihood of success. Patient behavior is extraordinary complex and reasons for patients to not taking their medications are varied and individual. This thesis will address the problem of classifying the reason for secondary medication non-adherence utilizing smart home sensor data, data from electronic health records and questionnaires.

The student is expected to perform literature review on the subject of secondary medication adherence, smart home environments and machine learning on medical data (especially electronic health records). In the medical domain, it is quite important to explain the reasons for a particular classification. The student should analyze the tradeoffs between better classification performance and interpretability.

Alexander Galozy

2018-07-06T19:45:17Z

Alexander:

{{Person
|Family Name=Galozy
|Given Name=Alexander
|Title=M. Sc.
|Position=PhD Student
|Email=alexander.galozy@hh.se
|Image=Head-unknown.jpg
|Street Address=Södra Vägen 9
|Country=Schweden
}}
{{AssignProjects
|project=IMedA
}}
{{AssignSubjectAreas
|SubjectArea=AI
}}
{{AssignSubjectAreas
|SubjectArea=Data Mining
}}
{{AssignSubjectAreas
|SubjectArea=Learning Systems
}}
{{AssignSubjectAreas
|SubjectArea=Mechatronics
}}
{{AssignApplicationAreas
|ApplicationArea=Health Technology
}}
{{AssignApplicationAreas
|ApplicationArea=Robotics
}}
{{StaffTemplate
|head_image=Head-unknown.jpg
}}

[[Category:Staff]]


{{ShowPerson}}
{{InsertProjects}}

Alexander Galozy

2018-07-02T11:10:56Z

Alexander: Created page with "{{Person |Family Name=Galozy |Given Name=Alexander |Title=M. Sc. |Position=PhD Student |Email=alex_galozy@yahoo.de |Street Address=Södra Vägen 9 |Country=Schweden }} {{Assig..."

{{Person
|Family Name=Galozy
|Given Name=Alexander
|Title=M. Sc.
|Position=PhD Student
|Email=alex_galozy@yahoo.de
|Street Address=Södra Vägen 9
|Country=Schweden
}}
{{AssignProjects
|project=IMedA
}}
{{AssignSubjectAreas
|SubjectArea=AI
}}
{{AssignSubjectAreas
|SubjectArea=Data Mining
}}
{{AssignSubjectAreas
|SubjectArea=Learning Systems
}}
{{AssignSubjectAreas
|SubjectArea=Mechatronics
}}
{{AssignApplicationAreas
|ApplicationArea=Health Technology
}}
{{AssignApplicationAreas
|ApplicationArea=Robotics
}}
[[Category:Staff]]


{{ShowPerson}}
{{InsertProjects}}