Efficient information dissemination is crucial for supporting critical operations across domains like disaster response, autonomous vehicles, and sensor networks. This paper introduces a Multi-Agent Reinforcement Learning (MARL) approach as a significant step forward in achieving more decentralized, efficient, and collaborative information dissemination. We propose a Partially Observable Stochastic Game (POSG) formulation for information dissemination empowering each agent to decide on message forwarding independently, based on the observation of their one-hop neighborhood. This constitutes a significant paradigm shift from heuristics currently employed in real-world broadcast protocols. Our novel approach harnesses Graph Convolutional Reinforcement Learning and Graph Attention Networks (GATs) with dynamic attention to capture essential network features. We propose two approaches, L-DyAN and HL-DyAN, which differ in terms of the information exchanged among agents. Our experimental results show that our trained policies outperform existing methods, including the state-of-the-art heuristic, in terms of network coverage as well as communication overhead on dynamic networks of varying density and behavior.
@misc{galliera2023learning,title={Collaborative Information Dissemination with Graph-based Multi-Agent Reinforcement Learning},author={Galliera, Raffaele and Venable, Kristen Brent and Bassani, Matteo and Suri, Niranjan},year={2024},month=feb,eprint={2308.16198},archiveprefix={arXiv},primaryclass={cs.LG},}
Deep Reinforcement Learning for Communication Networks
Raffaele Galliera
Feb 2024
Research abstract published at the 29th AAAI/SIGAI Doctoral Consortium.
This research explores optimizing communication tasks with (Multi-Agent) Reinforcement Learning (RL/MARL) in Point-to-Point and Group Communication (GC) networks. The study initially applied RL for Congestion Control in networks with dynamic link properties, yielding competitive results. Then, it focused on the challenge of effective message dissemination in GC networks, by framing a novel game-theoretic formulation and designing methods to solve the task based on MARL and Graph Convolution. Future research will deepen the exploration of MARL in GC. This will contribute to both academic knowledge and practical advancements in the next generation of communication protocols.
@misc{galliera2024aaaiDC,title={Deep Reinforcement Learning for Communication Networks},author={Galliera, Raffaele},year={2024},month=feb,}
2023
MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks
Raffaele Galliera, Alessandro Morelli, Roberto Fronteddu, and 1 more author
In NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium, May 2023
Presented at the AAAI 2023 Workshop on Reinforcement Learning Ready for Production
Fast and efficient transport protocols are the foundation of an increasingly distributed world. The burden of continuously delivering improved communication performance to support next-generation applications and services, combined with the increasing heterogeneity of systems and network technologies, has promoted the design of Congestion Control (CC) algorithms that perform well under specific environments. The challenge of designing a generic CC algorithm that can adapt to a broad range of scenarios is still an open research question. To tackle this challenge, we propose to apply a novel Reinforcement Learning (RL) approach. Our solution, MARLIN, uses the Soft Actor-Critic algorithm to maximize both entropy and return and models the learning process as an infinite-horizon task. We trained MARLIN on a real network with varying background traffic patterns to overcome the sim-to-real mismatch that researchers have encountered when applying RL to CC. We evaluated our solution on the task of file transfer and compared it to TCP Cubic. While further research is required, results have shown that MARLIN can achieve comparable results to TCP with little hyperparameter tuning, in a task significantly different from its training setting. Therefore, we believe that our work represents a promising first step towards building CC algorithms based on the maximum entropy RL framework.
@inproceedings{10154210,author={Galliera, Raffaele and Morelli, Alessandro and Fronteddu, Roberto and Suri, Niranjan},booktitle={NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium},title={MARLIN: Soft Actor-Critic based Reinforcement Learning for Congestion Control in Real Networks},year={2023},volume={},number={},pages={1-10},keywords={},doi={10.1109/NOMS56928.2023.10154210},issn={2374-9709},month=may,note={Presented at the AAAI 2023 Workshop on Reinforcement Learning Ready for Production},}
Learning to Sail Dynamic Networks: The MARLIN Reinforcement Learning Framework for Congestion Control in Tactical Environments
Raffaele Galliera, Mattia Zaccarini, Alessandro Morelli, and 4 more authors
In MILCOM 2023 - 2023 IEEE Military Communications Conference (MILCOM), May 2023
Conventional Congestion Control (CC) algorithms,such as TCP Cubic, struggle in tactical environments as they misinterpret packet loss and fluctuating network performance as congestion symptoms. Recent efforts, including our own MARLIN, have explored the use of Reinforcement Learning (RL) for CC, but they often fall short of generalization, particularly in competitive, unstable, and unforeseen scenarios. To address these challenges, this paper proposes an RL framework that leverages an accurate and parallelizable emulation environment to reenact the conditions of a tactical network. We also introduce refined RL formulation and performance evaluation methods tailored for agents operating in such intricate scenarios. We evaluate our RL learning framework by training a MARLIN agent in conditions replicating a bottleneck link transition between a Satellite Communication (SATCOM) and an UHF Wide Band (UHF) radio link. Finally, we compared its performance in file transfer tasks against Transmission Control Protocol (TCP) Cubic and the default strategy implemented in the Mockets tactical communication middleware. The results demonstrate that the MARLIN RL agent outperforms both TCP and Mockets under different perspectives and highlight the effectiveness of specialized RL solutions in optimizing CC for tactical network environments.
@inproceedings{galliera2023dynamic,author={Galliera, Raffaele and Zaccarini, Mattia and Morelli, Alessandro and Fronteddu, Roberto and Poltronieri, Filippo and Suri, Niranjan and Tortonesi, Mauro},booktitle={MILCOM 2023 - 2023 IEEE Military Communications Conference (MILCOM)},title={Learning to Sail Dynamic Networks: The MARLIN Reinforcement Learning Framework for Congestion Control in Tactical Environments},year={2023},volume={},number={},pages={424-429},doi={10.1109/MILCOM58377.2023.10356270},archiveprefix={arXiv},primaryclass={cs.LG},}
2022
Object Detection at the Edge: Off-the-shelf Deep Learning Capable Devices and Accelerators
Raffaele Galliera, and Niranjan Suri
Procedia Computer Science, Sep 2022
2022 International Conference on Military Communication and Information Systems (ICMCIS)
The advent of energy efficient, embedded Deep Learning accelerators have brought inference capabilities in conjunction with Internet of Things devices in a pervasive manner. Once limited to passive data-gathering, such devices are now able to actively take part in data processing operations and predictive tasks with acceptable performance. Shifting such computation to the edge allows the creation of interconnected environments able to achieve efficient and low powered inference capabilities at the edge, without being dependent on external services in the cloud. Despite significant recent advancements, the field of Edge-ML is still maturing. Therefore, it is important to develop a framework to evaluate the performance of off-the-shelf hardware and state-of-the-art Deep Learning models suited for low-powered devices. Such a framework can be applied to new devices and models as they become available in the future. This paper describes such an evaluation framework, as well as a broad study of different Edge-ML devices, with comparisons in terms of performance, capabilities, limitations, and possible applications with a focus on deploying state-of-the-art Object Detection models. The end objective is enabling the deployment of low-latency and independent decision making processes in both civilian and military contexts.
@article{GALLIERA2022239,title={Object Detection at the Edge: Off-the-shelf Deep Learning Capable Devices and Accelerators},journal={Procedia Computer Science},volume={205},pages={239-248},year={2022},month=sep,note={2022 International Conference on Military Communication and Information Systems (ICMCIS)},issn={1877-0509},doi={https://doi.org/10.1016/j.procs.2022.09.025},url={https://www.sciencedirect.com/science/article/pii/S1877050922008900},author={Galliera, Raffaele and Suri, Niranjan},keywords={IoT, Object Detection, EdgeML, Edge Computing, Deep Learning},}
A Machine Learning Approach to the Determination of Value of Information to Operators and Applications on the Tactical Edge
Jacques M. Perry, Raffaele Galliera, and Niranjan Suri
Procedia Computer Science, Sep 2022
2022 International Conference on Military Communication and Information Systems (ICMCIS)
Value of Information (VoI) is critical to the operators in the field and the backbone of a communications network. With proper determination of VoI the correct information can be routed to the operators that need it and not burden those who don’t. VoI is used to determine the relevance, importance, and utility of information objects (discrete elements of information) to an entity (ranging from an individual person to even a software component, such as an agent). The value is always dependent on the context of the receiving entity. In the military domain, context can be defined by the identity, role, current and future activity, current status, current and future location, and type of mission of an operator. Being able to determine the VoI allows systems to prioritize and filter information, thereby reducing information overload on the recipient, as well as network load on the communications infrastructure. The concept of VoI is not new, but has recently begun to be applied to military information systems.
@article{PERRY2022137,title={A Machine Learning Approach to the Determination of Value of Information to Operators and Applications on the Tactical Edge},journal={Procedia Computer Science},volume={205},pages={137-146},year={2022},month=sep,note={2022 International Conference on Military Communication and Information Systems (ICMCIS)},issn={1877-0509},doi={https://doi.org/10.1016/j.procs.2022.09.015},url={https://www.sciencedirect.com/science/article/pii/S1877050922008808},author={Perry, Jacques M. and Galliera, Raffaele and Suri, Niranjan},keywords={Value of Information, Information Prioritization, Information Filtering, Tactical Networks, Bandwidth Utilization},}
2021
Marine Vessel Tracking using a Monocular Camera
Tobias Jacob., Raffaele Galliera., Muddasar Ali., and 1 more author
In Proceedings of the 2nd International Conference on Deep Learning Theory and Applications - DeLTA, Jun 2021
@inproceedings{delta21,author={Jacob., Tobias and Galliera., Raffaele and Ali., Muddasar and Bagui., Sikha},title={Marine Vessel Tracking using a Monocular Camera},booktitle={Proceedings of the 2nd International Conference on Deep Learning Theory and Applications - DeLTA},year={2021},month=jun,pages={17-28},publisher={SciTePress},organization={INSTICC},doi={10.5220/0010516000170028},isbn={978-989-758-526-5},issn={2184-9277},}
An Introduction to Data Encryption and Future Trends in Lightweight Cryptography and Securing IoT Environments
Sikha Bagui, and Raffaele Galliera
Transactions on Engineering and Computing Sciences, Dec 2021