Ahmed E Kamal

Title Actor Critic Algorithms for Efficient Communications in IoT Systems

Two of the envisioned characteristics of future 6G networks are battery less operation, and using artificial intelligence to achieve optimal operation. Based on this vision, in this work we consider communications in IoT systems that harvests energy from ambient sources, and therefore does not require battery replenishing. Moreover, transmission decisions by IoT devices are generated by a reinforcement learning (RL) mechanism in order to maximize throughput. We present two RL approaches that mimic rational humans in the way of analyzing the available information and making decisions. The proposed algorithms are called selector-actor-critic (SAC) and tuner-actor-critic (TAC). They are obtained by modifying the well-known actor-critic (AC) algorithm. SAC consists of an actor, a critic, and a selector. The role of the selector is to determine the most promising action at the current state based on the last estimate from the critic. TAC is model based, and consists of a tuner, a model-learner, an actor, and a critic. After receiving the approximated value of the current state-action pair from the critic and the learned model from the model-learner, the tuner uses the Bellman equation to tune the value of the current state-action pair. This state-action pair is used by the actor to optimize the policy. The performance of the proposed algorithms are evaluated using numerical simulations and are compared to that of the AC algorithm to show the advantages of the proposed algorithms.


Ahmed E. Kamal is a professor and Director of Graduate Education in the Department of Electrical and Computer Engineering at Iowa State University in the USA. He received a B.Sc. (distinction with honors) and an M.Sc. both from Cairo University, Egypt, and an M.A.Sc. and a Ph.D. both from the University of Toronto, Canada, all in Electrical Engineering. He is a Fellow of the IEEE and a senior member of the Association of Computing Machinery. He was an IEEE Communications Society Distinguished Lecturer for 2013 and 2014. Kamal's research interests include cognitive radio networks, optical networks, wireless sensor networks, and performance evaluation. He received the 1993 IEE Hartree Premium for papers published in Computers and Control in IEE Proceedings, and the best paper awards of the IEEE Globecom Symposium on Ad Hoc and Sensors Networks Symposium in 2008 and 2018. He also received the 2016 Outstanding Technical Achievement Award from the Optical Networks Technical Committee of the IEEE Communications Society. Kamal chaired or co-chaired Technical Program Committees of several IEEE sponsored conferences including the Optical Networks and Systems Symposia of the IEEE Globecom 2007 and 2010, the Cognitive Radio and Networks Symposia of the IEEE Globecom 2012 and 2014, and the Access Systems and Networks track of the IEEE International Conference on Communications 2016. He was also the chair of the IEEE Communications Society Technical Committee on Transmission, Access and Optical Systems (TAOS) for 2015 and 2016. He serves or served on the editorial boards of a number of journals including IEEE Communications, the IEEE Communications Surveys and Tutorials, the Elsevier Computer Networks journal, the Elsevier Optical Switching and Networking journal and the Arabian Journal of Science and Engineering.