Q-learning Based Ad-Hoc Network Formation Strategy for Wireless Nodes with Random Mobility Models

Since a decentralized network can be formed and maintained by multiple nodes without direct control from network infrastructure, it can be deployed in a wide range of applications. As more nodes need to be taken into account, however, computational complexity required for an optimal network significantly increases. It is more challenging to maintain higher network throughput because of node mobility. For a mobile ad-hoc network that consists of battery-powered mobile devices, in particular, it is even more important to consider power consumption required for node mobility and data transmission. In this paper, we propose an algorithm for a mobile ad-hoc network formation based on Q-learning. The proposed algorithm enables relay nodes with random movement to learn how to manage transmission power for minimizing total power consumption and maximizing network throughput without a central coordination. We confirm that the proposed solution can achieve high network throughput while minimizing power consumption regardless of network size and mobility patterns.