Abstract
Learning in multi-agent systems, especially with adversarial behavior being exhibited, is difficult and challenging. The learning within these complicated environments is often muddied by the multitudinous conflicting or poorly correlated data coming from the multiple agents and their diverse goals. This should not be compared against well-known flocking-type behaviors where each agent has the same policy; rather, in our scenario each agent may have their own policy, sets of behaviors, or overall group strategy. Most learning algorithms will observe the actions of the agents and inform their algorithm which seeks to form the models. When these actions are consistent a reasonable model can be formed; however, eSense was designed to work even when observing complicated and highly-interactive must-agent behavior. eSense provides a powerful yet simplistic reinforcement learning algorithm that employs model-based behavior across multiple learning layers. These independent layers split the learning objectives across multiple layers, avoiding the learning-confusion common in many multi-agent systems. We examine a multi-agent predator-prey biomimetic sensing environment that simulates such coordinated and adversarial behaviors across multiple goals. This work could also be applied to theater wide autonomous vehicle coordination, such as that of the hierarchical command and control of autonomous drones and ground vehicles.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ammari Habib, T.B., Garnier, J.: Modeling active electrolocation in weakly electric fish. SIAM J. Imaging Sci. 6(1), 285–321 (2013)
Batista, G.E.A.P.A., Prati, R.C., Monard, M.C.: A study of the behavior of several methods for balancing machine learning training data. SIGKDD Explor. Newslett. 6(1), 20–29 (2004). https://doi.org/10.1145/1007730.1007735
Boyer, F., et al.: Model for a sensor inspired by electric fish. IEEE Trans. Robot. 28(2), 492–505 (2012)
Coggan, M.: Exploration and Exploitation in Reinforcement Learning, 3(3), p. 1448. CRA-W DMP Project at McGill University, Scholarpedia (2008)
Franklin, D.M.: Strategy inference in stochastic games using belief networks comprised of probabilistic graphical models. In: Proceedings of FLAIRS (2015)
Franklin, D.M., Martin, D.: eSense: BioMimetic modeling of echolocation and electrolocation using homeostatic dual-layered reinforcement learning. Proc. ACM SE 2016 (2016)
Freedman, H., Waltman, P.: Persistence in models of three interacting predator-prey populations. Math. Biosci. 68(2), 213–231 (1984)
Hopkins, C.D.: Electroreception: Passive Electrolocation and the Sensory Guidance of Oriented Behavior. Springer, New York (2005)
Hussein, S.: Predator-prey modeling. Undergraduate J. Math. Model.: One + Two 3(1), 32 (2010)
Lima, S.L.: Putting predators back into behavioral predator-prey interactions. Trends Ecol. Evol. 17(2), 70–75 (2002)
Shieh, K.T., et al.: Short-range orientation in electric fish: an experimental study of passive electrolocation. J. Exp. Biol. 199(11), 2383–2393 (1996)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, Chaps. 4, 5, 8 (1998)
Taylor, M.E., Whiteson, S., Stone, P.: Comparing evolutionary and temporal difference methods in a reinforcement learning domain. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation. ACM (2006)
Woergoetter, F., Porr, B.: Reinforcement learning. Scholarpedia 3(3), 1448 (2008)
Yi, F., Wei, J., Shi, J.: Bifurcation and spatiotemporal patterns in a homogeneous diffusive predatorprey system. J. Differ. Equ. 246(5), 1944–1977 (2009). https://doi.org/10.1016/j.jde.2008.10.024, http://www.sciencedirect.com/science/article/pii/S0022039608004373
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Michael Franklin, D., Martin, D. (2020). eSense 2.0: Modeling Multi-agent Biomimetic Predation with Multi-layered Reinforcement Learning. In: Arai, K., Bhatia, R. (eds) Advances in Information and Communication. FICC 2019. Lecture Notes in Networks and Systems, vol 70. Springer, Cham. https://doi.org/10.1007/978-3-030-12385-7_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-12385-7_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-12384-0
Online ISBN: 978-3-030-12385-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)