ENHANCEMENTS OF FUZZY Q-LEARNING ALGORITHM
DOI:
https://doi.org/10.7494/csci.2005.7.4.77Keywords:
fuzzy models, reinforcement learning, Q-Learning, automatic generation of fuzzy modelsAbstract
Fuzzy Q-Learning algorithm combines reinforcement learning techniques with fuzzy modelling. It provides a flexible solution for automatic discovery of rules for fuzzy systems inthe process of reinforcement learning. In this paper we propose several enhancements tothe original algorithm to make it more performant and more suitable for problems withcontinuous-input continuous-output space. Presented improvements involve generalizationof the set of possible rule conclusions. The aim is not only to automatically discover anappropriate rule-conclusions assignment, but also to automatically define the actual conclusions set given the all possible rules conclusions. To improve algorithm performance whendealing with environments with inertness, a special rule selection policy is proposed.Downloads
References
Bonarini A.: Evolutionary Learning of Fuzzy rules: competition and cooperation. Fuzzy Modelling: Paradigms and Practice, Kluwer Academic Press, Norwell, MA (1997), 265–284
Glorennec P. Y., Jouffe L.: Fuzzy Q-learning. Proc. of the Sixth International Conference on Fuzzy Systems 1997
Glorennec P. Y.: Reinforcement Learning: an Overview. ESIT 2000, Aachen, Germany 2000
Jang J. R.: ANFIS: Adaptive-Neural-Based Fuzzy Inference System. IEEE Trans. on SMC, vol. 23 1993, 665–685
Jouffe L.: Fuzzy inference system learning by reinforcement methods, IEEE Trans. Syst. Man. Cybernet. Part C: Appl. Rev., 28, 3, 1998
Klawonn F., Kruse R.: Constructing a fuzzy controller from data. Fuzzy Sets and Systems, 85, 1997, 177–193
Mandani E. H.: Applications of fuzzy algorithms for simple dynamic plants. Proc. IEE, 121, 1974, 1585–1588
Sutton R. S.: Generalization in reinforcement learning: Successful examples using sparse coarse coding. Advances in Neural Information Processing Systems, vol. 8. MIT Press 1996
Takagi H., Sugeno M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on System, Man, and Cybernetics, 15(1) 1985, 116–132.
Thrun S. B.: Efficient exploration in reinforcement learning, Technical Report CMU-CS-92-102, School of Computer Science, Carnegie Mellon University 1992
Watkins C. J. C. H.: Learning from Delayed Rewards. King’s College, Cambridge 1989 (PhD Thesis)
