Study on Using Reinforcement Learning for the Monotone Boolean Reconstruction
Abstract
This paper investigates the feasibility of using reinforcement learning to solve combinatorial optimization problems, in particular, the problem of query-based monotone Boolean function reconstruction. The monotone Boolean function reconstruction problem is a typical combinatorial problem that reconstructs the function unambiguously with a minimum number of queries about the value of the function at the defined points, based on the monotonicity of the function. The Shannon complexity of the problem is of the order of 2^n/sqrt(n), and the solution algorithm relies on complex constructions, which also add complexity in the form of memory and time. Additionally, there are problems of partial reconstruction, e.g., in the mining of associative rules, which do not fit into the developed solution formats. This necessitates exploring heuristic domains to attract additional resources to solve the problem. To this end, all elements of reinforcement learning - environment, agent, policy, etc. - are designed, and both exact and approximate algorithms are given to perform the necessary structural data transformations, as well as to calculate the reward, the value, and other operational data of the algorithm. The focal point of the considerations is a subclass of monotone Boolean functions related to the well-known shadow minimization theorem of layer-by-layer characterized functions. Preliminary experiments have been started and they require follow-up intensive actions.References
L. Aslanyan, The discrete isoperimetry problem and related extremal problems for discrete spaces, Problemy kibernetiki, 36:85–128 (1979).
L. Aslanyan and H. Sahakyan. The splitting technique in monotone recognition. Discrete Applied Mathematics, 216:502–512, 2017.
L. Aslanyan, H. Sahakyan, V. Romanov, G. Da Costa, and R. Kacimi. Large network target coverage protocols. In 2019 Computer Science and Information Technologies (CSIT), pages 57–64. IEEE, 2019.
Mohammad Azad, Igor Chikalov, Shahid Hussain, Mikhail Moshkov, and Beata Zielosko, Decision Trees with Hypotheses for Recognition of Monotone Boolean Functions and for Sorting, arXiv 2203.08894v1 [cs.CC]. Book series “Synthesis Lectures on Intelligent Technologies”.
R. Bellman, A Markovian decision process. Indiana Univ. Math. J., 6:679–684, 1957. ISSN 0022-2518.
A. Blum. Learning a function of r relevant variables (open problem). In Proceedings of the 16th Annual Conference on Learning Theory and 7th Kernel Workshop, pages 731-733, 2003.
E. Boros, P. L. Hammer, T. Ibaraki and K. Kawakami, Identifying 2-monotonic positive Boolean functions in polynomial time, ISA’91, Algorithms, edited by W. L. Hsu and R. C. T. Lee, Springer Lecture Notes in Computer Science, 557 (1991) 104-115.
Bshouty, Nader H., and Christino Tamon, On the Fourier spectrum of monotone functions. Journal of the ACM (JACM)43, no. 4 (1996): 747-770.
R. Dedekind, Über Zerlegungen von Zahlen durch ihre größten gemeinsamen Teiler, Festschrift Hoch. Braunschweig u. ges. Werke, II (1897), 103-148.
Peter Frankl and Gyula OH Katona. On strengthenings of the intersecting shadow theorem. Journal of Combinatorial Theory, Series A, 184:105510, 2021.
D. N. Gainanov, On one criterion of the optimality of an algorithm for evaluating monotonic Boolean functions, Computational Mathematics and Mathematical Physics, 24 (1984), 176–181.
Mihály Gara, Tamás Sámuel Tasi, and Péter Balázs, Machine Learning as a Preprocessing Phase in Discrete Tomography, Lecture Notes in Computer Science book series (LNIP, volume 7346).
Katona, Gyula, A theorem of finite sets. in: Theory of Graphs. Academic Press - Akadémiai Kiadó, New York; Budapest, (1968), pp. 187-207.
Katona, Gyula, A theorem of finite sets. In Classic Papers in Combinatorics, (2009) pages 381–401. Springer.
G. Hansel. Sur le nombre des fonctions booléennes monotones de n variables. C. R. Acad. Sci., 262:1088–1090, 1966.
Hu Y., Yao Y., Lee W., A reinforcement learning approach for optimizing multiple traveling salesman problems over graphs. Knowl-Based Syst 204:106244, (2020).
Jackson, Jeffrey C., Homin K. Lee, Rocco A. Servedio, and Andrew Wan, Learning random monotone DNF. In Approximation, Randomization and Combinatorial Optimization. Algorithms and Techniques: 11th International Workshop, APPROX 2008, and 12th International Workshop, RANDOM 2008, Boston, MA, USA, August 25-27, 2008. Proceedings, pp. 483-497. Springer Berlin Heidelberg, 2008.
E. B. Khalil, H. Dai, Y. Zhang, B. Dilkina, L. Song. Learning combinatorial optimization algorithms over graphs. Advances in Neural Information Processing Systems, 30:6351–6361, 2017.
V. K. Korobkov, “On monotone functions of the algebra of logic”, Problemy kibernetiki, 13:5–28 (1965).
D. Korshunov, “On the number of monotone boolean functions”, Problemy kibernetiki, 38:5–109, 1981.
J. Kruskal. The number of simplices in a complex. Mathematical Optimization Techniques, University of California Press, 251–278, 1963.
Zhuwen Li, Qifeng Chen, and Vladlen Koltun. Combinatorial optimization with graph convolutional networks and guided tree search. Advances in neural information processing systems, 31, 2018.
Mazyavkina N., Sviridov S., Ivanov S., and Burnaev E. Reinforcement learning for combinatorial optimization: A survey. Computers and Operations Research, 134, 2021.
S. Muroga, Threshold Logic and Its Applications, Wiley-Interscience, 1971.
O’Donnell, Ryan, and R. Servedio, Learning monotone functions from random examples in polynomial time, Citeseer, 2005.
André L. C. Ottoni, Erivelton G. Nepomuceno, Marcos S. de Oliveira, Daniela C. R. de Oliveira, Reinforcement learning for the traveling salesman problem with Refueling, Complex & Intelligent Systems (2022) 8:2001–2015 https://doi.org/10.1007/s40747-021-00444-4.
N. A. Sokolov, On optimal deciphering of monotone functions of logic algebra, Journal of Computational Mathematics and Math. Physics, 22(12):1878–1887, 1982.
N. A. Sokolov, Optimal reconstruction of monotone Boolean functions, USSR Computational Mathematics and Mathematical Physics, 27(6):181–187, 1987.
Schröder, Bernd SW. Ordered sets. Springer 29 (2003): 30.
Richard S. Sutton and Andrew G. Barto. Reinforcement learning. The MIT Press, London, 2nd edition, 2018.
G. P. Tonoyan, Chain partitioning of n-cube vertices and deciphering of monotone Boolean functions, Computational Mathematics and Mathematical Physics, 19(6) (1979).
L. Valiant (1984) A theory of the learnable. Communications of the ACM, 27(11):1134{1142.
O. Vinyals, M. Fortunato, N. Jaitly. Pointer networks. NeurIPS, pages 2692–2700, 2015.
H. Sahakyan, L. Aslanyan and V. Ryazanov, “On the Hypercube Subset Partitioning Varieties” 2019
Computer Science and Information Technologies (CSIT), Yerevan, Armenia, 2019, pp. 83-88, doi:
1109/CSITechnol.2019.8895211.
Dohmen, K. Improved Bonferroni Inequalities with Applications: Inequalities and Identities of Inclusion-Exclusion Type. Berlin: Springer-Verlag, 2003. Galambos, J. and Simonelli, I. Bonferroni-Type Inequalities with Applications. New York: Springer-Verlag, 1996.
J. Galambos, J. and I. Simonelli, “Bonferroni-TypeInequalities with Applications”. New York: Springer-Verlag, 1996.
DOI:
https://doi.org/10.31449/inf.v48i4.4804Downloads
Published
How to Cite
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
