Hostname: page-component-78c5997874-94fs2 Total loading time: 0 Render date: 2024-11-04T20:23:30.637Z Has data issue: false hasContentIssue false
  • English
  • Français

Learning Distributional Programs for Relational Autocompletion

Published online by Cambridge University Press:  26 August 2021

NITESH KUMAR Open the ORCID record for NITESH KUMAR [Opens in a new window] ,
ONDŘEJ KUŽELKA  and
LUC DE RAEDT
NITESH KUMAR
Affiliation:
Department of Computer Science, KU Leuven, Belgium
ONDŘEJ KUŽELKA
Affiliation:
Department of Computer Science, Czech Technical University in Prague, Czechia
LUC DE RAEDT
Affiliation:
Department of Computer Science, KU Leuven, Belgium
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Relational autocompletion is the problem of automatically filling out some missing values in multi-relational data. We tackle this problem within the probabilistic logic programming framework of Distributional Clauses (DCs), which supports both discrete and continuous probability distributions. Within this framework, we introduce DiceML – an approach to learn both the structure and the parameters of DC programs from relational data (with possibly missing data). To realize this, DiceML integrates statistical modeling and DCs with rule learning. The distinguishing features of DiceML are that it (1) tackles autocompletion in relational data, (2) learns DCs extended with statistical models, (3) deals with both discrete and continuous distributions, (4) can exploit background knowledge, and (5) uses an expectation–maximization-based (EM) algorithm to cope with missing data. The empirical results show the promise of the approach, even when there is missing data.

Keywords

probabilistic logic programmingstatistical relational learningstructure learninginductive logic programming
Type
Original Article
Information
Theory and Practice of Logic Programming , Volume 22 , Issue 1 , January 2022 , pp. 81 - 114
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2021. Published by Cambridge University Press

Footnotes

*

This work has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No [694980] SYNTH: Synthesising Inductive Data Models) and the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” program. OK’s work has been supported by the OP VVV project CZ.02.1.01/0.0/0.0/16_019/0000765 “Research Center for Informatics”, by the Czech Science Foundation project “Generative Relational Models” (20-19104Y) and a donation from X-Order Lab. Part of this work was done while OK was with KU Leuven and was supported by Research Foundation – Flanders (project G.0428.15).

References

Alberti, M., Bellodi, E., Cota, G., Riguzzi, F. and Zese, R. 2017. cplint on swish: Probabilistic logical inference with a web browser. Intelligenza Artificiale, 11, 1, 47–64.CrossRefGoogle Scholar
Bekker, J. and Davis, J. 2020. Learning from positive and unlabeled data: a survey. Machine Learning, 109, 719–760.CrossRefGoogle Scholar
Blockeel, H. and De Raedt, L. 1998. Top-down induction of first-order logical decision trees. Artificial Intelligence, 101, 1–2, 285–297.Google Scholar
Boutilier, C., Friedman, N., Goldszmidt, M. and Koller, D. 1996. Context-specific independence in Bayesian networks. In Proceedings of the Twelfth International Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 115–123 Google Scholar
Breese, J. S., Heckerman, D. and Kadie, C. 1998. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 43–52 Google Scholar
Chickering, D. M., Heckerman, D. and Meek, C. 1997. A Bayesian approach to learning Bayesian networks with local structure. In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 80–89 Google Scholar
Choi, J., Amir, E. and Hill, D. J. 2010. Lifted inference for relational continuous models. In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence, UAI’10, AUAI Press, Arlington, Virginia, USA, 126–134.Google Scholar
Conniffe, D. 1987. Expected maximum log likelihood estimation. Journal of the Royal Statistical Society: Series D (The Statistician), 36, 4, 317–329.Google Scholar
De Raedt, L., Blockeel, H., Kolb, S., Teso, S. and Verbruggen, G. 2018. Elements of an automatic data scientist. In International Symposium on Intelligent Data Analysis, vol. 11191. Lecture Notes in Computer Science. Springer, 3–14.Google Scholar
De Raedt, L., Dries, A., Thon, I., Van Den Broeck, G. and Verbeke, M. 2015. Inducing probabilistic relational rules from probabilistic examples. In Proceedings of the 24th International Conference on Artificial Intelligence, IJCAI 2015. AAAI Press, 1835–1843.Google Scholar
De Raedt, L., Kimmig, A. and Toivonen, H. 2007. Problog: A probabilistic prolog and its application in link discovery. In Proceedings of the 20th International Joint Conference on Artifical Intelligence, IJCAI’07, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2468–2473Google Scholar
Dempster, A. P., Laird, N. M. and Rubin, D. B. 1977. Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39, 1, 1–22.Google Scholar
Diebolt, J. and Ip, E. H. 1995. A stochastic em algorithm for approximating the maximum likelihood estimate. Technical report, Sandia National Labs., Livermore, CA (United States).Google Scholar
Dos Martires, P. Z., Dries, A. and De Raedt, L. 2019. Exact and approximate weighted model integration with probability density functions using knowledge compilation. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 7825–7833.Google Scholar
Džeroski, S. 2009. Relational data mining. In Data Mining and Knowledge Discovery Handbook. Springer, 887–911.Google Scholar
Friedman, N. 1997. Learning belief networks in the presence of missing values and hidden variables. In Proceedings of the Fourteenth International Conference on Machine Learning, ICML ’97, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 125–133Google Scholar
Friedman, N., Getoor, L., Koller, D. and Pfeffer, A. 1999. Learning probabilistic relational models. In Proceedings of the 16th International Joint Conference on Artificial Intelligence, IJCAI’99. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1300–1307Google Scholar
Friedman, N. and Goldszmidt, M. 1998 Learning Bayesian networks with local structure. In Learning in Graphical Models, vol. 89. Springer, 421–459.Google Scholar
Getoor, L., Friedman, N., Koller, D. and Pfeffer, A. 2001. Learning probabilistic relational models. In Relational Data Mining. Springer, 307–335.Google Scholar
Gutmann, B., Jaeger, M. and De Raedt, L. 2010. Extending problog with continuous distributions. In International Conference on Inductive Logic Programming, vol. 6489. Lecture Notes in Computer Science. Springer, 76–91.Google Scholar
Gutmann, B., Thon, I., Kimmig, A., Bruynooghe, M. and De Raedt, L. 2011. The magic of logical inference in probabilistic programming. Theory and Practice of Logic Programming, 11, 4–5, 663–680.CrossRefGoogle Scholar
Ilyas, I. F. and Chu, X. 2015. Trends in cleaning relational data: Consistency and deduplication. Foundations and Trends in Databases, 5, 4, 281–393.CrossRefGoogle Scholar
Islam, M. A., Ramakrishnan, C. and Ramakrishnan, I. 2012. Inference in probabilistic logic programs with continuous random variables. Theory and Practice of Logic Programming, 12, 4–5, 505–523.CrossRefGoogle Scholar
Jaeger, M. 1997. Relational Bayesian networks. In Proceedings of the Conference on Uncertainty in Artificial Intelligence 1997, UAI’97. AUAI Press, 266–273.Google Scholar
Kersting, K. and De Raedt, L. 2007. Bayesian logic programming: Theory and tool. In Introduction to Statistical Relational Learning. MIT Press.Google Scholar
Kersting, K., Natarajan, S. and Poole, D. 2011. Statistical relational AI: Logic, probability and computation. In Proceedings of the 11th International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR’11), 1–9.Google Scholar
Kersting, K. and Raiko, T. 2005. ‘say em’ for selecting probabilistic models for logical sequences. In Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence. AUAI Press, 300–307.Google Scholar
Khot, T., Natarajan, S., Kersting, K. and Shavlik, J. 2012. Structure learning with hidden data in relational domains. In Proceedings of ICML Workshop on Statistical Relational Learning. 2012.Google Scholar
Khot, T., Natarajan, S., Kersting, K. and Shavlik, J. 2015. Gradient-based boosting for statistical relational learning: the markov logic network and missing data cases. Machine Learning, 100, 1, 75–100.Google Scholar
Kimmig, A., Bach, S. H., Broecheler, M., Huang, B. and Getoor, L. 2012. A short introduction to probabilistic soft logic. In In Proceedings of NIPS Workshop on Probabilistic Programming: Foundations and Applications (NIPS Workshop-12). Google Scholar
Kok, S. and Domingos, P. Learning the structure of Markov logic networks. In Proceedings of the 22nd International Conference on Machine Learning 2005. ACM, 441–448.CrossRefGoogle Scholar
Kolb, S., Teso, S., Dries, A. and De Raedt, L. 2020. Predictive spreadsheet autocompletion with constraints. Machine Learning, 109, 2, 307–325.CrossRefGoogle Scholar
Koller, D., Friedman, N., Džeroski, S., Sutton, C., McCallum, A., Pfeffer, A., Abbeel, P., Wong, M.-F., Heckerman, D., Meek, C. and et al. 2007. Introduction to Statistical Relational Learning. MIT Press.Google Scholar
Lavrac, N. and Dzeroski, S. 1994. Inductive Logic Programming: Techniques and Applications. Prentice Hall.Google Scholar
Michels, S., Hommersom, A. and Lucas, P. J. F. 2016. Approximate probabilistic inference with bounded error for hybrid probabilistic logic programming. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI’16. AAAI Press, 3616–3622.Google Scholar
Moldovan, B., Moreno, P., Nitti, D., Santos-Victor, J. and De Raedt, L. 2018. Relational affordances for multiple-object manipulation. Autonomous Robots, 42, 1, 19–44.CrossRefGoogle Scholar
Muggleton, S. 1991. Inductive logic programming. New Generation Computing, 8, 4, 295–318.CrossRefGoogle Scholar
Muggleton, S. 1995. Inverse entailment and progol. New Generation Computing, 13, 3–4, 245–286.CrossRefGoogle Scholar
Narman, P., Buschle, M., Konig, J. and Johnson, P. 2010. Hybrid probabilistic relational models for system quality analysis. In 2010 14th IEEE International Enterprise Distributed Object Computing Conference, pp. 57–66. IEEE.CrossRefGoogle Scholar
Natarajan, S., Tadepalli, P., Dietterich, T. G. and Fern, A. 2008. Learning first-order probabilistic models with combining rules. Annals of Mathematics and Artificial Intelligence, 54, 1, 223–256.CrossRefGoogle Scholar
Neville, J. and Jensen, D. 2007. Relational dependency networks. Journal of Machine Learning Research, 8, 653–692.Google Scholar
Neville, J., Jensen, D., Friedland, L. and Hay, M. Learning relational probability trees. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2003. ACM, 625–630.CrossRefGoogle Scholar
Ngo, L. and Haddawy, P. 1997. Answering queries from context-sensitive probabilistic knowledge bases. Theoretical Computer Science, 171, 1–2, 147–177.CrossRefGoogle Scholar
Nitti, D., Belle, V., De Laet, T. and De Raedt, L. 2017. Planning in hybrid relational mdps. Machine Learning, 106, 12, 1905–1932.CrossRefGoogle Scholar
Nitti, D., De Laet, T. and De Raedt, L. 2016a. Probabilistic logic programming for hybrid relational domains. Machine Learning, 103a, 3, 407–449.Google Scholar
Nitti, D., Ravkic, I., Davis, J. and De Raedt, L. 2016b. Learning the structure of dynamic hybrid relational models. In Proceedings of the Twenty-Second European Conference on Artificial Intelligence 2016b, ECAI’16. IOS Press, NLD, 1283–1290.Google Scholar
Pearl, J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Elsevier.Google Scholar
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M. and Duchesnay, E. 2011. Scikit-learn: Machine learning in python. Journal of Machine Learning Research, 12, 2825–2830.Google Scholar
Persson, A., Dos Martires, P. Z., De Raedt, L. and Loutfi, A. 2019. Semantic relational object tracking. IEEE Transactions on Cognitive and Developmental Systems, 12, 1, 84–97.CrossRefGoogle Scholar
Poole, D. 2008. The independent choice logic and beyond. In Probabilistic Inductive Logic Programming, vol. 4911. Lecture Notes in Computer Science,. Springer, 222–243.Google Scholar
Provost, F. and Domingos, P. 2000. Improving probability estimation trees. Machine Learning, 52, 3, 199–215.CrossRefGoogle Scholar
Quinlan, J. R. 1990. Learning logical definitions from relations. Machine Learning, 5, 3, 239–266.CrossRefGoogle Scholar
Quinlan, J. R. and Cameron-Jones, R. M. 1995. Induction of logic programs: Foil and related systems. New Generation Computing, 13, 3–4, 287–312.CrossRefGoogle Scholar
Ravkic, I., Ramon, J. and Davis, J. 2015. Learning relational dependency networks in hybrid domains. Machine Learning, 100, 2–3, 217–254.CrossRefGoogle Scholar
Rekatsinas, T., Chu, X., Ilyas, I. F., and Ré, C. 2017. Holoclean: holistic data repairs with probabilistic inference. Proceedings of the VLDB Endowment, 10, 11, 1190–1201.Google Scholar
Richardson, M. and Domingos, P. 2006. Markov logic networks. Machine Learning, 62, 1–2, 107–136.CrossRefGoogle Scholar
Riguzzi, F., Bellodi, E. and Zese, R. 2014. A history of probabilistic inductive logic programming. Frontiers in Robotics and AI, 1, 6.CrossRefGoogle Scholar
Sato, T. Prism: A symbolic-statistical modeling language. In Proceedings of the 15th International Joint Conference on Artificial Intelligence, 1330–1335.Google Scholar
Sato, T. and Kameya, Y. 2001. Parameter learning of logic programs for symbolic-statistical modeling. Journal of Artificial Intelligence Research, 15, 391–454.CrossRefGoogle Scholar
Schulte, O. and Routley, K. 2014. Aggregating predictions vs. aggregating features for relational classification. 2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), 121–128.Google Scholar
Schwarz, G. 1978. Estimating the dimension of a model. The Annals of Statistics, 6, 2, 461–464.CrossRefGoogle Scholar
Shachter, R. 1998. Bayes-ball: The rational pastime (for determining irrelevance and requisite information in belief networks and influence diagrams). In Proceedings of the Thirteenth Conference on Uncertainty in Artificial Intelligence, UAI’97, 480–487.Google Scholar
Speichert, S. and Belle, V. 2018. Learning probabilistic logic programs in continuous domains. arXiv preprint arXiv:1807.05527,.Google Scholar
Srinivasan, A. 2001. The aleph manual. Technical report, Computing Laboratory, Oxford University.Google Scholar
Taskar, B., Abbeel, P. and Koller, D. 2002. Discriminative probabilistic models for relational data. In Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence. Morgan Kaufmann Publishers Inc., 485–492 Google Scholar
Vennekens, J., Verbaeten, S. and Bruynooghe, M. 2004. Logic programs with annotated disjunctions. In 20th International Conference on Logic Programming, vol. 3132. Lecture Notes in Computer Science. Springer, 431–445.Google Scholar
Vens, C., Ramon, J. and Blockeel, H. 2007. Remauve: A relational model tree learner. In Inductive Logic Programming, vol. 4455. Lecture Notes in Computer Science. Springer, 424–438.Google Scholar
Wang, J. and Domingos, P. 2008. Hybrid markov logic networks. In Proceedings of the Twenty-Third National Conference on Artificial Intelligence 2008, AAAI’08. AAAI Press, 1106–1111.Google Scholar
Wu, Y., Srivastava, S., Hay, N., Du, S. and Russell, S. 2018. Discrete-continuous mixtures in probabilistic programming: Generalized semantics and inference algorithms. In International Conference on Machine Learning. PMLR, 5343–5352.Google Scholar
Yakout, M., Berti-Équille, L. and Elmagarmid, A. K. 2013. Don’t be scared: Use scalable automatic repairing with maximal likelihood and bounded changes. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, 553–564.Google Scholar
Zuidberg Dos Martires, P., Kumar, N., Persson, A., Loutfi, A. and De Raedt, L. 2020. Symbolic learning and reasoning with noisy data for probabilistic anchoring. Frontiers in Robotics and AI, 7.Google Scholar
Supplementary material: PDF

Kumar et al. supplementary material

Kumar et al. supplementary material

Download Kumar et al. supplementary material(PDF)
PDF 1.3 MB