Causality Modeling and Physical Commonsense in Grounded Language Understanding

Motivations and Objectives

This research project develops novel causality models for concrete action verbs to capture the intended change of state of the physical world.

It augments meanings of concrete verbs based on how they might change the environment (i.e., causality) and meanings of concrete nouns based on how they might be changed by actions (i.e., affordance). It incorporates causality models into learning and inference algorithms for grounding language to the physical world.

This work will provide a new dimension to connect verb semantics to perception and action. Verb causality models will allow the robot to predict a potential change of state from human linguistic utterances. This prediction will provide top-down information to guide visual processing and action modeling.

Related Papers

• Shane Storks, Qiaozi Gao, Yichi Zhang, & Joyce Chai. (2021). Tiered Reasoning for Intuitive Physics: Toward Verifiable Commonsense Language Understanding. Findings of EMNLP 2021.
• Shaohua Yang, Qiaozi Gao, Sari Saba-Sadiya, and Joyce Chai. Commonsense Justification for Action Explanation. Conference on Empirical Methods in Natural Language Processing (EMNLP), Brussels, Belgium, November 1-4, 2018.
• Q. Gao, S. Yang, J. Y. Chai, and L. Vanderwende. What Action Causes This? Towards Naive Physical Action-Effect Prediction. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), 2018.
• L. She and J. Chai. Interactive Learning of Grounded Verb Semantics towards Human-Robot Communication. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), Vancouver, Canada, July 31-August 4, 2017.
• Q. Gao, M. Doering, S. Yang, and J. Y. Chai. Physical Causality of Action Verbs in Grounded Language Understanding. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany, August 7-12, 2016.
• L. She and J. Y. Chai.Incremental Acquisition of Verb Hypothesis Space towards Physical World Interaction. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL), Berlin, Germany, August 7-12, 2016.
• S. Yang, Q. Gao, C. Liu, C. Xiong, S. Zhu, and J. Y. Chai. Grounded Semantic Role Labeling. Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), San Diego, CA, June 12-17, 2016.