Collaborative Task Planning and Learning through Language Communication

Last updated on Nov 11, 2021 2 min read projects

Motivations and Objectives

The overall goal of this project is to develop a framework that tightly links language and dialogue processing with the robot’s underlying planning system to support collaborative task planning and learning in a human-robot team.

Theory of Mind Modeling and Common Ground in Communication

asd

Christian-Paul Bara, Sky Wang, and Joyce Chai. Mindcraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks . Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021.
C. Liu and J. Y. Chai. Learning to Mediate Perceptual Differences in Situated Human-Robot Dialogue. Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI), Austin, Texas, January 25-30, 2015.
J. Y. Chai, L. She, R. Fang, S. Ottarson, C. Littley, C. Liu, and K. Hanson. Collaborative Effort towards Common Ground in Situated Human-Robot Dialogue. The 9th ACM/IEEE Conference on Human-Robot Interaction (HRI), Bielefeld, Germany, March 3-6, 2014.

Task and Action Learning

asd

Yichi Zhang and Joyce Chai. Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoring. Findings of ACL 2021.
Joyce Chai, Qiaozi Gao, Lanbo She, Shaohua Yang, Sari Saba-Sadiya, and Guangyue Xu. Language to Action: Towards Interactive Task Learning with Physical Agents. Proceedings of the 27th International Joint Conference on Artificial Intelligence and 23rd European Conference on Artificial Intelligence (IJCAI-ECAI), Stockholm, Sweden, July 13-19, 2018.
Q. Gao, S. Yang, J. Y. Chai, and L. Vanderwende. What Action Causes This? Towards Naive Physical Action-Effect Prediction. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL), 2018.
Changsong Liu, Shaohua Yang, Sari Sadiya, Nishan Shukla, Y. He, Song-Chun Zhu, and Joyce Chai. Jointly Learning Grounded Task Structures from Language Instruction and Visual Demonstration. Conference on Empirical Methods in Natural Language Processing (EMNLP), Austin, Texas, November 1-5, 2016.
L. She, S. Yang, Y. Cheng, Y. Jia, J. Y. Chai, and N. Xi. Back to the Blocks World: Learning New Actions through Situated Human-Robot Dialogue. Proceedings of 15th SIGDIAL Meeting on Discourse and Dialogue, Philadelphia, PA, June 18-20, 2014.

Collaborative Language Grounding

asd

Joyce Chai, Rui Fang, Changsong Liu, and Lanbo She. Collaborative Language Grounding towards Situated Human-Robot Dialogue. AI Magazine, Vol 37, No 4, pp. 32-45, 2016.
R. Fang, M. Doering, and J. Y. Chai. Collaborative Models for Referring Expression Generation towards Situated Dialogue. Proceedings of the 28th AAAI Conference on Artificial Intelligence (AAAI), Quebec, Canada, August, 2014.
C. Liu, L. She, R. Fang, and J. Y. Chai. Probabilistic Labeling for Efficient Referential Grounding based on Collaborative Discourse. Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics (ACL), Short Paper, Baltimore, MD, June 22-26, 2014.