The DARPA FACT (Friction for Accountability in Conversational Transactions) program seeks to develop AI agents that model and mediate conversational dynamics in teams by revealing implicit assumptions and unspoken conflicts between dialogue partners.

Our team's approach is based upon modeling the common ground, or set of shared beliefs, that emerge within an team as they undertake a shared task. Our goal is to build an AI agent that can interpret multimodal signals to track the common ground, determine when a group has differing assumptions about the task, and insert naturalistic, relevant, and organic "friction" to prompt reflective reasoning and deliberation, in order to build epistemic cohesion within a team and enable better collaboration.

In the image below, our system (TRACE) observes 3 participants deducing the weights of differently colored blocks using a balance scale, detects their deixis and gaze, processes their speech, and tracks the locations of objects, to determine evidence (EBank) and agreed-upon facts (FBank) regarding the weights of each block.

Embodied Computational Metacognition was a short-term innovative research (STIR) grant from the Army Research Office. We examined and assessed statistical, neural, information theoretic, and logical mechanisms by which "metacognitive" processes can be developed in computers. We focused on developing into AI systems the capacity to detect when their underlying models are inadequate to the situation in which they find themselves, expanding those underlying models to include novel types and concepts, and inferring partial information from novel entities encountered in the environment to achieve specified goals. We adopted an "embodied simulation" view toward processes like object grounding and built upon this to develop and test our method in physical object classification and reasoning tasks.

In the image below, an agent (the white cube) performs analogical reasoning over different object types in order to solve a problem in a real-time environment. The agent determines the configuration, or "habitat" the object requires to satisfy a stackability property, and places it, resulting in a sequence of events where it can climb 5 platforms to retrieve the green blocks shown.

AIDA (Active Interpretation of Disparate Alternatives) was a program funded by DARPA that sought to develop automated understanding of events and situations from different perspectives and diverse data sources.

My group's work in the context of AIDA followed on findings from CSU's vision group about the interchangeability of feature spaces in CNN models.

We explored these interchangeability findings in Transformer-based language models and found that using simple affine transformation techniques, we can transfer information between different language models in tasks like coreference resolution and cognate detection. This has protentially profound implications for the properties of language embedding spaces.