Congratulations to Shadi Manafi!
07 October 2024
Congratulations to Dr. Shadi Manafi for successfully defending her Ph.D. dissertation!
read moreBest Paper Award at EDM 2024
17 July 2024
Exciting news! Our paper, Propositional Extraction from Natural Speech in Small Group Collaborative Tasks, was awarded Best Student Paper at the International Conference on Educational Data Mining 2024. This paper addresses the problem of extracting the semantics expressed by spoken utterances in a multiparty context. In spoken dialogue, the same semantic content can be expressed in many different ways. While it’s easy for humans to interpret the many ways of saying the same thing, it is challenging for computers for many reasons, such as filler words, disfluencies, and overlapping utterances. We adapted a cross-encoding method from coreference research in NLP...
read moreCongratulations to Nada Alalyani!
25 June 2024
Congratulations to Dr. Nada Alalyani for successfully defending her Ph.D. dissertation!
read moreCongratulations to Hannah VanderHoeven!
01 May 2024
Congratulations to Hannah VanderHoeven on successfully defending her Master’s thesis!
read moreDARPA FACT AIE Award
23 February 2024
I am beyond excited that we have been awarded a contract on the DARPA Friction and Accountability in Conversational Transactions (FACT) AI Exploration program! Our project, TRACE: Transparency, Reflection, and Accountability in Conversational Exchanges, will address a lack of “friction” (deliberation and reflective reasoning) in LLMs that prevent them from being used reliably in mission-critical workflows.
read moreRETTL Grant Awarded
11 September 2023
Pleased to announce that our grant proposal to NSF’s RETTL program, An AI Tutoring System for Pollinator Conservation Community Science Training, has been awarded, for $849,890!
read moreI'm On NPR
11 August 2023
I’m on the latest episode of The Pulse, NPR’s science and health podcast, talking about AI and spatial reasoning and… plumbing?
read moreMore Media
04 April 2023
I was interviewed by Denver’s 5280 magazine about, what else, ChatGPT. The interview was done at the end of February and it’s finally out now. Amazing how much has changed in such a short time, and how little. People are still star-struck by generative AI for reasons I’m not entirely convinced by.
read moreCongratulations to Aniket Tomar!
06 March 2023
Congratulations to Aniket Tomar on successfully defending his Master’s thesis!
read moreIn the media about ChatGPT
25 February 2023
One thing about being the only faculty member at a big university doing research on a particular topic is that if your topic becomes hot, everyone wants to talk to you about it.
read moreGoodbye, Hex
20 February 2023
Yesterday, at about 4:37 pm Mountain Time, we had to say goodbye to our boy Hex. We estimate he was about 17 and was diagnosed with an aggressive, rare form of cancer and didn’t respond effectively to treatment. Thanks to the staff at the CSU Vet Hospital who did everything they could.
read moreCongratulations to Abhijnan Nath!
21 October 2022
Congratulations to Abhijnan Nath for defending his Master’s thesis!
read moreNew Marathon PR
20 October 2022
Not work-related. On October 9, I ran the Chicago Marathon in 3:05:07! This is a new personal record by over 4 minutes but I just barely missed a BQ (Boston Qualification) time by 7 seconds. So that was a bit of a bummer but now I at least still have a goal to chip away at. It was a great race, and my 1st World Marathon Majors!
read moreNew Frontiers in AI Paper
06 October 2022
I’m pleased to finally have this paper Affordance Embeddings for Situated Language Understanding published in the Language and Computation section of the Frontiers in AI journal, as part of the special issue Perspectives for Natural Language Processing between AI, Linguistics and Cognitive Science. We use vectorizations of VoxML affordance encodings to perform transfer learning to reason about the properties of unknown objects in a zero-shot condition. We’ve been trying to get this work published for a couple of years now, so it’s a very good feeling to finally have it out in a major journal. The paper is open access...
read moreWrap-up of the least relaxing summer of my life
16 August 2022
I went to LREC, presented The VoxWorld Platform for Multimodal Embodied Agents (yay!) and then got COVID-19 (boo!). AirFrance lost my luggage, and I wrote a massive Twitter thread about it here. Our paper, Multimodal Semantics for Affordances and Actions won best paper at HCII (yay!). Marseille was really hot.
read moreCongratulations to Dhruva Patil and Jason Garcia!
11 April 2022
Congratulations to Dr. Dhruva Patil for defending his Ph.D. dissertation and to Jason Garcia for defending his Master’s thesis!
read moreCracked 3:10 in the Marathon
26 October 2021
Not work-related. Just very happy to have broken 3:10:00 in the marathon at the Colorado Marathon here in Fort Collins this past Sunday. Everything came together just right—the pace plan, hydration, fueling, the weather—with one small exception of forgetting my headphones at home so I ended up just playing my playlist out loud out of my phone all the way down Poudre Canyon. I threw up afterwards and got a massive blood blister on my toe, but no complaints! With a time of 3:09:15 this is a new personal best by over 2 minutes, and should be enough for guaranteed...
read moreDiana and NSF iSAT on the NYC Local News
01 October 2021
NYC’s FOX 5 channel did a segment on AI in the classroom that focused in part on our Diana system and how we are expanding that expertise into situation-aware, interactive AI agents to help teachers in classrooms. I’m not interviewed directly but the “CSU/Brandeis” work is shown in multiple clips (mostly taken from last year’s AAAI demo video), and my advisor, James Pustejovsky, is interviewed extensively. Unfortunately, the piece doesn’t mention NSF iSAT directly (in fact James mentioned it repeatedly but the mentions were edited out in the final product!), but this is where the work is being applied and...
read moreKI Paper and ESSLLI 2022 Course
21 September 2021
I’m pleased to (finally) have our paper “Embodied Human Computer Interaction” published in KI - Künstliche Intelligenz, the German Journal of Artificial Intelligence, after a drawn-out, um… interaction with the typesetter. I’m also very happy to be co-teaching a week-long course at ESSLLI 2022, the 33rd European Summer School in Language, Logic, and Information, next year in Galway, Ireland, hosted by NUI Galway! The paper can be found here (paywall, I know, sorry, I’ll try to find a way to make it available), and more information about the course will be published as it becomes available. May we be rid...
read moreNLP @ CSU in Fall 2021
10 June 2021
I’m late to announce this since the course was approved months ago, but I’m happy to be teaching CS 542: Natural Language Processing in the Fall 2021 semester. This is, to my knowledge, the first dedicated NLP course to be taught at Colorado State, and interest seems high already. This is a grad-level, technical computer science course that complements offerings in AI, machine learning, and computer vision, but I plan to teach it with a healthy dose of both linguistic theory and application, and critical thinking about NLP technology and its uses in modern society. More information about the course...
read moreTwo Workshops at IWCS 2021
10 February 2021
I’m happy to be organizing the First Workshop on Multimodal Semantic Representations (MMSR I), to be co-located with IWCS 2021, online in June! This workshop intends to bring together researchers who aim to capture elements of multimodal interaction such as language, gesture, gaze, and facial expression with formal semantic representations. We provide a space for both theoretical and practical discussion of how linguistic co-modalities support, inform, and align with “meaning” found in the linguistic signal alone. Our website is here (currently undergoing a little bit of final construction).
read moreBest Demo at ICAT-EGVE 2020
08 December 2020
Our demo, Situational Awareness in Human Computer Interaction: Diana’s World has won the best demo award at ICAT-EGVE (International Conference on Artificial Reality and Telexistence & Eurographics Symposium on Virtual Environments), an ACM conference! Our paper is here and our demo video can be viewed here.
read moreSome Grant News
11 September 2020
I’m very pleased to be playing a role in a couple of new NSF grants. One is an NSF EAGER exploratory grant with my advisor/former PI James Pustejovsky at Brandeis University and Josh Hartshorne at Boston College, A Playground and Proposal for Growing an AGI. With support from the Robust Intelligence program at the IIS division at NSF, we are addressing the challege of creating an Artificial General Intelligence by (from the abstract) “synthesizing symbolic or logical reasoning, learning through interaction with the environment, as well as state-of-the-art neural networks. Inspired by the structure of natural (e.g., human) intelligence, the...
read moreNeurosymbolic AI at ACS 2020
10 August 2020
Today I presented a long paper, “Neurosymbolic AI for Situated Language Understanding” at the Advances in Cognitive Systems conference, held virtually and hosted by the Palo Alto Research Center.
read moreLive from Fort Collins
03 August 2020
Should have posted this two days ago, but after leaving Boston on July 27th with my wife, two cats, and one car, and stops in Connecticut, Pittsburgh, Chicago, and Omaha, we finally made it to Fort Collins, CO, and moved into our new home. Shout out to Hex and Bilbo, MVPs of the cross-country drive.
read moreVirtual Conferences Are Weird
02 July 2020
For my last act as a postdoc/first act as a professor, I presented our late-breaking report, Situated Multimodal Control of Mobile Robot: Navigation Through a Virtual Environmment at RoboDIAL: the Special Session on Situated Dialogue with Virtual Agents and Robots at the SigDIAL conference. The conference was originally slated to take place in Boise, Idaho, but due to the COVID-19 pandemic was moved to an all-virtual format. Not knowing what to expect from a virtual conference, this has been a new experience for me, and while it was odd attending a conference via Trello, YouTube, and Zoom, having to record...
read moreIntroducing SIGNAL Lab
01 July 2020
Now that I’m officially activated as CSU faculty, it gives me great pleasure to be able to announce my new lab, the Situated Grounding and Natural Language Lab! We tackle questions of language understanding through situatedness and embodiment using machine learning, simulation, and symbolic methods. The lab’s website can be found at SIGNALLab.ai, and updates will be posted both there and here.
read moreWelcome!
17 April 2020
The website has been rebooted. Welcome, and please enjoy the sleek new look as I attempt to subvert expectations for faculty webpages. Although the world may seem to be crumbling at the seams right now, I’m very excited to be joining the computer science faculty at Colorado State this summer, where I hope to build a strong natural language understanding group to complement CSU’s strengths in computer vision, data science, and human-computer interaction. I’m currently looking for students who are interested in computational linguistics, multimodality, and natural language processing, so please feel free to get in touch via the contact...
read more