DARPA FACT AIE Award

23 February 2024

I am beyond excited that we have been awarded a contract on the DARPA Friction and Accountability in Conversational Transactions (FACT) AI Exploration program! Our project, TRACE: Transparency, Reflection, and Accountability in Conversational Exchanges, will address a lack of “friction” (deliberation and reflective reasoning) in LLMs that prevent them from being used reliably in mission-critical workflows.

read more

RETTL Grant Awarded

11 September 2023

Pleased to announce that our grant proposal to NSF’s RETTL program, An AI Tutoring System for Pollinator Conservation Community Science Training, has been awarded, for $849,890!

read more

I'm On NPR

11 August 2023

I’m on the latest episode of The Pulse, NPR’s science and health podcast, talking about AI and spatial reasoning and… plumbing?

read more

More Media

04 April 2023

I was interviewed by Denver’s 5280 magazine about, what else, ChatGPT. The interview was done at the end of February and it’s finally out now. Amazing how much has changed in such a short time, and how little. People are still star-struck by generative AI for reasons I’m not entirely convinced by.

read more

Congratulations to Aniket Tomar!

06 March 2023

Congratulations to Aniket Tomar on successfully defending his Master’s thesis!

read more

In the media about ChatGPT

25 February 2023

One thing about being the only faculty member at a big university doing research on a particular topic is that if your topic becomes hot, everyone wants to talk to you about it.

read more

Goodbye, Hex

20 February 2023

Yesterday, at about 4:37 pm Mountain Time, we had to say goodbye to our boy Hex. We estimate he was about 17 and was diagnosed with an aggressive, rare form of cancer and didn’t respond effectively to treatment. Thanks to the staff at the CSU Vet Hospital who did everything they could.

read more

AACL and ACS

21 November 2022

Two back-to-back conference activities.

read more

Congratulations to Abhijnan Nath!

21 October 2022

Congratulations to Abhijnan Nath for defending his Master’s thesis!

read more

New Marathon PR

20 October 2022

Not work-related. On October 9, I ran the Chicago Marathon in 3:05:07! This is a new personal record by over 4 minutes but I just barely missed a BQ (Boston Qualification) time by 7 seconds. So that was a bit of a bummer but now I at least still have a goal to chip away at. It was a great race, and my 1st World Marathon Majors!

read more

New Frontiers in AI Paper

06 October 2022

I’m pleased to finally have this paper Affordance Embeddings for Situated Language Understanding published in the Language and Computation section of the Frontiers in AI journal, as part of the special issue Perspectives for Natural Language Processing between AI, Linguistics and Cognitive Science. We use vectorizations of VoxML affordance encodings to perform transfer learning to reason about the properties of unknown objects in a zero-shot condition. We’ve been trying to get this work published for a couple of years now, so it’s a very good feeling to finally have it out in a major journal. The paper is open access...

read more

Wrap-up of the least relaxing summer of my life

16 August 2022

I went to LREC, presented The VoxWorld Platform for Multimodal Embodied Agents (yay!) and then got COVID-19 (boo!). AirFrance lost my luggage, and I wrote a massive Twitter thread about it here. Our paper, Multimodal Semantics for Affordances and Actions won best paper at HCII (yay!). Marseille was really hot.

read more

Congratulations to Dhruva Patil and Jason Garcia!

11 April 2022

Congratulations to Dr. Dhruva Patil for defending his Ph.D. dissertation and to Jason Garcia for defending his Master’s thesis!

read more

Cracked 3:10 in the Marathon

26 October 2021

Not work-related. Just very happy to have broken 3:10:00 in the marathon at the Colorado Marathon here in Fort Collins this past Sunday. Everything came together just right—the pace plan, hydration, fueling, the weather—with one small exception of forgetting my headphones at home so I ended up just playing my playlist out loud out of my phone all the way down Poudre Canyon. I threw up afterwards and got a massive blood blister on my toe, but no complaints! With a time of 3:09:15 this is a new personal best by over 2 minutes, and should be enough for guaranteed...

read more

Diana and NSF iSAT on the NYC Local News

01 October 2021

NYC’s FOX 5 channel did a segment on AI in the classroom that focused in part on our Diana system and how we are expanding that expertise into situation-aware, interactive AI agents to help teachers in classrooms. I’m not interviewed directly but the “CSU/Brandeis” work is shown in multiple clips (mostly taken from last year’s AAAI demo video), and my advisor, James Pustejovsky, is interviewed extensively. Unfortunately, the piece doesn’t mention NSF iSAT directly (in fact James mentioned it repeatedly but the mentions were edited out in the final product!), but this is where the work is being applied and...

read more

KI Paper and ESSLLI 2022 Course

21 September 2021

I’m pleased to (finally) have our paper “Embodied Human Computer Interaction” published in KI - Künstliche Intelligenz, the German Journal of Artificial Intelligence, after a drawn-out, um… interaction with the typesetter. I’m also very happy to be co-teaching a week-long course at ESSLLI 2022, the 33rd European Summer School in Language, Logic, and Information, next year in Galway, Ireland, hosted by NUI Galway! The paper can be found here (paywall, I know, sorry, I’ll try to find a way to make it available), and more information about the course will be published as it becomes available. May we be rid...

read more

NLP @ CSU in Fall 2021

10 June 2021

I’m late to announce this since the course was approved months ago, but I’m happy to be teaching CS 542: Natural Language Processing in the Fall 2021 semester. This is, to my knowledge, the first dedicated NLP course to be taught at Colorado State, and interest seems high already. This is a grad-level, technical computer science course that complements offerings in AI, machine learning, and computer vision, but I plan to teach it with a healthy dose of both linguistic theory and application, and critical thinking about NLP technology and its uses in modern society. More information about the course...

read more

Two Workshops at IWCS 2021

10 February 2021

I’m happy to be organizing the First Workshop on Multimodal Semantic Representations (MMSR I), to be co-located with IWCS 2021, online in June! This workshop intends to bring together researchers who aim to capture elements of multimodal interaction such as language, gesture, gaze, and facial expression with formal semantic representations. We provide a space for both theoretical and practical discussion of how linguistic co-modalities support, inform, and align with “meaning” found in the linguistic signal alone. Our website is here (currently undergoing a little bit of final construction).

read more

Best Demo at ICAT-EGVE 2020

08 December 2020

Our demo, Situational Awareness in Human Computer Interaction: Diana’s World has won the best demo award at ICAT-EGVE (International Conference on Artificial Reality and Telexistence & Eurographics Symposium on Virtual Environments), an ACM conference! Our paper is here and our demo video can be viewed here.

read more

Some Grant News

11 September 2020

I’m very pleased to be playing a role in a couple of new NSF grants. One is an NSF EAGER exploratory grant with my advisor/former PI James Pustejovsky at Brandeis University and Josh Hartshorne at Boston College, A Playground and Proposal for Growing an AGI. With support from the Robust Intelligence program at the IIS division at NSF, we are addressing the challege of creating an Artificial General Intelligence by (from the abstract) “synthesizing symbolic or logical reasoning, learning through interaction with the environment, as well as state-of-the-art neural networks. Inspired by the structure of natural (e.g., human) intelligence, the...

read more

Neurosymbolic AI at ACS 2020

10 August 2020

Today I presented a long paper, “Neurosymbolic AI for Situated Language Understanding” at the Advances in Cognitive Systems conference, held virtually and hosted by the Palo Alto Research Center.

read more

Live from Fort Collins

03 August 2020

Should have posted this two days ago, but after leaving Boston on July 27th with my wife, two cats, and one car, and stops in Connecticut, Pittsburgh, Chicago, and Omaha, we finally made it to Fort Collins, CO, and moved into our new home. Shout out to Hex and Bilbo, MVPs of the cross-country drive.

read more

Virtual Conferences Are Weird

02 July 2020

For my last act as a postdoc/first act as a professor, I presented our late-breaking report, Situated Multimodal Control of Mobile Robot: Navigation Through a Virtual Environmment at RoboDIAL: the Special Session on Situated Dialogue with Virtual Agents and Robots at the SigDIAL conference. The conference was originally slated to take place in Boise, Idaho, but due to the COVID-19 pandemic was moved to an all-virtual format. Not knowing what to expect from a virtual conference, this has been a new experience for me, and while it was odd attending a conference via Trello, YouTube, and Zoom, having to record...

read more

Introducing SIGNAL Lab

01 July 2020

Now that I’m officially activated as CSU faculty, it gives me great pleasure to be able to announce my new lab, the Situated Grounding and Natural Language Lab! We tackle questions of language understanding through situatedness and embodiment using machine learning, simulation, and symbolic methods. The lab’s website can be found at SIGNALLab.ai, and updates will be posted both there and here.

read more

Welcome!

17 April 2020

The website has been rebooted. Welcome, and please enjoy the sleek new look as I attempt to subvert expectations for faculty webpages. Although the world may seem to be crumbling at the seams right now, I’m very excited to be joining the computer science faculty at Colorado State this summer, where I hope to build a strong natural language understanding group to complement CSU’s strengths in computer vision, data science, and human-computer interaction. I’m currently looking for students who are interested in computational linguistics, multimodality, and natural language processing, so please feel free to get in touch via the contact...

read more