CityTalk
CityTalk is a prototype dialogue system in which users interact with social robots that search knowledge graphs as a primary source of information. The aim is to make the dialogue system more generic by putting metadata about domain ontologies into knowledge graphs instead of into dialogue components. The knowledge graphs are stored in graph databases either locally or in the cloud. The interaction between users, robots, and databases is managed by a conversational AI framework.
The current version uses Furhat social robots, Rasa open source conversational AI, and knowledge graphs stored in Neo4j graph databases. The initial domain for the prototype is the classic hotel and restaurant search application. CityTalk with Furhat, Rasa and Neo4j has been presented at HRI 2022, RO-MAN 2022 and ACII 2022.
Conversational AI and knowledge graphs (4 minutes)
This video shows the first demo of CityTalk on a Furhat social robot using Rasa conversational AI. The robot answers user queries about restaurants and hotels in Tokyo Waterfront by accessing knowledge graphs in a Neo4j graph database. The video was shown at the IEEE International Conference on Human-Robot Interaction (HRI 2022) in Sapporo, Japan.
An advantage of using knowledge graphs in graph databases is that the graphs can be enriched by adding layers of semantic metadata, which can be used to generate more intelligent responses. For example, a taxonomy of cuisines was extracted from WikiData and added to the restaurants knowledge graph in Neo4j. When asked about restaurants that serve Asian cuisine, CityTalk uses subclassOf relationships in the taxonomy and finds restaurants serving Japanese, Chinese or Thai cuisines.
More intelligent responses with semantic metadata (3 minutes)
This video shows a robot giving more intelligent responses by using metadata in a knowledge graph. When asked about restaurants that serve European cuisine, CityTalk uses a taxonomy of cuisines and finds restaurants serving Italian cuisine. The video was shown at the IEEE International Conference on Robot & Human Interactive Communication (RO-MAN 2022), Napoli, Italy.
An earlier version of CityTalk used Nao robots with the PyDial statistical dialogue toolkit from University of Cambridge. CityTalk with Nao and PyDial was demonstrated at the International Joint Conference on Artificial Intelligence (IJCAI 2018) in Stockholm, Sweden.
CityTalk with PyDial (10 minutes)
This video shows a Nao robot giving information about hotels and restaurants in Tokyo Waterfront, the area near the Tokyo 2020 Olympics. The robot gives information about three hotels in different price ranges. When the person asks about restaurants, the robot tells about several restaurants in different price ranges and serving different types of food.
What robots does it work on?
The prototype is demonstrated using Furhat robots. An earlier version used Nao robots.
What languages are available?
CityTalk works in English. An experimental bilingual version works in Japanese and English.
What domains are available?
The prototype is demonstrated using knowledge graphs for hotels and restaurants in Tokyo Waterfront. New domains can be added by providing knowledge graphs for the new domains.