Workshop on Spoken Dialogue Systems for Cybernetic Avatars (SDS4CA)

Date: September 17, 2024 (Tuesday) Afternoon
Venue: International Science Innovation Building, Kyoto University, Kyoto, Japan
Fee: free of charge

Workshop of SIGDIAL 2024
Sponsor: JST Moonshot R&D Program: Avatar-Symbiotic Society

Background:
Spoken dialogue systems (SDS) have made dramatic advances thanks to improved speech technology and large language models. However, they still have limitations; Purely autonomous systems might be out of control and produce unexpected responses and actions. Meanwhile, avatars have become prevailing in online communications since the pandemic. They enable us to play a particular role (e.g., civil servant or salesperson) while staying in distant places such as home. A hybrid of avatars with AI and robotics, called cybernetic avatars (CA), will further expand the possibility by breaking the physical and temporal constraints and handicaps. A hybrid of avatars and spoken dialogue systems is expected to complement each other and provide human-level services to many people in parallel and simultaneously. The interface can be either a robot or a CG-based avatar. The workshop focuses on this new area of spoken dialogue technology for cybernetic avatars, which is sponsored by the Moonshot Research and Development Program in Japan.

Topic:
The workshop features the emergent framework of cybernetic avatars, a hybrid of avatars with AI and robotics, focusing on spoken dialogue technology. Specifically, it addresses the following topics:

• Spoken dialogue technology for humanoid robots and avatars
• Applications of conversational robots and avatars
• Ethical issues in avatars and AI-based dialogue services

Keynote Speakers:

• Hiroshi Ishiguro (Osaka University)
• Yukiko Nakano (Seikei University)
• Nancy F. Chen (A-STAR, I2R)

Organizers:

• Tatsuya Kawahara (Kyoto University)
• Ryuichiro Higashinaka (Nagoya University)
• Kazunori Komatani (Osaka University)

Program (tentative):

  13:15 Opening
  13:30 Keynote 1: Hiroshi Ishiguro (Osaka University)
  	Toward Avatar-Symbiotic Society
  14:00 Keynote 2: Yukiko Nakano (Seikei University)
	Avatar Social Ethics Design
  14:30 Project talk 1: Tatsuya Kawahara (Kyoto University)
	Semi-autonomous Dialogue for Cybernetic Avatars
  14:50 Project talk 2: Ryuichiro Higashinaka (Nagoya University)
	Autonomous Dialogue Agents, Summarization, and Anomaly Detection for Cybernetic Avatars in Parallel Conversations

  15:10 Poster Sessions 1 (& Coffee Break)

1. A Preliminary Examination of the Impact of Dialogue Partners Having the User's Own Face and Voice on Self-Disclosure
	Taiga Natori, Changzeng Fu, Hiroshi Ishiguro, Yuichiro Yoshikawa
3. Multimodal Human-Agent Dialogue Dataset with Brain Signal
	Shun Katada, Ryu Takeda, Kazunori Komatani
5. Embodied Autonomous Interview System with Attentive Listening Behavior
	Zi Haur Pang, Yahui Fu, Divesh Lala, Mikey Elmers, Koji Inoue, and Tatsuya Kawahara
7. Modeling of Touch Gestures during Human Hugging Interactions and Implementing on a Huggable Robot
	Takuto Akiyoshi, Hidenobu Sumioka, Junya Nakanishi, Hirokazu Kato, and Masahiro Shiomi
9. Wizard-of-Oz Dialogue Data Collection for a Mobile Guide Robot
	Ao Guo, Shota Mochizuki, Sanae Yamashita, Saya Nikaido, Tomoko Isomura, and Ryuichiro Higashinaka
13. Android Avatars' Motion Generation for Distance Learning
	Naoki Kodani, Takahisa Uchida, Nahoko Kameo, Kurima Sakai, Tomo Funayama, Takashi Minato, Akane Kikuchi, Hiroshi Ishiguro

  15:40 Poster Sessions 2 (& Coffee Break)

2. Out-of-Vocabulary Word Detection in Spoken Dialogues Based on Joint Decoding with User Response Patterns
	Miki Oshio, Ryu Takeda, Kazunori Komatani
4. An Android Avatar with Operator-like Conversational Abilities
	Yuya Komai, Takahisa Uchida, Naoki Kodani, Hiroshi Ishiguro
6. Ownership Information Acquisition of Objects in the environment by Active Question Generation with Multimodal Large Language Models and Probabilistic Generative Models
	Saki Hashimoto, Tomochika Ishikawa, Shoichi Hasegawa, Akira Taniguchi, Yoshinobu Hagiwara, Lotfi ElHafi, Tadahiro Taniguchi
8. Impact of Data Size and Recording Environments on the Performance of Anomaly Detection in Human-Robot Interaction
	Shota Mochizuki, Sanae Yamashita, Tomonori Kubota, Kohei Ogawa, Ryuichiro Higashinaka
12. Real-Time Framework for Speech Extraction Based on Independent Low-Rank Matrix Analysis with Spatial Regularization and Rank-Constrained Spatial Covariance Matrix Estimation
	Yuto Ishikawa, Tomohiko Nakamura, Norihiro Takamune, Hiroshi Saruwatari

  16:10 Keynote 3: Nancy F. Chen (A-STAR, I2R)
	Multimodal, Multilingual Generative AI for Education

  16:40 Project talk 3: Kazunori Komatani (Osaka University)
  	Detecting Non-Linguistic Situations in Dialogues

  17:00 Oral session

Investigating the Impact of Gender Stereotypes in Authority on Avatar Robots
	Yuan-Chia Chang, Daniel J. Rea, Takayuki Kanda
Inducing the Perception that a Chatbot is Operated by an Android
	Seiya Mitsuno, Ryunosuke Kawashima, Midori Ban, Takahisa Uchida, Hiroshi Ishiguro, Yuichiro Yoshikawa
Analysis and Detection of Differences in Spoken User Behaviors between Autonomous and Wizard-of-Oz Systems
	Mikey Elmers, Koji Inoue, Divesh Lala, Keiko Ochi, and Tatsuya Kawahara


  17:45 Panel discussions

  18:30 Reception

Contact: Tatsuya Kawahara (Kyoto University)
E-mail: kawahara@i.kyoto-u.ac.jp