2026 · Research appointment
Visiting Researcher at the Institute of Science Tokyo
Exploring human–AI collaboration grounded in physiological and multimodal sensing with A/Prof. Erwin Wu.
Yongquan Owen Hu — HCI and AI Researcher
Context-Aware Systems, Multimodal Interaction, Human–AI Collaboration
Singapore, Japan, Australia, and China
About Me
I'm Yongquan “Owen” Hu, an HCI and AI researcher building context-aware, trustworthy systems through computer vision, multimodal sensing, large language models, and immersive interaction.
I completed my Ph.D. in Computer Science and Engineering at the University of New South Wales in January 2026, advised by Prof. Aaron Quigley and Prof. Wen Hu. My dissertation, Empowering Context-Aware Applications with AI-Driven Vision-Based Multimodal Interfaces, connects sensing and AI reasoning with usable, interactive systems.
Most recently, I was a visiting researcher at the Institute of Science Tokyo, exploring human–AI collaboration grounded in physiological and multimodal sensing. Before that, I was a Research Fellow at the Smart Systems Institute, National University of Singapore, where I worked on multimodal affect sensing and LLM-supported wellbeing interactions.
Research direction
My research asks how an intelligent system can sense the partial signals around it, reason about the context that matters, and adapt with people rather than simply acting on them. This spans vision-based multimodal interfaces, ubiquitous and mobile computing, human-centered AI, and human–AI collaboration, with trust, privacy, accessibility, and human agency treated as design requirements.
Interactive research map
The Context Field
Move through the field, then switch stages. The three-part system loop stays consistent while the finer actions reveal what happens inside it.
Observe
Sample partial, multimodal signals from a changing environment.
Trust boundary · uncertainty remains visible · people retain control
News
Research & Work Experience
ST
Institute of Science Tokyo (Feb 2026 – May 2026)
Visiting Researcher
Japan
Explored human–AI collaboration prototypes grounded in physiological and multimodal sensing with A/Prof. Erwin Wu.
Focus: Human–AI Collaboration · Multimodal Sensing · Affective Computing
NUS
Smart Systems Institute, National University of Singapore (Mar 2025 – Dec 2025)
Research Fellow (Postdoctoral Researcher)
Singapore
Developed multimodal affect-sensing pipelines and LLM-supported wellbeing interactions with A/Prof. Suranga Nanayakkara.
Focus: Physiological Sensing · LLMs · Wellbeing Interfaces
HW
2012 Lab, Huawei (Jun 2024 – Sep 2024)
Research & Development Engineer Intern
Shanghai, China (Remote)
Developed an optimized LLM-based retrieval-augmented generation pipeline for document search using LangChain.
Focus: Large Language Models · RAG · Document Retrieval
DingOS, Lenovo (Dec 2021 – Mar 2022)
Visiting Scientist
Beijing, China
Led an adaptive interface project for pervasive wearable and human-motion scenarios.
Focus: Adaptive Interfaces · Context Awareness · Operating Systems
Pervasive HCI Group, Tsinghua University (Sep 2020 – Sep 2021)
Research Assistant
Beijing, China
Developed machine-learning-based authentication, tracking, and gesture interaction for large intelligent displays.
Focus: Ubiquitous Computing · Computer Vision · Intelligent Displays
Human–Computer Interaction Lab, University of Maryland (Aug 2019 – Feb 2020)
Research Intern
College Park, Maryland, United States
Designed and built a wall-climbing robot for spatial tangible interaction, including hardware and motion control.
Focus: Tangible Interaction · Robotics · Digital Fabrication
iFLYTEK (Jun 2018 – Sep 2018)
Algorithm Engineer Intern
Hefei, Anhui, China
Developed computer-vision algorithms for object recognition and classification.
Focus: Computer Vision · Deep Learning · Algorithm Development
SUGAR Expo, Stanford University (Jun 2017 – Jun 2017)
Visiting Student
Stanford, California, United States
Participated in the ME310 global collaborative design course and SUGAR Expo.
Focus: Global Design Innovation · Prototyping
ZTE (Sep 2015 – Oct 2015)
Course Intern
Taicang, Jiangsu, China
Developed an Arduino-based smart cane to support people with visual impairments.
Focus: Embedded Development · Accessibility · Wireless Communication
Education
University of New South Wales (UNSW) (Jun 2021 – Jan 2026)
Doctor of Philosophy, Computer Science & Engineering
Sydney, Australia
Dissertation: Empowering Context-Aware Applications with AI-Driven Vision-Based Multimodal Interfaces
University of Science and Technology of China (USTC) (Sep 2016 – Jun 2019)
Master of Engineering, Information and Communication Engineering
Hefei, Anhui, China
Donghua University (DHU) (Sep 2012 – Jun 2016)
Bachelor of Engineering, Electronic Information Engineering
Shanghai, China
Skills
ML & LLM FrameworksLangChain · PyTorch · TensorFlow · ML algorithms
Programming LanguagesPython · MATLAB · C/C++ · Java · JavaScript · HTML
User & Data AnalysisQuantitative · Qualitative · Power BI · SQL · Observer XT
Mobile & Embedded SystemsAndroid · MSP430 · PCB · Altium Designer · EAGLE
XR & Interaction DesignUnity · Blender · Photoshop · Illustrator · Figma
Selected Awards/Honors
UNSW Three Minute Thesis People’s Choice, 2025
Top 5%
UbiComp/ISWC Best Student Competition Award, 2025
SituGuard team
ACM Spatial User Interaction Student Volunteer Award, 2023
Academic service recognition
Shanghai Outstanding Graduate, 2016
Top 5%, Shanghai-wide
Donghua University Annual Top 5 Student, 2016
Top 5 of 30,000+ students
Mathematical Contest in Modeling, 2015
Meritorious Winner, first place tier
Freescale Cup Smart Car Contest, East China Division, 2015
Winner
Scholarships
UNSW CSE School Top-Up Scholarship, 2021–2024
Top 5%, university-wide
UNSW Tuition Fee Scholarship plus Research Stipend, 2021
Top 10%, university-wide
USTC Graduate Student Study Scholarship, 2016–2019
First place, top 5%
Shanghai Enterprise Scholarship, 2015
First place
National Scholarship, 2012–2015
Top 1%, national level
Highest University Scholarship, 2012–2015
Top 1%, university-wide
Academic Services
Workshop Organizer, UbiComp/ISWC 2026
Privacy, Security, Safety and Ethics of AI in Pervasive Environments.
Program Committee Member, ACM SIGSPATIAL 2026
Reviewed and discussed research submissions.
Reviewer, 2026
ICME, CHI, and IUI.
Workshop Organizer, 2025
WellComp at UbiComp/ISWC and Embodied World Models at NeurIPS.
Reviewer, 2025
UIST, CHI, SIGGRAPH Asia, and IMWUT.
Peer Review
30+ papers reviewed across HCI, ubiquitous computing, and multimedia venues.
Projects
Recent Publications
2026
Ambient2Hap: Rendering Haptic Feedback via Somatosensory Electrical Stimulation from Ambient Cues in Virtual Reality
IEEE International Conference on Multimedia and Expo (ICME) · Accepted, in press · First author
2026
Mediating Impulse and Intention: Reflective Conversational Food Recommendation with Large Language Models
CCF Transactions on Pervasive Computing and Interaction · Accepted, in press · Co-first author
2026
SpeechAgent: An End-to-End Mobile Infrastructure for Speech Impairment Assistance
Interspeech · Co-author
2026
EmoDrink: An Embodied MR Framework for Physiology-Informed Wellbeing Recommendations, Illustrated Through Beverage Choice
Augmented Humans International Conference (AHs) · Co-author
2025
Vision-based Multimodal Interfaces for Enhanced Context Awareness: A Taxonomy for System Design via a Data Modality Lens
ACM CHI Conference on Human Factors in Computing Systems · First author
2025
Towards Human–AI Synergy in UI Design: Supporting Iterative Generation with LLMs
ACM Transactions on Computer-Human Interaction (TOCHI) · Co-author
2025
Toward AI-driven UI Transition Intuitiveness Inspection for Smartphone Apps
International Journal of Human–Computer Studies · Co-author
2025
A Fine-grained Context Awareness with Radar-based and Vision-based Surface Sensing
Radar-Based Human–Computer Interaction, Springer Nature · Book chapter, accepted in press · First author
Selected Published Systems
Ambient2Hap (ICME ’26)
Transforms visual and auditory ambient cues into somatosensory electrical stimulation, extending haptic feedback in virtual reality.
#multimodal interaction #virtual reality #haptics
MultiEEG-GPT (UbiComp/ISWC ’24)
Explores how large language models can interpret EEG-based multimodal data for personalized mental-health applications.
#EEG #large language models #mental health
MultiSurf-GPT (MobileHCI ’24)
Uses large-language-model agents to reason over multimodal surface-sensing signals for context-aware applications.
#surface sensing #LLM agents #context awareness
MicroCam (IMWUT ’23)
Turns a smartphone microscope camera into a low-cost sensor for recognizing contact surfaces and their fine-grained context.
#vision-based sensing #mobile computing #context awareness
RadarFoot (UIST ’23)
Embeds radar sensing in a smart shoe to recognize fine-grained ground-surface context during everyday movement.
#radar #wearable sensing #surface context
Generative AI for Spatial AR (ISMAR ’23)
Investigates how generative AI can facilitate image creation for spatial augmented-reality experiences.
#generative AI #spatial AR #content creation
GenAIR (SUI ’23)
Explores the design factors involved in employing generative AI for augmented-reality content and interaction.
#generative AI #augmented reality #design factors
Motion-Adaptive GUI (IUI ’23)
Adapts mobile interfaces to a user’s motion state so interaction remains legible and usable in dynamic settings.
#adaptive interfaces #mobile HCI #human motion
SmartRecorder (IUI ’23)
Helps people without video-editing expertise create clear tutorial videos for smartphone interaction tasks.
#interactive tools #tutorial creation #mobile interaction
FootUI (ASSETS ’22)
Designs and recognizes foot gestures that help people with upper-body motor impairments use smartphones while in bed.
#accessibility #foot interaction #gesture sensing
Auth+Track (CHI ’21)
Combines sparse authentication with continuous user tracking to reduce repeated smartphone authentication in everyday use.
#authentication #user tracking #ubiquitous computing
WallBot (CHI ’21 EA)
Extends tangible interaction from horizontal surfaces to walls through a responsive, wall-climbing robotic interface.
#tangible interaction #robotics #interactive surfaces
Ongoing Work
EmotionalDiffusion (ongoing)
This project aims to facilitate content creation in projection scenarios by combining visual (depth dimension) and textual modalities.
#generative AI #context creation #projection
Survey on Radar Sensing in HCI (ongoing)
How is radar sensing employed in HCI? How is it different from vision-based sensing methods?
#radar #survey
Speech2Pro (ongoing)
This project aims to facilitate content creation in projection scenarios by combining visual (depth dimension) and textual modalities.
#generative AI #context creation #projection
Earlier Projects
Hierarchical Video Encryption with Quantum Key Distribution
Explored how limited quantum keys can protect the most semantically important information in video content.
#semantic encryption #quantum encryption
Touch to See (Stanford SUGAR Expo)
Created a pipeline for translating ordinary images into tactile graphics for accessible books.
#tactile graphics #accessibility
Dynamic Network Planning for Ebola Eradication
A mathematical model balancing disease spread, medicine supply, delivery systems, and manufacturing capacity.
#mathematical modeling
Remote Interactive RF Experimental Platform
Designed FM modulation and demodulation circuits and evaluated them with NI ELVIS virtual instruments.
#RF circuit #experimental platform
Networked Mobile-Robot Monitoring and Anti-Collision
Built a web monitoring tool for inspecting and controlling a mobile robot’s operating state.
#robotics #remote monitoring
RF Amplifier for Internet-of-Things Applications
Designed and evaluated a low-noise RF amplifier using vector network analysis.
#RF amplifier #hardware
Resume
Curriculum Vitae · Updated May 2026
Research, publications, teaching, mentoring, and academic service
A current CV is available on request. Please get in touch if you would like to discuss a faculty opportunity, research position, invited talk, or collaboration.
Contact meContact
Send Message
More Info
It is my great honor to have worked with those awesome people from different affiliations, and I am always eager to look for potential collaboration opportunities. 😊
If you are interested in working with me together, please feel free to contact me!!! 🧐