Intelligent sensing

MicroCam

Turns a smartphone microscope camera into a low-cost sensor for recognizing contact surfaces and their fine-grained context.

MicroCam (IMWUT ’23)

vision-based sensing
mobile computing
context awareness

Read the paper

MicroCam system overview — MicroCam overview. Study and evaluation in the linked paper.

The question

What can a close-up view reveal about the surface a device touches?

The approach

How the system works

MicroCam uses a smartphone microscope camera as a low-cost surface sensor. Fine-grained visual observations provide evidence for recognizing contact surfaces and their context.

Evidence & scope

What the study supports

Maximum available image resolution: 1920 × 1080. Model input: 224 × 224 RGB. These describe one processing pipeline, not a resolution ablation experiment.

Leave-one-person-out evaluation with 12 participants: 95.56% object accuracy and 96.96% material accuracy across the collected classes.

Evaluation used one phone model and a computer-hosted classifier. Cross-device generalization and full on-device deployment remain separate questions.

The broader direction.

This work brings context sensing to the physical detail at a contact point. It is one example of how a different sensing perspective can reveal information that a broad view of a scene may miss.

Explore this research direction

Ambient2Hap (ICME ’26)

Accepted, in press

Transforms visual and auditory ambient cues into somatosensory electrical stimulation, extending haptic feedback in virtual reality.

multimodal interaction
virtual reality
haptics

Explore the project

MultiEEG-GPT (UbiComp/ISWC ’24)

UbiComp/ISWC Companion · Exploratory study

Explores EEG with facial or speech-derived context in language-model classification benchmarks, without claiming clinical validation.

The related IntervEEG-LLM framework explores how multimodal interpretations can inform supportive dialogue. Its example cases are not evidence of clinical effectiveness.