Learning to Detect Touches on Cluttered Tables
Authors:
Norberto Adrian Goussies,
Kenji Hata,
Shruthi Prabhakara,
Abhishek Amit,
Tony Aube,
Carl Cepress,
Diana Chang,
Li-Te Cheng,
Horia Stefan Ciurdar,
Mike Cleron,
Chelsey Fleming,
Ashwin Ganti,
Divyansh Garg,
Niloofar Gheissari,
Petra Luna Grutzik,
David Hendon,
Daniel Iglesia,
** Kim,
Stuart Kyle,
Chris LaRosa,
Roman Lewkow,
Peter F McDermott,
Chris Melancon,
Paru Nackeeran,
Neal Norwitz
, et al. (6 additional authors not shown)
Abstract:
We present a novel self-contained camera-projector tabletop system with a lamp form-factor that brings digital intelligence to our tables. We propose a real-time, on-device, learning-based touch detection algorithm that makes any tabletop interactive. The top-down configuration and learning-based algorithm makes our method robust to the presence of clutter, a main limitation of existing camera-pro…
▽ More
We present a novel self-contained camera-projector tabletop system with a lamp form-factor that brings digital intelligence to our tables. We propose a real-time, on-device, learning-based touch detection algorithm that makes any tabletop interactive. The top-down configuration and learning-based algorithm makes our method robust to the presence of clutter, a main limitation of existing camera-projector tabletop systems. Our research prototype enables a set of experiences that combine hand interactions and objects present on the table. A video can be found at https://youtu.be/hElC_c25Fg8.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
Eliciting Gestures for Novel Note-taking Interactions
Authors:
Katy Ilonka Gero,
Lydia B. Chilton,
Chris Melancon,
Mike Cleron
Abstract:
Handwriting recognition is improving in leaps and bounds, and this opens up new opportunities for stylus-based interactions. In particular, note-taking applications can become a more intelligent user interface, incorporating new features like autocomplete and integrated search. In this work we ran a gesture elicitation study, asking 21 participants to imagine how they would interact with an imagin…
▽ More
Handwriting recognition is improving in leaps and bounds, and this opens up new opportunities for stylus-based interactions. In particular, note-taking applications can become a more intelligent user interface, incorporating new features like autocomplete and integrated search. In this work we ran a gesture elicitation study, asking 21 participants to imagine how they would interact with an imaginary, intelligent note-taking application. We report agreement on the elicited gestures, finding that while existing common interactions are prevalent (like double taps and long presses) a number of more novel interactions (like dragging selected items to hotspots or using annotations) were also well-represented. We discuss the mental models participants drew on when explaining their gestures and what kind of feedback users might need to move to more stylus-centric interactions.
△ Less
Submitted 22 December, 2021;
originally announced December 2021.
