Responsible AI and Robotics

How can we develop tools and tutorials to enable students and engineers to build AI and Robotics systems that behave responsibly and ethically?

true

Build a Better Book

How can we engage students in STEM learning by making tactile picture books in libraries and schools for children with visual impairments?

true

MindScribe

How can we develop a socially responsive robot to encourage preschoolers to tell stories of their creations?

true

Publications

2018

MindScribe: Toward Intelligently Augmented Interactions in Highly Variable Early Childhood Environments
Layne Hubbard, Boskin Erkocevic, Dylan Cassady, Chen Hao Cheng, Andrea Chamorro and Tom Yeh
In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion
link
What's at Stake: Characterizing Risk Perceptions of Emerging Technologies
Mike Skirpan, Tom Yeh and Casey Fiesler
In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
link
More Than a Show: Using Personalized Immersive Theater to Educate and Engage the Public in Technology Ethics
Mike Skirpan, Jacqueline Cameron and Tom Yeh
In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
link
Reactile: Programming Swarm User Interfaces through Direct Physical Manipulation
Ryo Suzuki, Jun Kato, Mark Gross and Tom Yeh
In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
link
CraftML: 3D Modeling is Web Programming
Tom Yeh and Jeeeun Kim
In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
link
Quantified Self: An Interdisciplinary Immersive Theater Project Supporting a Collaborative Learning Environment for CS Ethics
Mike Skirpan, Jacqueline Cameron and Tom Yeh
In Proceedings of the 49th ACM Technical Symposium on Computer Science Education
link
Ethics Education in Context: A Case Study of Novel Ethics Activities for the CS Classroom
Mike Skirpan, Nathan Beard, Srinjita Bhaduri, Casey Fiesler and Tom Yeh
In Proceedings of the 49th ACM Technical Symposium on Computer Science Education
link

2017

Designing a Moral Compass for the Future of Computer Vision Using Speculative Analysis
Mike Skirpan and Tom Yeh
In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops
link
FluxMarker: Enhancing Tactile Graphics with Dynamic Tactile Markers
Ryo Suzuki, Abigale Stangl, Mark Gross and Tom Yeh
In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility
link
Understanding Uncertainty in Measurement and Accommodating its Impact in 3D Modeling and Printing
Jeeeun Kim, Anhong Guo, Tom Yeh, Scott Hudson and Jennifer Mankoff
In Proceedings of the 2017 Conference on Designing Interactive Systems
link
Facade: Auto-generating Tactile Interfaces to Appliances
Anhong Guo, Jeeeun Kim, Xiang 'Anthony' Chen, Tom Yeh, Scott Hudson and Jennifer Mankoff
In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
link
Machines as Co-Designers: A Fiction on the Future of Human-Fabrication Machine Interaction
Jeeeun Kim, Haruki Takahashi, Homei Miyashita, Michelle Annett and Tom Yeh
In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems
link

2016

Enticing Casual Nature Preserve Visitors into Citizen Science via Photos
Jenny Preece, Carol Boston, Tom Yeh, Jacqueline Cameron, Mary Lou Maher and Kazjon Grace
In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion
link
3D Folded PrintGami: Transforming Passive 3D Printed Objects to Interactive by Inserted Paper Origami Circuits
Claudia Daudén Roquet, Jeeeun Kim and Tom Yeh
In Proceedings of the 11th ACM SIGCHI Conference on Designing Interactive Systems (DIS)
link
Facade: Auto-generating Tactile Interfaces to Appliances
Anhong Guo, Jeeeun Kim, Xiang 'Anthony' Chen, Tom Yeh, Scott Hudson, Jennifer Mankoff and Jeffery Bigham
In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility
link
Inclusive Design of 3D Printed Tactile Picture Books in Indonesia
Abigale Stangl, Rebecca Johnson, Arum Putri Budiani and Tom Yeh
In FabLearn Europe

2015

Multimodal Frustration Detection on Smartphones
Esther Vasiete and Tom Yeh
In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems
link
Write, Design, and 3D Print Tactile Stories for Visually Impaired: Critical Making in a Middle School Classroom
Abigale Stangl, Brian Jernigan and Tom Yeh
In Proceedings of the Conference on Creativity and Fabrication in Education
Transcribing Across the Senses: Community Efforts to Create 3D Printable Accessible Tactile Pictures for Young Children with Visual Impairments
Abigale Stangl, Caleb Hsu and Tom Yeh
In Proceedings of the International ACM SIGACCESS Conference on Computers and Accessibility
link
Collect, Decompile, Extract, Stats, and Diff: Mining Design Pattern Changes in Android Apps
Khalid Alharbi and Tom Yeh
In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services Companion
link
Seen music: ambient music data visualization for children with hearing impairments
Jeeeun Kim, Swamy Ananthanarayan and Tom Yeh
In Proceedings of the 14th International Conference on Interaction Design and Children
link
Toward 3D-Printed Movable Tactile Pictures for Children with Visual Impairments
Jeeeun Kim and Tom Yeh
In Proceedings of the ACM Conference on Human Factors in Computing Systems
link
Beyond the Flipped Classroom: Learning by Doing Through Challenges and Hack-a-thons
Mike Skirpan and Tom Yeh
In Proceedings of the ACM Technical Symposium on Computer Science Education
link
A Study to Empower Children to Design Movable Tactile Pictures for Children with Visual Impairments
Jeeeun Kim, Hyunjoo Oh and Tom Yeh
In Proceedings of the International Conference on Tangible, Embedded, and Embodied Interaction
link
Like This, But Better: Supporting Novices' Design and Fabrication of 3D Models Using Existing Objects
Patrick Carrington, Shannon Hosmer, Tom Yeh, Amy Hurst and Shaun Kane
In Proceedings of the iConference

2014

Tactile Picture Books for Young Children with Visual Impairment
Jeeeun Kim, Abigale Stangl, Ann Eisenberg and Tom Yeh
In Proceedings of the International Conference on Tangible, Embedded, and Embodied Interaction
link
Android apps consistency scrutinized
Khalid Alharbi, Sam Blackshear, Emily Kowalczyk, Atif Memon, Evan Chang and Tom Yeh
In Proceedings of the ACM Conference on Human Factors in Computing Systems
link
Abstracting Perception and Manipulation in End-User Robot Programming using Sikuli
Mike Kasper, Nikolaus Correll and Tom Yeh
In IEEE International Conference on Technologies for Practical Robot Applications
link
Evaluating Tactile User Experience with Tactile Picture Books for Children with Visual Impairment
Jeeeun Kim, Abigale Stangl, Ann Eisenberg and Tom Yeh
In "Touch Me" - Tactile Evaluation Methods Workshop
SikuliBot: Automating Physical User Interface Using Images
Jeeeun Kim, Mike Kasper, Tom Yeh and Nikolaus Correll
In Proceedings of the ACM Symposium on User Interface Software and Technology
link
Using LEGO to Model 3D Tactile Picture Books by Sighted Children for Blind Children
Jeeeun Kim, Abigale Stangl and Tom Yeh
In Proceedings of the 2nd ACM symposium on Spatial User Interaction
link
Technology to Support Emergent Literacy Skills in Young Children with Visual Impairments
Abigale Stangl, Jeeeun Kim and Tom Yeh
In Proceedings of the ACM Conference on Human Factors in Computing Systems
link
Screen-based active user authentication
Mohammed Fathy, Vishal Patel, Tom Yeh, Yangmuzi Zhang, Rama Chellappa and Larry Davis
Pattern Recognition Letters
link
NatureNet: a model for crowdsourcing the design of citizen science systems
Mary Lou Maher, Jenny Preece, Tom Yeh, Carol Boston, Kazjon Grace, Abhijit Pasupuleti and Abigale Stangl
In Proceedings of the ACM conference on Computer supported cooperative work
link
3D Printed Tactile Picture Books for Children with Visual Impairments: A Design Probe
Abigale Stangl, Jeeeun Kim and Tom Yeh
In Proceedings of the International Conference on Interaction Design and Children
link
Toward a non-intrusive, physio-behavioral biometric for smartphones
Esther Vasiete, Yan Chen, Ian Char, Tom Yeh, Vishal Patel, Larry Davis and Rama Chellappa
In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services Companion
link
A process model for crowdsourcing design: A case study in citizen science
Kazjon Grace, Mary Lou Maher, Jenny Preece, Tom Yeh, Abigale Stangl and Carol Boston
In Design Computing and Cognition
link
Crowdsourcing Design and Citizen Science Data using a tabletop in a nature preserve
Jenny Preece, Kazjon Grace, Carol Boston, Mary Lou Maher, Tom Yeh and Abigale Stangl
In European Conference on Social Media

2013

Predicting Users' First Impressions of Website Aesthetics With a Quantification of Perceived Visual Complexity and Colorfulness
Katharina Reinecke, Tom Yeh, Luke Miratrix, Rahmatri Mardiko, Yuechen Zhao, Jenny Liu and Krzysztof Z. Gajos
In Proceedings of the ACM Conference on Human Factors in Computing Systems
link
Exploring the internal state of user interfaces by combining computer vision techniques with grammatical inference
Paul Givens, Aleksandar Chakarov, Sriram Sankaranarayanan and Tom Yeh
In NIER
link
Screen Fingerprints: A Novel Modality for Active Authentication
Vishal Patel, Tom Yeh, Mohammed Fathy, Yangmuzi Zhang, Yan Chen, Rama Chellappa and Larry Davis
IT Professional
link

2012

Leveraging user comments for aesthetic aware image search reranking
Jose San Pedro, Tom Yeh and Nuria Oliver
In Proceedings of the International World Wide Web Conference
link
Co-designing an e-health tutorial for older adults
Bo Xie, Tom Yeh, Greg Walsh, Ivan Watkin and Man Huang
In Proceedings of the iConference
link

Grants

PI

NSF CAREER: Adaptive Tactile Picture Books for Blind Children during Emergent Literacy
Tom Yeh (PI), $549,999, 2015-2020.
Piton Foundation Early Childhood Innovation Prize : MindScribe: Reflective Inquiry through Scaffolded Storytelling for Low-Income and Multilingual Early Childhood Communities
Tom Yeh (PI), Layne Hubbard (Student Lead), $80,000, 2018-2019.
NSF ITEST: Designing Tactile Picture Books: Critical Making in Libraries to Broaden Participation in STEM
Tom Yeh (PI), Bridget Dalton and Stacey Forsyth (co-PIs), $1,199,883, 2016-2019.
CU Outreach Award: Custom 3D Books for 50 Families Worldwide
Tom Yeh (PI), Tanya Ennis, Stacey Forsyth and Bridget Dalton (co-PIs), $24,000, 2017-2018.
NSF CHS: REU Supplement: CAREER: Adaptive Tactile Picture Books for Blind Children during Emergent Literacy
Tom Yeh (PI), $16,000, 2017-2018.
NSF AISL: Supplement: AISL: Innovations in Development: Community-Driven Projects That Adapt Technology for Environmental Learning in Nature Preserves
Tom Yeh (PI), $50,000, 2017-2018.
Knight Foundation News Challenge: Quantified Self Data Experience: Understanding Your Data and the World it Creates
Tom Yeh (PI), Mike Skirpan (Student Lead), $35,000, 2016-2017.
CU Outreach Award: Tactile Picture Books: 50 Books Project
Tom Yeh (PI), Tanya Ennis (co-PI), $8,000, 2016-2017.
CU ATLAS Mini-Grant: Data Science for Social Good: Hackathons and Public Exhibits to Engage Students and Increase Campus Awareness
Tom Yeh (PI), $8,000, 2014-2015.
CU President's Fund for the Humanities: Public Tactile Art on Campus: Integrating Art and Engineering to Promote Universal Access
Tom Yeh (PI), $3,000, 2014-2015.
NSF AccessComputing Mini-Grant: Teaching Blind High School Students Programming via a Guessing Game
Tom Yeh (PI), $5,000, 2013-2014.
CU Outreach Award: Teaching 3D Printing to the Teachers and Parents of Blind Children
Tom Yeh (PI), $8,000, 2013-2014.
CU Diversity and Excellence Grant: Broadening Participation of Blind Students in Computer Science at CU through Scalable Game Design
Tom Yeh (PI), $3,000, 2012-2013.

Co-PI

NIH R21: Developing an eHealth Tutorial for Older Adults to Use Online Health Resources
Bo Xie (PI), Tom Yeh (co-PI) , Total: $425,208 (Yeh's Share: $104,415), 2017-2019.
NSF AISL: Innovations in Development: Community-Driven Projects That Adapt Technology for Environmental Learning in Nature Preserves
Jenny Preece (PI), Tom Yeh, Tammy Cleggs and Mary Lou Maher (co-PIs) , Total: $2,390,552 (Yeh's Share: $306,508), 2015-2018.
DARPA MUSE: Mining and Understanding Bug Fixes to Address Application-Framework Protocol Defects
Evan Chang (PI), Sriram Sankaranarayanan, Tom Yeh, Pavol Cerny and Ken Anderson (co-PIs) , Total: $1,599,343 (Yeh's Share: $319,865), 2014-2018.
NSF EAGER: Computational Model for Evaluating the Quality of Citizen Science Contributions
Mary Lou Maher (PI), Tom Yeh and Jenny Preece (co-PIs) , Total: $175,099 (Yeh's Share: $39,999), 2014-2015.
DARPA APAC: Specialized Binary Analysis for Vetting Android Apps Using GUI Logic
Atif Memon (PI), Tom Yeh and Evan Chang (co-PIs) , Total: $483,734 (Yeh's Share: $202,774), 2013-2015.
DARPA Active Authentication: Visual Fingerprint for Active Authentication
Rama Chellappa (PI), Tom Yeh and Larry Davis (co-PIs) , Total: $500,000 (Yeh's Share: $100,000), 2013-2014.
NSF VOSS: Crowdsourcing Interaction Design for Citizen Science Virtual Organizations
Mary Lou Maher (PI), Tom Yeh and Jenny Preece (co-PIs) , Total: $399,874 (Yeh's Share: $52,168), 2012-2014.
DARPA Active Authentication: Screen Fingerprint for Active Authentication
Rama Chellappa (PI), Tom Yeh and Larry Davis (co-PIs) , Total: $500,000 (Yeh's Share: $124,205), 2012-2013.