Our research on pervasive sensing focuses around the concept of Computational Ethnography which borrows the in-depth understanding of activity and people’s behavior from ethnography and anthropology, and combine it with the potential of capturing large sets of rich multimodal data from environmental and body-worn sensors and technology. Through the use of a variety of tracking technology (body-tracking, eye-tracking, social media, etc.) we study human behavior in context and the interaction of individuals with technology.
Pervasive Sensing and Health
To support pervasive sensing we developed Lab-in-a-Box — a novel multimodal data collection infrastructure based on eye, body, gesture, voice and activity tracking — and we enhanced our analysis and visualization tools to better understand large amount of collected data. This includes the extension and integration of ChronoViz, our analysis tool for multimodal time-synchronized data, as well as novel web-based visualization and data analysis tools.
Our tools and techniques are used for in-depth study of activity in the wild, for instance we characterize interaction with Electronic Medical Records and understand patient-physician communication, to study sign language, to understand the ergonomics of laparoscopic and robotic surgery, to uncover multimodal cues of neurological disorders such as stroke, and to exploit social interactions online as a support for real-world intervention in the context of HIV/AIDS.
Ubiscope
Ubiscope, the Ubiquitous Computing Microscope is a new kind of microscope for understanding people through the unobtrusive and pervasive sensing of physiological and behavioral data. As the microscope enabled uncovering new understanding and supported key scientific advances, UbiScope acts as a similar proxy for understanding and supporting human activity.
- UbiStroke: Multimodal Computational Assessment of Stroke
- ErgoKinect: Detection of Ergonomically Incorrect Posture during Laparoscopic Surgery
- Effect of Packaging on Smoking Perception and Behavior: a Randomized Control Trial
Funding
 |
Stroke-Kinect: A Sensor-Based Approach Towards Creating a Multimodal Stroke Signature |
 |
Effect of Packaging on Smoking Perception and Behavior: a Randomized Control Trial |
 |
SL-CN: Learning to Move and Moving to Learn |
Publications
Computational Ethnography
The goal of this project is to accelerate observational analysis by employing multi-modal pattern recognition capabilities to pre-segment and tag records and increase analysis power by collecting multimodal activity and enable the investigation of macro-micro relationships of people’s interaction sin the wild, among themselves and with technology.
- QUICK: Quantifying Electronic Medical Record Usability to Improve Clinical Workflow
- Hands That Speak: Studying Complex Human Communicative Body Movements
Funding
|
A Multiscale Framework for Analyzing Activity Dynamics (US NSF IIS-0729013). October 2009 – August 2011 |
 |
Flight Crew Performance Data Collection and Analysis Tool (UCSD – Boeing Project Agreement 2011-012 / Boeing Project Agreement 2012) January 2011 – September 2012 |
 |
QUICK – Quantifying Electronic Medical Record Usability to Improve Clinical Workflow. (AHRQ R01 – HS 021290) September 2012 – August 2016. |
Publications
Connected and Open Research Ethics
The employment of pervasive sensing technology opened up the way for a critical consideration of research ethics. This is especially important with regards to the collection of behavioral, social, and personal data in clinical or health-related settings. To study these important ethical dilemmas, we created the CORE (Connected and Open Research Ethics) initiative and are actively investigating how to redefine research ethics.
Funding
 |
CORE – Designing, building, and testing a Web-based prototype to foster the ethical design and review of health research (RWJF Pilot Pioneer, November 2015 – October 2017) |
Publications
Social Media as a Sensor
We live in an era where much of our lives are represented online through a variety of social media such as Facebook, Twitter, Instagram, etc. The intense communication that happens across friends and acquaintances and the networks that are created around those exchanges of textual and multimedia messages create incredible opportunities to better understand a variety of behavioral clue that might help resolve health and healthcare related problems. We are using a mixed approach based on natural language processing and network analysis to capitalize on these issues to help drive intervention, specifically in the setting of HIV and Physical Activity.
- PIRC-Net: Characterizing HIV at-risk
- SMART: Using Social Media and Mobile Technologies to Promote Improved Health Behaviors
Funding
 |
PIRC-Net: PIRC-NET: Analyzing Social Media to characterize HIV at-risk populations among MSM in San Diego. (CFAR Developmental Grant, January 2015 – March 2017) |
 |
Detecting HIV at-risk MSM in San Diego through Social Networks (FISP 2014, UCSD Frontiers of Innovation Scholars Program, January 2015 – October 2016) |
 |
Characterizing the Relationship of METH Use and Neurocognitive Impairment in HIV At-risk Online Networks (TMARC Pilot PST7TP2/2016, Translational Methamphetamine AIDS Research Center, March 2016 – February 2018) |
Publications
Understanding Patient-Physician Communication
Communication in the medical office between physician and patient is challenging. This is often complicated by third party in the room such as relatives or an interpreter if the patient is not mother language and by the presence of the Electronic Health Record (EHR). We want to better understand the system defined by the medical office and the role that people, artifacts and technology play in this environment to inform the design of the next generation interactive tools and visualization.
Funding
 |
QUICK – Quantifying Electronic Medical Record Usability to Improve Clinical Workflow (AHRQ R01 – HS 021290, September 2012 – June 2016) |
 |
DEUCE – Design and Evaluation of User Centered Electronic Health Records (VA Merit 1 I01-HX000982-01A1) April 2015 – Ongoing |
Publications
Archived Projects
Publications – Pervasive Sensing and Health
