1. Activity Monitoring in Multi-Inhabitant Smart Home Enviornment

2. An Energy Efficient Quality Adaptive Multimodal Sensor Network Framework for Context Recognition

3. PAP-C: Design and Implementation of a Fine-Grained Power-line Based Appliance Profiling & Control Architecture

4. QoI-aware Resource and Task allocation for multiple applications in wireless sensor networks

5. Internet of Things: Accessing Sensors through the Web

Project Descriptions:

For each individual research project details please check out our Lab Homepage.

A. Building Energy Analytics

A1. Acoustic Enabled Fine-Grained Energy Analytics

Fine-grained monitoring of everyday appliances can provide better feedback to the consumers and motivate them to change behavior in order to reduce their energy usage. It also helps to detect abnormal power consumption events, long-term appliance malfunctions and potential safety concerns. Commercially available plug meters can be used for individual appliance monitoring but for an entire house, each such individual plug meters are expensive and tedious to setup. Alternative methods relying on Non-Intrusive Load Monitoring techniques help disaggregate electricity consumption data and learn about the individual appliance’s power states and signatures. However fine-grained events (e.g., appliance malfunctions, abnormal power consumption, etc.) remain undetected and thus inferred contexts (such as safety hazards etc.) become invisible. In this project, we correlate an appliance’s inherent acoustic noise with its energy consumption pattern individually and in presence of multiple appliances. Our approach helps to improve the performance of energy disaggregation algorithms and provide critical insights on appliance longevity, abnormal power consumption, consumer behavior and their everyday lifestyle activities.

A2. AARPA: Mobile and Power-line Sensing for Fine-grained Appliance Usage and Energy Monitoring

To promote energy-efficient operations in residential and office buildings, non-intrusive load monitoring (NILM) techniques have been proposed to infer the fine-grained power consumption and usage patterns of appliances from powerline measurement data. Fine-grained monitoring of everyday appliances (such as toasters and coffee makers) can not only promote energy-efficient building operations, but also provide unique insights into the context and activities of individuals. Current building-level NILM techniques are unable to identify the consumption characteristics of relatively low-load appliances, whereas smart-plug based solutions incur significant deployment and maintenance costs. In this project, we investigate an intermediate architecture, where smart circuit breakers provide measurements of aggregate power consumption at room (or section) level granularity. We then investigate techniques to identify the usage and energy consumption of individual appliances from such measurements. To further improve the accuracy of appliance level usage estimation, we have been investigating a hybrid system called AARPA, which uses mobile sensing to first infer high-level activities of daily living (ADLs), and then uses knowledge of such ADLs to effectively reduce the set of candidate appliances that potentially contribute to the aggregate readings at any point.

Datasets:

A3. Energy Disaggregation

Energy Disaggregation gives the itemized energy consumption of the appliances. Research suggested that the itemized energy consumption may help reduce 15% of total residential energy consumption. Most of the commercially available energy analytic systems are intrusive and expensive. In the US about 33 million smart meters have been deployed which gives us huge amount of data. Even if we narrow our scope to a city, the number of houses are above 500000, and the utility providers only provide feedback about the cumulative monthly consumption and some comparative analytic with neighbors’ total consumption. In these cases itemized consumption and comparative analytic will prove more helpful. Our objective is to look for a scalable disaggregation algorithm for big data. The underlying assumption is appliances might have similar energy patterns and we can learn those characteristics from a set of appliances and transfer the knowledge for discovering the energy patterns in a larger set.

People:

Nilavra Pathak (Ph.D. student), Hafiz Khan (Ph.D. student), Joseph Taylor (undergraduate student), Dr. Nirmalya Roy (MPSC director), Dr. George Karabatis (collaborator at UMBC), Dr. Aryya Gangopadhyay (collaborator at UMBC) and Dr. Archan Misra (collaborator at SMU).

Papers:

1. Nirmalya Roy, Nilavra Pathak, and Archan Misra, “Fine-grained Appliance Usage and Energy Monitoring through Mobile and Power-line Sensing”, Pervasive and Mobile Computing (PMC) Journal, Elsevier 2016 [pdf]
2. Nilavra Pathak, Nirmalya Roy, and Animikh Biswas “Iterative Signal Separation Assisted Energy Disaggregation”, IEEE 6th International Green and Sustainable Computing Conference (IGSC 2015), Dec 2015. Las Vegas, USA [pdf]
3. Md. Abdullah Al Hafiz Khan, H M Sajjad Hossain, and Nirmalya Roy. “Infrastructure-less Occupancy Detection and Semantic Localization in Smart Environments”, 12th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous) July 2015, Coimbra, Portugal. [pdf]
4. Nirmalya Roy, Nilavra Pathak, and Archan Misra. “AARPA: Combining Mobile and Power-line Sensing for Fine-grained Appliance Usage and Energy Monitoring”, IEEE International Conference on Mobile Data Management (MDM), June 2015.
5. Md. Abdullah Al Hafiz Khan, H M Sajjad Hossain, Nirmalya Roy. “SensePresence: Infrastructure-less Occupancy Detection for Opportunistic Sensing Applications”,  International Workshop On Human Mobility Computing And Privacy (HuMoComP’15) Pittsburgh.  June, 2015. [pdf]
6. Nilavra Pathak, Md. Abdullah Al Hafiz Khan, and Nirmalya Roy. “Acoustic based appliance state identifications for fine grained energy analytics”, IEEE International Conference on Pervasive Computing and Communications (PerCom), March 2015.
7. Md. Abdullah Al Hafiz Khan, Sheung Lu, Nirmalya Roy, and Nilavra Pathak. “Demo Abstract: A Microphone Sensor based System for Green Building Applications”, IEEE International Conference on Pervasive Computing and Communications Demonstrations  (PerCom), March 2015.
8. Nirmalya Roy, David Kleinschmidt, Joseph Taylor, and Behrooz Shirazi. “Performance of the Latest Generation Powerline Networking for Green Building Applications,” 5th ACM Workshop on Embedded Systems For Energy-Efficient Buildings (BuildSys), November 2013.
9. Joseph Taylor, Nirmalya Roy, David Kleinschmidt, and Behrooz Shirazi “Demo Abstract: Performance of the Latest Generation Powerline Networking for Green Building Applications,” 5th ACM Workshop on Embedded Systems For Energy-Efficient Buildings Demonstrations (BuildSys), November 2013.

Press:

Constellation Awards E2 Energy to Educate Grant to UMBC’s Information Systems Department, UMBC’s Giving Blog, April 21, 2015

COEIT Uses Constellation Energy’s Education Award For Undergraduate Research, UMBC Insights, March 27, 2014

Undergrad researchers succeed with the smart plug work, WSU News, August 12, 2013

WSU Smart Environments Summer REU Explores Green Living and Healthcare Applications, WSU News, July 11, 2013

 

B. Smart Home Activity and Health Analytics

B1. Multi-inhabitant Activity Recognition

Recognizing activities of multiple inhabitants poses fundamental challenges of defining, classifying, training, building, and scaling the underlying state space model in an effective way. Scaling the activity recognition techniques beyond a single inhabitant requires efficient design of novel algorithmic techniques to handle the exponential increase in the number of states present in a multi-inhabitant scenario. In this project, we have been exploiting the spatiotemporal constraints and correlations for recognizing intra- and inter-relationships across multiple inhabitants at the micro, meso and macro-level of activities to build an activity model which scale beyond an individual inhabitant and single smart home.

B2. Real Time Activity Recognition

Automated activity recognition enables a wide variety of applications related to functional, behavioral, and cognitive health, disease diagnosis and treatment, personal health and wellness management to name a few. Given the proliferation of smart and wearable devices and their greater acceptance in human lives, the need for developing real time lightweight activity recognition algorithms has become a viable and urgent avenue. Although variants of online and offline lightweight activity recognition algorithms have been developed, realizing them on real time to recognize people’s activities is still a challenging research problem due to the computational complexity of building, training, learning and storing activity models in resource constrained smart and wearable devices in presence of an impeding scarcity of just-in-time annotated datasets. To navigate the above challenges, we have been exploring to build lightweight activity recognition algorithm and design their use cases in practice.

B3. Sleep Monitoring for Community Scaling

Following healthy lifestyle is a key for active living. Regular exercise, controlled diet and sound sleep play an invisible role on the wellbeing and independent living of the people. Sleep being the most durative activities of daily living (ADL) has a major synergistic influence on people’s mental, physical, and cognitive health. Understanding the sleep behavior and its underpinning clausal relationships with physiological signals and contexts (such as eye or body movement etc.) responsible for a sound or disruptive sleep pattern help provide meaningful information for promoting healthy lifestyle and designing appropriate intervention strategy. In this project, we are working on detecting the microscopic states of the sleep which fundamentally constitute the components of a good or bad sleeping behavior and help shape the formative assessment of sleep quality. For a larger deployment of our proposed model across a community of individuals we have been investigating an active learning-based methodology for reducing the effort of ground truth data collection.

B4. Behavioral Health Prediction

To promote independent living for elderly population activity recognition based approaches have been investigated deeply to infer the activities of daily living (ADLs) and instrumental activities of daily living (I-ADLs). Deriving and integrating the gestural activities (such as talking, coughing, and deglutition etc.) along with activity recognition approaches can not only help identify the daily activities or social interaction of the older adults but also provide unique insights into their long-term health care, wellness management and ambulatory conditions. Gestural activities (GAs), in general, help identify fine-grained physiological symptoms and chronic psychological conditions which are not directly observable from traditional activities of daily living. In this project, we are developing GeSmart, an energy efficient wearable smart earring based gestural activity recognition model for detecting a combination of speech and non-speech events.

Dataset: Data-Set.zip.

Sources: change_detection.zip.

Project Sponsor:

This project is funded, in part, by the UMB-UMBC Research and Innovation Partnership Grant. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Sponsored Agency.

People:

Mohammad Arif Ul Alam (Ph.D. student), H M Sajjad Hossain (Ph.D. student), Md Abdullah Al Hafiz Khan (Ph.D. student), Joseph Taylor (undergraduate student), Dr. Nirmalya Roy (MPSC director).

Collaborators: Dr. Elizabeth Galik (Prof. of Nursing at UMB), Dr. Aryya Gangopadhyay (Prof. of Information Systems at UMBC), Dr. Zhiyuan Chen (Prof. of Information Systems at UMBC), Dr. Gunes Koru (Prof. of Information Systems at UMBC), Dr. Nilanjan Banerjee (Prof. of CSEE at UMBC), Dr. Judah Ronch (Prof. at Erickson School of Aging), and Dr. Diane Cook (Prof. of EECS at WSU).

Papers:

1. Mohammad Arif Ul Alam, Nirmalya Roy, Aryya Gangopadhyay and Elizabeth Galik, “A Smart Segmentation Technique Towards Improved Infrequent Non-Speech Gestural Activity Recognition Model”, Pervasive and Mobile Computing (PMC) Journal, Special Issue on Gerontechnology, Elsevier 2016 (in press)
2. Nirmalya Roy, Archan Misra, Sajal K. Das and Christine Julien, “Quality and Context Aware Smart Healthcare”, IEEE Systems, Man, and Cybernetics Magazine 2016 (in press)
3. Nirmalya Roy, Archan Misra, Sajal K. Das and Christine Julien, “Determining Quality- and Energy-Aware Multiple Contexts in Pervasive Computing Environments”, IEEE/ACM Transactions on Networking 2016 [pdf]
4. Nirmalya Roy, Archan Misra, and Diane Cook, “Ambient and Smartphone Sensor Assisted ADL Recognition in Multi-Inhabitant Smart Environments”, Journal of Ambient Intelligence and Humanized Computing, 7(1): 1-19 Springer 2016 [pdf]
5. Mohammad Arif Ul Alam, Nirmalya Roy, Michelle Petruska, Andrea Zemp, “Smart-Energy Group Anomaly Based Behavioral Abnormality Detection”, in Proc. of IEEE Wireless Health 2016, Washington D.C. October 2016
6. Mohammad Arif Ul Alam, Nirmalya Roy, Archan Misra, Joseph Taylor, “CACE: Exploiting Behavioral Interactions for Improved Activity Recognition in Multi-Inhabitant Smart Homes”, in Proc. 36th IEEE International Conference on Distributed Computing Systems (ICDCS) Nara, Japan, June 2016 [acceptance rate: 17%] [pdf]
7. Mohammad Arif Ul Alam, Nirmalya Roy, Sarah Holmes, Aryya Gangopadhyay, Elizabeth Galik, “Automated Functional and Behavioral Health Assessment of Older Adults with Dementia”, in Proc. First IEEE Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), Washington D.C. June 2016 [pdf]
8. Md Abdullah Al Hafiz Khan, Ruthvik Kukkapalli, Piyush Waradpande, Sekar Kulandaivel, Nilanjan Banarjee, Nirmalya Roy, Ryan Robucci “RAM: Radar based Activity Monitor”. IEEE International Conference on Computer Communications (INFOCOM), April 2016 [acceptance rate: 18%] [pdf]
9. H M Sajjad Hossain, Nirmalya Roy, Md Abdullah Al Hafiz Khan. “Active Learning Enabled Activity Recognition” IEEE International Conference on Pervasive Computing and Communications (PerCom), March 2016 [acceptance rate < 15%] [pdf]
10. Mohammad Arif Ul Alam, Nilavra Pathak, and Nirmalya Roy. “Mobeacon: An iBeacon-Assisted Smartphone-based Real Time Activity Recognition Framework”, in Proc. of the 12th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous) July 2015, Coimbra, Portugal. [pdf]
11. H M Sajjad Hossain, Nirmalya Roy, and Hafiz Khan. “Sleep Well: A Sound Sleep Monitoring Framework for Community Scaling”, in Proceeding of the IEEE International Conference on Mobile Data Management (MDM), June 2015.
12. Mohammad Arif Ul Alam and Nirmalya Roy, “Gesmart: A gestural activity recognition model for predicting behavioral health”, in Proceeding of the IEEE International Conference on Smart Computing (SmartComp), November 2014.
13. Nirmalya Roy and Christine Julien, “Immersive physiotherapy: Challenges for smart living environments and inclusive communities”, in Proceeding of the 12th International Conference on Smart Homes and Health Telematics (ICOST), June 2014.
14. Nirmalya Roy and Brooks Reed Kindle, “Monitoring patient recovery using wireless physiotherapy devices”, in Proceeding of the 12th International Conference on Smart Homes and Health Telematics (ICOST), June 2014.

Patent

Nirmalya Roy, H M Sajjad Hossain, and Mohammad Arif Ul Alam. “Scalable Activity Recognition for Independent Living Applications”, UMBC Ref. No. 2015-004.

Press

UMBC-UMB Partnership Awards a Catalyst for Collaboration, Research at UMBC News, Feb 17, 2015

UMBC and UMB celebrate research partnerships with symposium and innovation grants, UMBC News, Feb 12, 2015

Using Sensor Technology to Lower Elder Care Costs, The Wall Street Journal, July 28, 2014

Movement sensor’s feedback acts as physical therapist, WSU News, July 30, 2013

WSU Smart Environments Summer REU Explores Green Living and Healthcare Applications, WSU News, July 11, 2013

 

C. Development of Pervasive Low-cost Sensor Technologies

C1. Power Line Modems for Low-Volume Telemetry

Research areas like in-home monitoring require a robust infrastructure for communicating sensor telemetry. Common, off-the-shelf ISM-band radio communications devices are commonly used in practice to fill this need. However, issues with building architecture or radio interference may cause unexpected data loss and “black spots” that are hard to locate and characterize, and may result in anomalous data loss that stifles research progress. This project is investigating power line communications (“PLC”) as a more consistent alternative to technologies like 802.15.4 and Bluetooth.

C2. Embedded Sensors and Computing Devices for Cost-Sensitive Research Applications

Sensing research often necessitates the development of custom devices to enable the collection of data, but full-custom devices are both expensive and time-consuming to design and manufacture. Minimizing hardware costs is critical to enabling cutting-edge research. This project aims to meet the needs of researchers while also allowing hardware cost reduction via novel modification of off-the-shelf hardware in combination with low-volume custom hardware supplements.

C3. A Sensor Technology Box for Smart Health (SenseBOX)

The U.S. Census Bureau reports that the U.S. population of people aged 65 and up will grow more than double in between 2010 and 2050. The market for remote patient monitoring is expected to grow from $10.6 billion in 2012 to $21.2 billion in 2017. Inspired by this growing market need, we propose to build an integrated system consisting of commercially available off the-shelf sensors and servers that can be deployed for independent living applications across multiple homes for monitoring activities of daily living (ADLs) of older adults. Existing smart home technologies are mostly targeted at technology enthusiasts, with some limited offerings in smart health. They provide simple monitoring and control, e.g., times of activity and the ability to turn on and off lights. They range from fairly inexpensive to very expensive while being produced in limited quantity. We propose to develop a system which can be very price-competitive, especially by partnering with people already producing devices that can be easily modified to run our software (removing most of the need for full-custom hardware development associated with a substantial cost). Existing technologies also fail to analyze data in real time; while they can provide information about e.g., weather a person has been active or taking medications, we can mine data for more subtle trends that may indicate depression or illness that could warrant intervention. When compared to existing smart heath technologies, we can provide more useful information (detection and assessment of both ADLs and cognitive impairments) while still meeting or beating the price of the competition and promote the large scale deployment in real-life setting.

Easy-to-construct Raspberry Pi-based Outreach Device

 

 

 

 

 

 

 

 

 

System Development

A Linux kernel and base system has been built for the CloudEngines PogoPlug and tools to communicate with the Cao Gadgets Wireless Sensor Tags have been developed. The PogoPlug is currently acting as a bridge between the Sensor Tags and a standard 802.11N network in an in-lab test configuration. Tests have indicated the device is stable in this capacity, and recovers from power loss and outages on the external networks without issue. The PogoPlug is also successfully doing NAT translation in order to bridge additional Ethernet devices on to the network. Devices like an IP camera used for collection of ground truth benefit from such bridging.

People:

Nirmalya Roy, Joseph Taylor (Undergraduate Student).

 

The research projects are funded and sponsored by the following federal, public, and private agencies.