↓ Skip to main content

Health Smart Home (HIS) datasets

HIS project partners

A French corpus of audio and multimodal interactions in a health smart home.

Important notice
#

These data have been acquired in the context of a PhD thesis, in a laboratory that works on Health Smart Homes. These databases, devoted to activity recognition and distress recognition, are distributed free of charge, for academic and research use only, in order to be able to compare the results obtained. By downloading these anonymous data, you agree to the following limitations:

  • Commercial use: this database is not designed for any commercial use and should only be used by academic researchers.
  • Publications: any publication using the activity database should include a citation of the article: Fleury, A.; Vacher, M. and Noury, N.; « SVM-Based Multi-Modal Classification of Activities of Daily Living in Health Smart Homes: Sensors, Algorithms and First Experimental Results ». IEEE Transactions on Information Technology in Biomedicine, Vol. 14(2), March 2010, pp. 274-283 (DOI: 10.1109/TITB.2009.2037317).
  • Redistribution: this database cannot be, totally or partially, further distributed, published, copied or disseminated, in any way or form, for profit or not.
  • In the same way, all the publications that use the distress call dataset must include a reference to the following book chapter: Vacher, M., Fleury, A., Portet, F., Serignat, J.F., Noury, N.: “Complete Sound and Speech Recognition System for Health Smart Homes: Application to the Recognition of Activities of Daily Living”, in New Developments in Biomedical Engineering, pp. 645–673. Intech Book (2010).

For all questions or concerns, do not hesitate to contact us by email: anthony.fleury@mines-douai.fr.

Information on these datasets, their contents and the way they are indexed can also be found in the article: Fleury, A.; Vacher, M.; Portet, F.; Chahuara, P. and Noury, N.: “A French corpus of audio and multimodal interactions in a health smart home”. Journal on Multimodal User Interfaces, Springer, 2013, 7 (1), pp. 93-109 (DOI: 10.1007/s12193-012-0104-x).

Context of the acquisitions
#

The acquisitions presented here were made during a PhD thesis, in the Health Smart Home of the TIMC-IMAG Laboratory (UMR CNRS 5525), team AFIRM (now part of the AGIM Laboratory, FRE 3405, CNRS-UJF-EPHE). The PhD thesis was advised by Norbert Noury (TIMC-IMAG, now member of the INL Laboratory in Lyon) and Michel Vacher (LIG Laboratory, team GETALP).

The smart home of the TIMC-IMAG laboratory
#

The smart home of the TIMC-IMAG lab is a flat containing a kitchen, a living room, a bedroom, a bathroom, a hall and toilets. This flat is completely equipped and contains everything that can be used for the different activities in the home (including books, clothes, TV, radio, microwave oven, fridge, food, etc.).

It is also equipped with several sensors such as microphones, presence infra-red sensors and webcams. It also contains a temperature and hygrometry sensor inside the bathroom, and the dweller wears a home-made accelerometer and magnetometer circuit placed under the left armpit.

The HIS flat and its sensors

Finally, all the data are stored on different computers placed in the technical room, just next to the flat.

Distress call dataset
#

Experimental protocol
#

The protocol was quite simple. Each participant was alone in the flat. The participant had to go to the living room and close the door. Then, they had to move to the bedroom and read aloud the first half of one of the five successions of sentences, out of 10 normal and 20 distress sentences. Afterwards, the second half of the set of sentences had to be uttered in the living room. Finally, each participant was called 3 times and had to answer the phone and read the predefined phone conversation (5 sentences each). The sentences were uttered in the flat, with the participant sitting down or standing up. They had no instructions on how to orient themselves relative to the microphones or on the way they pronounced the different sentences.

Dataset content and organisation
#

This dataset has been acquired on 10 different subjects, all native French speakers:

IDAgeGenderWeight (kg)Height (m)Sentence set
161M821.701
222M771.772
353M671.683
447M561.704
531M621.705
625M781.842
751F641.654
829F531.603
928F611.691
1025M901.885

The last column gives the number of the sentence set each subject pronounced: there were 5 different sets of sentences (normal, distress and phone conversations). The sets were distributed as a spreadsheet, one sheet per set (no longer available online).

The dataset is organised as follows:

  • A directory containing the complete dataset for each subject, with ALL the sounds captured during the sessions. For each subject:
    • 8 directories, one per channel;
    • for each channel, the files of each acquisition: mainly the WAV file with the acquired sound and the XML file with its analysis by our system (segmentation as sound or speech; for sounds, the sound class; for speech, the five most probable sentences);
    • a text file (Resultats.txt) with a first analysis: for sounds, the SNR, the ID of the corresponding WAV/XML files and the sound class; for speech, the SNR, the ID of the WAV/XML files and the three most probable sentences.
  • A directory containing, for each speaker, only the files of the sounds with the best SNR, for each sentence uttered during the experiment (all other sounds have been deleted). The order of the sentences follows the spreadsheet.
  • A directory containing all the sound (and associated) files with the best SNR and all sounds with an SNR above 80% of that value. The order of the sentences is kept, and the XML files contain the timestamps and the identification of all simultaneous sounds, so that each sound can be matched to its sentence.

More information on the experiment, the datasets and the processing is given in the references [Vacher2008], [Vacher2010] and [Vacher2011] below.

Data
#

IDWhole setBest SNRSNR ≥ 80% of best
1zip (19 MB)zip (2.1 MB)zip (5 MB)
2zip (22 MB)zip (2.2 MB)zip (4.9 MB)
3zip (21 MB)zip (2.1 MB)zip (5.3 MB)
4zip (22 MB)zip (2.2 MB)zip (5 MB)
5zip (20 MB)zip (2.1 MB)zip (4.6 MB)
6zip (32 MB)zip (2.1 MB)zip (4.9 MB)
7zip (32 MB)zip (2.2 MB)zip (4.2 MB)
8zip (27 MB)zip (1.7 MB)zip (3.6 MB)
9zip (21 MB)zip (2.1 MB)zip (4.4 MB)
10zip (19 MB)zip (1.8 MB)zip (3.3 MB)
Allzip (231 MB)zip (21 MB)zip (45 MB)

Activities of Daily Living dataset
#

Experimental protocol
#

The first step of the experiment was a thorough visit of the flat, to allow the person to act as much as possible as if they were at home. After this visit, the only requirement was to perform a set of 7 activities, in the order and for the duration they wanted, and as many times as they wanted:

  • Sleeping: lie in the bed and stay “asleep” for a reasonable time (of your choice).
  • Resting: sit on the couch or a chair in the living room and do what you like when relaxing (watching TV, reading, doing nothing…).
  • Feeding: prepare a breakfast with the equipment and ingredients in the kitchen cupboard, then eat at the kitchen table, then wash the dishes in the kitchen sink.
  • Hygiene: wash your hands and face, and pretend to brush your teeth in the bathroom.
  • Toilets: pretend to go (or go) to the toilet (sit on the toilet, flush…).
  • Dressing: put on and remove the clothes in the chest of drawers near the bed (over your own clothes), before and after sleeping or after the hygiene task.
  • Communication: you will be called on the phone (in the living room) several times; each time, read the sentences indicated near the phone (one long phone conversation and three short ones).

For more information about the dataset and the experiments, see the references [Fleury2010], [Fleury2011], [Vacher2011] and [Noury2011] below.

Dataset content, organisation and download
#

The data of 2 of the 15 subjects are available on this page. To obtain the complete dataset with all participants, fill in the agreement form (PDF) and send it to anthony.fleury@mines-douai.fr, committing to use the dataset correctly; you will then receive the download link for the remaining participants.

IDAgeGenderHeight (m)Weight (kg)Phone conversationNative French speaker
125F1.77631yes
225F1.78702yes
332M1.92803yes
457F1.48504no
525M1.72815yes
626M1.82661no
728M1.78682yes
831M1.70633yes
926M1.84784yes
1024F1.62685no
1129F1.63571yes
1240F1.76602yes
1333M1.75793no
1434M1.85764yes
1543M1.72695yes

The phone conversations were distributed as a spreadsheet (no longer available online).

Download
#

All files are gzipped tar archives.

IDVideoAnnotationsAudioActimetry (raw)SensorsComplete set
202_video (719 MB)02_annotations (6.3 kB)02_son (46 MB)02_actim6d (4.1 MB)02_sensors (36 kB)02_all (768 MB)
909_video (528 MB)09_annotations (5.8 kB)09_son (38 MB)09_actim6d (2.8 MB)09_sensors (35 kB)09_all (568 MB)

Video
#

Video mosaic, subject 2
Video mosaic, subject 9

The videos come from 4 cameras, compressed in MPEG-4 (Xvid) and mixed into a single video: the kitchen at the top left, the kitchen/hall/bathroom door at the bottom left, the bedroom and living room at the top right, and the living room only at the bottom right. Above, subject 2 (left) and subject 9 (right).

For ethical reasons there is no camera in the bathroom and toilets. When entering this part of the flat, the dweller partially closed the door to go to the bathroom (hygiene activity) and closed it completely to go to the toilets.

Activity annotation
#

An XML file annotates the activities performed by the dweller during the session.

It first gives the information needed to synchronise the actimetry sensor (ACTIM6D) with the other data: all data are synchronised because the clocks of all acquisition computers were synchronised to the same NTP server. For this, the position corresponds to the date given in the ACTIM6D_Synchro section of the XML file.

It then gives the Activities of Daily Living performed during the session. For each activity, two timestamps are given: the beginning (debut) and the end (fin). In Activite, the id identifies the annotated activity:

IDActivity
1Sleeping
2Resting (TV, reading, radio…)
3Dressing/undressing
4Eating
5Toilet use
6Hygiene
7Communication

In addition, an Advene annotation file contains:

  • the ground-truth annotation of activities (as above, with file positions instead of timestamps);
  • the ground-truth annotation of posture (for some of the subjects, see Actimetry below);
  • the ground-truth annotation of the use of furniture (see below).

Both files are in the Annotations archive: Y.xml and AdveneAnnotation_Y.xml, where Y is the subject ID.

Audio
#

As for the distress call dataset, there is one directory per microphone and a text file summarising the classification of each sound that occurred in the flat. All files are in one archive.

Actimetry
#

ACTIM6D is a home-made circuit board with a 3-axis accelerometer and a 3-axis magnetometer, fixed under the left armpit of the subject in a pocket of a specific T-shirt. Its data are used to detect postural transitions and walking.

The raw data are given so that users can implement their own postural-transition detection (transitions can be annotated using the video). Each line of the file contains:

  • the experiment number (an integer from the card’s flash memory);
  • the second of the recording;
  • the accelerometer values on the X, Y and Z axes (hexadecimal);
  • the magnetometer values on the X, Y and Z axes (hexadecimal);
  • the temperature measured by the board (decimal).

For more details about this sensor and how to process its data, see [Fleury2009].

Presence infra-red sensors
#

Each movement detected in one of the rooms is logged in a text file (HIS_Y.txt, in the Sensors archive) with the date (DD/MM/YYYY), the time (HH:MM:SS) and the room.

Hygrometry and temperature
#

A device in the bathroom gives the temperature and hygrometry every 5 minutes, in a text file (Tempature.temp_Y.txt, in the Sensors archive) with a timestamp (DD.MM.YYYY-HH:MM), the temperature (°C) and the hygrometry (%).

Chest of drawers, fridge and cupboard
#

Sensors record when the chest of drawers, the fridge and the cupboard (containing everything needed to prepare meals) are opened or closed. Each text file (Cupboard_Y.txt, Drawers_Y.txt, Fridge_Y.txt, in the Sensors archive) lists the changes of state of the door: timestamp (DD-MM-YYYY-HH-MM-SS) and new state (0 closed, 1 open).

References
#

Distress call dataset
#

  • [Vacher2008] Vacher, M.; Fleury, A.; Serignat, J.-F.; Noury, N.; Glasson, H. « Preliminary Evaluation of speech/sound recognition for telemedicine application in a real environment ». Interspeech'08, Brisbane, Australia, 2008, pp. 496-499.
  • [Vacher2010] Vacher, M.; Fleury, A.; Portet, F.; Serignat, J.-F.; Noury, N. « Complete Sound and Speech Recognition System for Health Smart Homes: Application to the Recognition of Activities of Daily Living ». In New Developments in Biomedical Engineering, ISBN 978-953-7619-57-2, pp. 645-673, 2010.
  • [Vacher2011] Vacher, M.; Portet, F.; Fleury, A.; Noury, N. « Development of Audio Sensing Technology for Ambient Assisted Living: Applications and Challenges ». International Journal of E-Health and Medical Communications, 2(1):35-54, 2011. [pdf]

Activities of Daily Living dataset
#

  • [Fleury2009] Fleury, A.; Noury, N.; Vacher, M. « A wavelet-based pattern recognition algorithm to classify postural transitions in humans ». EUSIPCO 2009, 17th European Signal Processing Conference, Glasgow, Scotland, 2009, pp. 2047-2051.
  • [Fleury2010] Fleury, A.; Vacher, M.; Noury, N. « SVM-Based Multi-Modal Classification of Activities of Daily Living in Health Smart Homes: Sensors, Algorithms and First Experimental Results ». IEEE Transactions on Information Technology in Biomedicine, 14(2), 2010, pp. 274-283 (DOI: 10.1109/TITB.2009.2037317).
  • [Fleury2011] Fleury, A.; Noury, N.; Vacher, M. « Improving Supervised Classification of Activities of Daily Living Using Prior Knowledge ». International Journal of E-Health and Medical Communications, 2(1):17-34, 2011. [pdf]
  • [Noury2011] Noury, N.; Poujaud, J.; Fleury, A.; Nocua, R.; Haddidi, T.; Rumeau, P. « Smart Sweet Home… a pervasive environment for sensing our daily activity? ». In Activity Recognition in Pervasive Intelligent Environments, Atlantis Ambient and Pervasive Intelligence, vol. 4, Atlantis Press, 2011, ISBN 978-90-78677-42-0.
  • [Chahuara2012] Chahuara, P.; Fleury, A.; Portet, F.; Vacher, M. « Using Markov Logic Network for On-line Activity Recognition from Non-Visual Home Automation Sensors ». Lecture Notes in Computer Science vol. 7683, pp. 177-192, 2012.