Health Smart Home (HIS) datasets

A French corpus of audio and multimodal interactions in a health smart home

Important Notice

These data have been acquired in the context of a PhD. thesis, in a laboratory that work on Health Smart Home. These databases, devoted to activity recognition and distress recognition, are distributed free of charge, for an academic and research use only, in order to be able to compare the results obtained. By downloading these anonymous data, you agree to the following limitations:

  • Commercial use: This database is not designed for any commercial use and should only be used for academic researchers.
  • Publications: Any publication using the activity database should include a citation of the article: Fleury, A.; Vacher, M. and Noury, N.; « SVM-Based Multi-Modal Classification of Activities of Daily Living in Health Smart Homes: Sensors, Algorithms and First Experimental Results ». IEEE Transactions on Information Technology in Biomedicine, Vol. 14(2), March 2010, pp. 274-283 (DOI:10.1109/TITB.2009.2037317). In the same way, all the publications that use the
  • Redistribution: This database cannot be, totally or partially, further distributed, published, copied or disseminated, in any way or form, for profit or not.
  • In the same way, all the publications that use the distress call dataset must include a reference to the following book chapter: Vacher, M., Fleury, A., Portet, F., Serignat, J.F., Noury, N.: “New Developments in Biomedical Engineering, chap. Complete Sound and Speech Recognition System for Health Smart Homes: Application to the Recognition of Activities of Daily Living“, pp. 645 – 673. Intech Book (2010).

For all questions or concerns, do not hesitate to contact us by email: anthony.fleury@mines-douai.fr

Information on these datasets concerning their contents and their way to be indexed etc. can also be found in the article: Fleury, A.; Vacher, M.; Portet, F.; Chahuara, P. and Noury, N.: “A french corpus of audio and multimodal interactions in a health smart home” . Journal on Multimodal User Interfaces, Springer Berlin / Heidelberg, 2012, 17p (DOI:10.1007/s12193-012-0104-x).

A copy of this website may be found Here.

Context of the acquisitions

The following presented acquisitions were made during a PhD Thesis, in the Health Smart Home of the TIMC-IMAG Laboratory (UMR CNRS 5525), Team AFIRM (that is now part of the AGIM Laboratory,FRE 3405, CNRS-UJF-EPHE). The PhD thesis were advised by Norbert Noury (TIMC-IMAG, now member of INL Laboratory in Lyon), and Michel Vacher (LIG Laboratory, Team GETALP).

Presentation of the Smart Home of the TIMC-IMAG Laboratory

The smart home of the TIMC-IMAG lab is a flat containing a kitchen, a living room, a bedroom, a bathroom, a hall and toilets. This flat is completely equipped and contains all the materiels that can be used for the different activities in the home (including books, clothes, TV, radio, microwave oven, fridge, food, etc.).

It is also equipped of several sensors as microphones, presence infra-red sensors, and webcams. It also contains a temperature and hygrometry sensor that is inside the bathroom, and the dweller is equipped with an home-made accelerometer and magnetometer circuit that is placed under the left armpit.

 

Finally, all the data are stored on different computers that are placed in the technical room, just next the flat.

Data Sets

Distress call dataset

Experimental protocol

The protocol was quite simple. Each participant was alone in the flat. The participant had to go to the living room and to close the door. Then, s/he had to move to the bedroom and read aloud the first half of one of the five successions of sentences, out of 10 normal and 20 distress sentences. Afterwards, the second half of the set of sentences had to be uttered in the living room. Finally, each participant was called 3 times and had to answer the phone and read the predefined phone conversation (5 sentences each). The sentences were uttered in the flat, with the participant sat down or stood up. They have no orders on the way they orient themselves comparing to the microphones or on the way they pronounced the different sentences.

Dataset content and organization

This dataset has been acquired on 10 different subjects. The main informations on the different subject are given in the following table (in this experiment, all the subject were native French speakers).

 ID Age Gender Weight (kg) Height (m) Speech Number
 1  61  M 82 1,7  1
 2  22  M 77 1,77  2
 3  53  M 67 1,68  3
 4  47  M 56 1,7  4
 5  31  M 62 1,7  5
 6  25  M 78 1,84  2
 7  51  F 64 1,65  4
 8  29  F 53 1,6  3
 9  28  F 61 1,69  1
 10  25  M 90 1,88  5

For each subject, the last column of the table gives the sequence number of the different sentences that have been pronounced. Indeed, we had 5 different sequences of sentences (with normal, distress and phone conversations). The different sequences can be found in an Excel file, by consulting the appropriate sheet (corresponding to the given number).

On the server, the dataset is organized as follow:

  • A directory containing the complete dataset for each subject. This full dataset contains ALL the sounds that have been captured during the different sessions. For each subject, we can find:
    • 8 directories that correspond to the different channels,
    • For each channel, the different files that correspond to each acquisition (the most important are first the wave file that contains the acquired sound and the xml file that give the analysis of this sound by our system, with its segmentation as sound or speech and if sound, the classification in one of the sound classes and if speech the five more probable sentences that fit this sound).
    • With these 8 directories, a text file (named Resultats.txt), that contains a first analysis with the following information: for sound, the SNR, the ID of corresponding wav/xml files, and the class of the sound), and for speech, the SNR, the ID of the xml/wav files, and the three first most probable sentences that have been determined by the system.
  • A directory, containing for each speaker, only the files that correspond to the sounds with the best SNR, for only each of the sentences that have been uttered during the experimentation (all the other sounds have been deleted). The order of the sentences is kept comparing to the XLS spreadsheet given before.
  • A directory containing all the sounds (and associated) files that correspond to the best SNR and to all the sounds with an SNR upper that 80% of this first value. In the same way than before, the order of the sentences is kept and the XML files contains the timestamps and the identification of all the simultaneous sounds, allowing to determine which sound correspond to which sentence.

More information on the experiment, on the datasets and on the process are given in the references [Vacher2008, Vacher2010, Vacher2011].

Data
 ID  Whole Set RSB Max 80 %
 1  Zip File
(19 Mo)		  Zip File
(2,1 Mo)		  Zip File
(5 Mo)
 2  Zip File
(22 Mo)		  Zip File
(2,2 Mo)		  Zip File
(4,9 Mo)
 3  Zip File
(21 Mo)		  Zip File
(2,1 Mo)		  Zip File
(5,3 Mo)
 4  Zip File
(22 Mo)		  Zip File
(2,2 Mo)		  Zip File
(5 Mo)
 5  Zip File
(20 Mo)		  Zip File
(2,1 Mo)		  Zip File
(4,6 Mo)
 6  Zip File
(32 Mo)		  Zip File
(2,1 Mo)		  Zip File
(4,9 Mo)
 7  Zip File
(32 Mo)		  Zip File
(2,2 Mo)		  Zip File
(4,2 Mo)
 8  Zip File
(27 Mo)		  Zip File
(1,7 Mo)		  Zip File
(3,6 Mo)
 9  Zip File
(21 Mo)		  Zip File
(2,1 Mo)		  Zip File
(4,4 Mo)
 10  Zip File
(19 Mo)		  Zip File
(1,8 Mo)		  Zip File
(3,3 Mo)
 All Zip File
(231 Mo)		 Zip file
(21 Mo)		 Zip file
(45 Mo)

Activities of Daily Living dataset

Experimental protocol

The experimental protocol was quite simple. The first step of the experimentation was a deep visit of the flat, to allow the person to act as much as possible as if s/he was home. After this visit, the only requirement to the subject was to perform a set of 7 activities, in the order and for the duration that s/he wanted, and as many times as s/he wanted.

The list of activities to perform was:

  • Sleeping: you lie in the bed and stay “asleep” for a reasonable time (of your choice)
  • Resting: you sit on the couch or a chair in the living room and perform the activity that you like when you are relaxing (watching TV, reading, doing nothing…)
  • Feeding: you prepare a breakfast with the equipment and the ingredients in the kitchen cupboard and then eat (using the kitchen table). Then, you wash the dishes in the kitchen sink.
  • Hygiene: you wash your hands, face, and pretend to brush your teeth in the bathroom.
  • Toilets: you pretend to go (or you do) to the bathroom (sit on the toilet, flush the toilet…).
  • Dressing: you put and remove the clothes in the chest of drawers near the bed (over your own clothes). You can do this activity before going to sleep and after sleep or after performing the hygiene task.
  • Communication: you will be called over the phone (in the living room) several times. Each time you will read the sentences indicated near the phone (one long phone conversation and three small ones).

For more information about the dataset and the experiments, feel free to consult the references [Fleury2010, Fleury2011, Vacher2011, Noury2011].

Dataset content, organization and downloading

The data of 2 of the subjects are for the moment made available on this web page.

To retrieve the complete dataset including all the participants, please fill-in the PDF form and sent it to Anthony.Fleury@mines-douai.fr to engage yourself to use this dataset in a correct way. When done, you will receive the link to download the remain participants of the dataset.

The subjects have the following information:

 ID Age Gender Height (m) Weight (kgs)  Phone conversations ID Native French Speaker ?
1 25 F 1.77 63 1 yes
2 25 F 1.78 70 2 yes
3 32 M 1.92 80 3 yes
4 57 F 1.48 50 4 no
5 25 M 1.72 81 5 yes
6 26 M 1.82 66 1 no
7 28 M 1.78 68 2 yes
8 31 M 1.7 63 3 yes
9 2