WEAR Dataset Challenge
@HASCA 2024

Goal of the Challenge

The First WEAR Dataset Challenge is a Human Activity Recognition prediction challenge based on the inertial data of the original WEAR dataset [1] publication.
Challenge participants are tasked to predict the activity label of a yet unreleased test dataset of newly and re-recorded participants. Winners are determined based on the sample-wise macro F1-score averaged across all activities and participants. The challenge is part of the HASCA Workshop at UbiComp/ ISWC 2024.

The Data & Submission Format

The challenge dataset download contains accelerometer data of 18 particpants performing multiple outdoor fitness workouts. In total each participant performed 18 workout activities. Data was captured at the four limbs of each participant, being sampled at 50 Hz with a sensitivity of ±8G. The test dataset contains data of rerecorded and new participants performing the same workout and activities as the original participants. Participants are tasked to predict the activity label of each data record of the new participants using their prediction algorithm of choice. Submissions should contain the predicted activity label for each data record of the test dataset (see below for more details).

Submission to HASCA @ UbiComp/ ISWC 2024

To be part of the final ranking, participants will be required to submit a detailed paper to the HASCA workshop. The paper should contain technical description of the processing pipeline, the algorithms and the results achieved during the original WEAR dataset. Submissions must follow the HASCA format (see below for more details).

About the WEAR Dataset

The WEAR dataset is an outdoor sports dataset for inertial- and video-based human activity recognition (HAR). The dataset comprises data from 18 participants performing a total of 18 different workout activities with untrimmed inertial (acceleration) and egocentric video data recorded at 10 different outside locations. WEAR provides a challenging prediction scenario marked by purposely introduced activity variations as well as an overall small information overlap across modalities. This challenge focuses on the inertial data only.

• Registration open: April 12, 2024
• Challenge duration: April 12, 2024 - July 04, 2024
• HASCA-WEAR paper submission: July 04, 2024 AoE
• HASCA-WEAR review notification: July 11, 2024
• HASCA-WEAR camera ready submission: July 19, 2024
• HASCA Workshop: October 05-06, 2024 in Melbourne, Australia

tba

Registration is now open! In order to participate, each team must send a registration email to wear.challenge@gmail.com, stating the:

• Name of the team + e-mail adress of one participant representing the team
• Full Names of the participants in the team
• Affiliations of each participant (non-affiliated persons are also encouraged to participate)

• Training Data: download [approx. 600 MB]
• Test Data: download [approx. 150 MB]

Evaluation of submissions will be based on the sample-wise, macro F1-score averaged across all participants. The macro F1-score is the average of the F1-scores of each activity class. Please check the WEAR dataset repository for sample code on how to translate windowed data back to record-wise data and calculate the macro F1-score.

Submission of Final Predictions:

To submit your final predictions of the test dataset, please send a ZIP-archive file “{Your_Team_Name}.zip" (or a download link to it) to wear.challenge@gmail.com. The ZIP-archive should contain the 6 test participant CSV files appended with an additional column label being the predicted recordwise labels. Note that we will assume same label names as in the train dataset and same ordering - thus make sure to keep the order of the records and use the same label names as in the training data. An example submission (with random labels) can be downloaded here.

Submission of Technical Report:

Submission of the technical report will happen via PCS (select SIGCHI / UbiComp 2024 / UbiComp 2024 Workshop - HASCA-WEAR). Your technical report should detail your solution as well as provide preliminary results you achieved on the train dataset. ACM requires UbiComp/ISWC 2024 workshop submissions to use the double-column template. Your technical report must be between 3 to 6 pages including references. Submissions do not need to be anonymous. For details on the template please refer to the UbiComp/ISWC website.

Activities & Recording Scenario

Each participant performed a set of 18 workout activities. These activities include running-, stretching- and strength-based exercises, with base activities like push-ups being complemented with complex variations that alter and/ or extend the performed movement during the exercise. Activities were divided across multiple recording sessions, with each session consisting of uninterrupted data streams of all modalities. Each participant was tasked to perform each exercise for at least 90 seconds, but had the freedom to choose the order of activities and take breaks as desired.

Training Data

The original dataset comprises of outdoor workouts of 18 participants. Each workout is divided across multiple session. In total more than 15 hours were recorded at 10 outdoor locations. The training data contains the raw sensor data of the 18 partipants which were part of the original WEAR dataset [1]. The sensors were placed at four body locations (right wrist, left wrist, right ankle and left ankle). Each sensor sampled 3D-accelerometer data. During all recording sessions sensor orientation was fixed according to one pre-defined sensor placement. Each sampled data record is labeled as one of the 18 (+ null-class) possible activities.

Dataset Overview

Training Data: The training data is provided in a seperate CSV files per subject. Each file contains the following columns:

• sbj_id: subject identifier (int between 0 and 17)
• right_arm_acc_x: right arm acceleration x-axis (float between -8.0 and +8.0)
• right_arm_acc_y: right arm acceleration y-axis (float between -8.0 and +8.0)
• right_arm_acc_z: right arm acceleration z-axis (float between -8.0 and +8.0)
• right_leg_acc_x: right leg acceleration x-axis (float between -8.0 and +8.0)
• right_leg_acc_y: right leg acceleration y-axis (float between -8.0 and +8.0)
• right_leg_acc_z: right leg acceleration z-axis (float between -8.0 and +8.0)
• left_leg_acc_x: left leg acceleration x-axis (float between -8.0 and +8.0)
• left_leg_acc_y: left leg acceleration y-axis (float between -8.0 and +8.0)
• left_leg_acc_z: left leg acceleration z-axis (float between -8.0 and +8.0)
• left_arm_acc_x: left arm acceleration x-axis (float between -8.0 and +8.0)
• left_arm_acc_y: left arm acceleration y-axis (float between -8.0 and +8.0)
• left_arm_acc_z: left arm acceleration z-axis (float between -8.0 and +8.0)
• label: activity label (str being one of the 18 workout activities or null during breaks)

Testing Data

The test dataset follows the same structure as the training data. In total the test dataset consists of 6 new and existing participants performing the same 18 activities as the original participants. The test dataset is divided into multiple CSV files, each containing the sensor data of one participant. Participants of the test dataset are:

• sbj_0_2 (112050 records): additional session of participant with id 0 from the original dataset.
• sbj_14_2 (167750 records): additional session of participant with id 14 from the original dataset.
• sbj_18 (131600 records): session of a new participant (id 18).
• sbj_19 (110050 records): session of a new participant (id 19).
• sbj_20 (95400 records): session of a new participant (id 20).
• sbj_21 (127800 records): session of a new participant (id 21).

• You do not work in or collaborate with the WEAR dataset project (http://mariusbock.github.io/wear/)
• If you submit an entry, but are not qualified to enter the contest, this entry is voluntary. The organizers reserve the right to evaluate it for scientific purposes. If you are not qualified to submit a contest entry and still choose to submit one, under no circumstances will such entries qualify for sponsored prizes.
• Only registered teams will be considered for the final ranking. Teams must register via an e-mail to wear.challenge@gmail.com (see above for details).
• To be part of the final ranking, participants will be required to publish a detailed paper in the proceedings of the HASCA workshop; The dates will be set during the competition. Publishing of the paper also requires that at least one team member is registered for the HASCA workshop.
• The participants' predictions must be submitted online by sending an email to wear.challenge@gmail.com. The submission e-mail should contain a link to the predictions file, using services such as Dropbox, Google Drive, etc. In case the participants cannot provide link using some file sharing service, they should contact the organizers via email wear.challenge@gmail.com.
• Only one single submission is allowed per team. In case of multiple submission only the one which was submitted last will be considered for the final ranking. The same person cannot be in multiple teams, except if that person is a supervisor.

Challenge related questions: wear.challenge@gmail.com
All other questions: marius.bock@uni-siegen.de

Marius Bock, University of Siegen
Christina Runkel, University of Cambridge
Alexander Hoelzemann, University of Siegen
Mathias Ciliberto, University of Sussex
Prof. Dr. Kristof Van Laerhoven, University of Siegen
Prof. Dr. Michael Moeller, University of Siegen

[1] Marius Bock, Hilde Kuehne, Kristof Van Laerhoven and Michael Moeller. 2023. WEAR: An Outdoor Sports Dataset for Wearable and Egocentric Activity Recognition. CoRR abs/2304.05088. https://arxiv.org/abs/2304.05088

WEAR is offered under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. You are free to use, copy, and redistribute the material for non-commercial purposes provided you give appropriate credit, provide a link to the license, and indicate if changes were made. If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original. You may not use the material for commercial purposes.