[2023] Sparse spiking neural networks for radar-based human activity recognition

Human activity recognition (HAR) bears key importance for several fields, perhaps most notably healthcare and security. In the case of healthcare, the continuous monitoring of patients’ activities and behavioural patterns provides medical professionals with valuable information. Additionally, situations where immediate medical attention is required can be identified by anomalous behaviour such as falling or attempts at self-harm or suicide.
In order to best suit patient monitoring use cases, radar sensing allows for a non-invasive setup that preserves privacy and can function in darkness. Furthermore it is a contactless technology, with no wires or sensors attached to the body of the person.
To enable an always-on deployment of these use cases, the computational footprint of HAR should be minimized. In this project, we will take inspiration from the brain, where neurons transiently exhibit high spike-based activity patterns only when unexpected input is received, which is known in neuroscience research as excitatory-inhibitory balance (EIB) [1-2]. This follows a three-fold rationale:
- unexpected input is likely to represent transitions between two different human activity patterns,
- complex human activity recognition can be triggered only when necessary as transitions occur,
- the network activity is kept sparse outside of transitions, thereby minimizing the system-level power consumption.

[1] https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003258
[2] https://ojs.aaai.org/index.php/AAAI/article/view/11320

Assignment

1. Development of a machine-learning framework for training EIB networks with radar-based activity and/or gesture data.
2. Analysis and discussion of energy/accuracy tradeoffs at the system level, including spike-based encoding aspects.
3. If time allows, development of a digital architecture for always-on HAR.

Requirements

For this ambitious and interdisciplinary project, previous experience with machine learning (incl. frameworks such as PyTorch or Tensorflow) is necessary.
Familiarity with digital circuit design is recommended.
No previous expertise with neuroscience and radar signal processing is expected, support will be provided.

Interested students should send a motivation letter together with their CV (incl. course transcripts and grades) to Dr. Francesco Fioranelli (f.fioranelli@tudelft.nl) and Dr. Charlotte Frenkel (c.frenkel@tudelft.nl)

More MSc proposals for Dr. Charlotte Frenkel will appear in the coming weeks, interested students are encouraged to reach out by e-mail to enquire about upcoming projects.

Contact

Last modified: 2023-12-02