Abstract
Quantitative behavioral measurements are important for answering questions across scientific disciplines—from neuroscience to ecology. State-of-the-art deep-learning methods offer major advances in data quality and detail by allowing researchers to automatically estimate locations of an animal’s body parts directly from images or videos. However, currently available animal pose-estimation methods have limitations in speed and robustness. Here, we introduce a new easy-to-use software toolkit, DeepPoseKit, that addresses these problems using an efficient multi-scale deep-learning model, called Stacked DenseNet, and a fast GPU-based peak-detection algorithm for estimating keypoint locations with subpixel precision. These advances improve processing speed >2× with no loss in accuracy compared to currently available methods. We demonstrate the versatility of our methods with multiple challenging animal pose-estimation tasks in laboratory and field settings — including groups of interacting individuals. Our work reduces barriers to using advanced tools for measuring behavior and has broad applicability across the behavioral sciences.
Full text
Introduction
Understanding the behavioural repertoire of an animal — what it does, when, and why — is a central goal of neuroscience, ecology and evolutionary biology. The rapid availability of inexpensive, high-resolution cameras has made it routine to collect hours or days of video of behaving animals, but turning raw video into the high-resolution kinematic measurements needed for downstream analyses remains a bottleneck. Deep-learning approaches have recently shown that user-defined keypoints can be tracked through video without specialised hardware or markers, opening up new experimental designs. We build on this progress to deliver a toolkit that is both fast and easy to apply across species and recording conditions.
Results
DeepPoseKit packages an end-to-end annotation, training, and prediction workflow into a single Python library. At its core is the Stacked DenseNet backbone, a multi-scale architecture inspired by DenseNet that propagates features across scales while keeping parameter count low. Combined with a fast GPU-based subpixel peak-detection algorithm, the model produces keypoint estimates with subpixel precision at over twice the throughput of prior tools.
We evaluate DeepPoseKit on four datasets that span the practical range of animal pose-estimation problems — from a tethered Drosophila setup to overhead recordings of interacting groups of zebras in the field. Across datasets,Stacked DenseNet matches or exceeds the accuracy ofLEAP and DeepLabCut while running between 2× and 5× faster on the same hardware.
Discussion
The combination of an efficient backbone, a high-throughput peak detector and a small, opinionated Python API lets researchers move from raw video to keypoint trajectories without first becoming experts in deep-learning infrastructure. We expect this to broaden the use of pose estimation across behavioural sciences, and to make it practical to study behaviour in groups and natural environments at high temporal resolution.
Materials and methods
All datasets, code and trained models are publicly available under permissive licences. The software is distributed through github.com/jgraving/deepposekit and the documentation, including a step-by-step tutorial, lives at docs.deepposekit.org.
Data availability
Training images, annotations and example notebooks are released with the paper. See the published article for the complete data-availability statement.
Citation
Graving JM, Chae D, Naik H, Li L, Koger B, Costelloe BR, Couzin ID. DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning. bioRxiv 620245 (2019). doi: 10.1101/620245. Published version: eLife 8:e47994. doi:10.7554/eLife.47994.
Information / History
| Version | Date | Title |
|---|---|---|
| v7current | September 4, 2019 | DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning |
| v6 | August 31, 2019 | DeepPoseKit: a software toolkit for fast and robust animal pose estimation using deep learning |
| v5 | August 20, 2019 | DeepPoseKit: a software toolkit for fast and robust animal pose estimation using deep learning |
| v4 | August 19, 2019 | DeepPoseKit: a software toolkit for fast and robust animal pose estimation using deep learning |
| v3 | May 3, 2019 | Fast and robust animal pose estimation |
| v2 | April 27, 2019 | Fast and robust animal pose estimation |
| v1 | April 26, 2019 | Fast and robust animal pose estimation |
Submitted on April 26, 2019 · Last revised September 4, 2019 · License: CC BY-NC-ND 4.0