Paper in IEEE CVPR 2013 "Decoding Children's Social Behavior"

  • J. M. Rehg, G. D. Abowd, A. Rozga, M. Romero, M. A. Clements, Stan Sclaroff, I. Essa, O. Y. Ousley, Yin Li, C. Kim, H. Rao, J. C. Kim, L. L. Presti, J. Zhang, D. Lantsman, J. Bidwell, and Z. Ye (2013), “Decoding Children’s Social Behavior,” in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013. [PDF] [WEBSITE] [DOI] [BIBTEX]
    @InProceedings{ 2013-Rehg-DCSB,
    author  = {James M. Rehg and Gregory D. Abowd and Agata Rozga
    and Mario Romero and Mark A. Clements and Stan
    Sclaroff and Irfan Essa and Opal Y. Ousley and Yin
    Li and Chanho Kim and Hrishikesh Rao and Jonathan C.
    Kim and Liliana Lo Presti and Jianming Zhang and
    Denis Lantsman and Jonathan Bidwell and Zhefan Ye},
    booktitle  = {{IEEE Conference on Computer Vision and Pattern
    Recognition (CVPR)}},
    doi = {10.1109/CVPR.2013.438},
    month = {June},
    organization  = {IEEE Computer Society},
    pdf = {},
    title = {Decoding Children's Social Behavior},
    url = {},
    year = {2013}


We introduce a new problem domain for activity recognition: the analysis of children’s social and communicative behaviors based on video and audio data. We specifically target interactions between children aged 1-2 years and an adult. Such interactions arise naturally in the diagnosis and treatment of developmental disorders such as autism. We introduce a new publicly-available dataset containing over 160 sessions of a 3-5 minute child-adult interaction. In each session, the adult examiner followed a semi-structured play interaction protocol which was designed to elicit a broad range of social behaviors. We identify the key technical challenges in analyzing these behaviors, and describe methods for decoding the interactions. We present experimental results that demonstrate the potential of the dataset to drive interesting research questions, and show preliminary results for multi-modal activity recognition.

Full database available from

via IEEE Xplore – Decoding Children’s Social Behavior.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.