Identifying shoplifting behaviors and inferring behavior intention based on human action detection and sequence analysis

Siyeon Kim, Sungjoo Hwang, Seok Hwan Hong

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


Identification of abnormal behaviors affecting public safety (e.g., shoplifting, robbery, and stealing) is essential for preventing human casualties and property damage. Many studies have attempted to automatically identify abnormal behaviors by detecting relevant human actions by developing intelligent video surveillance systems. However, these studies have focused on catching predefined actions associated explicitly with the target abnormal behavior, which can lead to errors in judgment when such actions are undetected or inaccurately detected. To better identify abnormal behaviors, it is essential to understand a series of performed actions to capture behaviors’ pre- and post-indications (e.g., repeatably looking around and spotting CCTVs) and infer the intentions underlying such behaviors. Thus, in the present study, we propose a framework to identify abnormal behaviors through deep-learning-based detection of non-semantic-level human action components segmented with a window size of several seconds (e.g., walking, standing, and watching) and performing sequence analyses of the detected action components to infer behavior intentions. Then, we tested the applicability of the framework to the specific scenario of shoplifting, one of the most common crimes. Analysis of actual incident data confirmed that shoplifting intentions could be effectively gauged based on distinct action sequence features, and the intention inference results are continuously updated with the accumulated series of detected actions during the course of the input video stream. The results of this study can help enhance the ability of intelligent surveillance systems by providing a new means for monitoring abnormal behaviors and deeply understanding the underlying intentions.

Original languageEnglish
Article number101399
JournalAdvanced Engineering Informatics
StatePublished - Oct 2021

Bibliographical note

Funding Information:
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: [Sungjoo Hwang reports administrative support, equipment, drugs, or supplies, and statistical analysis were provided by National Research Foundation of Korea (NRF) and the Ministry of SMEs and Startups (MSS, Korea)].

Funding Information:
This research was supported by the National Research Foundation of Korea (NRF) grant (2020R1F1A1073178) funded by the Korean government, and the Technology Development Program (S2658843) funded by the Ministry of SMEs and Startups (MSS, Korea). The authors also would like to acknowledge anonymous participants who participated in the survey.

Publisher Copyright:
© 2021 Elsevier Ltd


  • Abnormal behavior detection
  • Action sequence analysis
  • Behavioral intention
  • Public safety
  • Shoplifting
  • Video surveillance


Dive into the research topics of 'Identifying shoplifting behaviors and inferring behavior intention based on human action detection and sequence analysis'. Together they form a unique fingerprint.

Cite this