Human action recognition: A paradigm of best deep learning features selection and serial based extended fusion

Seemab Khan, Muhammad Attique Khan, Majed Alhaisoni, Usman Tariq, Hwan Seung Yong, Ammar Armghan, Fayadh Alenezi

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

Human action recognition (HAR) has gained significant attention recently as it can be adopted for a smart surveillance system in Multimedia. However, HAR is a challenging task because of the variety of human actions in daily life. Various solutions based on computer vision (CV) have been proposed in the literature which did not prove to be successful due to large video sequences which need to be processed in surveillance systems. The problem exacerbates in the presence of multi-view cameras. Recently, the development of deep learning (DL)-based systems has shown significant success for HAR even for multi-view camera systems. In this research work, a DL-based design is proposed for HAR. The proposed design consists of multiple steps including feature mapping, feature fusion and feature selection. For the initial feature mapping step, two pre-trained models are considered, such as DenseNet201 and InceptionV3. Later, the extracted deep features are fused using the Serial based Extended (SbE) approach. Later on, the best features are selected using Kurtosis-controlled Weighted KNN. The selected features are classified using several supervised learning algorithms. To show the efficacy of the proposed design, we used several datasets, such as KTH, IXMAS, WVU, and Hollywood. Experimental results showed that the proposed design achieved accuracies of 99.3%, 97.4%, 99.8%, and 99.9%, respectively, on these datasets. Furthermore, the feature selection step performed better in terms of computational time compared with the state-of-the-art.

Original languageEnglish
Article number7941
JournalSensors (Switzerland)
Volume21
Issue number23
DOIs
StatePublished - 1 Dec 2021

Keywords

  • Deep learning
  • Features fusion
  • Features selection
  • Human action recognition
  • Recognition

Fingerprint

Dive into the research topics of 'Human action recognition: A paradigm of best deep learning features selection and serial based extended fusion'. Together they form a unique fingerprint.

Cite this