Human Actions

Recognizing, localizing, and detecting human actions in video, from fully-supervised to weakly- and semi-supervised settings.

(Kumar & Rawat, 2022) (Rana & Rawat, 2021) (Rana & Rawat, 2022) (Rana & Rawat, 2023) (Rana et al., 2025) (Dave et al., 2022) (Rizve et al., 2020) (Tirupattur et al., 2021) (Tirupattur et al., 2021) (Kerrigan et al., 2021) (Ahmad et al., 2025) (Ahmad et al., 2023) (Singh et al., 2024) (Modi et al., 2023) (Grover et al., 2023) (Modi et al., 2022) (Sia & Rawat, 2025) (Abdullah et al., 2025) (Abdullah et al., 2026) (Swetha et al., 2021) (Demir et al., 2020) (Xu et al., 2022) (Kumar et al., 2025) (Dave et al., 2021)

References

2026

Learning to Deny: Action Denial in Multimodal Large Language Models

Raiyaan Abdullah, Shehreen Azad, and Yogesh Singh Rawat

In European Conference on Computer Vision (ECCV), 2026

ECCV arXiv Bib Code

@inproceedings{abdullah2026deny,
  title = {Learning to Deny: Action Denial in Multimodal Large Language Models},
  author = {Abdullah, Raiyaan and Azad, Shehreen and Rawat, Yogesh Singh},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year = {2026},
}

2025

OmViD: Omni-supervised active learning for video action detection

Aayush Rana, Akash Kumar, Vibhav Vineet, and Yogesh S Rawat

In IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2025

ICCVW Bib Code

@inproceedings{rana2025omvid,
  title = {OmViD: Omni-supervised active learning for video action detection},
  author = {Rana, Aayush and Kumar, Akash and Vineet, Vibhav and Rawat, Yogesh S},
  booktitle = {IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)},
  pages = {6911--6921},
  year = {2025},
}

T2L: Efficient Zero-Shot Action Recognition with Temporal Token Learning

Shahzad Ahmad, Sukalpa Chanda, and Yogesh S. Rawat

Transactions on Machine Learning Research (TMLR), 2025

TMLR Bib HTML

@article{ahmad2025t2l,
  title = {T2L: Efficient Zero-Shot Action Recognition with Temporal Token Learning},
  author = {Ahmad, Shahzad and Chanda, Sukalpa and Rawat, Yogesh S.},
  journal = {Transactions on Machine Learning Research (TMLR)},
  year = {2025},
}

Scaling Open-Vocabulary Action Detection

Zhen Hao Sia and Yogesh Singh Rawat

In IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2025

ICCVW arXiv Bib Code Website

@inproceedings{sia2025scaling,
  title = {Scaling Open-Vocabulary Action Detection},
  author = {Sia, Zhen Hao and Rawat, Yogesh Singh},
  booktitle = {IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)},
  pages = {6351--6361},
  year = {2025},
}

Punching Bag vs. Punching Person: Motion Transferability in Videos

Raiyaan Abdullah, Jared Claypoole, Michael Cogswell, Ajay Divakaran, and Yogesh Rawat

In IEEE/CVF International Conference on Computer Vision (ICCV), 2025

ICCV arXiv Bib HTML Code Website

@inproceedings{abdullah2025punching,
  title = {Punching Bag vs. Punching Person: Motion Transferability in Videos},
  author = {Abdullah, Raiyaan and Claypoole, Jared and Cogswell, Michael and Divakaran, Ajay and Rawat, Yogesh},
  booktitle = {IEEE/CVF International Conference on Computer Vision (ICCV)},
  year = {2025},
}

Stable Mean Teacher for Semi-supervised Video Action Detection

Akash Kumar, Sirshapan Mitra, and Yogesh Singh Rawat

In AAAI Conference on Artificial Intelligence, 2025

AAAI arXiv Bib Video Code Website

@inproceedings{kumar2025stable,
  title = {Stable Mean Teacher for Semi-supervised Video Action Detection},
  author = {Kumar, Akash and Mitra, Sirshapan and Rawat, Yogesh Singh},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
}

2024

Semi-supervised active learning for video action detection

Ayush Singh, Aayush J Rana, Akash Kumar, Shruti Vyas, and Yogesh Singh Rawat

In Proceedings of the AAAI Conference on Artificial Intelligence, 2024

AAAI Bib PDF Code

@inproceedings{singh2024semi,
  title = {Semi-supervised active learning for video action detection},
  author = {Singh, Ayush and Rana, Aayush J and Kumar, Akash and Vyas, Shruti and Rawat, Yogesh Singh},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
  volume = {38},
  number = {5},
  pages = {4891--4899},
  year = {2024},
}

2023

Hybrid active learning via deep clustering for video action detection

Aayush J Rana and Yogesh S Rawat

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

Bib PDF Code

@inproceedings{rana2023hybrid,
  title = {Hybrid active learning via deep clustering for video action detection},
  author = {Rana, Aayush J and Rawat, Yogesh S},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages = {18867--18877},
  year = {2023},
}

EZ-CLIP: Efficient Zero-shot Video Action Recognition

Shahzad Ahmad, Sukalpa Chanda, and Yogesh S Rawat

arXiv preprint arXiv:2312.08010, 2023

arXiv Bib PDF Code

@article{ahmad2023ez,
  title = {EZ-CLIP: Efficient Zero-shot Video Action Recognition},
  author = {Ahmad, Shahzad and Chanda, Sukalpa and Rawat, Yogesh S},
  journal = {arXiv preprint arXiv:2312.08010},
  year = {2023},
}

On occlusions in video action detection: Benchmark datasets and training recipes

Rajat Modi, Vibhav Vineet, and Yogesh Rawat

Advances in Neural Information Processing Systems, 2023

NeurIPS Bib HTML Code

@article{modi2023occlusions,
  title = {On occlusions in video action detection: Benchmark datasets and training recipes},
  author = {Modi, Rajat and Vineet, Vibhav and Rawat, Yogesh},
  journal = {Advances in Neural Information Processing Systems},
  volume = {36},
  pages = {57306--57335},
  year = {2023},
}

Revealing the unseen: Benchmarking video action recognition under occlusion

Shresth Grover, Vibhav Vineet, and Yogesh Rawat

Advances in Neural Information Processing Systems, 2023

NeurIPS Bib HTML Website

@article{grover2023revealing,
  title = {Revealing the unseen: Benchmarking video action recognition under occlusion},
  author = {Grover, Shresth and Vineet, Vibhav and Rawat, Yogesh},
  journal = {Advances in Neural Information Processing Systems},
  volume = {36},
  pages = {65642--65664},
  year = {2023},
}

2022

End-to-End Semi-Supervised Learning for Video Action Detection

Akash Kumar and Yogesh Singh Rawat

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2022

CVPR Bib

@inproceedings{kumar2022end,
  title = {End-to-End Semi-Supervised Learning for Video Action Detection},
  author = {Kumar, Akash and Rawat, Yogesh Singh},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year = {2022},
}

Are all Frames Equal? Active Sparse Labeling for Video Action Detection

Aayush Rana and Yogesh S Rawat

In Advances in Neural Information Processing Systems, 2022

NeurIPS Bib HTML Code

@inproceedings{rana2022all,
  title = {Are all Frames Equal? Active Sparse Labeling for Video Action Detection},
  author = {Rana, Aayush and Rawat, Yogesh S},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
}

GabriellaV2: Towards better generalization in surveillance videos for action detection

Ishan Dave, Zacchaeus Scheffer, Akash Kumar, Sarah Shiraz, Yogesh Singh Rawat, and Mubarak Shah

In IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), 2022

WACV Bib PDF Website

@inproceedings{dave2022gabriellav2,
  title = {GabriellaV2: Towards better generalization in surveillance videos for action detection},
  author = {Dave, Ishan and Scheffer, Zacchaeus and Kumar, Akash and Shiraz, Sarah and Rawat, Yogesh Singh and Shah, Mubarak},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW)},
  pages = {122--132},
  year = {2022},
}

Video Action Detection: Analysing Limitations and Challenges

Rajat Modi, Aayush Jung Rana, Akash Kumar, Praveen Tirupattur, Shruti Vyas, Yogesh Singh Rawat, and Mubarak Shah

In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2022

Bib PDF Website

@inproceedings{modi2022video,
  title = {Video Action Detection: Analysing Limitations and Challenges},
  author = {Modi, Rajat and Rana, Aayush Jung and Kumar, Akash and Tirupattur, Praveen and Vyas, Shruti and Rawat, Yogesh Singh and Shah, Mubarak},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year = {2022},
}

Don’t Pour Cereal into Coffee: Differentiable Temporal Logic for Temporal Action Segmentation

Ziwei Xu, Yogesh S Rawat, Yongkang Wong, Mohan Kankanhalli, and Mubarak Shah

In Advances in Neural Information Processing Systems, 2022

NeurIPS Bib HTML Code

@inproceedings{xu2022don,
  title = {Don't Pour Cereal into Coffee: Differentiable Temporal Logic for Temporal Action Segmentation},
  author = {Xu, Ziwei and Rawat, Yogesh S and Wong, Yongkang and Kankanhalli, Mohan and Shah, Mubarak},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
}

2021

We Don’t Need Thousand Proposals: Single Shot Actor-Action Detection in Videos

Aayush J Rana and Yogesh S Rawat

In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2021

WACV Bib PDF Code Website

@inproceedings{rana2021we,
  title = {We Don't Need Thousand Proposals: Single Shot Actor-Action Detection in Videos},
  author = {Rana, Aayush J and Rawat, Yogesh S},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  pages = {2960--2969},
  year = {2021},
}

Modeling Multi-Label Action Dependencies for Temporal Action Localization

Praveen Tirupattur, Kevin Duarte, Yogesh Rawat, and Mubarak Shah

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

CVPR Bib PDF Code

@inproceedings{tirupattur2021modeling,
  title = {Modeling Multi-Label Action Dependencies for Temporal Action Localization},
  author = {Tirupattur, Praveen and Duarte, Kevin and Rawat, Yogesh and Shah, Mubarak},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  year = {2021},
}

Tinyaction challenge: Recognizing real-world low-resolution activities in videos

Praveen Tirupattur, Aayush J Rana, Tushar Sangam, Shruti Vyas, Yogesh S Rawat, and Mubarak Shah

arXiv preprint arXiv:2107.11494, 2021

arXiv Bib

@article{tirupattur2021tinyaction,
  title = {Tinyaction challenge: Recognizing real-world low-resolution activities in videos},
  author = {Tirupattur, Praveen and Rana, Aayush J and Sangam, Tushar and Vyas, Shruti and Rawat, Yogesh S and Shah, Mubarak},
  journal = {arXiv preprint arXiv:2107.11494},
  year = {2021},
}

Reformulating zero-shot action recognition for multi-label actions

Alec Kerrigan, Kevin Duarte, Yogesh Rawat, and Mubarak Shah

Advances in Neural Information Processing Systems, 2021

NeurIPS Bib PDF Slides

@article{kerrigan2021reformulating,
  title = {Reformulating zero-shot action recognition for multi-label actions},
  author = {Kerrigan, Alec and Duarte, Kevin and Rawat, Yogesh and Shah, Mubarak},
  journal = {Advances in Neural Information Processing Systems},
  volume = {34},
  pages = {25566--25577},
  year = {2021},
}

Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activities

Sirnam Swetha, Hilde Kuehne, Yogesh S Rawat, and Mubarak Shah

In 2021 IEEE International Conference on Image Processing, 2021

ICIP Bib

@inproceedings{swetha2021unsupervised,
  title = {Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activities},
  author = {Swetha, Sirnam and Kuehne, Hilde and Rawat, Yogesh S and Shah, Mubarak},
  booktitle = {2021 IEEE International Conference on Image Processing},
  year = {2021},
}

"Knights": First Place Submission for VIPriors21 Action Recognition Challenge at ICCV 2021

Ishan Dave, Naman Biyani, Brandon Clark, Rohit Gupta, Yogesh Rawat, and Mubarak Shah

arXiv preprint arXiv:2110.07758, 2021

arXiv Bib

@article{dave2021knights,
  title = {"Knights": First Place Submission for VIPriors21 Action Recognition Challenge at ICCV 2021},
  author = {Dave, Ishan and Biyani, Naman and Clark, Brandon and Gupta, Rohit and Rawat, Yogesh and Shah, Mubarak},
  journal = {arXiv preprint arXiv:2110.07758},
  year = {2021},
}

2020

Gabriella: An Online System for Real-Time Activity Detection in Untrimmed Security Videos

Mamshad Nayeem Rizve, Ugur Demir, Praveen Tirupattur, Aayush Jung Rana, Kevin Duarte, Ishan Dave, Yogesh Singh Rawat, and Mubarak Shah

In 25th International Conference on Pattern Recognition (ICPR), 2020

ICPR Awarded Bib PDF Video Website

Best Scientific Paper Award, ICPR 2020

@inproceedings{rizve2020gabriella,
  title = {Gabriella: An Online System for Real-Time Activity Detection in Untrimmed Security Videos},
  author = {Rizve, Mamshad Nayeem and Demir, Ugur and Tirupattur, Praveen and Rana, Aayush Jung and Duarte, Kevin and Dave, Ishan and Rawat, Yogesh Singh and Shah, Mubarak},
  booktitle = {25th International Conference on Pattern Recognition (ICPR)},
  pages = {4237--4244},
  year = {2020},
}

TinyVIRAT: Low-resolution Video Action Recognition

Ugur Demir, Yogesh S Rawat, and Mubarak Shah

In 25th International Conference on Pattern Recognition (ICPR), 2020

ICPR arXiv Bib Website

@inproceedings{demir2020tinyvirat,
  title = {TinyVIRAT: Low-resolution Video Action Recognition},
  author = {Demir, Ugur and Rawat, Yogesh S and Shah, Mubarak},
  booktitle = {25th International Conference on Pattern Recognition (ICPR)},
  year = {2020},
}