Before Northeastern, I spent 8 months at the Vision and AI Lab at IISc developing diffusion-based monocular depth estimation, 6 months at Dashtoon shipping production diffusion pipelines for animated character generation, over a year at Collablens building deployed computer vision systems for Indian manufacturing lines, and a year and a half at EZ Works developing neural machine translation and document intelligence pipelines.
I am actively seeking Summer 2026 research and engineering internships.
A video large language model can watch two clips of the same person in the same room and name every object in both, yet fail to tell you which one moves faster, which way a hand travels, or how hard a door is shut. That gap matters because Video-LLMs are increasingly the perceptual front end of world models: auto-labeling state transitions, acting as reward and success detectors for model-based RL, grounding instructions in what an agent sees. Each of those roles presupposes a competence at reading motion that, we show, these models do not have. MotionBlind is a contrastive, minimal-pairs benchmark of self-recorded clip pairs that are near-identical except in motion, each paired with two complementary yes/no questions, so a model must answer all four items correctly (Instance Accuracy, I_Acc) to score the instance. Single-frame, appearance and language-only shortcuts all collapse toward the 6.25% chance floor. It complements TimeBlind with physically grounded categories (speed, magnitude, direction) that are hard to source and label from uncontrolled internet video but are precisely the variables a world model must predict. Across two suites we run a controlled study of six open Video-LLMs and two frontier models over frame budgets 1-24 and four frame-selection strategies. Open models sit near the chance floor; a larger model does no better; and neither more frames nor dynamic selection, which changes which frames are seen and not whether motion is read, closes the gap. The task genuinely needs video in the right order: removing it zeroes every model, and shuffling frames collapses accuracy to chance. Only Gemini 3.1 Pro clears the benchmark overall, yet it too fails on the purely rate-defined categories.
@inproceedings{bhatia2026motionblind,title={MotionBlind: Probing the Illusion of Motion Understanding in Video-LLMs},author={Bhatia, Dhairya and Galoaa, Bishoy and Fritsche, Oliver and Kamal, Shahid and Abdul Salam, Muhammad Obaidullah and Saleem, Umer and Rastogi, Om and Chettiar, Frania Felix and Erdoğmuş, Nesli and Ostadabbas, Sarah},booktitle={Under review, NeurIPS 2026 Workshop on World Models in Physical AI},year={2026},}
AdSelect: Fine-Tuning Multimodal LLMs for Advertisement Shot Selection
Om Rastogi, Bishoy Galoaa, Wen Xie, and 3 more authors
AdSelect frames advertisement editing as a set-prediction problem, mapping shots from a focal (long) video to the subset retained in the target (short) edit. We introduce a large-scale dataset of 4,800 long-short ad pairs mined from YouTube across 17 industries, and propose AdCraft, a fine-tuning method that addresses the over-selection failures of zero-shot multimodal LLMs by combining LoRA with explicit complement-shot supervision. AdCraft reaches 0.771 precision and 0.688 IoU on the 800-pair benchmark, showing that in zero-shot MLLMs selection is not understanding.
@article{rastogi2026adselect,title={AdSelect: Fine-Tuning Multimodal LLMs for Advertisement Shot Selection},author={Rastogi, Om and Galoaa, Bishoy and Xie, Wen and Overgoor, Gijs and Bart, Yakov and Ostadabbas, Sarah},journal={Under review, AAAI 2027},year={2026},}
AdShot: A Benchmark for Advertisement Video Clipping with Multimodal LLMs
Wen Xie, Om Rastogi, Sai Siddhartha Vivek Dhir Rangoju, and 3 more authors
AdShot is a benchmark for evaluating multimodal large language models on advertisement video clipping, the task of shortening a 30-second ad to a 15-second cut while preserving brand messaging and narrative coherence. It comprises 823 video pairs across 194 brands sourced from real advertisement archives, with an automated evaluation harness scoring temporal coherence, brand retention, and narrative quality across models including Qwen-Omni, MiniCPM-o, and Aria.
@article{rastogi2026adshot,title={AdShot: A Benchmark for Advertisement Video Clipping with Multimodal LLMs},author={Xie, Wen and Rastogi, Om and Rangoju, Sai Siddhartha Vivek Dhir and Overgoor, Gijs and Bart, Yakov and Ostadabbas, Sarah},journal={Under review, NeurIPS 2026},year={2026},}
Color Masking Method for Variable Luminosity in Videos with Application in Lane Detection Systems
Om Rastogi and others
In Proceedings of International Conference on Machine Intelligence and Data Science Applications, 2021
The paper is focused on providing a color thresholding method, that is independent of lighting conditions in a video. A two-layered process is proposed, a luminosity filter followed by a variable color threshold. The approach is non-parametric and deterministic. For applications like Lane Detection System, there can be multiple lighting conditions depending on the weather conditions and time of day. Being able to use a color mask, instead of edging techniques for road markers would be robust.
@inproceedings{rastogi2021color,title={Color Masking Method for Variable Luminosity in Videos with Application in Lane Detection Systems},author={Rastogi, Om and others},booktitle={Proceedings of International Conference on Machine Intelligence and Data Science Applications},year={2021},publisher={Springer},doi={10.1007/978-981-33-4087-9_24},}
