Edson Araujo

Omni-R1 fine-tunes the Qwen2.5-Omni multi-modal LLM on an audio QA dataset using GRPO, achieving SOTA on the MMAU benchmark. Surprisingly, much of this improvement stems from enhanced text-based reasoning, and fine-tuning on a text-only dataset also boosted audio-based performance.

We improved audio-visual learning by treating audio as a temporal sequence aligned with video frames instead of global representations, and by separating competing objectives with dedicated tokens. Our approach outperforms more complex architectures on retrieval, classification, and localization tasks across multiple datasets.

This paper introduces a novel weakly-supervised reinforcement learning method to accelerate instructional videos using text, addressing limitations of current summarization techniques. The proposed approach uses a new joint reward function and the Extended Visually-guided Document Attention Network (VDAN+) to effectively control video length while achieving state-of-the-art performance.

This paper presents a novel reinforcement learning methodology to accelerate instructional videos by adaptively selecting and removing irrelevant frames, addressing visual gaps in traditional summarization. The approach utilizes a textually and visually oriented agent with a new Visually-guided Document Attention Network (VDAN) for creating shorter, coherent videos and achieves state-of-the-art F1 score and segment coverage.

This paper introduces an approach for automatically creating personalized fast-forward videos from First-Person Videos (FPVs) by leveraging text-centric data from a user's social networks to infer their topics of interest. The method assigns scores to input frames based on user preferences and significantly outperforms competitors in F1 score, with its effectiveness demonstrated through extensive experiments and a user study.

This study investigates the impact of acoustic annoyance on drivers in real-world scenarios, revealing significant differences in driving styles and presenting an online classifier that detects driver annoyance from inertial measurements with 77% accuracy. While empirically confirming a passenger dynamics model using pressure mat data, the study couldn't establish a change in self-reported passenger comfort due to acoustically induced driving styles not being sufficiently distinct.