Dr. Hila Chefer | Tel Aviv University

Abstract:
Despite remarkable advances, visual generative models are still far from faithfully modeling the world, struggling with fundamental aspects such as spatial relations, physics, motion, and dynamic interactions.
In this talk, I present a line of work that tackles these challenges, based on a deep understanding of the inner mechanisms that drive models. I will begin by analyzing state-of-the-art visual generators, gaining insights into the underlying reasons for their limited understanding. Building upon these insights, I will demonstrate methods that significantly enhance both spatial and temporal reasoning in image and video generation, surpassing even resource-intensive proprietary models without relying on additional data or model scaling. I will conclude the talk by discussing open challenges and future directions for advancing faithful world modeling in visual generative models.

Bio:
Dr. Hila CheferExternal link is a Research Scientist at Black Forest Labs and an incoming Assistant Professor (Senior Lecturer) at Tel Aviv University. Her research focuses on architecture development and interpretability for visual foundation models, aiming to both understand and advance their generative capabilities.

Dr. Chefer earned her PhD from Tel Aviv University under the supervision of Prof. Lior Wolf, where she developed the de facto leading methods for Transformer interpretability and the Attend-and-Excite framework for text-to-image control. Her work in video generation includes the development of Lumiere (a foundational video generation model) during her time at Google Research, and VideoJAM (a joint appearance-motion framework for motion and physics fidelity in video generation) at Meta AI. Dr. Chefer`s contributions have been recognized with several prestigious honors, including the Fulbright Postdoctoral Fellowship, the Eric and Wendy Schmidt Postdoctoral Award, and the Council for Higher Education (VATAT) Award for Outstanding PhD Students.