Projects Proposed Projects

Long-range optical flow and dense tracking are powerful tools for applications such as 3D reconstruction, animal behavior analysis, and motion analysis. Another promising application is video editing, where edits made to a single frame can be propagated through a video sequence using optical flow, simplifying the task to image-level editing. However, current optical flow systems often suffer from drift errors and struggle with long-term occlusions, resulting in editing “holes” and inconsistencies. The recent Dense Optical Tracking (DOT) method has achieved state-of-the-art performance in long-range optical flow by combining a sparse set of long-range tracks with iterative refinement, providing superior occlusion handling and improved long-term stability.  This project aims to address the limitations of existing optical flow systems by leveraging DOT’s accurate long-range correspondences and, potentially, refining the final edits using VDM-based methods. This approach promises to improve occlusion recovery, reduce drifting errors, and enhance the quality of propagated edits. Students involved in this project will engage with cutting-edge advancements in video processing and generative modeling, building a strong foundation for further research and gaining valuable theoretical and practical skills.

Assaf Singer

Establishing dense point correspondences in video has seen remarkable progress in recent years, expanding recently to long-range point tracking.  DINO-Tracker, we introduced a novel self-supervised method for long-range dense video tracking, harnessing DINO's powerful visual priors. Our approach combines test-time training on individual videos with the highly localized semantic features learned by a pre-trained DINO-ViT model, yielding state-of-the-art results, particularly in tracking across extended occlusions. Further details can be found on our project page. While DINO-Tracker shows promise, its optimization process—taking 1-2 hours per video—limits its practicality for real-time applications. This project aims to address this by exploring feed-forward, real-time solutions that enhance temporal consistency across frames and increase DINO's spatial resolution, enabling high-resolution tracking capabilities. This project offers students the opportunity to engage with cutting-edge advancements in video tracking and processing, providing a solid foundation for further research in this dynamic field. Participants will gain expertise in both theoretical and practical aspects of video tracking and representation learning.

Assaf Singer

We recently introduced a novel photo-realistic image restoration algorithm based on flow matching, a generalization of diffusion. For more details, visit our project page: https://pmrf-ml.github.io.

We would like to continue with this line of work by:
1. Designing our algorithm to trade distortion (e.g., MSE) with visual quality at test time (by tuning a test time hyper-parameter).
2. Considering recent and more advanced schemes of flow matching, potentially improving both the theoretical guarantees and the practical results of our method.
This project provides an opportunity for students to engage with cutting-edge developments in photo-realistic image restoration, paving their way for further research in this dynamic field. The project will emphasize learning, brainstorming, and developing new mathematical insights rather than extensive coding hours. Participants will gain expertise for conducting both theoretical and practical research in the endless field of image restoration

Guy Ohayon

Daniel Silver

As a continuation of our latest publication, we thought of an elegant, theoretically-based solution to assess the determinism & robustness of an image restoration algorithm via the diversity of its outputs. 

Guy Ohayon

עריכת תמונות באמצעות מודלי דיפוזיה התפתחה מאוד בשנים האחרונות. שיטות רבות מצריכות סימון ידני היכן האובייקט יתווסף. שיטה חדשה שהצענו לאחרונה מבוססת על טקסט בלבד להוספת האובייקטים לתמונה. בפרוייקט נחליף את הטקסט המתאר את האובייקט שנרצה, בתמונת של האובייקט עצמו. The field of image editing with diffusion models has advanced significantly in recent years. Traditionally, many techniques necessitated manual annotation to designate the insertion point of an object within an image. Our newly proposed method, however, relies solely on textual descriptions to integrate objects into images. In our project, we aim to substitute the textual descriptions of desired objects with their corresponding images.

Noam Rotstein

Roee Ganz

Alzheimer's disease dementia (AD) is a debilitating and prevalent neurodegenerative disease in older adults worldwide. Cognitive impairment, a hallmark of AD, is assessed through verbal tests that require high specialization, and while accepted as screening tools for AD, general practitioners seldom use them. Recent evidence indicates that early detection of mild cognitive impairment (MCI) can enable interventions that slow the rapid decline in functioning.
In this project, the students will utilize CNN-based methods to develop and test an application tailored to differentiate between drawings of individuals with MCI, AD, and healthy controls (HC).

Noam Rotstein

Alon Zvirin

Developing an image editing pipeline that allows the user to edit personal photos using a user-friendly interface (dragging and text prompts). Photo Editing is a universally used tool, but few can master complex tools like Photoshop. We aim to develop an intuitive user-friendly method of editing. Where users can drag points around the picture and add text prompts to generate high-quality edited pictures. While this is possible using previously developed tools, we propose using a personal generative prior to constraining the images to the space of the person's images, thus providing a more desirable output.

Sari Hleihil

Coronary heart disease is one of the diseases with the highest mortality rate. Due to the important position of cardiovascular disease prevention and diagnosis in the medical field, the segmentation of cardiovascular images has gradually become a research hotspot. How to segment accurate blood vessels from coronary angiography videos to assist doctors in making accurate analysis has become the goal of our research. In this project, the student will be required to implement a new method based on the U-net architecture. The proposed method includes three modules: the sequence encoder module, the sequence decoder module, and the sequence filter module. The high-level information of the feature is extracted in the encoder module. Multi-kernel pooling layers suitable for the extraction of blood vessels are added before the decoder module. In the filter block, we add a simple temporal filter to reduce inter-frame flickers.

Noam Rotstein

Asi Elad

Instance segmentation is the task of detecting and masking objects in an image and distinguishing between instances of the same class. Accurate segmentation of leaves in plant images is important in many agricultural applications. These include early detection of water and heat stress, identification of biological infection, monitoring of plant growth, and prediction of harvest yields. Our purpose is to use an existing instance segmentation deep neural network, and integrate it with image processing tools for better performance. In this project we will improve upon previous results, participate in an international leaf segmentation challenge, and attempt to achieve best scores

Alon Zvirin

Yaron Honen

Chest motion abnormalities and sporadic breathing rates are often associated with respiratory system diseases. In this project, the students will use computer vision tools to calculate breaths per minute and other vital respiratory parameters.

Yaniv Wolf

A few years ago, we showed for the first time that the molecular profile of cancer can be predicted by analysis of biopsy images, without using molecular assays. In other words, the shape of the tumor cells and the tissue architecture hold information that allows to accurately predict molecular expression, even though such molecules cannot be seen by humans by visual examination of biopsy images.  Based on these findings, in the past two years we established collaborations with several medical data hospitals in Israel and abroad and extended the scope of our research to different prediction tasks in breast cancer, lung cancer, and Leukemia. We collected and scanned high quality well annotated tens of thousands of histology slides, and extended our research team, consisting of data scientists, graduate and postdoctoral students and clinical collaborators and advisors. We have recently shown how to steer our technology into assisting personalized medicine procedures. Take a part in developing an AI-based framework for analysis of histology images for improving personalized oncology  -  Link to the article 

Gil Shamai