Projects Last Projects

Image inpainting is an extensively studied problem in image processing, and various tools have been brought to serve it over the years. Recently, effective solutions to this problem based on deep-learning have been added to this impressive list. This paper offers a novel and unconventional solution to the image inpainting problem, still in the context of deep-learning. As opposed to a direct solution of training a CNN to fill-in missing parts in images, this work promotes a solution based on pre-trained classification-oriented CNN. The proposed algorithm is based on the assumption that such CNN's have memorized the visual information they operate upon, and this can be leveraged for our inpainting task. The main theme in the proposed solution is the formulation of the problem as an energy-minimization task in which the missing pixels in the input image are the unknowns. This minimization aims to reduce the distance between the true image label and the one resulting from the network operating on the completed image. A critical observation in our work is the fact that for better inpainting performance, the pre-training of the CNN should be applied on small portions of images (patches), rather than the complete images. This ensures that the network assimilates small details in the data, which are crucial for the inpainting needs. We demonstrate the success of this algorithm on two datasets: MNIST digits and face images (Extended Yale B), showing in both the tendency of this method to operate very well.

Please, see project report.

Please, see final presentation.