Teck-Yian Lim

2022

1. NeurIPS

   

   Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networks

   Renán A. Rojas-Gómez, Teck-Yian Lim, Alexander G. Schwing, Minh N. Do, and Raymond A. Yeh

   Advances in Neural Information Processing Systems, 2022

   Abs Project Page Code

   We propose learnable polyphase sampling (LPS), a pair of learnable down/upsampling layers that enable truly shift-invariant and equivariant convolutional networks. LPS can be trained end-to-end from data and generalizes existing handcrafted downsampling layers. It is widely applicable as it can be integrated into any convolutional network by replacing down/upsampling layers. We evaluate LPS on image classification and semantic segmentation. Experiments show that LPS is on-par with or outperforms existing methods in both performance and shift consistency. For the first time, we achieve true shift-equivariance on semantic segmentation (PASCAL VOC), i.e., 100% shift consistency, outperforming baselines by an absolute 3.3%.

2018

1. ICASSP

   

   Image restoration with deep generative models

   Raymond A. Yeh, Teck-Yian Lim, Chen Chen, Alexander G. Schwing, Mark Hasegawa-Johnson, and 1 more author

   In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018

   Abs Project Page PDF Code Slides

   Many image restoration problems are ill-posed in nature, hence, beyond the input image, most existing methods rely on a carefully engineered image prior, which enforces some local image consistency in the recovered image. How tightly the prior assumptions are fulfilled has a big impact on the resulting task performance. To obtain more flexibility, in this work, we proposed to design the image prior in a data-driven manner. Instead of explicitly defining the prior, we learn it using deep generative models. We demonstrate that this learned prior can be applied to many image restoration problems using an unified framework.

2017

1. CVPR

   

   Semantic image inpainting with deep generative models

   Raymond A. Yeh, Chen Chen, Teck-Yian Lim, Alexander G. Schwing, Mark Hasegawa-Johnson, and 1 more author

   In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2017

   Abs Project Page PDF Code Poster

   Semantic image inpainting is a challenging task where large missing regions have to be filled based on the available visual data. Existing methods which extract information from only a single image generally produce unsatisfactory results due to the lack of high level context. In this paper, we propose a novel method for semantic image inpainting, which generates the missing content by conditioning on the available data. Given a trained generative model, we search for the closest encoding of the corrupted image in the latent image manifold using our context and prior losses. This encoding is then passed through the generative model to infer the missing content. In our method, inference is possible irrespective of how the missing content is structured, while the state-of-the-art learning based method requires specific information about the holes in the training phase. Experiments on three datasets show that our method successfully predicts information in large missing regions and achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.