We propose a method of using a focal stack, i.e., a set of differently focused images, as the input for a novel light field display called a “tensor display.” Although this display consists of only a few light attenuating layers located in front of a backlight, it can be viewed from many directions (angles) simultaneously without the resolution of each viewing direction being sacrificed. Conventionally, a transmittance pattern is calculated for each layer from a light field, namely, a set of dense multi-view images (typically dozens) that are to be observed from different directions. However, preparing such a massive amount of images is often cumbersome for real objects. We developed a method that does not require a complete light field as the input; instead, a focal stack composed of only a few differently focused images is directly transformed into layer patterns. Our method greatly reduces the cost of acquiring data while also maintaining the quality of the output light field. We validated the method with experiments using synthetic light field datasets and a focal stack acquired by an ordinary camera.
Yuto Kobayashi, Keita Takahashi, Toshiaki Fujii: "From Focal Stacks to Tensor Display: A Method for Light Field Visualization without Multi-View Images," Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2017), New Orleans, USA (2017.3.5-9). [ IEEE Xplore ]
Our source codes written in C++. [ Get our software ] [ Get our software v2 ]
Version 2 is written using cv::Mat instead of the obsolete IplImage.
Multi-view Camera to Tensor Display
ProFUSION25/Lytro Illum to Tensor Display (Experimental results only)
Development of an end-to-end system
Our system was demonstrated at International Display Workshop 2016.
Process pipeline from capture to display.
We have developed a prototype of a layered light-field (3-D) display, where three liquid crystal display (LCD) panels are stacked in front of a backlight.
We have also created an end-to-end system where a real 3D scene captured by a multi-view camera can be reproduced in 3D on this prototype display.
Our main contribution lies in the data conversion method using image-based rendering,
by which a set of sufficiently dense light field data, which is required for high quality displaying of a 3-D scene, is generated from a sparser set of data that is captured by the multi-view camera.
Theoretical Analysis and Computer Simulation
We have analyzed two limitation factors (upper-bound spatial frequency and anti-aliase condition) of a layered light-field display to derive suitable configurations for the input multi-view images. We have also proposed to use image-based rendering to synthetically generate a set of multi-view images that is suitable as the input.
Yuto Kobayashi, Shu Kondo, Keita Takahashi, Toshiaki Fujii: "A 3-D Display Pipeline: Capture, Factorize, and Display the Light Field of a Real 3-D Scene", ITE Transactions on Media Technology and Applications Vol. 5, No. 3, pp. 88--95, (2017.7.1) [ J-Stage (Open Access)]
Keita Takahashi, Yuto Kobayashi, Toshiaki Fujii: "Displaying Real World Light Fields using Stacked LCDs," The 23rd International Display Workshops in conjunction with Asia Display 2016, DES4/3D8 - 1, pp. 1300--1303, invited talk & Innovative Demonstration Session (I-DEMO), Fukuoka, Japan (2016.12.7-9).
Toyohiro Saito, Yuto Kobayashi, Keita Takahashi, Toshiaki Fujii: "Displaying Real-World Light-Fields with Stacked Multiplicative Layers: Requirement and Data Conversion for Input Multi-view Images," IEEE/OSA Journal of Display Technology, Volume 12, Issue 11, pp. 1290--1300, Print ISSN: 1551-319X, Online ISSN: 1558-9323, (2016.11) [ IEEE Xplorer ]
Keita Takahashi, Toyohiro Saito, Mehrdad Panapour Tehrani, Toshiaki Fujii, "Rank Analysis of a Light Field for Dual-Layer 3D Displays", IEEE International Conference on Image Processing (ICIP), NEW-P1.4, Poster, Quebec City, Canada, (2015.9.27-30).
Toyohiro Saito, Keita Takahashi, Mehrdad Panahpour Tehrani, Toshiaki Fujii: "Data Conversion from Multi-View Cameras to Layered Light Field Display for Aliasing-Free 3D Visualization", IS&T/SPIE Electronic Imaging, Stereoscopic Displays and Applications XXVI, Paper 9391-42, San Francisco, USA (2015.2.8-12).