Lightweight All-Focused Light Field Rendering

• https://www.sciencedirect.com/science/article/abs/pii/S1077314224001127

light field, plane sweep, image-based rendering, GPU, focusing

This paper proposes a novel real-time method for high-quality view interpolation
from light field. The proposal is a lightweight method, which can be used with
consumer GPU, reaching same or better quality than existing methods, in a shorter
time, with significantly smaller memory requirements. Light field belongs to
image-based rendering methods that can produce realistic images without
computationally demanding algorithms. The novel view is synthesized from multiple
input images of the same scene, captured at different camera positions. Standard
rendering techniques, such as rasterization or ray-tracing, are limited in terms
of quality, memory footprint, and speed. Light field rendering methods often
produce unwanted artifacts resembling ghosting or blur in certain parts of the
scene due to unknown geometry of the scene. The proposed method estimates the
geometry for each pixel as an optimal focusing distance to mitigate the
artifacts. The focusing distance determines which pixels from the input images
are mixed to produce the final view. State-of-the-art methods use a constant-step
pixel matching scan that iterates over a range of focusing distances. The scan
searches for a distance with the smallest color dispersion of the contributing
pixels, assuming that they belong to the same spot in the scene. The paper
proposes an optimal scanning strategy of the focusing range, an improved color
dispersion metric, and other minor improvements, such as sampling block size
adjustment, out-of-bounds sampling, and filtering. Experimental results show that
the proposal uses less resources, achieves better visual quality, and is
significantly faster than existing light field rendering methods. The proposal is
8× faster than the methods in the same category. The proposal uses only four
closest views from the light field data and reduces the necessary data transfer.
Existing methods often require the full light field grid, which is typically 8×8
images large. Additionally, a new 4K light field dataset, containing scenes of
various types, was created and published. An optimal novel method for light field
acquisition is also proposed and used to create the dataset.