CVPR 2022

RealMCVSR Dataset

Real-world Multi-Camera Video Super-Resolution Dataset

Junyong Lee, Myeonghee Lee, Sunghyun Cho, Seungyong Lee
POSTECH

Download Link

Dropbox OneDrive

The RealMCVSR dataset provides real-world HD video triplets concurrently recorded by Apple iPhone 12 Pro Max equipped with triple cameras.

About the dataset

Our RealMCVSR dataset provides real-world HD video triplets concurrently recorded by Apple iPhone 12 Pro Max equipped with triple cameras having fixed focal lengths: ultra-wide (30mm), wide-angle (59mm), and telephoto (147mm).

To concurrently record video triplets, we built an iOS app that provides full control over exposure parameters (i.e., shutter speed and ISO) of the cameras. For recording each scene, we set the cameras in the auto-exposure mode, where the shutter speeds of the three cameras are synced to avoid varying motion blur across a video triplet. ISOs are adjusted accordingly for each camera to pick up the same exposure. Each video is saved in the MOV format using HEVC/H.265 encoding with the HD resolution (1080 x 1920).

The dataset contains triplets of 161 video clips with 23,107 frames in total. The video triplets are split into training, validation, and testing sets, each of which has 137, 8, and 16 triplets with 19,426, 1,141, and 2,540 frames, respectively.

Video Samples

Dataset Structure

For each video in ultra-wide (UW), wide-angle (W), and telephoto (T) video triplets concurrently recorded from iPhone 12 Pro Max, we extract video frames and save them in PNG format. We divide the video frames into training, test, and validation sets, and for each of the divided sets, we provide video frames in original (HR), 2x (LRx2), and 4x (LRx4) downsampled resolutions. Note that for downsampling operation, we used imresize function (bicubic) provided in MATLAB. The following diagram shows the detailed file structure of the RealMCVSR dataset. 

RealMCVSR
├── train                       # a training set
│   ├── HR                      # videos in original resolution 
│   │   ├── T                   # telephoto videos
│   │   │   ├── 0002            # a video clip 
│   │   │   │   ├── 0000.png    # a video frame
│   │   │   │   └── ...         
│   │   │   └── ...            
│   │   ├── UW                  # ultra-wide-angle videos
│   │   └── W                   # wide-angle videos
│   ├── LRx2                    # 2x downsampled videos
│   └── LRx4                    # 4x downsampled videos
├── test                        # a testing set
└── valid                       # a validation set

BibTeX

@InProceedings{Lee2022RefVSR,
    author    = {Junyong Lee and Myeonghee Lee and Sunghyun Cho and Seungyong Lee},
    title     = {Reference-based Video Super-Resolution Using Multi-Camera Video Triplets},
    booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2022},
}