Nowadays, Satellite images are used for various analysis, including building detection and road extraction, which are directly beneficial to governmental applications, such as urbanization, and monitoring the environment. Spatial resolution is an element of crucial impact on the usage of remote sensing imagery.  High spatial resolution means satellite images provide more detailed information. To improve the spatial resolution at the sensor level, many factors are ought to be taken into consideration, such as the manufacturing process. Moreover, once the satellite is launched, no further action can be taken from the perspective of hardware.  Therefore, a more practical solution to improve the resolution of a satellite image is to use Single Image Super Resolution (SISR) techniques.

Methodology and results:

This research proposal deals with the re-design, implementation, and evaluation of SISR technique using Deep Convolutional Neural Network with Skip Connections and Network in Network (DCSCN) for enlarging multispectral remote sensing images captured by DubaiSat-2 (DS-2) and estimating the missing high frequency details. The goal is to achieve high performance in terms of quality, and to test whether training the network using luminance channel only, which is extracted from YCbCr domain, can achieve high quality results. For this purpose, DCSCN is trained, evaluated, and tested using a dataset collected from DS-2.  A single low resolution DS-2 image is used to construct its high resolution version by training the model from scratch and fine-tuning its hyper-parameters to produce optimal results. The performance is evaluated using various quality indices, such as Structural Similarity Index Measurement (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Wavelet Signal-to-Noise Ratio (WSNR). The performance is compared to other state-of-the-art methods, such as Bilinear, Bi-cubic, and Lanczos interpolation.

Figure 1- (left) result of RGB super resolution vs. (right) result of Y channel super resolution

From the results seen in Tables 1, 2, and 3, it can be concluded that the hypothesis is correct. Training the network over Y channel only, and using Bi-cubic interpolation to enlarge Cb and Cr, gives overall similar results to those of RGB in terms of SSIM and WSNR. In some cases, SSIM for YCbCr prevails over RGB. What is interesting to note is that in Table 3, as the scaling factor becomes larger, training with YCbCr shows more resilience. That is, the gap between RGB and YCbCr results becomes smaller in all metrics, and this is especially clear in SSIM. Moreover, the results from YCbCr also show superiority against commonly used interpolation methods, such as Bi-linear, Bi-cubic, and Lanczos, as seen in Table 4.  From Figure 1, it can be seen that, visually, RGB and YCbCr produce the same quality.

Duration:

5 month (Feb,2019 – July,2019)

Team members:

  • Eng. Nour Aburaed
  • Dr. Alavi Panthakkan
Single Image Super Resolution of DS-2 Satellite Imagery Using Deep Convolutioal Neural Network