Grayscale To Colorization Using Machine Learning

Amit Prajapati
Electroics and Communication Engineering
Galgotia College of Engineering and Technology
Uttar Pradesh, India
Avijeet Pandey
Electroics and Communication Engineering
Galgotia College of Engineering and Technology
Uttar Pradesh, India
Akash Singh
Electroics and Communication Engineering
Galgotia College of Engineering and Technology
Uttar Pradesh, India
Akash Yadav
ABSTRACT: We represent an efficient and effective new deep learning method for image colorization.
Our approach uses a generative adversarial network (GAN) that has been trained on a sizable dataset of
colorized and grayscale photos. There are various ramifications for this work. It first shows that the col-
orization of images may be automated using deep learning. This is an important development since it may
allow for the colorization of enormous historical image archives, which would be beneficial for both re-
search and teaching. Secondly, this approach is more effective than earlier approaches, which may in -
crease its usability for academics and developers. Finally, our research shows how GANs can be used for
a variety of other image processing applications. We consider our study to be a significant step toward the
creation of more potent and effective image colorization techniques.

Keywords: Deep Learning, U-Net, Convolutional Neural Networks (CNNs), Image Colorization, Re-
search, Education, Potential, Inspiration, and Visualization; Efficiency, Fidelity; Historical Images;

1. INTRODUCTION

This study report provides comprehensive details on sophisticated methods for converting grayscale pho-
tos into color images. Recently, there has been a lot of activity in the field of study on using Generative
Adversarial Networks (GANs) to convert grayscale photos to colorful ones. Deep learning models called
GANs are able to produce new data that closely resembles the input data. The process of giving a
grayscale image color is known as image colorization. This task has been around for centuries and has
been used in many other fields, including science, art, and photography. Image colorization used to be a
labor-intensive manual procedure that took a lot of experience. Nevertheless, image colorization may now
be automated thanks to deep learning. There are many potential applications for image colorization. For
example,it could be used to:

● Add color to historical photos to assist make them more readable and engaging for those who are not
familiar with the era.
● Add color to medical pictures to aid in better disease diagnosis and treatment for medical professionals.

● colorize satellite photos to enable closer examination of the Earth's ecosystem by scientists. Product
photos can be made more colorful to help businesses sell more goods. In advance of picture colorization
becoming extensively employed, a number of issues must be resolved. One problem is that it might be
hard to know which colors are right for an image. For instance, one cannot always identify what color an
automobile in a photograph is meant to be because they can be any color. Colorizing images can be com-
putationally costly, which presents another difficulty. This is due to the fact that deep learning models
need a lot of data and processing power to be trained. In this research, we propose an efficient and novel
deep learning approach for image colorization. A generative adversarial network (GAN), a kind of deep
neural network that can be trained to produce realistic images, is the foundation of our approach. We use
a sizable collection of grayscale and correspondingly colored photos to train our GAN. Compared to ear-
lier techniques, our technology colorizes photos more accurately and faithfully while also being more ef-
ficient. We consider our work to be a major advancement in the field of image colorization technology. It
is our aim that our technique will be applied to colorize a large range of photos and contribute to a more
vibrant world. Our Methodology: Our method of colorizing images is predicated on generative adversar -
ial networks (GANs). One kind of deep neural network that can produce realistic-looking images is a
GAN. A discriminator network plus a generator network make up a GAN. The discriminator is in charge
of determining which photos are fraudulent and which are real, while the generator creates new ones. The
generator has been trained to produce visuals that are identical to actual ones. The discriminator receives
training to discern between authentic and counterfeit photographs. The generator gradually gains the abil -
ity to produce more lifelike images as a result of the two networks being trained simultaneously.

2. LITERATURE REVIEW AND RELATED WORK

There has been numerous research and methods proposed to address the problem of transforming
grayscale images into color. Among the popular ways are example-based approaches, gradient-based
methods, and colorization utilizing deep learning. A literature study on the application of generative ad-
versarial networks, or GANs, for image colorization would entail an analysis of the prior studies on the
subject. (Source:) Tung Nguyen, Kazuki Mori, and Ruck Thawonmas’ article "Image Colorization Using
a Deep Convolutional Neural Network.” This study offers a fresh method for colorizing grayscale photos
using deep learning algorithms.by applying a convolutional neural network that has been trained before-
hand and was initially intended for picture classification. This makes it possible for us to take the infor -
mation and style of various photographs and mix them again into one. Next a method is suggested that al -
lows colors to be introduced to a grayscale image by fusing the content of a color image with a style that
shares semantic similarities with the grayscale image. [/2] Nazeri, Kamyar, Ng, Eric, and Ebrahimi,
Tehran, "Image Colorization Using Generative Adversarial Networks." This research provides a method
of learning deep representations using Generative Adversarial Networks (GANs) without heavily annotat-
ing training material. They accomplish this by using a competitive approach utilizing two networks to de-
rive backpropagation signals. Many applications, such as image synthesis, semantic picture editing, style
transfer, image super-resolution, and classification, can make use of the representations that GANs can
learn. With the use of well-known metaphors and ideas, this work seeks to give the signal processing
community an overview of GANs. [/3]By Klaus Mueller, Michael Ashikhmin, and Tomihisa Welsh,
"Transferring Color to Grayscale Images." This paper describes a general method for transferring color
from a source color image to a destination grayscale image, so "colorizing" grayscale images. By match-
ing brightness and texture information across the images, they transfer the full color "mood" of the source
to the target image instead of selecting RGB colors from a palette to color specific components. They de-
cide to keep the target image's original brightness values and simply transfer chromatic information. Al-
lowing the user to match regions of the two photos with rectangular swatches improves this process even
further. They demonstrated that, as long as the texture and luminance are sufficiently different, this
straightforward technique may be successfully used on a wide range of photos and videos. The resulting
photographs show our technique's potential and usefulness across a wide range of application sectors.
*[4] Ian Goodfellow, "Generative Adversarial Networks.” This research provides a method of learning
deep representations using Generative Adversarial Networks (GANs) without heavily annotating training
material. They accomplish this by using a competitive approach utilizing two networks to derive back -
propagation signals. Many applications, such as image synthesis, semantic picture editing, style transfer,
image super-resolution, and classification, can make use of the representations that GANs can learn. With
the use of well-known metaphors and ideas, this work seeks to give the signal processing community an
overview of GANs. (5) Mehdi Mirza and Simon Osindero's work "Conditional Generative Adversarial
Net. “In this study, we provide a general introduction to the conditional form of generative adversarial
nets, which may be built by feeding the generator and discriminator the data, y, that we want to condition.
They demonstrate how this approach is able to produce MNIST digits that are dependent on class labels.
They also show how a multi-modal model may be learned using this model, and they offer early instances
of an application to picture tagging where we show how this method can produce descriptive tags that are
not included in the training labels.[6] “GAN Faculty of Science Image Colorization" by Eric Ng, Mehran
Abrahami, and Kamyar Nazeri. The automatic picture colorization procedure, which has been of great in-
terest for several application areas, including the restoration of old or damaged photographs, is repre-
sented in this work.

3. METHODOLOGY

In this paper, we will go over the study paper's methodology. The model architecture will be explained
first, followed by the dataset that was used, and then the training process.

A. Dataset

The COCO dataset, which has 8,000 images, is the dataset that was employed for this study. A training
set and a validation set are the two sets of photos in the dataset. The model is trained on the training set,
and its performance is assessed on the validation set.

B. Model Architecture

● Generator

U-Net serves as our generator. Convolutional neural networks like U-Net are employed in image segmen-
tation applications. It is made up of a decoder and an encoder. After downsampling the input image, the
encoder extracts its features. The feature maps are upsampled by the decoder, which then produces a seg -
mentation mask. U-Net links the encoder and decoder layers together using skip connections. The de -
coder can create fine-grained segmentation masks using the high level features that the encoder taught it
thanks to the skip connections. When it comes to medical image segmentation tasks, such as separating
tumors or organs from MRI or CT scans, U-Net performs exceptionally well. Other image segmentation
tasks, such segmenting cells in microscopy pictures or creating segmentation from satellite photos, have
also been accomplished with it.

● Discriminator

The discriminator's architecture is straightforward. To determine if an input image is real or false, the
convolution-batch normalization blocks of LeakyReLU layers are layered together. Note that the last
block has no activation function since it is part of the loss function that will be employed; the first and last
blocks do not contain normalization.

● Activation functions

ReLU: This is a popular activation function that gives the input values a threshold. Positive values remain
unaltered while all negative values are set to zero. Deep neural networks make extensive use of this non -
linear function, which aids in introducing nonlinearity into the network. BreachReLU: For negative input
values, this version of the ReLU activation function adds a tiny negative slope. In addition to enabling the
network to learn from negative input values, it helps to reduce the "dying ReLU" issue. {f(x) = max(x, al -
pha*x)} is the definition of the function, and {alpha} is a hyperparameter that regulates the function's
slope for negative input values.

C. Instructional Method

With a learning rate of 0.0001, the Adam optimizer was used to train the model. Training the model
yielded 100 periods. The loss function that was applied was a mix of L1 loss and loss of GAN. The L1
loss aids in guaranteeing that the correct colors are produced, and the GAN loss aids in making certain
that the photos produced are accurate.

D. Assessing

The model's effectiveness was assessed using the subsequent metrics: The Mean Absolute Error (MAE)
metric is utilized to quantify the mean discrepancy between the anticipated colors and the colors of real -
ity. The statistic known as the Structural Similarity Index (SSIM) assesses the degree of overlap between
the anticipated and the photos that are based on ground truth. The peak signal-to-noise ratio, or PSNR, is
a measurement. evaluates the expected photos' quality.