XJTLU Entrepreneur College (Taicang) Cover Sheet

Module code and Title DTS304TC Machine Learning

School Title School of AI and Advanced Computing
Assignment Title Assessment Task 2
Submission Deadline 23:59, 14th April (Sunday), 2024
(China Time, GMT + 8)
Final Word Count N/A
If you agree to let the university use your work anonymously for teaching and
learning purposes, please type “yes” here.

I certify that I have read and understood the University’s Policy for dealing with Plagiarism, Collusion
and the Fabrication of Data (available on Learning Mall Online). With reference to this policy I certify
that:
1) My work does not contain any instances of plagiarism and/or collusion.
My work does not contain any fabricated data.

By uploading my assignment onto Learning Mall Online, I formally declare that all of the
above information is true to the best of my knowledge and belief.
Scoring – For Tutor Use
Student ID

Stage of Marking Marker Learning Outcomes Achieved （F/P/M/D） Final

Code (please modify as appropriate) Score
A B C
Marker – red pen
1st
Moderation The original mark has been accepted by the moderator (please circle Y/N
IM as appropriate):
– green pen Initials
Data entry and score calculation have been checked by another tutor Y
(please circle):
2nd Marker if
needed – green pen
For Academic Office Use Possible Academic Infringement (please tick as appropriate)
Date Days Late ☐ Category A
Received late Penalty Total Academic Infringement Penalty (A,B,
☐ Category B C, D, E, Please modify where necessary)
_____________________
☐ Category C
☐ Category D
☐ Category E
 DTS304TC Machine Learning
Coursework – Assessment Task 2
Submission deadline: TBD
Percentage in final mark: 50%
Learning outcomes assessed: C, D
⚫ Learning outcome C: apply ML algorithms for specific problems.
⚫ Learning outcome D: Demonstrate proficiency in identifying and customizing aspects on ML
algorithms to meet particular needs.
Individual/Group: Individual
Length: This assessment comprises a guided coding and experimentation project, along with an
associated research report.
Late policy: 5% of the total marks available for the assessment shall be deducted from the assessment
mark for each working day after the submission date, up to a maximum of five working days
Risks:

• Please read the coursework instructions and requirements carefully. Not following these instructions
and requirements may result in loss of marks.
• The formal procedure for submitting coursework at XJTLU is strictly followed. Submission link on
Learning Mall will be provided in due course. The submission timestamp on Learning Mall will be
used to check late submission.
__________________________________________________________________

Guided Project: Street-view Semantic Segmentation with U-Net

In this project, you will evaluate, fine-tune, and perform unsupervised domain adaptation of a
pretrained U-Net model for street-view semantic segmentation. You will encounter some challenges
associated with street-view segmentation, such as segmenting objects in diverse driving scenes and
classifying each pixel into 14 categories (sky, road, building, car and etc). The given pretrained model
is trained on bright sunny images, which may not perform as well on darker cloudy images. Your task
is to fine-tune this model on such images. Furthermore, you will explore unsupervised domain
adaptation, assuming no ground truth is available for cloudy images due to the high cost of
segmentation annotation. We will employ two methods: (1) pseudo ground truth generation, and (2)
adapted batch normalization, to enhance the model's performance on cloudy image set.
The assessment consists of two segments: (1) coding and experimentation, and (2) the composition of a
research report. As you progress through the coding and experimentation phase, it is important to
record and document all pertinent findings, which will be integrated into your research report.

Before starting this project, please ensure that you have downloaded the pretrained model, assessment
codebase, and the example Jupyter notebook.

Notes:
1. Although a GPU-powered machine is preferable for running this project, you can efficiently
complete it on a CPU-only machine, such as those provided in school labs. This is due to the small size
of both the model and the fine-tuning dataset.
2. You are strongly encouraged to read the provided code and accompanying training notebook as
you will be referring to this code and modifying it during the project. You are encouraged to utilize code
that was covered during our Lab sessions, as well as other online resources for assistance. Please ensure
that you provide proper citations and links to any external resources you employ in your work. However,
the use of Generative AI for content generation (such as ChatGPT) is not permitted on all assessed
coursework in this module.

Coding and Experimentation Guideline

The experiments are divided into four parts:
1) Evaluate the pretrained model on bright sunny test images (20 marks):
Finish the supplied notebook. The pretrained model and testing dataloader code are provided. You are
required to implement the testing loop and evaluation metrics.
a. Testing Loop. Implement the testing loop to predict the testing images using the pretrained model.
Ensure the model is in eval mode and use the torch.no_grad() decorator for accuracy and efficiency.
b. Accuracy Metrics. Within the testing loop, calculate the global image accuracy and per-class accuracy
for each image and for the overall dataset.
a) Global image accuracy is the percentage of correctly classified pixels per image.
The overall dataset global image accuracy is calculated by averaging the global image accuracy
across all images in the dataset.
b) Per-class accuracy is the Intersection-Over-Union (IOU) score for each of the 14 categories per
testing image, with mean-IOU being the average across these categories.
 Handle categories not present in both the ground truth and predictions correctly by assigning
NaN IOU scores and use numpy.nanmean function for mean-IOU calculation
The overall dataset per-class IOU scores are obtained by computing the average per-class IOU
for each category across all images in the dataset.
Present the overall dataset global image accuracy and overall dataset per-class IOU scores in the report.
Please observe that there may be significant variation in IOU scores across different classes.
Investigate the reasons for this discrepancy and provide explanations in the report.
c. Ranking and Analysis. Rank the testing images by global image accuracy and mean-IOU. Observe that
the rankings for global image accuracy and mean-IOU may differ significantly, such that the top or
worst-ranked images based on global image accuracy may not be the same when ranked according to
mean-IOU. Provide explanations for these differences with example images where rankings vary
between the two metrics and state them in the report.
d. Analysis the challenges associated with street-view segmentation in the project report and discuss the
difficulties using specific image examples.
2) Evaluate and fine-tune the pretrained model on darker cloudy dataset (12 marks):
Finish the supplied notebook. The pretrained model and testing dataloader code for cloudy dataset are
provided in the notebook.
a. Evaluate Cloudy Dataset Performance. Utilize the code from Part 1 to assess the cloudy dataset's
performance, which will likely be lower, indicating the need for improvement.
b. Fine-tuning. Adapt the training notebook code to fine-tune the model on the cloudy dataset, adhering
to the suggested learning rate and epoch count. Monitor and plot training/validation loss to determine
the best fine-tuned model. The reporter should show training/validation loss curves to demonstrate how
best fine-tuned model is selected.
c. Re-evaluation. Re-evaluate the best fine-tuned model to confirm performance improvements. The
report should state overall dataset global image accuracy and overall dataset per-class IOU scores
before and after finetuning.

3) Unsupervised domain adaptation using pseudo-ground truth generation (18 marks):

As the creation of semantic segmentation ground truth is both time-consuming and labor-intensive, we often
encounter a surplus of unlabeled images. The challenge of leveraging such unlabeled images to enhance
the performance of a model—pretrained on a source domain with characteristics like bright, sunny
images—when applied to a target domain containing darker, cloudy images is known as unsupervised
domain adaptation. In this part, and subsequently in part (4), we will address the challenge of unsupervised
domain adaptation for our cloudy image dataset under the assumption that no ground truth is available for
the training set of these images. It is important to note that while ground truth data does exist for the cloudy
training dataset, its use is strictly prohibited as it contradicts the purpose of this exercise. Employing
ground truth in the training set, specifically within the scope of unsupervised domain adaptation, will incur
substantial score penalties for both this segment and part (4).
Pseudo-ground truth generation is a technique commonly used in unsupervised domain adaptation to
leverage unlabeled data in the target domain. The idea is to use our pretrained model to predict labels for
the unlabeled target domain data. These predictions, despite not being as accurate as human-annotated
ground truth, provide a 'good enough' estimate that can be used for further training. The process typically
involves the following steps:
⚫ Model Prediction: Apply the model trained on the source domain (e.g., bright, sunny images) to the
target domain data (e.g., darker, cloudy images) to generate predictions. Since the model is not yet
adapted to the target domain, these predictions will not be perfect and will likely contain errors.
⚫ Confidence Thresholding: Evaluate the confidence of the model's predictions and retain only those
that exceed a certain confidence level. The threshold can be determined empirically. This step ensures
that only the predictions the model is most sure about are used in the next stage of training.
⚫ Further Training: Use the pseudo-ground truths as labels to further train the model on the target domain
data. This step is similar to standard supervised learning but uses generated labels instead of true
annotations. The model can be fine-tuned using these labels, allowing it to gradually learn the
characteristics of the target domain.

Complete the following tasks:

a. Experimentation with Model Confidence.
Entropy serves as a conventional indicator of model confidence; a higher entropy in the probabilistic
distribution output by a classifier, for instance, suggests uniformity across the scores for the 14 classes,
indicating a lower level of certainty in the classification and thus a reduced confidence level. Complete
the following experiments:
⚫ Compute the entropy map for the segmentation result. Begin by applying the softmax activation
function to the model's output to determine the probabilistic scores. Then, utilize the entropy
formula to calculate the entropy map for the model.
⚫ Compare this entropy map to the segmentation error map, which reflects whether the ground truth
matches the predictions at each respective location.
⚫ For your selected testing images, show both segmentation error map and entropy map examples
to demonstrate any findings you can find. Present the images and findings in your report.
⚫ Additionally, explore the relationship between model entropy and segmentation error. Flatten the
entire entropy map, concatenate the entropy values from across all dataset images, and categorize
the pixels into quartiles based on their entropy levels: top 25%, 25-50%, 50-75%, and 75-100%.
In the report, calculate the average segmentation error within these intervals to uncover any
patterns or insights.
b. Fine-tuning with Pseudo-ground Truth. A notebook has been provided for your use in the domain
adaptation process, specifically for fine-tuning the pretrained model to perform well on a cloudy dataset,
using the technique of pseudo-ground truth generation. Within this notebook, there are incomplete
segments of code related to the loss calculation that you will need to address. Carefully review the code
to ensure that after the application of the confidence threshold, the tensor shapes and pixel positions
remain correct and conform to the PyTorch requirements for cross-entropy loss (you may also choose
to implement a custom loss function if preferred). The fine-tuning should be carried out for 1-5 epochs
with a low learning rate, as indicated in the notebook. To maintain simplicity, you can skip the
validation step during the fine-tuning phase.
c. Performance Evaluation. Once you've completed the fine-tuning using pseudo-ground truth, evaluate
the model's accuracy on the cloudy test images to ascertain any performance improvements. State this
evaluation in your report.

4) Unsupervised domain adaptation using adaptive batch normalization (AdaBN) (10 marks):
Our pretrained UNet model, which includes batch normalization layers and was initially trained using
bright, sunny images, will undergo an adaptation process through Adaptive Batch Normalization
(AdaBN) to better perform on a target domain featuring dark, cloudy images. The adaptation process is
outlined as follows:
⚫ Model Preparation: Adjust the model in the training mode to allow the batch normalization layers
to update their running statistics, and at the same time ensure the pretrained convolution weights
remain untouched.
⚫ Statistics Update: Conduct forward passes with the target domain data, specifically using our
cloudy training dataloader, through the UNet model. This step aims to recalibrate the batch
normalization layers by modifying their running mean and variance to align with the target
domain's data distribution.
⚫ Switch to Inference: Once the batch normalization layers have been updated to reflect the target
domain statistics, transition the model to inference mode.

For more details refer to the paper Li, Y., Wang, N., Shi, J., Liu, J., & Hou, X. (2016). Revisiting batch
normalization for practical domain adaptation. arXiv preprint arXiv:1603.04779.

Complete the following tasks:

a. Implement AdaBN. Implement AdaBN to adapt the pretrained UNet model to the dataset of cloudy
images. Briefly explain how you implement AdaBN in the report.
b. Model Performance. Assess the adapted model's performance by evaluating its accuracy on cloudy
test images. State this evaluation in your report.

Report Guideline (40 Marks)

As part of your assessment for the topic of Street-view Semantic Segmentation with U-Net, you are required
to compose a detailed report that discuss the scope of the project, the methodologies employed, the results
from your experiments, and a critical analysis of these findings. Additionally, your report should
contemplate future research directions that could stem from your work. It is important that you refrain from
using ChatGPT for writing your report. Your report should be structured as follows:

Section Content Weighting

Introduction ⚫ Briefly introduce semantic segmentation and its 10%
significance in computer vision, particularly in street-view
imagery.
⚫ Outline the objectives of the project, including the
challenges of segmenting under varying lighting conditions
and the adaptation of the U-Net model.
Methodology ⚫ Briefly describe the pretrained U-Net model and its initial 20%
capabilities.
 ⚫ Detailed explain the fine-tuning process and unsupervised
domain adaptation techniques, including pseudo-ground
truth generation and adaptive batch normalization
(AdaBN).
⚫ Emphasize any method customization you have performed
(if any)
⚫ Discuss the experimental setup, including any software or
hardware used.
Results ⚫ Present the results (global accuracy and average mIoU 40%
scores) of the pretrained model evaluation on the bright
sunny.
⚫ Explain the differences between global accuracy and
average mIoU metric using image examples.
⚫ Detail the outcomes of the fine-tuning process on the
cloudy dataset, including training/validation loss curves
and any improvements observed.
⚫ Show how entropy served as a model confidence is
related to the model segmentation error, using examples
and interval statistics as instructed.
⚫ Show the effects of unsupervised domain adaptation
techniques (pseudo-ground truth generation and adaptive
batch normalization) on the model's performance.
Discussions ⚫ Discuss the challenge of street-view segmentation, using 25%
failure case images in both bright and cloudy dataset.
⚫ Discuss the effectiveness of model finetuning and domain
adaptation strategies, and any insights or experiences you
have learned.
⚫ Suggest potential avenues for future research based on
your experimental findings and literature understanding.

Conclusions ⚫ Summarize the key points from your report. 5%

and References ⚫ Provide a reference list of at least 3 related literatures

Writing and Submission Guidelines

⚫ Use a clear and concise academic writing style, with technical terms explained for the reader.
⚫ Ensure that the report is typed, spell-checked, and properly referenced.
⚫ Include any relevant images, diagrams, or tables that support your findings (not included in the word
count).
⚫ Submit your report via the designated online platform in PDF format.
⚫ Remember to attach all required components, including Jupyter notebooks, experimental code, and
the trained model.
⚫ In Jupyter notebooks: use notebook cells to structure their work with clear headings and explanations
and include comments within code cells to describe the purpose and functionality of key sections of
code, which will facilitate the grading process.
After submission, download and verify that your file is viewable to ensure a successful submission.

Project Material Access Instructions

To obtain the complete set of materials for our project, including the dataset, code, and Jupyter notebook
files, please use the links provided below:
⚫ (OneDrive Link): https://1drv.ms/u/s!AoRfWDkanfAYnvY7SJT0AoG8TYOtOg?e=iLyLpR
⚫ (Baidu Drive Link): https://pan.baidu.com/s/1lBbYzfwnCMV3di0DSn5EMg?pwd=9876
Download password: 9876
When prompted, use the following password to unlock the zip file: DTS304TC (please note that it is case-
sensitive and should be entered in all capital letters).
Additionally, for ease of reference, the project's Jupyter notebooks have been appended to the end of this
document.
2/9/24, 8:30 AM code_example_train_unet_sunny

In [1]: # example unet model training code for sunny bright image set
# major library dependencies: jupyter, numpy, matplotlib, pytorch, scikit-image,

# import related libraries

from dataset import camvidLoader
import numpy as np
from matplotlib import pyplot as plt
from dataset import from_label_to_rgb
import data_aug as aug
from skimage.io import imsave
import os
from unet import UNet
import torch
from torch.utils.data import Dataset, DataLoader

In [2]: # hyper-parameters
lr = 0.0002
epochs = 30
batch_size = 8
num_workers = 8
# we are training on the sunny camvid dataset.
data_root = './CamVid/sunny'
num_classes = 14 # number of classes is always 14 for this project.
labels = ['Sky', 'Building', 'Pole', 'Road', 'LaneMarking', 'SideWalk', 'Pavemen
'Fence', 'Car_Bus', 'Pedestrian', 'Bicyclist', 'Others']

In [3]: # we first double confirm the dataloader works correctly

# we disable data argumentation and pytorch transformation to check the dataload
img_label_data = camvidLoader(root=data_root, split='train', is_aug=False, img_s
img, label, img_path = img_label_data[0]
# this is to convert from the loaded label map to rgb label map visualization
label_rgb_vis = from_label_to_rgb(label)
plt.figure(0)
# original image visualization
plt.imshow(img)
plt.figure(1)
# rgb label map visualization
# note that we have performed certain label conversion, so the color map not exa
plt.imshow(label_rgb_vis)

Out[3]: <matplotlib.image.AxesImage at 0x237d6dedba0>

localhost:8888/lab/tree/code_example_train_unet_sunny.ipynb 1/7
2/9/24, 8:30 AM code_example_train_unet_sunny

In [4]: # during the training, we perform data argumentation.

# This is to check the data argumented image is also reasonable.
aug_obj = aug.Compose([aug.RandomHorizontalFlip(), aug.RandomResizedCrop(256),
aug.ColorJitter(brightness=0.4, contrast=0.4, saturation=0.4)
img_label_data_aug = camvidLoader(root=data_root, split='train', is_aug=True, im
is_pytorch_transform = False, aug = aug_obj)
for i in range(3):
img, label, img_path = img_label_data_aug[i]
label_rgb_vis = from_label_to_rgb(label) # this is to convert from the loade

localhost:8888/lab/tree/code_example_train_unet_sunny.ipynb 2/7
2/9/24, 8:30 AM code_example_train_unet_sunny

plt.figure(i * 2 + 0)
plt.imshow(img)
plt.figure(i * 2 + 1)
plt.imshow(label_rgb_vis)

localhost:8888/lab/tree/code_example_train_unet_sunny.ipynb 3/7
2/9/24, 8:30 AM code_example_train_unet_sunny

localhost:8888/lab/tree/code_example_train_unet_sunny.ipynb 4/7
2/9/24, 8:30 AM code_example_train_unet_sunny

In [5]: # we set is_pytorch_transform to true, in order to perform network training/test

# we also do data argumentation for training
train_dataset = camvidLoader(root=data_root, split='train', is_aug=True, img_siz
is_pytorch_transform = True, aug = aug_obj)
train_loader = DataLoader(train_dataset, num_workers=num_workers, batch_size=bat
# we do not do data argumentation for validation
val_dataset = camvidLoader(root=data_root, split='val', is_aug=False, img_size =
is_pytorch_transform = True, aug = None)
val_loader = DataLoader(val_dataset, num_workers=num_workers, batch_size=batch_s

localhost:8888/lab/tree/code_example_train_unet_sunny.ipynb 5/7
2/9/24, 8:30 AM code_example_train_unet_sunny

In [6]: # Define our U-Net model, loss function and optimizer

device = 'cpu' # can be either set to cuda or cpu
unet = UNet(3, num_classes, width=32, bilinear=True)
unet = unet.to(device)
loss_func = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(unet.parameters(), lr=lr)

In [ ]: # make the result directory

train_result_dir = '../train_internal_data/'
if not os.path.exists(train_result_dir):
os.makedirs(train_result_dir)

# finally!!! the training loop!!!

for epoch in range(epochs):
# training loop
unet.train()
train_loss = 0
count = 0
for idx_batch, (imagergb, labelmask, filename) in enumerate(train_loader):
# zero the grad of the network before feed-forward
optimizer.zero_grad()

# send to the device (GPU or CPU) and do a forward pass

x = imagergb.to(device)
y_ = labelmask.to(device)
y = unet(x)

# finally calculate the loss and back propagate

loss = loss_func(y, y_)
loss.backward()
optimizer.step()

# every 10 batches, print the loss function

if idx_batch % 10 == 0:
print("train epoch = " + str(epoch) + " | batch = " + str(idx_batch)
train_loss += loss.item()
count += 1
train_loss /=count

# validation loop
unet.eval()
val_loss = 0
count = 0
for idx_batch, (imagergb, labelmask, filename) in enumerate(val_loader):
with torch.no_grad(): # no gradient computation required during validati
x = imagergb.to(device)
y_ = labelmask.to(device)
y = unet(x)
loss = loss_func(y, y_)
val_loss += loss.item()
count += 1

# every 5 batches, we also saved the validation label map

if idx_batch % 5 == 0:
for idx in range(0, y.shape[0]):
max_index = torch.argmax(y[idx], dim=0).cpu().int().numpy()
label_rgb_vis = from_label_to_rgb(max_index)
gt_correct_format = y_[idx].cpu().int().numpy()
gt_vis = from_label_to_rgb(gt_correct_format)

localhost:8888/lab/tree/code_example_train_unet_sunny.ipynb 6/7
2/9/24, 8:30 AM code_example_train_unet_sunny

result_vis = np.concatenate((label_rgb_vis, gt_vis), axis=1)

result_vis_for_saving = (result_vis * 255.0).astype(np.uint8
imsave(train_result_dir + "valdation_seg_result_{}_{}_{}.png
val_loss /=count
print('epoch ' + str(epoch) + ', training loss = ' + str(train_loss) + ', va
model_location = train_result_dir + "model_file_epoch_" + str(epoch) + "_val
torch.save(unet, model_location)

localhost:8888/lab/tree/code_example_train_unet_sunny.ipynb 7/7
2/9/24, 8:30 AM part1

In [1]: # part 1 experimentation: Evaluate the pretrained model on bright sunny test ima
# major library dependencies: jupyter, numpy, matplotlib, pytorch, scikit-image,

import torch
from dataset import camvidLoader
import numpy as np

device = 'cpu' # can be set to "cuda" if you have a GPU

unet = torch.load('camvid_sunny_model.pt', map_location=torch.device(device))
data_root = './CamVid/sunny'
test_data = camvidLoader(root=data_root, split='test', is_aug=False, img_size =

num_classes = 14 # number of classes is always 14 for this project.

labels = ['Sky', 'Building', 'Pole', 'Road', 'LaneMarking', 'SideWalk', 'Pavemen
'Fence', 'Car_Bus', 'Pedestrian', 'Bicyclist', 'Others']

In [2]: # implement your evaluation metric functions here.

def global_accuracy_metric(y_true, y_pred):
pass

def IoU_metric(y_true, y_pred):

pass

In [3]: # write your testing loop in this cell

In [4]: # calculate global image accuracy and per-class accuracy for the entire dataset

In [5]: # implement image ranking code according to the per-image global image accuracy

localhost:8888/lab/tree/part1.ipynb 1/1
2/9/24, 8:31 AM part2

In [1]: # part 2 experimentation: Evaluate and fine-tune the pretrained model on darker
# major library dependencies: jupyter, numpy, matplotlib, pytorch, scikit-image,

import torch
from dataset import camvidLoader
import numpy as np

device = 'cpu' # can be set to "cuda" if you have a GPU

unet = torch.load('camvid_sunny_model.pt', map_location=torch.device(device))

data_root = './CamVid/cloudy'
test_data = camvidLoader(root=data_root, split='test', is_aug=False, img_size =

num_classes = 14 # number of classes is always 14 for this project.

labels = ['Sky', 'Building', 'Pole', 'Road', 'LaneMarking', 'SideWalk', 'Pavemen
'Fence', 'Car_Bus', 'Pedestrian', 'Bicyclist', 'Others']

In [ ]: # Evaluate pretrained Cloudy Dataset Performance

In [ ]: # Implement your model finetuning code

In [ ]: # Plot training/validation loss curves and determine the best fine-tuned model.

In [ ]: # Evaluate Cloudy Dataset Performance after Finetuning

localhost:8888/lab/tree/part2.ipynb 1/1
2/9/24, 8:31 AM part3a

In [1]: # part 3 experimentation: Unsupervised domain adaptation using pseudo-ground tru

# major library dependencies: jupyter, numpy, matplotlib, pytorch, scikit-image,

# Experimentation with Model Confidence

import torch
from dataset import camvidLoader
import numpy as np

device = 'cpu' # can be set to "cuda" if you have a GPU

unet = torch.load('camvid_sunny_model.pt', map_location=torch.device(device))

data_root = './CamVid/cloudy'
test_data = camvidLoader(root=data_root, split='test', is_aug=False, img_size =

num_classes = 14 # number of classes is always 14 for this project.

labels = ['Sky', 'Building', 'Pole', 'Road', 'LaneMarking', 'SideWalk', 'Pavemen
'Fence', 'Car_Bus', 'Pedestrian', 'Bicyclist', 'Others']

In [ ]: # write the function that computes the entropy map for the unet output
def compute_entropy_map(model_output):
pass

In [ ]: # write the function that computes the segmentation error map for the unet outpu
def compute_segmentation_error_map(model_output, gt):
pass

In [ ]: # write a evaluation loop to calculate the entropy map and segmenation error map
# observe the relationship between entropy map and segmenation error map by visu

In [ ]: # flattening the entropy map into a one-dimensional array

# concatenating these arrays from all images to form an overall entropy array
# categorizing the pixels into quartiles based on their entropy levels: top 25%,
# calculate the average segmentation error within these intervals
# to discover the relationship between entropy(model confidence) and segmenation

localhost:8888/lab/tree/part3a.ipynb 1/1
2/9/24, 8:31 AM part3b

In [1]: # part 3 experimentation: Unsupervised domain adaptation using pseudo-ground tru

# major library dependencies: jupyter, numpy, matplotlib, pytorch, scikit-image,

# Fine-tuning with Pseudo-ground Truth

import torch
from dataset import camvidLoader
from torch.utils.data import Dataset, DataLoader
import numpy as np
import data_aug as aug
from copy import deepcopy
import torch.nn.functional as F

device = 'cpu' # can be set to "cuda" if you have a GPU

unet = torch.load('camvid_sunny_model.pt', map_location=torch.device(device))

data_root = './CamVid/cloudy'
test_data = camvidLoader(root=data_root, split='test', is_aug=False, img_size =

num_classes = 14 # number of classes is always 14 for this project.

labels = ['Sky', 'Building', 'Pole', 'Road', 'LaneMarking', 'SideWalk', 'Pavemen
'Fence', 'Car_Bus', 'Pedestrian', 'Bicyclist', 'Others']

In [2]: batch_size = 4
num_workers = 8
# we set is_pytorch_transform to true, in order to perform network training/test
train_dataset = camvidLoader(root=data_root, split='train', is_aug=False, img_si
is_pytorch_transform = True, aug = None)
train_loader = DataLoader(train_dataset, num_workers=num_workers, batch_size=bat

In [ ]: # This is the code of model fine-tuning on the cloudy dataset with pseudo-ground
# Note that loss calculation part of the code is imcomplete.
lr = 5e-6
epochs = 1
loss_func = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(unet.parameters(), lr=lr)

# finally!!! the training loop!!!

unet.train()
for epoch in range(epochs):
# training loop
train_loss = 0
count = 0
for idx_batch, (imagergb, _, filename) in enumerate(train_loader):
# note that we do not utilize the groundtruth labels from the training d

# zero the grad of the network before feed-forward

optimizer.zero_grad()

# send to the device (GPU or CPU) and do a forward pass

x = imagergb.to(device)
y = unet(x)

# calculate entropy map here.

entropy = your_entropy_calculation_function(y)
confidence = -entropy # confidence is the negative entropy
pseudo_labels = torch.argmax(y, dim=1)

# loss calculation part of the code is imcomplete.

localhost:8888/lab/tree/part3b.ipynb 1/2
2/9/24, 8:31 AM part3b

# y_high_conf = your_code(y, confidence)

# pseudo_labels_high_conf = your_code(pseudo_labels, confidence)

# finally calculate the loss and back propagate

loss = loss_func(y_high_conf, pseudo_labels_high_conf)
loss.backward()
optimizer.step()

if idx_batch % 2 == 0:
print("train epoch = " + str(epoch) + " | batch = " + str(idx_batch)

In [ ]: # Evaluate Cloudy Dataset Performance after domain adaptation

localhost:8888/lab/tree/part3b.ipynb 2/2
2/9/24, 8:31 AM Q4

In [1]: # part 4 experimentation: Unsupervised domain adaptation using adaptive batch no

# major library dependencies: jupyter, numpy, matplotlib, pytorch, scikit-image,

# Fine-tuning with Pseudo-ground Truth

import torch
from dataset import camvidLoader
from torch.utils.data import Dataset, DataLoader
import numpy as np
import data_aug as aug
import torch.nn.functional as F

device = 'cpu' # can be set to "cuda" if you have a GPU

unet = torch.load('camvid_sunny_model.pt', map_location=torch.device(device))

data_root = './CamVid/cloudy'
test_data = camvidLoader(root=data_root, split='test', is_aug=False, img_size =

num_classes = 14 # number of classes is always 14 for this project.

labels = ['Sky', 'Building', 'Pole', 'Road', 'LaneMarking', 'SideWalk', 'Pavemen
'Fence', 'Car_Bus', 'Pedestrian', 'Bicyclist', 'Others']

In [ ]: # Implement your model finetuning using adaptive batch normalization

In [ ]: # Evaluate Cloudy Dataset Performance after domain adaptation

localhost:8888/lab/tree/Q4.ipynb 1/1