Scribd
Upload
64 views18 pages

Lec18 Denoising+SparseAutoencoders

This document discusses denoising and sparse autoencoders. It begins with an overview of autoencoders, denoising autoencoders, and sparse autoencoders. It then provides examples of applying denoising autoencoders to handwritten digit recognition and optical coherence tomography tissue characterization challenges. The document describes the challenges, network architectures used, interpretation of learned representations, and performance results.

Uploaded by

rakesh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
64 views18 pages

Lec18 Denoising+SparseAutoencoders

This document discusses denoising and sparse autoencoders. It begins with an overview of autoencoders, denoising autoencoders, and sparse autoencoders. It then provides examples of applying denoising autoencoders to handwritten digit recognition and optical coherence tomography tissue characterization challenges. The document describes the challenges, network architectures used, interpretation of learned representations, and performance results.

Uploaded by

rakesh
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

NPTEL ONLINE

CERTIFICATION COURSES

Denoising and Sparse Autoencoders


Dr. Debdoot Sheet
Assistant Professor, Department of Electrical Engineering
Principal Investigator, Kharagpur Learning, Imaging and Visualization Group
Indian Institute of Technology Kharagpur

www.facweb.iitkgp.ernet.in/~debdoot/
NPTEL ONLINE
CERTIFICATION COURSES

Organization
• Autoencoder
• Denoising AE
• Sparse AE
• Examples

Denoising and Sparse Autoencoders / Debdoot Sheet 2


NPTEL ONLINE
CERTIFICATION COURSES

Autoencoder

Denoising and Sparse Autoencoders / Debdoot Sheet 3


NPTEL ONLINE
CERTIFICATION COURSES

Denoising Autoencoder

Denoising and Sparse Autoencoders / Debdoot Sheet 4


NPTEL ONLINE
CERTIFICATION COURSES

Sparsity in Autoencoder

Denoising and Sparse Autoencoders / Debdoot Sheet 5


NPTEL ONLINE
CERTIFICATION COURSES

Deep Neural Network

Denoising Auto Encoder

Denoising Auto Encoder

Logistic Reg.
Denoising and Sparse Autoencoders / Debdoot Sheet 6
NPTEL ONLINE
CERTIFICATION COURSES

Weight Refinement

Denoising and Sparse Autoencoders / Debdoot Sheet 7


NPTEL ONLINE
CERTIFICATION COURSES

Vincent, Pascal, et al. "Stacked denoising autoencoders: Learning useful representations in


a deep network with a local denoising criterion." The Journal of Machine Learning
Research 11 (2010): 3371-3408.

HANDWRITTEN DIGIT RECOGNITION

Denoising and Sparse Autoencoders / Debdoot Sheet 8


NPTEL ONLINE
CERTIFICATION COURSES

About the Challenge


• MNIST
– Handwritten digit
recognition challenge
• Train set
– 60,000 samples
• Test set
– 10,000 samples
• Patch size
– 28 x 28 pixels
– uint8 gray valued
– Binary images
Denoising and Sparse Autoencoders / Debdoot Sheet 9
NPTEL ONLINE
CERTIFICATION COURSES

Interpreting Weights

0% Noise added 25% Noise added


Denoising and Sparse Autoencoders / Debdoot Sheet 10
NPTEL ONLINE
CERTIFICATION COURSES

Dynamics of Weights with Noise

Neuron A

Neuron B

0% 10% 20% 50%


Denoising and Sparse Autoencoders / Debdoot Sheet 11
NPTEL ONLINE
CERTIFICATION COURSES

Appearance Model Variations

Denoising and Sparse Autoencoders / Debdoot Sheet 12


NPTEL ONLINE
CERTIFICATION COURSES

Performance Metrics

Denoising and Sparse Autoencoders / Debdoot Sheet 13


NPTEL ONLINE
CERTIFICATION COURSES

Sheet, Debdoot, et al. "Deep learning of tissue specific speckle representations in optical
coherence tomography and deeper exploration for in situ histology.“ IEEE 12th
International Symposium on Biomedical Imaging (ISBI), 2015.

OCT TISSUE CHARACTERIZATION

Denoising and Sparse Autoencoders / Debdoot Sheet 14


NPTEL ONLINE
CERTIFICATION COURSES

About the Challenge


• Data Collection • DNN architecture
– School of Medical Science – Patch size – 36 × 36 px
and Technology, Indian – DAE1 – 400 nodes
Institute of Technology – DAE2 – 100 nodes
Kharagpur
– 1300 nm (HPBW 100 nm) – Target – Logistic Reg.
Swept Source OCT System • 5 outputs
• OCS 1300 SS, ThorLabs, NJ, – Sparsity – 20%
USA – Mini-batch training
• 8 bit bitmap images
• In situ Histology Performance
– Histology for ground truth
– Epithelium – 96%
• HE stained
– Papillary dermis – 93%
• Samples
– Dermis – 99%
– Mus musculus (small mice)
– Adipose tissue – 98%
– 16 healthy skin
– 2 wounds on skin

Denoising and Sparse Autoencoders / Debdoot Sheet 15


NPTEL ONLINE
CERTIFICATION COURSES

Unfurling the Deep Network

Denoising and Sparse Autoencoders / Debdoot Sheet 16


NPTEL ONLINE
CERTIFICATION COURSES

Learning of Representations

Sparsity
Representation
of representations
of speckle
learned by
appearance modelsDAE2learned by DAE1

Denoising and Sparse Autoencoders / Debdoot Sheet 17


NPTEL ONLINE
CERTIFICATION COURSES

Take Home Messages


• Palm, Rasmus Berg, Prediction as a candidate for learning
deep hierarchical models of data, MS Thesis, TU Denmark,
2012.
• Vincent, Pascal, et al. "Stacked denoising autoencoders:
Learning useful representations in a deep network with a
local denoising criterion." The Journal of Machine Learning
Research, 11 (2010): 3371-3408.

Denoising and Sparse Autoencoders / Debdoot Sheet 18

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy