ISSCC 2023 Tutorial #6

Solid-State CMOS LiDAR Sensors

Seong-Jin Kim
Ulsan National Institute of Science and Technology
kimsj@unist.ac.kr

Feb 19, 2023

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 1 of 95

© 2023 IEEE International Solid-State Circuits Conference

Self Introduction
 Education
 Ph.D. degree from KAIST, Korea, in 2008
 Work Experiences
 Samsung Advanced Institute of
Technology, Korea, 2008-2012
 Institute of Microelectronics, A*STAR,
Singapore, 2012-2015
 Associate Professor, UNIST, Korea, 2015-
Present
 Co-founder at SolidVue, Korea, in 2020
 Research Interests
 CMOS image sensors and LiDAR systems
 Brain and body interface circuits
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 2 of 95

© 2023 IEEE International Solid-State Circuits Conference

Outline
 Introduction to LiDAR sensors

 Indirect Time-of-Fight (iToF) Technique

 Direct Time-of-Fight (dToF) Technique

 Fundamental Principles
 Single-Photon Avalanche Diodes
 Time-to-Digital Converters (TDC)
 On-chip Histogramming TDC

 Summary

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 3 of 95

© 2023 IEEE International Solid-State Circuits Conference

Cameras in Our Life

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 4 of 95

© 2023 IEEE International Solid-State Circuits Conference

CIS Market

6.1B units in 2019  11B units in 2024!

A. Theuwissen, ISSCC 2021
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 5 of 95

© 2023 IEEE International Solid-State Circuits Conference

Emerging Applications: Ranging

Light Detection And Ranging (LiDAR)

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 6 of 95

© 2023 IEEE International Solid-State Circuits Conference

LiDAR Market Forecast

Ref: Yole
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 7 of 95

© 2023 IEEE International Solid-State Circuits Conference

LiDAR Performances
Depth Range Depth Error

×
Dmin ×× ×
Dmax ×
×× ××
×
×
×
××
×
A few meters to Precise Precise Not precise
a few kilometers Accurate Not accurate Not accurate

Spatial Resolution Frame Rate

Wide FoV
High resolution

Narrow FoV Motion

Low resolution
Distance
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 8 of 95

© 2023 IEEE International Solid-State Circuits Conference

Time-of-Flight for LiDAR

IR Emitter

Controller
R

Object
Detector Depth map
 To measure the round-trip time of emitted light for acquiring
the distance
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 9 of 95

© 2023 IEEE International Solid-State Circuits Conference

Indirect Measurement
f
IR Emitter

Controller
Object

Δφ Sensor array
c ∆ϕ (Demodulation pixel)
D=
2 2πf
Detecting phase difference
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 10 of 95

© 2023 IEEE International Solid-State Circuits Conference

 Phase Delay Estimation
Emitted Light …
TTOF
Reflected Light …
TX0/2 TX1/3
TX0
…
TX1 …
TX2
…
TX3 …
 π
Simplified pixel w/
N 0 = niir ∫ (a sin (θ − TTOF ) + a )dθ = 2niir a cos TTOF + 
π
N − N3 sin TTOF
0
 2 ⇒ 2 = = tanTTOF two taps
N 0 − N 1 cos TTOF
 π
N 1 = niir ∫ (a sin (θ − TTOF ) + a )dθ = 2niir a − cos TTOF + 
2π

2  N − N3 
 ⇒ TTOF = arctan 2 
π


3π
 π
N 2 = niir ∫π 2 (a sin (θ − TTOF ) + a )dθ = 2niir a sin TTOF + 
 N 0 − N1 
 2  N − N3 
2
c c
π
 π ∴R = TTOF = arctan 2 

N 3 = niir ∫ 2π (a sin (θ − TTOF ) + a )dθ = 2niir a − sin TTOF +  4πf 4πf  N 0 − N1 
−
2  2

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 11 of 95

© 2023 IEEE International Solid-State Circuits Conference

Direct Measurement
Time-to-Digital converter
∆t
IR Emitter

Controller
Δt
Object
c
D = ∆t Sensor array
2 (Sensitive diode)
Detecting direct time difference
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 12 of 95

© 2023 IEEE International Solid-State Circuits Conference

Direct ToF (dToF)

 ToF is directly measured like a basic principle.

 Highly sensitive detectors are required to know the
time when photons arrive.
 Large avalanche photodiodes (APD) are traditionally
employed.
 Complex front-end electronics to handle APDs are also
necessary.
 It is suitable for a single-point range finder and 3D
scanner with a line sensor.

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 13 of 95

© 2023 IEEE International Solid-State Circuits Conference

Block Diagram of dToF Sensor

High-speed ADC

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 14 of 95

© 2023 IEEE International Solid-State Circuits Conference

Scanner-type Initial LiDAR

 Velodyne HDL-64e
 64 APDs aligned with 64
lasers
 1.3 Mpoints/sec
 360° horizontal FoV
 28.6° vertical FoV
 120m distance
 Up to 900rpm (15Hz)
 Class 1 eye safety

B. Schwartz, Nat. Photonics 2010

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 15 of 95

© 2023 IEEE International Solid-State Circuits Conference

Demonstration of Initial LiDAR

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 16 of 95

© 2023 IEEE International Solid-State Circuits Conference

Commercial Product
20.3 cm
28.3 cm

64-channel, 120 m range LiDAR sensor

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 17 of 95

© 2023 IEEE International Solid-State Circuits Conference

3D Point Cloud from LiDAR

https://www.youtube.com/watch?v=KxWrWPpSE8I
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 18 of 95

© 2023 IEEE International Solid-State Circuits Conference

Miniaturization of LiDAR

 Issues of LiDARs w/
discrete components
 Limited productivity
Discrete components
(high cost)
 Low durability
 Low scalability
 Low spatial
resolution

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 19 of 95

© 2023 IEEE International Solid-State Circuits Conference

Miniaturization of LiDAR

Discrete components Sony

SOSLAB

Small-size Low-cost Solid-State LiDAR has emerged!

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 20 of 95

© 2023 IEEE International Solid-State Circuits Conference

Single Photon Avalanche Diode (SPAD)
Pixel
Reflected IR Output
Rq Vop

p+ n+
p- p-
well well
Avalanche n-well

 Geiger mode operation (virtually infinite gain)

 Possible to detect a single photon

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 21 of 95

© 2023 IEEE International Solid-State Circuits Conference

SPAD Bias Point

+
I
p
V
n
-

 Biasing above
breakdown
 Gain variation is
meaningless

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 22 of 95

© 2023 IEEE International Solid-State Circuits Conference

SPAD Operation

1. Steady-state at Geiger region VSPAD – ID

2. Avalanche by photon I APD Geiger
3. Quenching PVDD
4. Recharging to 1
VSPAD

Vout

Rq

VSPAD
VBD VBD+VE
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 23 of 95

© 2023 IEEE International Solid-State Circuits Conference

SPAD Operation

1. Steady-state at Geiger region VSPAD – ID

2. Avalanche by photon I APD Geiger
3. Quenching
4. Recharging to 1
PVDD Vout

ID Rq CD
VSPAD Avalanche
Vout

Rq
VD = IDRq Photon

Vout = VD(1-exp(-t/RqCD)) VSPAD

VBD VBD+VE
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 24 of 95

© 2023 IEEE International Solid-State Circuits Conference

SPAD Operation

1. Steady-state at Geiger region VSPAD – ID

2. Avalanche by photon I APD Geiger
3. Quenching
4. Recharging to 1
PVDD Vout

Quench
0 Rq CD
VSPAD Avalanche
Vout

Rq
VD = 0 Photon

Vout ~ VE VSPAD
VBD VBD+VE
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 25 of 95

© 2023 IEEE International Solid-State Circuits Conference

SPAD Operation

1. Steady-state at Geiger region VSPAD – ID

2. Avalanche by photon I APD Geiger
3. Quenching
4. Recharging to 1
PVDD Vout

Quench
Rq CD
VSPAD Avalanche
Vout

Rq
VD = 0 Recharge Photon

Vout = VEexp(-t/RqCD) VSPAD

VBD VBD+VE
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 26 of 95

© 2023 IEEE International Solid-State Circuits Conference

Simple pn Junction for SPAD?

Cathode Anode Anode Cathode

n+ p+ n+
Vbd1 = 25 V
n-well

Vbd2 = 20 V

High E-field
p-sub

 High electric field is at the edges of the junction.

 Premature breakdown at the periphery.
 Guardring structure with circular shape is needed.

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 27 of 95

© 2023 IEEE International Solid-State Circuits Conference

Desirable Active Region
Small active
region!

Active region

Premature breakdown

 Guardring prevents SPAD from premature breakdown at

the periphery and makes whole area as an active region.
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 28 of 95

© 2023 IEEE International Solid-State Circuits Conference

Guardring Structures

Cathode Anode Anode Cathode

n+ p+ n+
p-well Vbd1 = 25 V p-well
V = 30 V
deep n-well bd2

p-sub

 Low-doping layer is inserted at the periphery to remove abrupt

pn junction.
 It makes the breakdown voltage at the periphery higher than
active region.
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 29 of 95

© 2023 IEEE International Solid-State Circuits Conference

Guardring Structures
Cathode Anode Anode Cathode

n+ p+ n+
n-well
deep n-well

p-sub
Cathode Anode Anode Cathode

n+ p+ n+
STI STI
n-well

p-sub

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 30 of 95

© 2023 IEEE International Solid-State Circuits Conference

Today’s Typical SPAD for LiDAR

Cathode Anode Anode Cathode

n+ p+ n+
STI STI STI STI
p-well
deep retrograde n-well

p-sub

 Suitable for deep-submicron technologies

 Compatible with any triple-well process
 Excellent performance in dark environment
J. Richardson, PTL 2009
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 31 of 95

© 2023 IEEE International Solid-State Circuits Conference

TCAD Simulation

SPAD structure

Multiplication region

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 32 of 95

© 2023 IEEE International Solid-State Circuits Conference

SPAD Pixel Evolution

Diffuser

Time
M.-J. Lee, JSTQE 2018 Technology
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 33 of 95

© 2023 IEEE International Solid-State Circuits Conference

Photon Detection Probability (PDP)
FSI SPAD BSI SPAD

C. Veerappan, TED 2016 E. Charbon, ISSW 2020

PDP = (# of SPAD pulses - # of DCR) / # of photons

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 34 of 95

© 2023 IEEE International Solid-State Circuits Conference

Sony’s State-of-the-Art SPAD

• 40nm CMOS
• Dead time ~5ns
• DCR (Dark count rate) < 2kcps@60°C
• PDE (PDP × Fill Factor) ~22%@905nm
O. Kumagai, ISSCC 2021
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 35 of 95

© 2023 IEEE International Solid-State Circuits Conference

Even Smaller Pitch

OCL: On-Chip Lens

PSD: Pyramid Surface
for Diffraction
FTI: Full Trench
Isolation

S. Shimada, IEDM 2021

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 36 of 95

© 2023 IEEE International Solid-State Circuits Conference

Performances

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 37 of 95

© 2023 IEEE International Solid-State Circuits Conference

SPAD High Level Behavior
Photons Response
Deadtime
P1P2 P3 Missed
SPAD-
based
Pixel

t DCR t
Detected!
 Detected photons generate digital pulses.
 Not all the photons are detected.
 Pulses can also be generated in the dark.
 Jitter noise should be added.
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 38 of 95

© 2023 IEEE International Solid-State Circuits Conference

Time-Correlated Single Photon Counting

 Repeat and histogram

SPAD pulses.
 Finally, a peak value is
detected as the ToF.

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 39 of 95

© 2023 IEEE International Solid-State Circuits Conference

 Block Diagram of SPAD-Based LiDAR
SPAD Front-End Time-to-Digital Converter Histogram
Photon VSPAD
ToF = 0011(2)

Counts
SPAD Clock freq. x N M-bit Counter
SPAD
Pulse ToF x 2N
QCH start time

t
Detecting photons Converting arrival time of Extracting the peak
SPAD pulse to digital bit value (ToF)

Distance
𝒄𝒄
(𝑫𝑫𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊𝒊 = 𝜟𝜟𝜟𝜟)
𝟐𝟐
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 40 of 95

© 2023 IEEE International Solid-State Circuits Conference

Time-to-Digital Converter

 Velocity of light = 3 × 108 m/s

 1 ns time = 15 cm depth
 15 GHz clock is necessary for
1 cm resolution!
 Coarse-fine conversion is a
popular solution.
 Clk counting followed by sub-
phase

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 41 of 95

© 2023 IEEE International Solid-State Circuits Conference

Column-level TDC Architecture
Coarse 2b Medium Fine 4b by
by 4b by DLL Delay line
Counter w/ 16 w/
w/ 40MHz phases registers
(25ns) (1.56ns) (97.6ps)

10b TDC

100ns range
97.6ps resolution

Large size
No histogram circuit
C. Niclass, ISSCC 2008
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 42 of 95

© 2023 IEEE International Solid-State Circuits Conference

Pixel-level TDC: GRO

3-bit
Thermometer
Gated Ring Coder
Start
Oscillator
Stop (8-stages) 7-bit Ripple
SPAD Counter

440ps from coarse counter

55ps from fine phase coder
Small size
No histogram circuit C. Veerappan, ISSCC 2011
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 43 of 95

© 2023 IEEE International Solid-State Circuits Conference

On-Chip Histogram DSP

1×16 ToF w/ 24 SiPM per pixel

64 12-b TDC and 768 kb SRAM (2048 bins)
Integrated histogram circuit
Large size DSP and SRAM C. Niclass, JSSC 2014
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 44 of 95

© 2023 IEEE International Solid-State Circuits Conference

1-D Line LiDAR
Polygonal mirror 200×96 pixel (55°×9° FoV)
10 fps, 100-m distance

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 45 of 95

© 2023 IEEE International Solid-State Circuits Conference

Background Light Suppression

 Multiple SPADs’
events in a
single pixel is
only counted
for histogram.
 It extracts
spatial
temporal
correlation of
photons!

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 46 of 95

© 2023 IEEE International Solid-State Circuits Conference

Coincidence Detection Circuit

 Find out two

simultaneous
events.
 Carry or two
sums make
output high.

C. Niclass, JSSC 2013

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 47 of 95

© 2023 IEEE International Solid-State Circuits Conference

OR Tree Compression & Counting

 Shorten pulse
width and
serialize pulses.
 Count pulses for
background
suppression.

L. Braga, ISSCC 2013

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 48 of 95

© 2023 IEEE International Solid-State Circuits Conference

XOR Tree Edge Combining

 Odd numbers of
pulses  High
 Even numbers
of pulses  Low
 Count edges for
background
suppression.

N. Dutton, ISSCC 2015

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 49 of 95

© 2023 IEEE International Solid-State Circuits Conference

Data Rate of Color CIS
1st frame 2nd frame nth frame
 Assumption
 Resolution: 160×120
 Frame rate: 30
 ADC: 10b
 Synchronous readout

 Data rate
=160×120×30fps×10-b
=5.76Mbps

33ms

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 50 of 95

© 2023 IEEE International Solid-State Circuits Conference

Data Rate of Line dToF
1st line 2nd line nth line

 Assumption
 Resolution: 160×120
 Frame rate: 30fps
 TDC: 10b
 Repetition: 1MHz
 Synchronous readout

 Data rate
=160×30fps×10-b×1M
=48Gbps
1st frame
33ms
1µs
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 51 of 95

© 2023 IEEE International Solid-State Circuits Conference

Data Rate of Flash dToF
1st frame 2nd frame nth frame
 Assumption
 Resolution: 160×120
 Frame rate: 30fps
 TDC: 10b
 Repetition: 1MHz
 Synchronous readout

 Data rate
=160×120×30fps×10b×
1M
=5.76Tbps!!
33ms
1µs
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 52 of 95

© 2023 IEEE International Solid-State Circuits Conference

On-Chip Histogram

 Each time bin has

a corresponding
counter.
 TDC and
histogramming
are done at the
same time.
 Multi-echo can be
recorded.

N. Dutton, ISSCC 2015

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 53 of 95

© 2023 IEEE International Solid-State Circuits Conference

Flash Histogramming TDC

 1024 SPAD to single TDC channel

 264 counters (Memories)
 71ps resolution w/ 18.8ns
dynamic range (8b)
 Conversion rate = 11.7Gs

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 54 of 95

© 2023 IEEE International Solid-State Circuits Conference

In-pixel Histogramming TDC

[R. Henderson, ISSCC 2019]

3D stacking process
• Front wafer: SPAD array
• Back wafer: Logic circuits for TDC and histogramming
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 55 of 95

© 2023 IEEE International Solid-State Circuits Conference

3D Stacked LiDAR

 Tier 1 wafer: SPAD

 Tier 2 wafer: TDC and
Memory
 128 SPADs share a
TDC.
 256 × 256 pixel array
 Up to 430m range

A. Ximenes, ISSCC 2018

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 56 of 95

© 2023 IEEE International Solid-State Circuits Conference

Shared TDC and Histogramming

 A shared single TDC w/ histogramming memory for 8 x 16 SPAD array

 Employing asynchronous readout to utilize the shared TDC effectively

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 57 of 95

© 2023 IEEE International Solid-State Circuits Conference

Interference Suppression
Phase shift keying
(Pulse position modulation)

8-PSK example:
1. Applying -6, -4, -2, 0, 2, 4, 6, 8ns delays
2. Subtracting the same delays in TDC

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 58 of 95

© 2023 IEEE International Solid-State Circuits Conference

Interference Filtering

Conceptual operation

H. Seo, VLSI 2020

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 59 of 95

© 2023 IEEE International Solid-State Circuits Conference

In-pixel Multi-event Histogramming
Spatiotemporal
correlation

4-b time bin

Multi-
event TDC
R. Henderson, ISSCC 2019
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 60 of 95

© 2023 IEEE International Solid-State Circuits Conference

TDC Architecture and Operation

Detect four
SPADs’ responses

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 61 of 95

© 2023 IEEE International Solid-State Circuits Conference

Multi-event Clocked Flash TDC

TDC
SPAD PhotonEdges
+ + + + + + + +
STOPd
Count

TDCClk

TDC<3> TDC<2> TDC<1> TDC<0> t (ns)

0 1 0 1

CNT1 CNT2 CNT3 CNT4

t (ns)
ToF Histogram

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 62 of 95

© 2023 IEEE International Solid-State Circuits Conference

 Multi-event Clocked Flash TDC
TDC<0> TDC<1> TDC<15>

STOPd
PhotonEdges
Rst Rst Rst

Time info D Q D Q D Q
STOPd

TDCClk

TDC<3> TDC<2> TDC<1> TDC<0> t (ns)

PhotonEdges
D Q D Q D Q D Q
0 1 0 1
Rst Rst Rst Rst

Start
TDCClk
CNT1 CNT2 CNT3 CNT4

Histogram

Multiple bins are updated simultaneously!

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 63 of 95

© 2023 IEEE International Solid-State Circuits Conference

 Multi-event 4-bin TDC

TDC<0> TDC<1> TDC<2> TDC<3> TDC<0> TDC<1> TDC<2> TDC<3>

0 0 0 0 0 0 0 0
STOPd STOPd
Rst Rst Rst Rst Rst Rst Rst Rst

D Q D Q D Q D Q D Q D Q D Q D Q
0 0 0 0 1 0 0 0
1st event
0 0 0 0 0 1 0 0 0 0
PhotonEdges PhotonEdges
D Q D Q D Q D Q D Q D Q D Q D Q D Q D Q

Rst Rst Rst Rst Rst Rst Rst Rst Rst Rst
Clk edge
Start Start
TDCClk TDCClk

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 64 of 95

© 2023 IEEE International Solid-State Circuits Conference

 Multi-event 4-bin TDC

TDC<0> TDC<1> TDC<2> TDC<3> TDC<0> TDC<1> TDC<2> TDC<3>

0 0 0 0 0 0 0 0
STOPd STOPd
Rst Rst Rst Rst Rst Rst Rst Rst

D Q D Q D Q D Q D Q D Q D Q D Q
0 1 0 0 1 0 1 0
2nd event
1 1 0 0 0 0 1 1 0 0
PhotonEdges PhotonEdges
D Q D Q D Q D Q D Q D Q D Q D Q D Q D Q

Rst Rst Rst Rst Rst Rst Rst Rst Rst Rst
Clk edge Clk edge
Start Start
TDCClk TDCClk

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 65 of 95

© 2023 IEEE International Solid-State Circuits Conference

 Two-Step Histogram
1st step: course range

photons

time
photons

2nd step: fine range

S. Hutchings, JSSC 2019 time

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 66 of 95

© 2023 IEEE International Solid-State Circuits Conference

 Histogram Scanning
1st scan
photons

2nd scan time

photons

3rd scan time

photons

time D. Stoppa, IISW 2021

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 67 of 95

© 2023 IEEE International Solid-State Circuits Conference

SAR hTDC Architecture

Conventional TDC Proposed TDC

3-b 3-b

Counters: 2N Counters: 1
8 for 3-b 1 for n-b

Histogram Histogram
B. Kim, ISSCC 2021
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 68 of 95

© 2023 IEEE International Solid-State Circuits Conference

Operating Principles [1/4]
IR 5.34
Start ToF=000
DN[2] UP[2] DN<UP  ToF=100
Zoom TDC
Coarse

d-ToF 0 1 2 3 4 5 6 7

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 69 of 95

© 2023 IEEE International Solid-State Circuits Conference

Operating Principles [2/4]
IR 5.34
Start ToF=000
DN[2] UP[2] DN<UP  ToF=100
Zoom TDC
Coarse

DN[1] UP[1] DN>UP  ToF=100

d-ToF 0 1 2 3 4 5 6 7

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 70 of 95

© 2023 IEEE International Solid-State Circuits Conference

Operating Principles [3/4]
IR 5.34
Start ToF=000
DN[2] UP[2] DN<UP  ToF=100
Zoom TDC
Coarse

DN[1] UP[1] DN>UP  ToF=100

DN[0] UP[0] DN>UP  ToF=101

d-ToF 0 1 2 3 4 5 6 7 Coarse ToF = 5

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 71 of 95

© 2023 IEEE International Solid-State Circuits Conference

Operating Principles [4/4]
IR 5.34
Start ToF=000
DN[2] UP[2] DN<UP  ToF=100
Zoom TDC
Coarse

DN[1] UP[1] DN>UP  ToF=100

DN[0] UP[0] DN>UP  ToF=101

d-ToF 0 1 2 3 4 5 6 7 Coarse ToF = 5

In-ph=101100010
Zoom TDC

Out-ph=010110101
Fine

In Phase Out Phase

CNTPhase180 Fine ToF = 0.34

iToF = TON ×
CNTPhase0 + CNTPhase180
ToF = 5.34
Time
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 72 of 95

© 2023 IEEE International Solid-State Circuits Conference

Two-Step Zoom hTDC
IR 5.34
DN[2] UP[2]
Zoom TDC
Coarse

DN[1] UP[1]

DN[0] UP[0] SAR coarse TDC

d-ToF 0 1 2 3 4 5 6 7
+
Zoom TDC

iToF fine TDC

Fine

In Phase Out Phase

CNTPhase180
iToF = TON ×
CNTPhase0 + CNTPhase180

Time B. Kim, ISSCC 2021

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 73 of 95

© 2023 IEEE International Solid-State Circuits Conference

Background Light Suppression

 Common background light is

cancelled by the up-down
counting.
 Only shot noise remains, which
is a sqrt(# of elec).
 The time duration is shrunk by
half in each step.
 Time-gating is inherently
applied to the fine hTDC.

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 74 of 95

© 2023 IEEE International Solid-State Circuits Conference

Chip Photograph

Process 110-nm CIS CMOS

Pixel Array 48 × 40
Pixel Size 110 × 80 µm2
PWR
840 mW
Consumption
Chip Size 5900 × 5240 µm2
ZH-TDC 4000 µm2
TDC Depth Coarse 7-b / Fine 6-b

TDC resolution Coarse 5 ns / Fine 90 ps

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 75 of 95

© 2023 IEEE International Solid-State Circuits Conference

Depth Image
Target objects

Zoomed short range Long range (0~9m)

(0~0.75cm)

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 76 of 95

© 2023 IEEE International Solid-State Circuits Conference

Issues in Zoom SA-hTDC

 Synthesis of 9
sub-frames: low
frame rate,
motion artifact
 Low signal to
background
ratio (SBR)
 Slow SBR
improvement

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 77 of 95

© 2023 IEEE International Solid-State Circuits Conference

Quaternary Search hTDC

 The whole range is divided by four.

 Both counters are compared to determine ToF[MSB-1].
 A sign of the winning counter determines ToF[MSB].
S. Park, ISSCC 2022
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 78 of 95

© 2023 IEEE International Solid-State Circuits Conference

Δ-Intensity Phase Detection

 Four phase data is utilized to generate the fine ToF.

 The bin width is 2.5ns, which is 32 times smaller than the first
bin width, achieving high SBR.
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 79 of 95

© 2023 IEEE International Solid-State Circuits Conference

Δ-Intensity Phase Detection

ToFFine
cTLSB −1
CNTB − CNTA
= � tan
2π CNTB + CNTA
Improving background light immunity!
Reducing motion artifact!
Achieving a high frame rate!
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 80 of 95

© 2023 IEEE International Solid-State Circuits Conference

Pixel Architecture

 6 SPADs w/ AFE &

masking mem
 Coincidence
detection circuit
 Two 9-b UDC for
quaternary search
 Timing generator
for time bin
management
 Clock repeater
shared by four
neighbor pixels
S. Park, JSSC 2022
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 81 of 95

© 2023 IEEE International Solid-State Circuits Conference

Fabricated Prototype

 110-nm BSI
process
 75-µm pixel
pitch
 6 SPADs per
pixel

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 82 of 95

© 2023 IEEE International Solid-State Circuits Conference

Measurement Results

Coarse TDC
Success rate: Matched distance / 100 trials

Fine TDC
(3m ~ 4.5m)

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 83 of 95

© 2023 IEEE International Solid-State Circuits Conference

Depth Image in a Short Range

Captured in 30fps w/ 30klux background light

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 84 of 95

© 2023 IEEE International Solid-State Circuits Conference

Depth Image in a Long Range

Captured in 1.5fps at night

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 85 of 95

© 2023 IEEE International Solid-State Circuits Conference

Demo Video

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 86 of 95

© 2023 IEEE International Solid-State Circuits Conference

iToF vs dToF
Specification iToF dToF
Maximum range Few meter Few kilometers
Spatial resolution ~ Mpixel < QVGA
Depending on shot Depending on TDC
Depth precision
noise resolution
Several options
Common-mode Coincidence detection /
Background resilience
rejection circuit Common-mode rejection /
Time gating
Multi-event detection Difficult to achieve Possible by histogramming
Coded pulse /
Interference Phase shift keying /
Pseudo-random
cancellation Interference filtering
modulation

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 87 of 95

© 2023 IEEE International Solid-State Circuits Conference

Line vs Flash LiDAR
1-D Line 2-D Flash
TDC Column-parallel Pixel-parallel
Emitter Beamforming for focusing Beam spreading
Range Long Short
Background
Good Bad
resilience
Frame rate Slow Fast
Spatial
High Low
resolution
Application Outdoor, self-driving cars Indoor, mobile devices
Scanner Motor MEMS Solid-State N/A
Cost/Size High Medium Low Low

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 88 of 95

© 2023 IEEE International Solid-State Circuits Conference

Summary
 Basic principles, building blocks, and architectures of SPAD-
based CMOS LiDAR sensors have been presented.
 The direct ToF technique is the best candidate for LiDAR to
provide a long detectable range with strong background noise
resilience.
 No winning architecture for solid-state LiDAR sensors has been
decided yet, so we still have many research opportunities aiming
at high spatial resolution, large FoV, small size, high depth
accuracy, high frame rate, multi-echo detection, low power
consumption, etc.
 Emitters and depth image processing units are also crucial parts
of a LiDAR, although they are not covered in this tutorial.
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 89 of 95

© 2023 IEEE International Solid-State Circuits Conference

LiDAR / SPAD Papers at ISSCC 2023
 Unfortunately, no paper related to LiDAR or ToF sensors
will be presented at this year’s ISSCC, but a couple of other
things you might be interested in will appear.

 5.6: A 400x200 600fps 117.7dB-DR SPAD X-ray Detector with

Seamless Global Shutter and Time-Encoded Extrapolation
Counter
 Forum 2.6: Long-Range High-Resolution LiDAR Technology
 Forum 5.7: Extremely Compact Data Converters for Future 3D
Sensing

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 90 of 95

© 2023 IEEE International Solid-State Circuits Conference

References
 J. A. Richardson, et al., “Low Dark Count Single-Photon Avalanche Diode Structure Compatible With
Standard Nanometer Scale CMOS Technology,” IEEE Photon. Technol. Lett., vol. 21, no. 14, pp. 1020–
1022, Jun. 2009.
 E. Charbon, “Single-photon imaging in complementary metal oxide semiconductor processes,” Phil.
Trans. R. Soc. A: Mathematical, Physical and Engineering Sciences, vol. 372, no. 2012, pp. 20130100–
20130100, Feb. 2014.
 C. Niclass, et al., “A 128×128 Single-Photon Image Sensor With Colum-Level 10-bit Time-to-Digital
Converter Array,” IEEE J. Solid-State Circuits, vol. 43, no. 12, pp. 2977-2989, Dec. 2008.
 M.-J. Lee, et al., “High-Performance Back-Illuminated Three-Dimensional Stacked Single-Photon
Avalanche Diode Implemented in 45-nm CMOS Technology,” IEEE J. Select. Topics Quantum Electron.,
vol. 24, no. 6, p. 3801809, May 2018.
 C. Veerappan and E. Charbon, “A Low Dark Count p-i-n Diode Based SPAD in CMOS Technology,” IEEE
T. Electron Devices, vol. 63, no. 1, pp. 65-71, Jan. 2016.
 E. Charbon, “Small and Smart SPAD Pixels,” in Proc. of Int. SPAD Sensors Workshop (ISSW), Jun.
2020.
 S. Shimida, et al., “A Back Illuminated 6 µm SPAD Pixel Array with High PDE and Timing Jitter
Performance,” in IEEE Int. Electron Devices Meeting (IEDM) Dig. Tech. Papers, Dec. 2021, pp. 445-449.
 C. Veerappan, et al., "A 160×128 single-photon image sensor with on-pixel 55ps 10b time-to-digital
converter," in IEEE Int. Solid-State Circuits Conf. (ISSCC) Dig. Tech. Papers, Feb. 2011, pp. 312–313.
 C. Niclass, M. Soga, H. Matsubara, S. Kato, and M. Kagami, “A 100-m Range 10-Frame/s 340×96-Pixel
Time-of-Flight Depth Sensor in 0.18-µm CMOS,” IEEE J. Solid-State Circuits, vol. 48, no. 2, pp. 559–
572, Feb. 2013.
S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 91 of 95

© 2023 IEEE International Solid-State Circuits Conference

References
 C. Niclass, M. Soga, H. Matsubara, M. Ogawa, and M. Kagami, “A 0.18-µm CMOS SoC for a 100-m-
Range 10-Frame/s 200×96-Pixel Time-of-Flight Depth Sensor,” IEEE J. Solid-State Circuits, vol. 49, no.
1, pp. 315–330, Jan. 2014.
 A. R. Ximenes, P. Padmanabhan, M. Lee, Y. Yamashita, D. Yaung and E. Charbon, "A Modular, Direct
Time-of-Flight Depth Sensor in 45/65-nm 3-D-Stacked CMOS Technology," IEEE J. of Solid-State
Circuits, vol. 54, no. 11, pp. 3203–3214, Nov. 2019.
 L. Braga, et al., "An 8×16-pixel 92kSPAD Time-Resolved Sensor with On-Pixel 64ps 12b TDC and
100MS/s Real-Time Energy Histogramming in 0.13μm CIS Technology for PET/MRI Applications," in
IEEE Int. Solid-State Circuits Conf. (ISSCC) Dig. Tech. Papers, Feb. 2013, pp. 486–487.
 H. Seo, H. Yoon, D. Kim, S.-J. Kim, J.-H. Chun, and J. Choi, “A 36-channel SPAD-Integrated Scanning
LiDAR Sensor with Multi-Event Histogramming TDC and Embedded Interference Filter,” in Proc. of IEEE
Symp. VLSI Circuits, Jun. 2020.
 O. Kumagai, et al., "A 189×600 Back-Illuminated Stacked SPAD Direct Time-of-Flight Depth Sensor for
Automotive LiDAR Systems," in IEEE International Solid-State Circuits Conference (ISSCC) Dig. Tech.
Papers, Feb. 2021, pp. 110–111.
 N. Dutton, et al., “A Time-Correlated Single-Photon-Counting Sensor with 14GS/S Histogramming Time-
to-Digital Converter,” in IEEE Int. Solid-State Circuits Conf. (ISSCC) Dig. Tech. Papers, Feb. 2015, pp.
204–205.
 S. Shimada, et al., “A Back Illuminated 6 µm SPAD Pixel Array with High PDE and Timing Jitter
Performance,” in IEEE International Electron Device Meeting (IEDM) Dig. Tech. Papers, Dec. 2021, pp.
446-449.

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 92 of 95

© 2023 IEEE International Solid-State Circuits Conference

References
 C. Zhang, et al., “A 30-frames/s, 252×144 SPAD Flash LiDAR With 1728 Dual-Clock 48.8-ps TDCs, and
Pixel-Wise Integrated Histogramming,” IEEE J. Solid-State Circuits, vol. 54, no. 4, pp. 1137–1151, Apr.
2019.
 S. W. Hutchings, et al., “A Reconfigurable 3-D-Stacked SPAD Imager With In-Pixel Histogramming for
Flash LIDAR or High-Speed Time-of-Flight Imaging,” IEEE J. Solid-State Circuits, vol. 54, no. 11, pp.
2947–2956, Oct. 2019.
 D. Stoppa, et al., “A reconfigurable QVGA/Q3VGA Direct Time-of-Flight 3D Imaging System with On-
chip Depth-map Computation in 45/40 nm 3D-Stacked BSI SPAD CMOS,” in Proc. Int. Image Sensor
Workshop (IISW), 2021.
 B. Kim, S. Park, J.-H. Chun, J. Choi, and S.-J. Kim, “A 48×40 13.5mm Depth Resolution Flash LiDAR
Sensor with In-Pixel Zoom Histogramming Time-to-Digital Converter,” in IEEE International Solid-State
Circuits Conference (ISSCC) Dig. Tech. Papers, Feb. 2021, pp. 108–109.
 S. Park, B. Kim, J. Cho, J.-H. Chun, J. Choi, and S.-J. Kim, “An 80×60 Flash LiDAR Sensor with In-Pixel
Histogramming TDC Based on Quaternary Search and Time-Gated Δ-Intensity Phase Detection for 45m
Detectable Range and Background Light Cancellation,” in IEEE International Solid-State Circuits
Conference (ISSCC) Dig. Tech. Papers, Feb. 2022, pp. 98–99.
 Seonghyeok Park, Bum-Jun Kim, Su-Hyun Han, Junhee Cho, Jung-Hoon Chun, Jaehyuk Choi,
and Seong-Jin Kim, “An 80×60 Flash LiDAR Sensor with In-Pixel Delta-Intensity Quaternary Search
Histogramming TDC,” IEEE J. Solid-State Circuits, vol. 57, no. 11, pp. 3200-3211, Nov. 2022.

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 93 of 95

© 2023 IEEE International Solid-State Circuits Conference

More References
 M. Perenzoni, D. Perenzoni, and D. Stoppa, “A 64×64-pixels Digital Silicon Photomultiplier Direct TOF
sensor with 100-MPhotons/s/pixel Background Rejection and Imaging/Altimeter Mode with 0.14%
Precision up to 6 km for Spacecraft Navigation and Landing,” IEEE J. Solid-State Circuits, vol. 52, no. 1,
pp. 151–160, Jan. 2017.
 C. Zhang, et al., “A 240×160 3D Stacked SPAD dToF Image Sensor with Rolling Shutter and In-Pixel
Histogram for mobile devices,” IEEE Open J. Solid-State Circuits Soc., vol. 2, pp. 3-11, 2022.
 K. Yoshioka et al., “A 20-ch TDC/ADC Hybrid Architecture LiDAR SoC for 240 x 96 Pixel 200-m Range
Imaging With Smart Accumulation Technique and Residue Quantizing SAR ADC,” IEEE J. Solid-State
Circuits, vol. 53, no. 11, pp. 3026–3038, Nov. 2018.
 S. Kondo, et al., “An Automotive LiDAR SoC for 240 × 192-Pixel 225-m-Range Imaging With a 40-
Channel 0.0036-mm2 Voltage/Time Dual-Data-Converter-Based AFE,” IEEE J. Solid-State Circuits, vol.
55, no. 11, pp. 2866–2877, Nov. 2020.
 I. Gyongy, N. A. W. Dutton, and R. K. Henderson, “Direct Time-of-Flight Single-Photon Imaging,” IEEE
Trans. Electron Dev, vol. 69, no. 6, pp. 2794–2805, 2022.
 P. Padmanabhan, et al., “A 256×128 3D-stacked (45 nm) SPAD FLASH LiDAR with 7-Level Coincidence
Detection and Progressive Gating for 100 m Range and 10 klux Background Light,” in IEEE Int. Solid-
State Circuits Conf. (ISSCC) Dig. Tech. Papers, Feb. 2021, pp. 111–112.
 E. Manuzzato, et al., “A 64×64-Pixel Flash LiDAR SPAD Imager with Distributed Pixel-to-Pixel
Correlation for Background Rejection, Tunable Automatic Pixel Sensitivity and First-Last Event Detection
Strategies for Space Applications,” in IEEE Int. Solid-State Circuits Conf. (ISSCC) Dig. Tech. Papers,
Feb. 2022, pp. 96–97.

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 94 of 95

© 2023 IEEE International Solid-State Circuits Conference

Acknowledgment

 Contributions from a LiDAR research team

 Bumjun Kim
 Su-Hyun Han
 Seonghyeok Park

 Special thanks to
 Leonardo Gasparini
 Naveen Verma

S.-J. Kim T6: Solid-State CMOS LiDAR Sensors 95 of 95

© 2023 IEEE International Solid-State Circuits Conference