Background: Physics and Math of Shading

Naty Hoffman
2K

Hi. Over the next 25 minutes or so I’ll be going from the physics underlying shading, to the math used to describe it in the kind of
shading models we’ll see in the rest of the course.
 2

We’ll start with the physics of what happens when light interacts with matter.
 Light Through Homogeneous Medium

The simplest case is light propagating through a homogeneous medium, or one that is the same everywhere. In this case light moves
in a straight line.
 Transparent media

Some homogeneous media (like clean water or glass) barely absorb light in the visible spectrum.
 Clear Absorbent Media

Other homogeneous media noticeably absorb light in the visible spectrum - in this case more blue light than red is absorbed.
 Scale

Scale is important. For example, water does absorb a little light in the red end of the visible spectrum, but it’s not noticeable over a few
inches.
 Scale

But this absorption is quite significant over distances of dozens of yards.

 What if the medium isn’t homogeneous?

?
8

That means the index of refraction (the property of matter that affects light) changes. Will it still move in a straight line?
 Scattering

Abrupt changes in the index of refraction cause scattering, which changes the direction of light propagation; amount of light stays the
same (unlike absorption, which changes amount but not direction)
 Interactions of Light and Matter
•Absorption: intensity / color changes,
direction unchanged

•Scattering: direction changes,

intensity & color unchanged

•Emission (new light created; not

relevant for most materials)

10

To summarize, we have three basic modes of light / matter interaction: absorption, scattering, and emission (the last one will not be
discussed further).
 Cloudy Media

11

Cloudy media cause enough scattering to somewhat randomize the light direction.
 Scale Matters For Scattering Too

12

Like absorption, scale matters for scattering: air doesn’t scatter noticeably over a few yards, but it does over a scale of miles.
 Opaque or Translucent Media

13

Opaque (or translucent) media cause enough scattering to completely randomize the light direction.
 Absorption and Scattering
Absorption (color)

Scattering (cloudiness)
14

The appearance of a homogeneous medium is determined by the combination of its absorption and scattering properties.
 Scattering at a Plane: Reflection & Refraction
•This special scattering
splits light in two
directions: reflection and
refraction
•Refracted light may be
absorbed, scattered
under the surface

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

15

Important scattering special case: a perfectly flat (up to the size of light waves) plane between two indices of refraction. Instead of
splitting up into an arbitrary number of scattering directions (as would happen in the general case), in this case light splits into exactly
two directions: reflection and refraction.
 Non-Optically-Flat Surfaces: Microgeometry
•Each surface point reflects light in one direction
•Surface appearance is the aggregate result of
many points with different surface orientations

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

16

A surface which is flat up to the size of light wavelengths (e.g. high-end telescope optics) is optically flat - however, most surfaces
have microgeometry (bumps bigger than a light wavelength but smaller than a pixel) that affect light reflection.
 Rougher = Blurrier Reflections

Images from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

17

Both of these surfaces look equally smooth to the naked eye; however under a microscope we can see the difference in roughness.
The surface on the top is only slightly bumpy; the incoming light rays hit bits of the surface that are angles slightly differently and get
reflected to slightly different outgoing directions. The surface on the bottom is very rugged on the microscopic scale - each incoming
light ray hits a piece of surface that is oriented very differently, spreading the outgoing light rays over a wide cone of directions. We
can see the results in the differences between the reflections off the two objects.
 Macroscopic View

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

18

In the macroscopic view, we treat the microgeometry statistically and view the surface as reflecting (and refracting) light in multiple
directions.
 ?
19

What happens to the refracted light? It depends what kind of material the object is made of.
 Metals

20

Metals immediately absorb all refracted light.

 Non-Metals (Insulators, Dielectrics)

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

21

Non-metals behave like those cups of liquid we saw earlier. They cause various degrees of absorption and scattering; the refracted
light gets scattered and absorbed inside the material and some of it comes back out.
 Scale and Subsurface Scattering

22

The re-emitted light comes out at varying distances from the entry point. The distribution of distances depends on the density and
properties of the scattering particles.
 Scale and Subsurface Scattering

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

23

If the pixel size (or distance between samples) is large (green circle) compared to the entry-exit distances, we can assume the
distances are effectively zero.
 Scale and Subsurface Scattering

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

24

By ignoring the entry-to-exit distance, we can then compute all shading locally at a single point. The shaded color is only affected by
light hitting that surface point.
 Shading Terms
diffuse
specular

25

It is convenient to split these two very different light-material interactions (surface reflection and refraction-absorption-scattering) into
different shading terms - the familiar “specular” and “diffuse”.
 Scale and Subsurface Scattering

26

If the pixel is small (red circle) compared to the entry-exit distances, you can’t ignore the distances and need specialized rendering
techniques. These are referred to as “subsurface scattering”, but diffuse is always subsurface scattering - the difference is the scale.
So distant skin or marble can use normal rendering techniques, and plastic in extreme close up would need “subsurface scattering”
techniques.
 Math of Physically-Grounded Shading
5
FSchlick (cspec , l, n) = cspec + (1 cspec )(1 (l · n))
Z
Lo (v) = f (l, v) ⌦ Li (l)(n · l)d!i
⌦

F (l, h)G(l, v, h)D(h)

f (l, v) =
4(n · l)(n · v)
27

Math of physical shading - to turn light physics into a mathematical model, the first step is to quantify light as a number.
 Radiance
Single Ray
Spectral/RGB
28

Radiometry is the measurement of light. Of the various radiometric quantities, we’ll use radiance, which measures the intensity of light
along a single ray. Radiance is spectral (varies with wavelength) - it’s technically a continuous spectral power distribution but for
production purposes it’s represented as an RGB value.
 Local Shading

29

Given the assumption that shading can be handled locally, light response at a surface point only depends on the light and view
directions.
 Bidirectional f (l, v)
Reflectance
Distribution
Function

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

30

We represent this variation with the BRDF, a function of light and view direction. In principle, the BRDF is a 4D or 3D function. In
practice, a varying number of angles are used to compute it. The BRDF is only defined for light and view vectors above the
macroscopic surface. Backfacing light directions should be avoided or at least have their contribution multiplied by 0. Backfacing view
directions are theoretically impossible for opaque surfaces but may occur with interpolated normals or normal mapping; these are
typically handled either by clamping the n-dot-v product to 0 or taking its absolute value to avoid negative values.
 The Reflectance Equation

Z
Lo (v) = f (l, v) ⌦ Li (l)(n · l)d!i
⌦

31

Outgoing radiance equals the integral (over all directions above the surface) of incoming radiance times BRDF and cosine factor; “X in
circle” denotes component-wise vector (RGB) multiplication.
 Surface Reflection (Specular Term)

32

We’ll start by looking at the specular term.

 Microfacet Theory

33

Microfacet theory is a way to derive BRDFs for surface (specular) reflection from general (non-optically flat) surfaces. It assumes the
surface is composed of many microfacets. Each facet is a perfect mirror (optically flat), so it reflects each incoming ray of light into
only one outgoing direction, which depends on the light direction l and the microfacet normal m.
 The Half Vector

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

34

Only those microfacets which happen to have their surface normal m oriented exactly halfway between l and v will reflect any visible
light - this direction is the half-vector h.
 Shadowing and Masking

shadowing masking

Images from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

35

Not all microfacets with m = h will contribute - some will be blocked by other microfacets from either l (shadowing) or v (masking).
 Multiple Surface Bounces

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

36

In reality, blocked light continues to bounce; some will eventually contribute to the BRDF. Microfacet BRDFs ignore this, and assume
all blocked light is lost
 Microfacet BRDF

F (l, h)G(l, v, h)D(h)

f (l, v) =
4(n · l)(n · v)

37

This is the microfacet BRDF. I’ll go over its various parts, explaining each.
 Fresnel Reflectance

F (l, h)G(l, v, h)D(h)

f (l, v) =
4(n · l)(n · v)

38

The Fresnel reflectance is the fraction of incoming light reflected (vs. refracted) from an optically flat surface of a given substance
given light direction and surface normal. This tells us much how much of the light hitting the relevant microfacets (the ones facing in
the half-angle direction) is reflected.
 Fresnel
Reflectance

Image from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

39

Fresnel reflectance depends on refraction index (in other words, what the object’s made of) and light angle. In this graph, substances
with three lines are R, G and B – substances with one line have uncolored reflectance.
 Fresnel
Reflectance
barely changes

changes somewhat

goes rapidly to 1

40

As angle increases, at first the Fresnel reflectance stays almost the same, then for very glancing angles it goes to 100% at all
wavelengths.
 Fresnel
Reflectance

F(0°)
Is the surface’s
characteristic
specular color:
cspec
41

Since the reflectance over most angles is close to that at normal incidence, the normal-incidence reflectance F(0) is the surface’s
characteristic specular color.
 Normal-Incidence Fresnel for Metals

Table from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

42

Note that metals have no subsurface term, so the surface Fresnel reflectance is the material’s only source of color. The “linear” and
“sRGB” columns refer to whether the values are in linear or gamma space. Note that for metals the specular colors are all relatively
high.
 Normal-Incidence Fresnel for Non-Metals

Table from “Real-Time Rendering, 3rd Edition”, A K Peters 2008

43

Most non-metals also have a subsurface (diffuse) color in addition to this Fresnel (specular) reflectance. Note that for non-metals the
specular colors are achromatic (gray) and are relatively low (especially if excluding gems and crystals).
 The Schlick Approximation to Fresnel
•Pretty accurate, cheap, parameterized by cspec
5
FSchlick (cspec , l, n) = cspec + (1 cspec )(1 (l · n))

•For microfacet BRDFs (m = h):

5
FSchlick (cspec , l, h) = cspec + (1 cspec )(1 (l · h))

44

The Schlick approximation to Fresnel is commonly used. It is cheap and reasonably accurate - more importantly it has a convenient
parameter (normal incidence Fresnel reflectance, or specular color).
 Microfacet Normal Distribution

F (l, h)G(l, v, h)D(h)

f (l, v) =
4(n · l)(n · v)

45

The microfacet normal distribution tells us what is the concentration of microfacet normals pointing in a given direction (in this case,
the half-angle direction). The NDF determines the size and shape of the highlight.
 Geometry Factor
F (l, h)G(l, v, h)D(h)
f (l, v) =
4(n · l)(n · v)

46

The geometry factor gives the chance that a microfacet of the given orientation is lit and visible (e.g., not shadowed and/or masked)
from the given l and v directions.
 Subsurface Reflection (Diffuse Term)

47

Next, we’ll take a quick look at the diffuse or subsurface term.

 Lambert
•Constant value (n•l is part of reflection equation):
cdi↵
fLambert (l, v) =
⇡
•cdiff: fraction of light reflected, or diffuse color

48

The Lambert model is the most common diffuse term used in game and film production. By itself, it’s the simplest possible BRDF - a
constant value (the well-known cosine factor is part of the reflection equation).
 Beyond Lambert: Diffuse-Specular Tradeoff

49

There are a few important physicaly phenomena that Lambert doesn’t account for. Diffuse comes from refracted light - in a sense the
specular term gets “dibs” on the incoming light and diffuse gets the leftovers. Since surface reflection goes to 100% at glancing
angles, it follows that diffuse should go to 0%. There are various ways to account for this - perhaps the simplest is to multiply diffuse
by (1-F) (this turns Fresnel into a “lerp factor” between specular and diffuse).
 Beyond Lambert - Surface Roughness

50

Another phenomenon not modeled by Lambert is the effect of surface roughness. Roughness typically only affects specular; since
diffuse reflectance at a point comes from incoming light over an area, the microgeometry variations tend to be averaged out for diffuse
reflectance. But some surfaces have microgeometry larger than the scattering distance, and these do affect diffuse. These are the
surfaces that should be modeled with specialized diffuse models like Oren-Nayar or Minneart.
 Implementing Physical Shading Models
•To implement a shading equation in a renderer,
combine it with various illumination models:
–General Lighting
–Image-Based Lighting
–Area Light Sources
–Punctual Light Sources
–Ambient Light

51

General lighting: integrate BRDF against continuous incoming light from all directions (area light sources, skylight, indirect reflections)
- requires GI algorithm like ray-tracing. Image-based lighting refers to environment maps and the like. Area light sources are starting to
be commonly used in film but not in games, which typically use punctual light sources (still used in film as well). Ambient is the
simplest lighting model - just constant light from all directions. We will talk more about punctual lights and image-based lighting due to
their common use in game and film production rendering.
 Punctual Light Sources
•Parameterized by light color clight and direction to
the light (center) position lc
•clight equals radiance from a white Lambertian
surface illuminated by the light at 90 degrees

52

These are commonly used in games and film - infinitely small and infinitely bright, they are not realistic, but are the easiest type of light
to implement with arbitrary BRDFs. They look OK for rough surfaces, not so much for smooth ones. Punctual lights are parameterized
by color and direction.
 Punctual Light Reflection Equation

Lo (v) = ⇡f (lc , v) ⌦ clight (n · lc )

53

The course notes have the derivation. Games often clamp the dot product as a convenient method of skipping backfacing lights.
 Image Based Lighting (Environment Maps)
•Perfectly smooth surface is easy - just multiply by
Fresnel function; same cspec, different angle:
–F(v, n) instead of F (l, h) (or F (v, h); same)
•Accurate glossy/diffuse requires many samples
–Importance sampling helps
–Prefiltering (alone or with importance sampling)

54

Image-based lighting is useful for all materials, but especially for smooth or metallic ones. Games use prefiltering
heavily - film tends to use importance sampling (sometimes combined with prefiltering).
 Building a Microfacet Model
•The choice of D() and G() are independent; you can
mix and match from different models
•Most papers proposing a new BRDF model are
really introducing a new D() or a new G()

55

We will now look at some options for building a microfacet model. Don’t worry about copying the equations down - they will all be in
the course notes.
 Normalizing a Normal Distribution Function
Z
1= D(m)(n · m)d!m
⇥

56

To be used in a microfacet BRDF, an NDF should be normalized based on the equation above. If you have a non-normalized term, just
divide it by its cosine-weighted integral. In the general case the integral is over the sphere but we will only deal with heightfield
surfaces where you want to integrate over the hemisphere. Heightfield NDFs should always evaluate to 0 for backfacing directions; if
you may evaluate it for these directions (e.g. normal mapping can sometimes cause backfacing half-vectors) then you should ensure it
evaluates to 0 in these cases. For NDFs that go to 0 at 90 degrees this can simply be done by clamping the n-dot-m dot product to 0;
other NDFs may require a conditional expression.
 Phong NDF

↵p + 2 ↵p
Dp (m) = (n · m)
2⇡

57

Correctly normalized, the Phong (cosine power lobe) becomes a valid NDF.
 Beckmann NDF
0 1
2
1 (n · m)
@ A
1 2
↵b (n · m) 2
Db (m) = 2 4
e
⇡↵b (n · m)

58

The Beckmann NDF is similar to Phong in some important ways and different in others - see the course notes for more details.
 Trowbridge-Reitz (GGX) NDF

2
↵tr
Dtr (m) = 2
⇡ ((n · 2 2
m) (↵tr 1) + 1)

59

The Trowbridge-Reitz NDF has a fundamentally different shape than Phong & Beckmann, with a narrower “peak” and broader “tails”.
More details can be found in the course notes.
 ABC NDF
1
Duabc (m) = ↵abc2
(1 + ↵abc1 (1 (n · m)))
2 ↵abc2
↵abc1 (1
+ ↵abc1 ) ( 2 + ↵abc2 )( 1 + ↵abc2 )
kabc = 2 ↵
2⇡ ((1 + ↵abc1 ) + (1 + ↵abc1 ) abc2 ( 1 + ↵abc1 ( 2 + ↵abc2 )))

Dabc (m) = kabc Duabc (m)

60

The ABC function has been around for a while in the optical engineering community, not as an NDF but as a spectral power
distribution for the heightfield of the surface. Recent work has proposed using this function as an NDF. I got the normalization factor
from Mathematica and didn’t spend any time simplifying or optimizing it; note that it has singularities at αabc2=1.0 and αabc2=2.0. I
suspect a much cheaper curve could be fit to it. More details can be found in the course notes.
 Shifted-Gamma NDF
2
↵sgd1 +x
↵sgd2 1
↵sgd1 e ↵sgd1

p22[x] = ⇣ ⌘↵sgd2
(1 ↵sgd2 , ↵sgd1 ) ↵2 + x
sgd1
h 2
i
1 (n·m)
p22 (n·m) 2
Dsgd (m) = 4
⇡(n · m)
61

The shifted-Gamma NDF is a relatively new one. Like ABC it has two parameters, rather than the single roughness parameter most
isotropic NDFs have. Like ABC, its normalization factor looks expensive, but a cheap approximation could likely be found. Again
similarly to ABC, the authors who proposed this NDF suggest that it is a closer fit to measured BRDFs (such as the Matusik database)
than previous models.
 The Visibility Term
F (l, h)G(l, v, h)D(h)
f (l, v) =
4(n · l)(n · v)
G(l, v, h)
V (l, v) =
(n · l)(n · v)
62

In some cases, expressions are found for the geometry factor divided by the n-dot-l-times-n-dot-v “foreshortening term”. We’ll call this
combined term the “visibility term”. The following slides will describe some choices for representing either just the geometry factor or
the entire visibility term.
 The Simplest Possible Visibility Term

G(l, v, h)
=1
(n · l)(n · v)
•Equivalent to:

Gimplicit (lc , v, h) = (n · lc )(n · v)

63

Many BRDFs (especially game ones) have no visibility term at all. This means the visibility term is effectively equal to one, which
implies a geometry factor equal to n-dot-l-times-n-dot-v. This “implicit” geometry factor is plausible - it equals one when view and light
are in the normal direction, and zero when either one of them is at 90 degrees. And it is “cheaper than free” since the entire visibility
term is missing. But it goes to 0 too fast at glancing angles (compared to real-world materials) and it isn’t affected by roughness, which
is implausible.
 The Cook-Torrance Geometry Factor
✓ ◆
2(n · h)(n · v) 2(n · h)(n · l)
Gct (l, v, h) = min 1, ,
(v · h) (v · h)

64

This was actually invented by Blinn in 1975 as a reformulation of the Torrance-Sparrow geometry factor. It’s been used a lot over the
years (especially in film), but it has some problems: it’s based on an unrealistic microgeometry model (isotropic surface composed of
infinity long grooves) and it’s not affected by roughness, which is implausible.
 Kelemen-Szirmay-Kalos Geometry Factor

Gct (l, v, h) 1
⇡ 2
(n · l)(n · v) (l · h)

65

In addition to its other issues, Cook-Torrance is a little on the expensive side for games. There is a very cheap and effective
approximation for it however; almost as cheap as the implicit geometry factor - you just need to divide by the square of a dot product
that you need to compute anyway for Fresnel.
 Smith Geometry Factor

D() G()

66

This is probably the best one to use today. It is affected by the roughness, and it has been generalized to a process for generating a
matching geometry factor for any NDF. You can think of it as a “black box” where you plug in an NDF and get a matching geometry
factor. The process may not result in an analytical expression but that’s fine since you most likely want to fit a simpler curve to the
result in any case. Note that the Schlick approximation to this function (which has been referenced in many places - including my own
book!) is not correct since it approximates the wrong version of Smith. More details on the Smith shadowing factor can be found in the
course notes.
 Acknowledgements
•A K Peters for permission to use RTR3 images
•Brent Burley, Paul Edelstein, Yoshiharu Gotanda,
Christophe Hery, Dimitar Lazarov, and Brian Smits
for thought-provoking discussions

67

I’d like to close by thanking some people who helped me with this talk.