Shadows are a great way to relay information in a 3D rendering. They can help demonstrate distances between two objects such as a bouncing ball and the ground. They also relay further information to the structure of an object as they give a second silhouette from the perspective of the light casting the shadow. In this article, I will demonstrate a very small but important improvement for THREEjs’s shadow rendering, a one line change to the shader code.
THREEjs, our chosen graphics engine, like many realtime 3D renderers, casts shadows using a technique called shadow maps. A map is a term used to describe image data used on a GPU. A map can be generated by rendering a given scene and then storing the color information for that created image. A shadow map is a generated map that instead of storing color, stores the distance from a light in the scene to the surface of a rendered object.
Correctly generating the shadow map can be non-obvious since there are ways to get values that may be visibly close until you put it through some scrutiny. One part of this work is often referred to as packing where we take a piece of data and store it using a format other than its native representation. Specifically for shadow maps we need to pack the depth, represented as a decimal number between 0, really close, and 1, really far.
Packing is somewhat tricky in GLSL, the programming language for shaders in WebGL. The earlier versions of GLSL, and the ones available on most mobile devices and in WebGL, did not allow bitwise operators that frequent more general programming languages and are the normal choice for performing packing work. Instead, using a floating point modulus function is the only way.
It is not unusual for a piece of software to use published code from others, especially open source software like THREEjs. People figured out one way to pack the depth value in GLSL a while ago and it seems to have circulated a lot either without people questioning or possibly accepting the error it produces to a more accurate and slightly more costly source.
Fract, shorthand for the word “fraction” or the part of a decimal value after the dot, is equivalent to a modulus function with a predefined mod of value 1. This may open it to optimizations that the general modulus function cannot achieve. However its use in this specific case will cause errors in the less precise components being stored. Maps can have different precisions depending on how much space a developer wants it to take up in the GPU’s memory (RAM) versus the available precision. An individual cell in our shadow map has 4 components, each storing 256 possible values. On the CPU side these values would be interpreted as binary values of 0 to 255. On the GPU side they are interpreted as decimal values of 0 to 1. Since fract is a mod 1 function, no component in our map will store a value of 1 instead storing a value of 0.
Instead we want to multiply our components by 255 and mod by 256 and then divide back to 255.
So first we can say this will cause some values to be incorrectly stored at the highest component value of 255. But if you consider more values in the lower components, some of them will be stored as something lower than it should.
We can see from the above image, that using fract, the image on the left, adds an implicit and likely, in our case definitely, undesired offset. If this offset was a fixed amount maybe it would be an acceptable optimization for the error but it is actually dependent on the higher precision components created by packing the float value.
This chart shows what happens to the second component if the largest precision part equates to 128 in integer value or 0.501… in floating point. There is a shift in the third component, the second largest precision part as it is in little endian, when using the fract function equal to the stored value of the last component, the largest precision part. If we consider this difference with a scene that is 256 units in depth, around halfway through the scene, the shadow values shift from a half world unit deeper to a half world unit closer. On a self-shadowing surface, this would cause the half to be computed to be in shadow that should not be in shadow and another half to not be in shadow when it should.
On average this shift is worse halfway through the scene. But the jumps between those shifts will be possibly jarring effect depending on what you are rendering. In our case where most rendered objects have hard edges, it becomes regularly obvious.
A Physics Use Case
Another great use case for this change in value packing is using the GPU for calculating physics collisions. This is actually why I stumbled upon this error in THREEjs. In one of my off hour experiments I needed to read and store values for a particle based physics simulation I wanted to execute on the GPU. So I used the code THREEjs used to do that same job.
I started with just reading and then writing the same values out to see if I had set it up correctly in my toy example. When some values were returned as very different values I thought I had something wrong but after running the math by hand and thinking about the source I could explain the above errors. Using the change I made in my toy physics example I found this had obvious graphical changes to results from THREEjs in our game.
Why Should I Write About This Tiny Change?
The code for packing depth information in THREEjs did not originate from the THREEjs’s authors. The depth packing snippet of GLSL has been floating around the internet for at least 5 years.
Links referenced in THREEjs’s source:
Another source I found:
Looking around further you can find some other slightly different versions but they all seem to use fract. Maybe I am wrong in thinking this is a better solution since so many others seem to have found fract suitable for their needs but I thought I would share my lengthy ideas on why fract should not be used in the case of GLSL depth packing.
Here is a video to try and help explain further.