How to texture for cel shading

This is the first post in a series of tutorials designed to bring you up to speed with the cel shader and learn all the tricks along the way. This time, we'll be covering texturing.

Fundamentals

One of the most basic aspects of the cel shader - detail shading - depends entirely on differences in pixel hue or luminosity; that is: colour and brightness. Where there is a rapid change in colour and brightness between two pixels, a line is drawn. The bigger the difference, the darker the line. A great example can be seen here, where two subtly different sand textures meet:

Although the textures are similar enough to see the line dividing them, they're different enough in brightness that the cel shader applies a soft line.

The other aspect to consider is that cel-shading takes place in screen-space, so different details will be cel shaded at different distances from the camera:

As the camera zooms out, different aspects of the texture smoothly sharpen and become cel-shaded, while smaller details fade.

In traditional cel-shaded texture where the line art is baked in, you either find yourself with heavy, pixellated lines close up or missing detail further away.

This isn't the case with the Advanced Cel Shader. You can take advantage of the way the lines are drawn by creating textures that apply specific levels of artistic detail at different sizes and distances from the object

Levels of detail

You can use this to create textures that are appealing at several distances. This isn't just a good technique for a cel shader - this is a good idea for any textures you're creating.

Here we have three dirt textures with different levels of detail:

They don't look so great on their own. The first one is blurry, the second one is repetitive and the third is too repetitive AND too detailed.

However, by applying the second two textures over the first with 50% transparency, we get this:

This is a marked improvement. Even better, this is what we get in-game using the Anime preset, which heavily favours detail shading. This is a 50/50 shot with an un-celshaded region on the left:

Straight away you can see how the cel shader adds and sharpens detail. it's not just about heavy black lines - the details of the texture itself have been improved. Whereas the texture is actually kind of fuzzy at this level, the cel-shaded version is crisp and artistic.

Further out, other details begin to be traced. In this case perhaps our smallest detail example texture is a bit too strong! We can fix this by making it less opaque in photoshop or by using a smart blur filter. That aside, let's continue zooming out:

Getting a bit small now, but the type of detail being outlined has switched again. Now we're seeing details in the middle detail texture being shaded. You can see the change between these two shots.

This process continues over and over until the object can barely be seen, with the detail being drawn adapting and continuing to look sharp and pixel-perfect at every distance.

Make use of this in your textures. Consider the far, middle and near (or big, medium and small) details, particularly with repetitive surfaces such as dirt, grass and other environmental textures.