SHAD3S: A model to Sketch, Shade and Shadow

Download paper here

Hatching is a common method used by artists to accentuate the third dimension of a sketch, and to illuminate the scene. Our system attempts to compete with a human at hatching generic three-dimensional (3d) shapes, and also tries to assist her in a form exploration exercise. The novelty of our approach lies in the fact that we make no assumptions about the input other than that it represents a 3d shape, and yet, given a contextual information of illumination and texture, we synthesise an accurate hatch pattern over the sketch, without access to 3d or pseudo 3d. In the process, we contribute towards a) a cheap yet effective method to synthesise a sufficiently large high fidelity dataset, pertinent to task; b) creating a pipeline with conditional generative adversarial network (cgan); and c) creating an interactive utility with gimp, that is a tool for artists to engage with automated hatching or a form-exploration exercise. User evaluation of the tool suggests that the model performance does generalise satisfactorily over diverse input, both in terms of style as well as shape. A simple comparison of inception scores suggest that the generated distribution is as diverse as the ground truth.

Recommended citation: R. B. Venkataramaiyer, A. Joshi, S. Narang, and V. P. Namboodiri (2021). “SHAD3S : A model to Sketch, Shade and Shadow”, Proceedings of 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)” Hawaii, USA, Jan. 2021.