In order to visualize the techniques, process, and emotions of sketch artists, we have sought to display elements of traditional drawing processes. To do so, we created an interactive system that unobtrusively tracks the freehand drawing process (movement and pressure of artist’s pencil) on a traditional easel. The system outputs recorded information using video renderings and 3D-printed sculptures.
To test our system, we held a user study with 6 experienced artists who created multiple pencil drawings using our easel. The resulting digital and physical outputs from our system revealed vast differences in drawing speeds, styles, and techniques. The easel, video renderings, and bas-relief sculptures will be presented at the ACM Twelfth International Conference on Tangible, Embedded and Embodied Interactions (TEI 2018) in Stockholm, Sweden. You can read the write up here: (TEI 2018 publication)
System
Our interactive system is a traditional drawing easel which has been augmented with a pencil tracking system and a pencil pressure sensing system.
Pencil Tracking
To track the movements of the pencil, our system uses two cameras, which are mounted on the top and left sides of the easel. Images captured by the cameras are used to determine the vertical and horizontal location of the pencil. To make the tracking easier, we covered the drawing pencils in a layer of blue ink and mounted green colored background strips along the bottom and right edges of the easel. The horizontal and vertical locations of the drawing pencil is determined by locating the blue color blob created by the pencil against the green background.
Pressure Sensing
The pencil pressure sensing system is based on acoustic sensing, since we observed that the sound created by friction between the pencil and paper can be used to approximate the pencil pressure. While this relationship is not reliable enough to measure subtle variations of pressure, it is sufficient for detecting the major changes. To record sound, we placed 12 modules (each containing a microphone and microcontroller) in a 3 X 4 grid on the back side of the easel. Weighted averages of the three sensors closest to the pencil are used to determine the pencil pressure exerted on the drawing surface.
You can read more about the development of the easel here
Visualizing the Data
To display the recorded data, we chose to render the pencil speed and the pressure as an animation. In the animations, the pencil speed is determined by calculating the distance between data points. The pencil strokes which were drawn in slow, medium, or high speeds are represented distinctly in the visualization using green, yellow, and red colors. The different pressure levels are depicted using different line thicknesses.
In addition, we created another program that generates 3D bas-relief models displaying the drawing data. Bas-relief is a type of sculpture that consists of a projected image with little overall depth, such as Egyptian hieroglyphs or coins. In our models, the thickness of the ridges is based on the speed of the drawing, while the height of the ridges is based on the pressure of the drawing stroke. The height of the ridge can be compounded if several lines are drawn over the same area.
Artists Exploring the System
To explore the possibilities of our system, we conducted a study with six local artists, including MFA students, cartoonists, and a primary school art teacher. Each artist was invited to a drawing session during which they created three sketches, two of objects in the room (a lamp and flower pot) and one of whatever they wanted. In between creating the sketches, the artists were shown the video rendering and the 3D bas-relief rendering of the sketch they had just completed. All artists who took part in our study considered our tracking system to be unobtrusive and were interested in seeing the visualizations of their pencil movements.
The video renderings revealed unique characteristics among the drawing styles of participants. For example, they clearly showed that some participants, particularly the cartoonists, tend to use thicker lines in their drawings when compared to the others. The artists felt that the system could be useful both for teaching beginning artists and as a tool to study the evolution of a particular artist’s style.