Category Archives: Uncategorized

Tracking, Animating, and 3D Printing the Freehand Drawing Process

Our interactive drawing setup(left), 2D visualization(middle) and 3D bas-relief model(right) generated by our system

In order to visualize the techniques, process, and emotions of sketch artists, we have sought to display elements of traditional drawing processes. To do so, we created an interactive system that unobtrusively tracks the freehand drawing process (movement and pressure of artist’s pencil) on a traditional easel. The system outputs recorded information using video renderings and 3D-printed sculptures.

To test our system, we held a user study with 6 experienced artists who created multiple pencil drawings using our easel. The resulting digital and physical outputs from our system revealed vast differences in drawing speeds, styles, and techniques. The easel, video renderings, and bas-relief sculptures will be presented at the ACM Twelfth International Conference on Tangible, Embedded and Embodied Interactions (TEI 2018) in Stockholm, Sweden. You can read the write up here: (TEI 2018 publication)

Link for the youtube video.


Our interactive system is a traditional drawing easel which has been augmented with a pencil tracking system and a pencil pressure sensing system.

Pencil Tracking

To track the movements of the pencil, our system uses two cameras, which are mounted on the top and left sides of the easel. Images captured by the cameras are used to determine the vertical and horizontal location of the pencil. To make the tracking easier, we covered the drawing pencils in a layer of blue ink and mounted green colored background strips along the bottom and right edges of the easel. The horizontal and vertical locations of the drawing pencil is determined by locating the blue color blob created by the pencil against the green background.

Pressure Sensing

The pencil pressure sensing system is based on acoustic sensing, since we observed that the sound created by friction between the pencil and paper can be used to approximate the pencil pressure. While this relationship is not reliable enough to measure subtle variations of pressure, it is sufficient for detecting the major changes. To record sound, we placed 12 modules (each containing a microphone and microcontroller) in a 3 X 4 grid on the back side of the easel. Weighted averages of the three sensors closest to the pencil are used to determine the pencil pressure exerted on the drawing surface.

You can read more about the development of the easel here

Visualizing the Data

To display the recorded data, we chose to render the pencil speed and the pressure as an animation. In the animations, the pencil speed is determined by calculating the distance between data points. The pencil strokes which were drawn in slow, medium, or high speeds are represented distinctly in the visualization using green, yellow, and red colors. The different pressure levels are depicted using different line thicknesses.

In addition, we created another program that generates 3D bas-relief models displaying the drawing data. Bas-relief is a type of sculpture that consists of a projected image with little overall depth, such as Egyptian hieroglyphs or coins. In our models, the thickness of the ridges is based on the speed of the drawing, while the height of the ridges is based on the pressure of the drawing stroke. The height of the ridge can be compounded if several lines are drawn over the same area.


Artists Exploring the System

To explore the possibilities of our system, we conducted a study with six local artists, including MFA students, cartoonists, and a primary school art teacher. Each artist was invited to a drawing session during which they created three sketches, two of objects in the room (a lamp and flower pot) and one of whatever they wanted. In between creating the sketches, the artists were shown the video rendering and the 3D bas-relief rendering of the sketch they had just completed. All artists who took part in our study considered our tracking system to be unobtrusive and were interested in seeing the visualizations of their pencil movements.

Corresponding 2D renderings of pencil drawings by 4 different artists

The video renderings revealed unique characteristics among the drawing styles of participants. For example, they clearly showed that some participants, particularly the cartoonists, tend to use thicker lines in their drawings when compared to the others. The artists felt that the system could be useful both for teaching beginning artists and as a tool to study the evolution of a particular artist’s style.

BioDesign Challenge

On July 12th,  the students of ASU Digital Culture and The Design School have presented their LIFE/LIGHT project at the Biogesign Challenge summit in MOMA, New York. The project was developed at AME 410 Interactive Materials course and finalized for the competition.

The summit happened for the third time, engaging enthusiasts that combine design with biotechnology. It is one of the largest biodesign events in the US and brings the attention of the growing community of designers and researchers.

Around twenty teams have participated in the Challenge this year from various schools and countries. The 1st prize was taken by the team from Central Saint Martins, UK, that have presented the concept of Quantumworm Mines. The runners-up were the students of  University of Edinburgh, UK with the research project “UKEW 2029” that showed parallels between biology and socio-political trends.


ASU project researched the potential of bioluminescent unicellular organisms and scrutinized the issue of co-habitat and control in a man-made environment. LIFE/LIGHT is an algae-driven living building system that produces fuel and light if properly taken care of.

We were designing in the middle-ground between an artifact, living nature, and humanity where the behavior of each component of the system influences its performance. (See figure 1.)


Figure 1. Concept diagram.

Choosing our components within the broad fields of nature and artifacts, we decided to look into the relationships between dinoflagellate, a capricious algae creature that illuminates ocean in a number of coastal cities, including San Diego, and architecture as a medium for most of the human activities.


With the increasing concerns about ecology, the notion of living architecture arises. In the age of Anthropocene, living buildings adapt to the constant flux of technological, social and environmental conditions through integration with living nature.

The best example of such thing, probably, would be the rice paddles in South Vietnam (image 1), a sustainable artifact of agriculture and built environment that existed for centuries.


Image 1. Rice paddles in South Vietnam, stock photo.

Among other inspirational examples are the Algae-fueled building in Hamburg designed by ARUP (image 2), the proposal by Mitchell Joachim for homes grown like plants (image 3), and the interactive installation by David Benjamin that visualized ecological conditions for the citizens of Seoul (image 4).


Image 2. BIQ algae-powered building in Hamburg, image courtesy of ARUP.


Image 3. FabTreeHub, image courtesy of Mitchell Joachim.


Image 4. Living Light, Seoul, image courtesy of David Benjamin and The Living New York.



Image 5. Bioluminescent dinoflagellate, stock photo.

Dinoflagellates are unicellular algae plankton chosen for the LIFE/LIGHT project due to the its qualities:



Image 5. Bioluminescent jellyfish, stock photo.

Bioluminescence is the ability of living organisms to produce light. The “cold light” produced by dinoflagellate is done without wasting energy compared to conventional
electrically generate light.

When agitated by movement, algae colony produces light for a short perious of time.

CO2 Consumption

Dinoflagellate photosynthesis is capable of converting CO2 in glucose. This provides residual potential energy within cultures longer after decay.

Conversion to biofuel

Dinoflagellates may contain large amounts of high-quality lipids, the principal component of fatty acid methyl esters. The harvest of these organisms provides a suitable choice as a bioresource for biodiesel production.

Natural medium

Dinoflagellates are marine organisms that thrive in the natural medium of marine water. It makes them suitable for growth in the coastal cities with the use of natural salt water resources only.


The dinoflagellates were grown in SANDS lab as part of the Digital Art Ranch at ASU.
This space supports DIY biology as well as other forms of researching interactive materials (image 6).


Image 6. The experience of working with dinoflagellates, photos from DAR.

Growing algae takes a lot of patience and attentiveness. Not only we had to keep in a specific medium for the lack of fresh marine water, but also synchronise its day and night cycles with the lab operating hours.

During the day cycle (~12 hours), photosynthesis happens, and algae transform CO2 into glucose. During the night cycle (~12 hours also), they multiply and show bioluminiscense if agitated. Like humans, dinoflagellates are active during the day, rest during the night, and are very irritated when their rest is interrupted.

The optimal living condition for dinoflagellate is a room temperature 18 to 24°C (65 to 75°F) and avoiding rapid temperature fluctuations. This was regulated using a white LED lamp, which can be changed for a cool white fluorescent light.


Time was a limiting factor as cultures would take a week or two to regain its properties from packaging. This opened for the possibility that cultures order may have been non-lively upon arrival.

Also the time for sub-dividing cultures takes 3-4 weeks, again letting the subcultures regain their properties. Then when testing cultures, this would have to be done over a span of days to few weeks to determine the necessary action for culturing.

A typical dinoflagellate flash of light contains about 100 million photons and lasts about a tenth of a second. In a testing format, it is suggested to use a control amount and compare the luminous value on the scale of 10. Also, one has to be very careful not to “stimulate” the culture before you actually measuring their light output because the first time they flash they produce a lot more light than each successive flash.


Patience and constantly being aware of the cultures. The cultures can be unforgiving when they begin to use bio luminescence and taking additional time to recharge before seeing the effect again. Also there was a problem to document the effect during an appropriate time.For circadian rhythms to

For circadian rhythms to be aligned with documentation during the day. The cultures would be in a night cycle during the day. Causing a problem of space, we had to devise a small container that would keep temperatures low and block enough light pollution from the room it was placed in.


The project is a living building system that is attached to buildings in coastal cities and relies on algae for light and fuel production. It utilizes ocean water resources as a medium for dinoflagellate. It consists of tubes filled with algae-infused fluid, distributed operational nodes that control the water flow and a controlling device.


Image 7. Facade system sketch

The system works in 3 different modus operandi: day, night and
harvesting organic residues for biofuel production.


System elements


Figure 2. System elements

Day mode

During the day, the water is supplied from the ocean water resources and distributed to the LIFE/LIGHT and other building systems, e.g. cooling. The algae-infused fluid flows into the tubes attached to a building facade and exposed to the sun.


Figure 3. Day mode

Night mode

At night, algae-infused water fills the interior tube system that prevents its exposure to
city night illumination. When moved, the fluid gives away cold light that supports quiet
night activities inside a building.

In this mode, the most interaction between a human and the system happens. Human and algae share the same habitat and have to live in harmony in order for the system to work. If the night cycle is distracted by a human’s late night activities, algae do not multiply. When a person moves within the space with the algae tubes, they also move, arousing bioluminescence and illuminating the space.


Figure 4. Night mode

Harvest mode

At the end of dinoflagellates life cycle, they become a residual organic matter that can be harvested in order to produce biofuel.


Figure 5. Night mode

Operational node

The node serves an illustration to a highway of tracks within a system.
There would be numerous tubes to ensure the cultures are filtering, harvesting and transport to the correct location.


Image 8. Operational node

Controlling node

Inspired by thermostats, the control unit provides a basis for displaying information and controlling additional systems in a house.

The 3 buttons would allow for the most critical options of the system to be chosen.
Additionally, the display provides a small sample of the dinoflagellates that would be tested. Depending on the condition of the sample and the previous sample was taken, the filter option could be accepted. Cycling the dinoflagellate culture and providing more medium.


Image 9. Controlling node

The origin of the design needed to resemble a simple form of communication to a user that performs maintenance with the architecture embedded system. Not only would it provide given information on the LCD screen but it has the ability to control other system operations as needed.


Image 10. Inspiration for the controlling node. Image courtesy of Honeywell.

The questions then are:

What is the boundary between an artifact and nature? Is the LIFE/LIGHT system alive?

Would you co-inhabit space in algae and adjust your habits so that both species thrive or control it remotely and transform living creatures into a utility?

The project was submitted by Loren Benally and Veronika Volkova and contributed by Jacob Sullivan and Ryan Wertz

AME to host TEI 2019

Our bid to host TEI 2019, the fourteenth International ACM Conference on Tangible, Embedded and Embodied Interaction at AME has been officially accepted!


We will host the conference in Tempe Arizona, a vibrant and growing Sonoran desert city. The main venue, the Tempe Mission Palms, is walking distance to the ASU Campus and in the heart of Tempe shops, bars, and galleries. The conference rooms, AV services, catering, palm courtyard, and rooftop pool reception deck will offer a flexible meeting space for academic and social gatherings during the conference. Phoenix is well known as an ideal winter destination, boasting average high temperatures in February ranging from 68 to 73 degrees Fahrenheit and a low chance of precipitation, while much of the rest of the northern hemisphere may be suffering from severe winter weather.


Hybrid Materials

Our bid proposed the theme of Hybrid Materials with the aim of strengthening transdisciplinary tries across the tangible interaction, HCI, material sciences, social sciences, and arts communities. Gaining increasing momentum over the last five years, the material turn and its effect within HCI has generated development in numerous fields of interest to the TEI community.


Over the past few years, TEI research has increasingly embraced hybridity, whether through material explorations of composites such as bioelectronic, on-body, or active materials, or theoretical inquiries into socio-technical systems as hybrid assemblies. The theme of Hybrid Materials will continue to catalyze this exciting trend of tangible interaction research at the intersection of social, technical, biological, and artistic systems. Topics focusing on hybridity in interaction design include but are not limited to:

  • active materials
  • materiality
  • material as interface
  • expressive computing
  • human perception
  • bioelectronic systems and interactions
  • on-body computing
  • new materialism
  • computer as material
  • sociotechnical assemblies
  • design things
  • seamful computing
  • hybrid sense-making
  • transdisciplinarity and HCI
  • rapid prototyping
  • participatory design
  • productive tensions in design

Thanks to everyone who helped and contributed to our bid. On behalf of AME, we are really excited and very honored to host the conference in 2019!

Our full bid document [PDF]

Bio workshop at HeatSync

Hi folks, as we are continuing our work in DIY biology with general public and non-professional biology hobbyists, last week Cass, Stacey and Me conducted a DIY bio workshop at HeatSync Labs in Meza, Az. HeatSync is a community driven maker-space, one of the coolest places I’ve ever been in Arizona. Unfortunately Matt couldn’t make it this time, even though he immensely contributed in organising and planning the workshop.DSC_0414.JPG

The first part of the workshop was about yoghurt fermentation. Cass explained the steps of yoghurt fermentation process and worked closely with the participants in the process. Here are some images.

Then we moved to the second part of the workshop – Gram staining. In this activity participants were asked to follow instructions printed on the card given by us, as well as Cass’ guidance. Everyone was so excited to see the microscopic images of the slides they have created, actually results were awesome!


While everyone was partying with bacterias in the downstairs, I was busy connecting the camera of our DIY incubator to the HeatSync WIFI network. Yes, now we have a WIFI camera inside our incubator as we promised in one of our earlier posts!

We left our incubator and some basic materials at heat syncs lab, so that they can play with them in the summer. Hopefully we will get some useful feeds from the camera too!DSC_0411

I’m Piyum, signing off and running to catch the flight to CHI 16 to present our Bio work there. More on that later! Thanks for reading.


Here we were at CHI 16 poster session.




Art/ Science of cooking and sustainability.

Hey’all, this is Sunny and I am a new member in the growing family of the SANDS. I am an industrial design student at the Herberger’s and I potentially help in coming up with concepts and aesthetic models for the sustainable solar cooking research project. I assist in designing and finding new ways of using rich solar energy to our advantage and help community find new ways to use this energy for their cooking needs.

The first set of concepts looks into how we could take the french cooking technique “Sous-vide” and figure concepts that could help us in this regard.

So, what is sous-vide? well, Sous-vide is a method of cooking in which food is sealed in airtight plastic bags then placed in a water bath or in a temperature-controlled steam environment for longer than normal cooking times—96 hours or more, in some cases—at an accurately regulated temperature much lower than normally used for cooking, typically around 55 to 60 °C (131 to 140 °F) for meat and higher for vegetables – our friends at Wikipedia.

I started off by sketching different concepts and making a 3D model on SolidWorks and rendered them on KeyShot. From there we moved on to 3D printing the design for testing. The prints and the forms have come out well and it is on to testing and validating.

Stay tuned for the results!

DIYbio Update!

Hi All! My name is Cass and I’m a new member of the SANDS Group here at ASU (pictured above explaining some cool science with my hands at the head of the table). I’m a PhD student in Biological Design, and will be helping out as a science advisor for some awesome new DIYbio projects at SANDS. Last week, we held our first bio workshop, doing a disc diffusion assay with some members of the community. This experiment allowed us to determine if different compounds have antibacterial properties, an important test when antibiotic resistance is on the rise.

To do the test, participants were each given a petri dish with agar media (bacteria food) and a tube of E. coli culture to spread on it. (*Fun fact! Almost all strains of E. coli, including the one we used, are harmless. In fact, there are millions of E. coli cells living happily in your gut right now). Once the bacteria were spread out, discs with antibiotics were placed on the plate along with materials participants were interested in testing. We used various soaps, a penny, and even garlic. These plates were then nestled cozily in the lab’s incubator, built previously at SANDS, so the bacteria could grow overnight.

After growing the bacteria, we can looked for zones of clearing in the lawn of bacteria around the disc/material placed on the plate. Are the bacteria growing right up to the disc? Then they’re resistant to whatever’s on it! Do you see a zone of clearing around the disc where the bacteria can’t grow? Then your compound has antibacterial properties! This test uses some basic microbiology to answer some important questions. It’s still used all the time in clinics to determine if a certain drug will work against a patient’s disease or not. We wanted to use this experiment for our first workshop to highlight the ability of any person to do such a powerful test, and bring attention to the important issue of antibiotic resistance. As our use of antibiotics increases, resistance to these drugs also increases. Searching for new antibiotics is an important area of research in which citizen scientists can participate. Just like bacteria are all around us, so are potential antibiotics.

We’re planning several more future bio workshops to involve the public in simple, empowering science, so keep an eye out!