Skip to content

Module 4

Sensing the body

About Citlali Hernandez
https://turbulente.net/
citlali.hs@gmail.com

Resources_

Recommended books_

Concepts introduced during the session_

What is the body: medium or tool of expression AND is also a topic of research. Citlali does performance art. The body is political and the body has something to tell. If the body is a territorry that we inhabit, how do we connect with it. Data can be translated into other types of signals. Emojis as a way of communication. What makes us emotional? physical event, social interactions, remembering/imagining specific event, etc. Affective Computing Robotics Madeline Ganon - make the robot move using a natural language that we can understand Expresivity in human environments: Digit - Agility Robotics

Some inspiring projects_

  • Live coding: visuals sharing code in real time. Normally it has 2 screens, i coding for coding and the other for visuals. 2 mobile phones, with an app with all the information about mobile phones (orientation, apps, etc).
  • Online algorithmic video performance: project Metonimia del cuerpo
  • Body algortihms: project cinestesia. Using a nano arduino with conductive fabrics. Exploracion de movimientos con artefactos robóticos.
  • Emodying memes small raspberry: pi zw has wifi and can connect a camera

Hands-on experimentation: Prototype your wearable and experiment with your training model

I joined forces with Jorge to develop a glove as a soft sensor prototype. We used Velostat as the main material. Velostat is a variable resistor that lets electricity flow through a system or not, functioning similarly to a potentiometer. It allows modulation of how electricity flows within a circuit. button1

The circuit

circuit

We designed this glove to measure the pressure an artist might apply to a material, such as clay. We sewed the neoprene to create a firm and consistent glove. Then, we connected the glove to an OSC (Open Sound Control) to receive the input and to be able to train the model. The glove sends data each time it touches the material, allowing for precise measurement of pressure. button1 button1 button1

Pressure Sensor CODE

  • Code for reading the change in values when pressing against the material such as.

int sensorPin = A0; // select the input pin for the potetntiometer

int sensorValue = 0; // variable to store the value coming from the sensor

int pinD = D8;
int b = 0;


void setup() {
  // declare the ledPin as an OUTPUT:
  pinMode(pinD, OUTPUT);
  pinMode(sensorPin, INPUT);
  Serial.begin(9600);

}

void loop() {
  // read the value from the sensor:
 // b = !b; // asigno el contrario
digitalWrite(pinD, HIGH);
sensorValue = analogRead(sensorPin) ;
Serial.println(sensorValue) ;
delay (200);
}
button1 button1

As an experiment, we trained the model to recognize a “surprised” face when the material is touched, adding an element of interaction and feedback to the process.

About Wekinator
Wekinator is a machine learning platform that enables users to create interactive systems by using algorithms to recognize patterns in real-time input data. It is designed to be accessible to non-experts and allows the rapid development of interactive systems without needing to write code. Wekinator can work with various inputs such as sensors, cameras, and microphones, and it outputs to different software and hardware platforms, enabling the creation of music, games, and interactive art installations. wekinator

FOTOS con sophie con wekinator

Developing the soft sensor glove showcased the versatility of soft robotics in creative applications. It demonstrated how responsive simple materials can be used to capture intricate details of artistic processes. This project has motivated me to delve deeper into the field of soft robotics and explore its diverse applications in both art and other domains.

Extra personal notes:

FaceOSC x Wekinator_ Configuration: 8338 66 xy -> 132 OSC message /raw output: 1 IP can be the addres of your barduino or localholst

About the answers: 1 message answer - We are going to train wekinator if we are in 1 move to 0. - We can also train 3 different emotions (1,2,3) 2 message answer - We can also say while x =0 and y=1, then Z, while x=1 and y =1 then W - Type custom > configure> osc message> /cara > with> integer (which means 2 answers)

Module 5

Extending the body

This module challenged us to create visuals to experience our trained model.

First, I explored Wekinator and its possibilities. Then, we also got to know MAX.

About MAX
Max, also known as Max/MSP, is a visual programming language for music and multimedia. Developed by Cycling ‘74, it is used by composers, performers, software designers, researchers, and artists for creating interactive audio and visual applications. Max provides a graphical user interface where users can connect objects to create custom signal processing, user interfaces, and other multimedia applications. It’s highly versatile and supports various types of media, making it a popular tool in the fields of electronic music, multimedia art, and sound design.

In the end, I realized that I felt much more comfortable using TouchDesigner with Mediapipe for extending the body. I was already in the process of understanding the model and how to make TouchDesigner work. It was a very fun process to play with this hand-tracking recognition. It recognized each part of my fingers, so I managed to set the direction of the resized photo from the tip of my ring finger.

Gif that shows TD