Processing Introduction

Processing

processing logo

Processing is an open-source programming language and development environment that is designed for creating visual art and interactive design. It is built on top of the Java programming language and is intended for beginners and non-programmers who want to learn programming for the purpose of creating interactive graphics, animations, and other visual media. Processing is often used in fields such as visual art, graphic design, and data visualization and it allows users to create interactive sketches that respond to user input and generate real-time graphics. The language is known for its simplicity and ease of use, making it a popular choice for artists and designers looking to create interactive digital works.

Here's the link for downloading it: https://processing.org/download/

Say Hello

setup() function:

only run once.

draw()function:

will continuously repeat the action inside of the {}

1void setup(){
2size(400,400);
3println("Hello!Nan");
4
5}
6void draw(){
7println("welcome to interaction station!");
8}

copy-paste the code above and observe the message from the console.

Now, let's try to draw something

 1void setup(){
 2size(1200,800);
 3background(0);
 4
 5}
 6void draw(){
 7stroke(250,250,140);
 8line(0,0,600,400);
 9//line(width/2, height/2, mouseX, mouseY);
10}

Sketch --> Run

Sketch --> Tweak

Sketch --> Present

Full screen!

 1void setup(){
 2//size(1200,800);
 3background(0);
 4fullScreen();
 5}
 6void draw(){
 7stroke(250,250,140);
 8//line(0,0,600,400);
 9line(width/2, height/2, mouseX, mouseY);
10}

Draw

 1void setup(){
 2
 3size(1200,800);
 4background(0); 
 5//fullScreen();
 6
 7}
 8void draw(){
 9
10background(0);
11noStroke();
12fill(0,80,80);
13ellipse(width/2,height/2,200,200);
14strokeWeight(5);
15stroke(250,250,140);
16line(width/2, height/2, mouseX, mouseY);
17
18}

1. Try to change the background(0); to the end of void draw() and see what happens.

2. Try to use noStroke(), fill(), strokeWeight().

3. Don't forget to use the reference page:

https://processing.org/reference/

4. Be patient carefully read the console and error message.

5. Let's use a simple if statement to clear out the lines when we mouse is pressed.

 1void setup(){
 2
 3size(1200,800);
 4background(0); 
 5//fullScreen();
 6
 7}
 8void draw(){
 9
10noStroke();
11fill(0,80,80);
12ellipse(width/2,height/2,200,200);
13strokeWeight(5);
14stroke(250,250,140);
15line(width/2, height/2, mouseX, mouseY);
16
17if(mousePressed){
18
19background(0);
20}
21
22}

Communication with Arduino

There are two ways to get processing sketches communicating with microcontrollers.

1. The first one is to use the microcontroller as an HID (Human Interface Device), like a mouse or keyboard.

We won't go into detail today about it, but you are interested check out Arduino HID

2. Sometimes, you want to send data directly from a microcontroller to Processing
In this case, we need to do write few lines of code in both Arduino and processing.

Send out data Arduino IDE

N.B. The following steps use the Adafruit Circuit Playground Express, a board with integrated sensors that comes really handy for wearables' projects. You can easily re-adapt it to be used to send sensor data from an Arduino to processing. You can find a more specific guide to do so here.

 1#include <Adafruit_CircuitPlayground.h>
 2float X, Y, Z;
 3void setup() {
 4 Serial.begin(9600);
 5 CircuitPlayground.begin();
 6}
 7 
 8void loop() {
 9 
10  int leftButton = CircuitPlayground.leftButton();
11  X = CircuitPlayground.motionX();
12  Y = CircuitPlayground.motionY();
13  Z = CircuitPlayground.motionZ();
14  
15  int xVal=map(X,-9,10,0,1920);
16 //somtime, if you move too fast the value will turn into a negative number.
17 //if the value is smaller than 0, we can multiply -1 (*=). and converts it back to a positive number
18  if(xVal<0){xVal*= -1;
19    }
20  int yVal=map(Y,-9,10,0,1080);
21  if(yVal<0){yVal*= -1;
22    }
23  Serial.print(xVal);
24  Serial.print("\t"); //instert a tap in between
25  Serial.print(yVal);
26  Serial.print("\t");//instert a tap in between
27  Serial.println(leftButton);
28 
29  delay(10);
30 }

Receive the value in Processing

1. First let's select the right serial Port

The communication between Arduino and processing is done by serial port(USB port).

When Arduino executing "Serial.print", it prints messages to the serial port, and processing needs to read the message from the serial port.

However, everyone have different USB port layouts, we need to know which usb port wen need to read.

 1import processing.serial.Serial;
 2//select serial port and set bauds rate
 3static final int PORT_INDEX=2,BAUDS=9600;
 4int[] vals= {};
 5void setup(){
 6  noLoop();
 7  frameRate(18);
 8  final String[] ports = Serial.list();
 9  printArray(ports);
10 
11}

In this code, we are printing all the ports in setup() function that's why it only runs once remember?

Now you can see a list of USB ports in your console, find out which port is your Adafruit circuit playground express.

Normally it is a /dev/cu.usbmodem.In my case it is "[2] "/dev/cu.usbmodem11201".

So I need to go to second line of my code:

1static final int PORT_INDEX=2,BAUDS=9600;

Make sure that the PORT_INDEX=2

For example, if YOUR Port is 3, you need to make sure that your PORT_INDEX=3

2. Now Processing is ready to receive messages!!!

 1import processing.serial.Serial;
 2//select serial port and set bauds rate
 3static final int PORT_INDEX=2,BAUDS=9600;
 4// create an array, a list of data will be used to store values from snesors.
 5int[] vals= {};
 6 
 7void setup(){
 8  noLoop();
 9  frameRate(12);
10  final String[] ports = Serial.list();
11  printArray(ports);
12  new Serial(this, ports[PORT_INDEX], BAUDS).bufferUntil(ENTER); 
13}
14 
15void draw(){
16 
17   print("x=");
18   println(vals[0]); 
19   print("Y=");
20   println(vals[1]); 
21   print("Button=");
22   println(vals[2]); 
23    }
24void serialEvent(final Serial s) {
25// split the string from the serial port into 3 integers and store it in the list.
26  vals = int(splitTokens(s.readString()));
27  redraw = true;
28}

Add the lines back to this code

Now, please try to add the animated lines back to the code and use X and Y values from the motion sensor as mouse positions to draw the line. As well as CPX button clicks to clear out the lines.

 1import processing.serial.Serial;
 2//select serial port and set bauds rate
 3static final int PORT_INDEX=2,BAUDS=9600;
 4// create an array, a list of data will be used to store values from snesors.
 5int[] vals= {};
 6 
 7void setup(){
 8  noLoop();
 9  frameRate(18);
10  final String[] ports = Serial.list();
11  printArray(ports);
12  new Serial(this, ports[PORT_INDEX], BAUDS).bufferUntil(ENTER); 
13  
14  size(1200,800);
15  background(0);
16}
17 
18void draw(){
19   
20   print("x=");
21   println(vals[0]); 
22   print("Y=");
23   println(vals[1]); 
24   print("Button=");
25   println(vals[2]); 
26
27 //the ellipse  
28   noStroke();
29   fill(0,80,80);
30   ellipse(width/2,height/2,200,200);
31   
32   
33// the lines
34   strokeWeight(5);
35   stroke(250,250,140);
36 //  line(width/2,height/2,mouseX,mouseY);
37   line(width/2,height/2,vals[0],vals[1]);
38   if(vals[2]>0){
39   background(0);
40   }
41
42    }
43void serialEvent(final Serial s) {
44// split the string from the serial port into 3 integers and store it in the list.
45  vals = int(splitTokens(s.readString()));
46  redraw = true;
47}

Draw a image

Let's try to draw an image with PImage!

https://processing.org/reference/PImage.html

First, we need to create empty processing and save it somewhere on your computer.

In this case, we created a sketch folder, and inside of the sketch folder, we need to create a new folder called "data" and save an image (jpg, png, gif,tga)in the "data" folder.

 1PImage images;
 2
 3void setup() {
 4
 5 fullScreen();
 6 images=loadImage("run.gif");
 7}
 8
 9void draw() { 
10 // background(0);
11  image(images,0,0, images.width, images.height );  
12}

Image sequence

Step two:

Let's make it move!

In order to do that we need to use a sequence of images. Here is how to use for loop to load a sequence of images:

1 int numImages=120;
2 PImage[] images= new PImage[numImages];
3 
4 for (int i = 0; i < numImages; i++) {
5   String imageName = "run-" + nf(i, 5) + ".png";
6  // images[i] = loadImage(imageName);
7   println(imageName);
8   println(nf(i,5));
9  }

Now we understand how the code is loading all the images sequence, we can start drawing the image sequence:

One important thing!

Make sure you put the data folder in the right place: the same folder where your processing sketch is saved.

Like this:

 1int numImages=23;
 2int startNumImages=1;
 3PImage[] images= new PImage[numImages];
 4int imgInSeq=1;
 5
 6void setup() {
 7 // size(1920, 1080);
 8 //let's give it a frame rate value so that we can change it later:-))
 9 frameRate(18);
10 fullScreen();
11
12 for (int i = 0; i < numImages; i++) {
13   String imageName = "run-" + nf(i, 5) + ".png";
14   images[i] = loadImage(imageName);
15  }
16}
17
18void draw() { 
19 background(0);
20//we can use % (modulo) to cycle through the images. from 1- to numImages
21imgInSeq= (imgInSeq+1) % numImages;
22image(images[imgInSeq],0,0,images[imgInSeq].width,images[imgInSeq].height);
23
24}

Now, let's make it interactive!

Remember the first example where we use mouse position to draw lines?

we can also use mouseX position to control the animation.

Let's comment out:

1imgInSeq= (imgInSeq+1) % numImages;

and replace it with :

1int imgInSeq = (int)map(mouseX, 0, width, 0, numImages - 1);

Now, the images always display from the top left corner. Let's move it to the center:

the center point should be:

x=(display window's width - the image width) divided by 2

y=(display window's Hight- the image height) divided by 2

1image(images[imgInSeq], (displayWidth-images[imgInSeq].width)/2, (displayHeight-images[imgInSeq].height)/2, images[imgInSeq].width, images[imgInSeq].height );

Sensor controlled image sequence

Let's use higher resolution images and try to control the animation with the light sensor from circuit Playground Express. We can use the lego image sequence from the data folder on teams.

1. Use the light sensor from CPX to animated the image sequences.

Send out light sensor value from Arduino

 1#include <Adafruit_CircuitPlayground.h>
 2
 3int value;
 4
 5void setup() {
 6  Serial.begin(9600);
 7  CircuitPlayground.begin();
 8}
 9
10void loop() {
11  value = CircuitPlayground.lightSensor();
12  
13 // Serial.print("Light Sensor: ");
14  Serial.println(value);
15  delay(50);
16}

receive and draw in Processing:

 1import processing.serial.Serial;
 2static final int PORT_INDEX=2,BAUDS=9600;
 3int [] vals= {};
 4
 5int numImages=97;
 6int startNumImages=1;
 7PImage[] images= new PImage[numImages];
 8int imgInSeq=1;
 9
10void setup() {
11  size(1920,1080);
12  noLoop();
13  final String[] ports = Serial.list();
14  printArray(ports);
15  new Serial(this, ports[PORT_INDEX], BAUDS).bufferUntil(ENTER);
16
17 //let's give it a frame rate value so that we can change it later:-))
18 frameRate(18);
19 //fullScreen();
20
21 for (int i = 0; i < numImages; i++) {
22   String imageName = "lego" + nf(i, 5) + ".png";
23   images[i] = loadImage(imageName);
24  }
25}
26
27void draw() { 
28  background(0);
29  println(vals[0]);
30
31  if( vals[0]>549){
32    
33   vals[0]=550;
34}
35  imgInSeq = (int)map(vals[0], 0, 550, 0, numImages - 1);
36  if( vals[0]>549){
37    
38   vals[0]=550;
39}
40
41  image(images[imgInSeq],0,0,images[imgInSeq].width,images[imgInSeq].height);
42
43}
44
45void serialEvent(final Serial s) {
46  vals = int(splitTokens(s.readString()));
47  redraw = true;
48}

Use the "if" statement

Now let's use other sensor values and an "if " statement to control the image sequences.

In this case, we can try to use the sound level value from CPX's mic. And say: if the sound input is louder than a certain level, play the image sequence. else, pulse and do nothing.

Send out sound value from Arduino

 1#include <Adafruit_CircuitPlayground.h>
 2
 3int value;
 4
 5void setup() {
 6  Serial.begin(9600);
 7  CircuitPlayground.begin();
 8}
 9
10void loop() {
11  // Take 10 milliseconds of sound data to calculate
12  value = CircuitPlayground.mic.soundPressureLevel(10);
13  
14 // Serial.print("Sound Sensor SPL: ");
15  
16  int newVal=map(value,58,102,0,94);
17if (newVal<0){
18  newVal=0;
19    }
20  Serial.println(newVal);
21
22  delay(90);
23}

Receive and draw in Processing

 1import processing.serial.Serial;
 2static final int PORT_INDEX=6,BAUDS=9600;
 3int [] vals= {};
 4
 5int numImages=97;
 6int startNumImages=1;
 7PImage[] images= new PImage[numImages];
 8int imgInSeq=1;
 9
10void setup() {
11  noLoop();
12  final String[] ports = Serial.list();
13  printArray(ports);
14  new Serial(this, ports[PORT_INDEX], BAUDS).bufferUntil(ENTER);
15
16 //let's give it a frame rate value so that we can change it later:-))
17 frameRate(20);
18 fullScreen();
19
20 for (int i = 0; i < numImages; i++) {
21   String imageName = "lego" + nf(i, 5) + ".png";
22   images[i] = loadImage(imageName);
23  }
24}
25
26void draw() { 
27 background(0);
28 println(vals[0]);
29
30if (vals[0]>40){
31imgInSeq= (imgInSeq+1) % numImages;
32}
33
34image(images[imgInSeq],0,0,images[imgInSeq].width,images[imgInSeq].height);
35
36}
37
38void serialEvent(final Serial s) {
39  vals = int(splitTokens(s.readString()));
40  redraw = true;
41}

What if you want the image sequence to go back to the first frame after each trigger? what do you need to add to this code?

1if (vals[0]>40){
2imgInSeq= (imgInSeq+1) % numImages;
3}else{
4  imgInSeq=1;
5}

And after each sound trigger, if you want processing to finish playing the entire sequence before go back to the first frame you can take a look at this:

1if (vals[0]>40){
2imgInSeq= (imgInSeq+1) % numImages;
3}else{
4  if(imgInSeq>5 && imgInSeq<97){
5  imgInSeq= (imgInSeq+1) % numImages;
6}else{
7  imgInSeq=1;
8  }
9}

Use other GIFs

Now, let's try it by search online and find your own GIF and make your own image mouse-controlled animated images.

You can use a free online tool GIF frame extractor to split the gif file in image sequence:

https://ezgif.com/split

About renaming the files in sequence:

If you are using macOS, you can just select all the images and right-click select "rename" and chose "format" to rename all your files in sequence.

Like this:

However:

If you are using Window 10 after you rename a sequence of files, they might look like this: run-(1).png run-(2).png run-(3).png ...

Don't panic, you can change the code from

1for (int i = 0; i < numImages; i++) {
2   String imageName = "run-" + nf(i, 5) + ".png";
3   images[i] = loadImage(imageName);
4  }

into

1for (int i = 0; i < numImages; i++) {
2   String imageName = "run-(" + i + ").png";
3   images[i] = loadImage(imageName);
4  }

---

If you are interested you can also check out how to write from Processing to Arduino. You can find a tutorial here