Mechatronics Art: Interface

This page documents my final project for DXARTS 471: Introduction to Mechatronics Art.

AVL-4274

Introduction

Throughout my life, I’ve always been fond of robots. Similar to how most people are far more emotionally attached to dogs in a movie than people, I am the same with robots. Below are a few clips of robots who, despite their good intentions, are punished. WARNING: Content is violent and saddening.

The piece: Meet AVL-4274

Speculation

This piece envisions a future in which the issues of mass incarceration, global warming, and the ethics of artificial intelligence are exacerbated to the extent where they intersect. AVL is a robot who was part of a farming robot colony, but malfunctioned and ended up destroying a few crops. However, AVL is extremely fond of plants - it’s all he cares about. Unfortunately, due to his glitches, he isn’t able to take care of them well. The engineers in the world have ceased to exist - they are just unempathetic technicians now who care only about efficiency. Outdated notions of corrections, which involve raw imprisonment, still exist in this world. Therefore, AVL was never fixed, he was simply locked up. Despite his good intentions, despite all the potential he has to sustain the dying world, he was just locked up, and forgotten. Now, AVL simply watches whatever plants he can from within his cage. He monitors them, but he’s chained up and can’t water them. He’s grateful for the kind souls who water the plants, whether that be the clouds above, or a passerby. He’s far more grateful if that passerby shows empathy.

This story isn’t far-fetched. Over two million human beings are incarcerated, with a recidivism rate of over 50% nationally. Our society is one that throws away that which doesn’t work even if fixing it takes less energy. This mentality applies directly to global warming, and how our throw-away society has polluted this Earth. It seems like something inextricable from our nature; perhaps robots are the only beings with absolutely pure intentions of maintenance and sustained care. However, the ethics of imprisonment are unjustified, and the ethics of intelligent robots are not established; it would make sense if the latter inherited the former philosophy: lock up robots instead of fixing them. Empathy is the key factor, the ultimate solution, to all these issues; it’s in short supply today, so perhaps AVL can demonstrate what it’s like to see through a perspective of pure, sustained, focused, good intention behind bars.

Interface

The goal of the piece is to take empathy to its furthest extent: literally putting you inside the robot software’s perspective, and fixing the robot’s glitched software through empathy. Ever since I began practicing fundamental engineering principles, I’ve always believed that empathy is the basis of every good engineering solution. Complex problem solving requires understanding the perspectives of multiple stakeholders. It’s honestly what the world needs more of. Unfortunately, given the issues described above, we’re in short supply. This interface implements empathy to the best of its ability. Ideally, I’d have some kind of transfer-of-consciousness like technology which would allow motor neurons to control the robot and electrodes to stimulate sensory neurons (like retinal neurons to see what the robot sees). Unfortunately, this level of empathy is way in the future. An immersive virtual reality interface like this one does its best to simulate empathy; the hardware connection between the VR game and the electronics in the cage allow the VR environment to affect the real world, making the simulation far more realistic.

As a sidenote: this robot was definitely inspired by these pure, innocent characters:

  • Wall-E - taking care of the plants and looking super cute

  • Chappie - creating a transfer-of-consciousness-like interface

  • BT-7274 - having pure intentions (3 protocols) and a cool name

Function

The head is an old black and white TV, which connects to a camera mounted on the top of the cage. The camera direction is controlled by a servo motor. There’s a soil moisture sensor in each plant. The camera pans from plant to plant, “scanning” each one (reading the moisture levels) and outputting its status (red = dry, yellow = good, green = currently being watered). The moment a plant is being watered, the camera will look at it and express joy (through LEDs or otherwise in a future implementation). The code omits delays, and replaces the functionality of delays with moisture level readings (in a parallel manner which doesn’t require interrupts).

The virtual reality interface is implemented by the Microsoft Mixed Reality headset, a VR-ready laptop, Unity, and the Mixed Reality Toolkit for Unity (on Github). The hardware connection is implemented by a Serial connection between Unity and an Arduino with a servo. After the game asks the user to complete tasks to fix the robot, the game instructs the user to open the cage by touching a handle on the back gate; when that handle is touched, a Serial message is sent to the Arduino that the cage should be opened, and the Arduino turns the servo to let the cage fall.

Circuit Schematic

(plus an additional servo motor, on another arduino, exactly as wired below)

specobj_bb.png

Code

Code for VR Interface

Kudos to the Microsoft Mixed Reality Toolkit for Unity, and various free non-commercial 3D models online.

Opening the Cage (Arduino): 

#include <Servo.h>
Servo myservo;
int pos = 0; // position of servo 

int data;

void setup() {
  Serial.begin(9600);
  pinMode(13, OUTPUT);
  myservo.attach(6);
  myservo.write(1); // reset the cage to closed
}

void loop() {
  if (Serial.available()) {
    data = Serial.read();
    if (data == 'A') {
      digitalWrite(13, HIGH);
      myservo.write(90); // open the cage
    } else {
      digitalWrite(13, LOW);
    }
  }
}

Opening the Cage (Unity):

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.IO.Ports;

public class collide_test : MonoBehaviour {

    public SerialPort serial = new SerialPort("COM5", 9600);
    private bool rotateBool = true;
    
    private void OnTriggerEnter(Collider other)
    {
        if (serial.IsOpen == false)
        {
            serial.Open();
        }

        if (rotateBool)
        {
            serial.Write("A");
            rotateBool = false;
        }
        else
        {
            serial.Write("B");
            rotateBool = true;
        }
    }
}

Plant Animation Trigger (Unity): [there are a few other small animations like this] 

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class animController : MonoBehaviour {

    public Animator anim; 

    private void OnTriggerEnter(Collider other)
    {
        anim.Play("plant-1-animation");
        Debug.Log("Seed planted!");
        cageScript.plantCount++;
    }
}

Code for Robot

Main Code (Arduino):

/*
 * The goal of this code is to pan the servo motor whilst reading 
 * sensor data, and react to that sensor data ASAP (as if they were 
 * occurring in parallel). This is done by substituting the delay between
 * servo turns with sensor checks. 
 * 
 * Now, we're checking the status of three water sensors by panning to each one. 
 * Meanwhile, if any one of them becomes wet, the camera will react by looking
 * at that plant for a short duration. 
 * 
 * Now, the reaction entails LED responses.
 */

#include <Servo.h>
Servo myservo;
int pos = 0; // position of servo  

// sensor IDs of the plants
const int S1 = A0;
const int S2 = A1; 
const int S3 = A2;

// angle values of the plants
const int P1 = 58;
const int P2 = 83;
const int P3 = 110;

// soil moisture sensor thresholds
const int WATER_THRESH = 750;
const int HUMID_THRESH = 500;
const int DRY_THRESH = 300;

// LEDs
const int R = 9;
const int G = 10;
const int Y = 11;

void setup() {
  Serial.begin(9600);
  myservo.attach(6);
  pinMode(R, OUTPUT);
  pinMode(G, OUTPUT);
  pinMode(Y, OUTPUT);
}

void loop() { 
  
  pan(P1, S1, P2);
  pan(P2, S2, P3); 
  pan(P3, S3, P2); 
  pan(P2, S2, P1);

}

void pan(int plant1, int sensor1, int plant2) {
  scan(plant1, sensor1);
  // pan from plant1 to plant 2
  if (plant2 > plant1) {
    for (int i = plant1; i <= plant2; i++) {
      myservo.write(i);
      check();
    }
  } else {
    for (int i = plant1; i >= plant2; i--) {
      myservo.write(i);
      check();
    }
  }
}

/* scans a single plant for approximately a second
 * whilst panning 
 */
void scan(int plant, int sensor) {
  myservo.write(plant);
  for (int i = 0; i <= 50; i++) {
    check();
    status(sensor);
  }
}

/*
 * Takes a little over 30ms to accomplish this
 * 
 * checks all the plants for watering
 */
void check() {
  for (int i = 0; i <= 100; i++) {
    checkPlant(P1, S1);
    checkPlant(P2, S2);
    checkPlant(P3, S3);
  }
}

// checks if an individual plant is
// being watered
void checkPlant(int plant, int sensor) {
  if (analogRead(sensor) > WATER_THRESH) {
    react(plant, G);
  } 
}

// reacts to a plant being watered
void react(int plant, int reactLED) {
  resetLEDs();
  myservo.write(plant);
  delay(1000);
  for (int i = 0; i < 5; i++) {
    Serial.println("Water!!!!");
    digitalWrite(reactLED, HIGH);
  }
  delay(2000);
  digitalWrite(reactLED, LOW);
}

// reports the status of the given plant 
// with LED 
void status(int sensor) {
  resetLEDs();
  if (analogRead(sensor) > HUMID_THRESH && analogRead(sensor) <= WATER_THRESH) {
    digitalWrite(Y, HIGH);
  } else {
    digitalWrite(R, HIGH);
  }
}

void resetLEDs() {
  digitalWrite(R, LOW);
  digitalWrite(G, LOW);
  digitalWrite(Y, LOW);
}

Calibration Code - Servo Motor Angle (Arduino):

/*
 * Helps callibrate the camera angle
 */

#include <Servo.h>
Servo myservo;
int pos = 0; // position of servo 

void setup() {
  Serial.begin(9600);
  myservo.attach(6);
}

void loop() {
  int angle = Serial.parseInt();
  if (angle != 0) {
    myservo.write(angle);
  }
}

Calibration Code - Water Sensor Check (Arduino):

/*
  MODIFIED BY: Anand Sekar (11/12/18) 
  
  # Example code for the moisture sensor
  # Editor     : Lauren
  # Date       : 13.01.2012
  # Version    : 1.0
  # Connect the sensor to the A0(Analog 0) pin on the Arduino board
  
  # the sensor value description
  # 0  ~300     dry soil
  # 300~700     humid soil
  # 700~950     in water
*/

void setup(){
  Serial.begin(57600);
}

void loop(){
  Serial.print("MSV 1:");
  Serial.println(analogRead(A0));  
  Serial.print("MSV 2:");
  Serial.println(analogRead(A1));
  Serial.print("MSV 3:");
  Serial.println(analogRead(A2));
  delay(100);
}
Anand Sekardxarts-471