Let's Interact

To commemorate my dinner with family and impending in-laws on the eve of my wedding, I simultaneously logged my galvanic skin response (GSR) and which way I was looking using a three-axis accelerometer mounted on my glasses (overkill, I know), which involved having my computer in my lap while wearing wired glasses and trying to carry on eleven conversations at once! It was a bold plan to discover whether I responded predictably (and differently) to my own family and my wife’s and it seemed to be working until three minutes in, an over-enthusiastic waitress howled at a joke nearby, which made me jump and pulled the wires out of my GSR sensor. Without a hot glue gun, I was helpless to continue, so I closed up the computer, resolved to continue next week, and enjoyed my beer.

Please note that my accelerometer and ribbon wire matched my shirt. Because that’s how I roll.

Click for a sample of the data I logged.

1254093432844,3,454,508,511
1254093432956,4,451,504,511
1254093433064,4,456,505,510
1254093433195,4,459,506,511
1254093433307,4,458,507,511
1254093433419,4,461,509,509
1254093433531,3,461,509,510
1254093433643,3,461,510,510
1254093433755,4,461,510,511
1254093433864,4,461,512,508
1254093433995,3,461,515,510
1254093434107,4,460,513,510
1254093434220,4,460,515,508
1254093434331,3,462,516,506
1254093434444,3,460,517,505
1254093434555,4,463,516,509
1254093434663,5,460,517,512
1254093434795,4,465,522,510
1254093434907,4,461,523,511
1254093435019,4,463,524,510
1254093435131,3,462,523,507
1254093435243,3,461,518,508
1254093435355,4,463,520,509
1254093435463,4,463,519,509
1254093435595,3,465,520,505
1254093435707,4,464,522,504
1254093435819,1,466,519,507
1254093435931,2,465,521,506
1254093436043,2,466,523,505
1254093436155,1,465,521,503
1254093436263,1,462,519,504
1254093436395,4,464,521,505
1254093436507,6,466,524,505
1254093436620,3,465,521,503
1254093436731,6,465,519,504
1254093436843,5,464,521,504
1254093436956,3,468,522,505
1254093437083,6,468,521,504
1254093437195,3,466,521,505
1254093437307,5,465,520,503
1254093437419,3,465,520,506
1254093437532,1,467,520,505
1254093437643,2,462,519,505
1254093437751,2,460,518,508
1254093437884,2,461,520,508
1254093437995,3,463,522,506
1254093438107,4,464,520,508
1254093438219,7,463,519,507
1254093438331,6,463,521,506
1254093438443,7,464,522,505
1254093438551,6,465,521,509
1254093438682,3,465,522,505
1254093438794,5,466,524,503
1254093438906,4,466,519,502
1254093439018,4,465,517,498
1254093439130,3,465,519,494
1254093439242,4,469,519,495
1254093439350,4,468,519,495
1254093439482,6,467,520,492
1254093439594,5,466,522,492
1254093439707,5,465,524,493
1254093439818,6,463,526,496
1254093439930,8,466,528,506
1254093440042,6,467,533,502
1254093440170,6,464,527,505
1254093440282,6,466,527,505
1254093440394,7,467,525,505
1254093440506,7,467,526,502
1254093440619,7,465,524,501
1254093440730,7,455,515,504
1254093440843,10,451,518,504
1254093440971,10,452,515,504
1254093441082,7,452,514,504
1254093441195,2,445,510,505
1254093441306,2,446,511,504
1254093441419,7,445,508,503
1254093441531,4,447,507,503
1254093441659,5,448,505,503
1254093441770,3,448,506,497
1254093441883,4,449,506,490
1254093441994,3,450,501,486
1254093442107,3,451,504,481
1254093442219,4,448,503,476
1254093442331,3,451,504,482
1254093442461,4,451,505,481
1254093442571,4,451,507,479
1254093442683,5,451,504,483
1254093442794,4,452,504,480
1254093442907,4,453,506,482
1254093443022,6,452,506,483
1254093443126,3,455,510,482
1254093443259,6,456,510,483
1254093443371,4,456,512,485
1254093443483,5,457,511,485
1254093443596,3,457,511,485
1254093443707,3,458,511,486
1254093443819,4,459,511,485
1254093443926,3,459,512,485
1254093444058,2,461,510,485
1254093444170,4,462,513,485
1254093444281,4,462,515,484
1254093444393,4,462,514,486
1254093444506,8,462,514,484
1254093444618,6,464,514,485
1254093444725,6,465,514,485
1254093444858,5,464,514,486
1254093444970,5,466,514,483
1254093445082,4,464,513,484
1254093445194,8,465,514,482
1254093445306,4,468,514,481
1254093445418,7,467,512,483
1254093445546,3,467,511,481
1254093445658,4,467,513,480
1254093445769,5,469,510,480
1254093445882,4,470,510,482
1254093445994,5,468,508,484
1254093446106,4,472,511,484
1254093446218,5,469,510,485
1254093446326,4,471,514,483
1254093446458,7,473,518,484
1254093446570,7,472,517,486

My glasses with accelerometer:

My trusty Arduino, wired up to lie flat:

My homemade GSR sensor:

DoorSob is a door that doesn’t want you to leave a room. A Processing sketch allows the playback on a screen of a human face’s progression from ecstatic happiness to utter misery to be controlled by a potentiometer activated by turning a doorknob. Depending on the state of the face (and by extension, the potentiometer), a voice repeats either “yes” or “no” more or less emphatically. The volume of the voice and the brightness of the face are affected by the amount of ambient light falling on a photoresistor. My intention is to install the photo sensor next to a doorknob so that when someone puts their hand on the knob, it blocks the light and brightens the screen so that the video is visible and the sound is audible. The pot is moved by the knob, so that as a person starts to move the knob to open the door, it reacts, getting more and more distraught the closer the person is to opening the door (and leaving the room).

A week reading about the location of consciousness (apparently behind the eyes according to most people with a minority locating it in their upper chest) and our dubious awareness of our own perceptual and cognitive shortcomings has left me scratching my head. I haven’t done huge amounts of reading in the cognitive sciences, but I’ve done enough to feel that Julian Jaynes’s arguments against the necessity of consciousness in “Origins of Consciousness” and Dan Ariely’s TEDtalk about the limits of free will are a series of cleverly erected straw men. I’ve never heard anyone claim that consciousness is as ubiquitous and constant as implied in Jaynes’ refutation, nor do I buy Ariely’s claims that people’s laziness and susceptibility to influence constitute proof of sensory and cognitive deficiencies. The self-awareness and introspection that these men refer to as consciousness seems to me a response to complicated social structures. It’s essential not to the survival of the individual but to the survival of the group. It’s no wonder then that it tends to lag a little when considered in conjunction with the senses.

And it was thinking about the conniving, scheming, backroom dealing, weighing, and planning to which consciousness presumably emerged as a response that I started thinking about all the unconscious social and physical cues that US Weekly body language experts and NLP practitioners are constantly harping on about. We like it when people laugh at our jokes and praise us, we don’t for the most part like making people unhappy or getting yelled at. How would we feel if everyday objects called our attention to the actions we perform unconsciously?