Lab 5: Talk to Me

The questions below are due on Saturday July 22, 2017; 11:59:00 PM.


You are not logged in.

If you are a current student, please Log In for full access to this page.

Talking Heads

Goals:

Today we're going to do some speech synthesis with our Raspberry Pi...and then use this new capability to have our Raspberry Pi tell us how slow/fast we are at responding to light and sound stimuli.

1) Getting Started

First let's download the espeak library onto your Pi found HERE (from here: https://github.com/relsi/python-espeak). Then, open up a terminal (found under Raspberry>Accessories>Terminal). Then in there type the following:

cd Downloads

Then:

unzip python-espeak-master.zip

Then

cd python-espeak-master

Then

sudo python3 setup.py install

After a bunch of text flies by you should be all set. Then do the following:

  • Make sure your system can play music through the LM386-based audio amplifier like last week!
  • Plug your audio amplifier into your Raspberry Pi
  • Set the volume to a reasonable level.
  • Open up a Python Instance, and create a new file. Copy/Paste in the following code and then run it:

import espeak
es = espeak.ESpeak()
es.say("Hello MOSTEC Student")

You should have heard the phrase "Hello, MOSTEC student." be spoken by a creepy man with a creepy English accent. Pressing submit ran these two simple lines of Python Code:

Now what this code did was the following:

import espeak

This line imports a Python library called espeak that let's Python control the espeak program (which is a speech-synthesis program and pretty neat). The next line creates an instance of a speech-synthesizer:

es = espeak.ESpeak()

And the final line uses the speech synthesizer, telling it to greet you:

es.say("Hello MOSTEC Student")

You'll always need to create a espeak object (and you can call it anything you want) before being able to make the computer speak.

This was accomplished by changing the voice attribute of the espeak object like so:

es.voice = "whisper"

You can also change the pitch and intonation of the voice by changing various things...for exampmle you can change the voice to be f4 (female pitch 4)

es = espeak.ESpeak()
es.voice = "f4"
es.say("Hi there!")

We can also make multiple simultaneous espeak objects like so:

person1 = espeak.ESpeak() #make one person
person2 = espeak.ESpeak() #make another person
person1.voice = 'de' #make the person German (deutsch)
person1.speed = 100 #speak slowly (1 through 500 for fast...1 is slow)
person2.voice = "f3" #female tone 3

person1.say("Hello. How are you?")
person2.say("I'm fine thank you.")

Give the code above a try...you should hear two people talking to eachother.

1.1) Formatting Strings

Now we've just seen how we can say strings...but what if we wanted to dynamically generate strings to say based off of inputs/values/etc... We can use Python to construct strings for us. An example is below:

x = 5
print("I wish I owned {} dogs".format(x))

When you run this you'll get the sentenct "I wish I had 5 dogs." printed out. What we did was use string formatting to insert variables into the string. This is a very powerful capability because it allows us to start to build up sentences/phrases/statements in a programmatic way. Perhaps you have a variable that is being used to count how many times you're going through a loop. Some code like the following:

count = 0
while some_condition()==True:
    print('I am currently on my {} th time through the loop!'.format(count))
    count +=1  

This would yield the output of:

'I am currently on my 0th time through the loop!'
'I am currently on my 1th time through the loop!'
'I am currently on my 2th time through the loop!'
'I am currently on my 3th time through the loop!'

...and on and on until you vomit.

So can we combine this with our voice thing? Absolutely. Give it a try.

Now you'll figure out soon enough that even though it is easy enough to call this voice thing it is limited. If you give this a multi-line string to readsuch as the following (where the \n is actually counted as one character not two, which is known as a new line):

multi_line_string = 'Hi there,\nI am a human.\nHave a great day!'
print (multi_line_string)

Hi there,
I am a human.
Have a great day!

You'll see that espeak will properly strip these new line characters so the message gets read out properly.

1.2) What to Speak?

The addition of speaking capabilities makes a massive difference in how we treat a piece of software. If you can have code that dynamically creates sentences based off of user input, your interaction with the device can change dramatically. Think about Siri...even though Siri/Cortana/Alexa (why are they always females?? I think that says something about gender roles in our society...think of ScarJo's role in Her...but I digress) is really just speaking Google results at you, the fact that it/she/etc speaks it changes how we feel/react.

We'll spend a lab or exercise next week messing with web-scraping...you could conceive a verbal announcer of tweets or emails or something once you have that accomplished.

I have some example code below that is another use for this speaking thing. Take a look through it and maybe use it as a starting point for experimenting in the final part below. It works with one pull-down push button switch wired to GPIO pin 12. It'll count (out loud) how many times you've pushed the button so far, and then reward you with music when you've done it multiples of five times. Is there a use for this (in real life)...probably not...but with a bit of tweaking this could turn into something pretty useful (automated DJ, radio station, etc...)

import RPi.GPIO as GPIO
import pygame
import time
import espeak

GPIO.setmode(GPIO.BCM)
GPIO.setup(12,GPIO.IN,pull_up_down=GPIO.PUD_DOWN)

#set up some stuff

#initialize the music Player
song_name = '/home/pi/Music/seeyouagain.mp3'
pygame.mixer.init()
pygame.mixer.music.load(song_name)

#Run some code that counts how many times you've pushed a button. A man with a British voice
#will count off the times for you. When you've pressed it an integer multiple of five times, he will
#play some music for you.  The default is the Tyler the Creator... file (which was put onto your machines for testing purposes
#but also as a joke to humor myself...but now I wish I had put something else on since I hate it after having heard
#it so many times....anywasy....the guy who speaks to you will play the song

#Take this code and combine it with previous code you've had. Come up with something original. It is WEIRD how when 
#you can have code that talks to you in a somewhat intelligent you suddenly feel different about it.

count= 0
try:
    es = espeak.ESpeak() #keep at default
    es.speed = 250 #decently fast
    while True:
        if GPIO.input(12) ==1:
            count +=1
            time.sleep(0.2)
            if count == 1: #What am I doing with this count variable in this line?
                plural=''
            else:
                plural = 's'
            statement = 'You have pressed the button {} time{}.'.format(count,plural)
            es.say(statement)
            if count%5 ==0:
                new_statement = "Pressing the button five, ten, etc.. times earns you twenty seconds of a song!"   
                es.say(new_statement)
                pygame.mixer.music.play() # begin playing music
                time.sleep(2) #wait while it starts
                es.say('Awwww yeah.') # say something stupid (over the music...potential to have automated MC)
                time.sleep(2) #wait again
                es.say('Uhhh') #Say something else stupid
                time.sleep(3) #wait more
                pygame.mixer.music.set_volume(0.2)   #fade music down
                es.say('This is my favorite song!')   # so he can say something
                time.sleep(1)   #wait for it
                es.say('Summer Jams!')   #say something stupid again.
                pygame.mixer.music.set_volume(1.0) #fade back up
                time.sleep(15) #let it soak in for 15 more seconds
                pygame.mixer.music.stop() # stop
except KeyboardInterrupt:
    GPIO.cleanup() #cleanup
    pygame.mixer.music.stop()
finally:
    GPIO.cleanup()
    pygame.mixer.music.stop()

Here's a similar code (2017 version) in action (playing Taylor Swift...another song which I ruined since I heard it so many times.)

You can go in and mess with espeak. Try to get different voices by reading throug their documentation and see what you can get!

2) Assignment: Reaction Timer!

For the rest of this lab, our goal is to build a reaction timer circuit. You can us this to develop statistics on how fast you can respond to light and sound signals. We'll use one new part in this lab, a piezo buzzer. We'll also use one of your regular switches (the one connected to GPIO12) as an input (just keep your old circuit too for right now!). Then we'll use a second GPIO pin (I chose #20) for a Piezo transducer output and a third GPIO pin (I used #21) for an LED.

Our Response Circuit (Keep your amplifier built...just add this in addition).

The only piece that should look new to you will be the part labeled B1. That is a Piezo Buzzer, which makes noise by expanding and contracting a crystal (which in turn pushes and pulls air molecules causing sound waves to form) It is different from a speaker in that it can be connected directly to the Pi, but this convenience comes at the cost of not being able to do much other than buzzing (no music coming out of this part):

A piezo buzzer.

The buzzer can be hooked up in either direction.

2.1) What are we Using this For?

What we want to do in this lab is built some code that will interface with this new hardware in such a way that it will:

  • Randomly present a user with a light or sound stimulus
  • Time how long it takes for the user to respond to that stimulus
  • Record the response time for each session
  • Possibly provide these numbers if so desired
  • Run indefinitely until we want it to end (called via Control C)
  • When Control C is pressed, before exiting completely, print the arithmetic mean of the response time for both sound and light stimuli!

An example of what we're going for is found here (where random roll of the die resulted in a lot of sound stimuli and fewer light stimuli) here:

We can generate LED flashes simply by turning on (GPIO.output(pin_number,1)) and off (GPIO.output(pin_number,0)) the LED of interest.

In order to generate a sound stimulus, we could do several things. You could run a music file through pygame, but that isn't a super time-sensitive piece of Python, so it'd be better if we could somehow generate sound more directly. We can achieve that by driving our Piezo unit with a Pulse Width Modulation signal. Basically what this is is a square wave of a certain frequency. For this lab, we're going to just use it as a tone generator, but in future labs you could use it as a dimmer control or as a way to drive motors, or something. We can use the Pulse Width Modulation (PWM) stuff by doing the following with a GPIO pin. At the top of the code have:

GPIO.setup(pin_number,GPIO.OUT)

like usual where pin_number is a valid pin number of the pin you want to be driving the Raspberry Pi. Then below that do:

pwm=GPIO.PWM(pin_number,800)
pwm.start(0)

What this does is make this pin (whatever pin_number is, into a tone generator!), which is sort of cool1.

If we want to start a tone playing in code, therefore we could do the following:

pwm.ChangeDutyCycle(50)

Don't worry about what the 50 means right now, just know that the number 50 will get this tone playing. Alternatively if we want to turn off the tone signal, you'll want to call:

pwm.ChangeDutyCycle(0)

2.2) Designing the Code

The code you need for this lab is going to take in all that you've learned so far. Whenever you start coding out a solution to a problem, often times the beginning is the worst since the problem can seem just so massive that you don't know where to start. What I usually do is try and sketch out an idea of my control flow...what paths will my code take as it runs...What ends up happening might not 100% match what you initially sketched out but at least this helps you get started. Here's the flow diagram I quickly sketched out when I was coming up with this lab.

Control Flow

I then use this (and you should to) to start and figure out where my code is. Often times decision points are going to be generated from if/elif/else statements. Loops in a flow diagram, generally are going end up manifesting in loops in your code (while loop, for loop).

A few pieces to help you implement some of the stuff you'll need:

2.2.1) random.random()

In Python there is a random library that provides a bunch of functions and tools that you can use to generate random things. The random.random() function will generate a pseudo-random float from 0.0 to 1.0 with relatively uniform distribution2. If you need to get a random delay time, for example you could use this function like so:

random_pause_duration = random.random()*8

This would give you a random number from 0 up to 8.0

You can also use this function to "flip a coin". For example if we know that a call on random.random() will be uniformally distributed from 0.0 to 1.0 we could get a 50/50 random True/False value by doing:

coin_flip = random.random() > 0.5

The variable coin_flip would then contain the result of that coin flip (True or False).

2.2.2) time.time()

time.time() function which you've probably already started messing with from Lab 04. When you call this it will return the number of seconds that have elapsed since Midnight on January 1st, 1970. Why is this useful? Well, while we rarely need the actual value returned by time.time() using multiple calls on it can be used to find the amount of time it took to run a piece of code. For example:

start_time = time.time()
for x in range(500):
    print('EECS is the best field of study.')
stop_time = time.time()
print(stop_time - start_time)

This code will print the amount of time it takes for that for loop to run 500 times. This is pretty cool.

You can also wait for a certain amount of time using time.sleep() which we've gotten lots of experience with so far!

2.2.3) Waiting Loops

Often times in code you need to just hang out and wait for a user input. This structure is known by many things, depending on the context, but we can just call it a "waiting loop" in our class. If you want to pause your code until something happens you can do the following:

print('Waiting for Button press')
while GPIO.input(pin_number) != 1:
    pass
print('Button pressed!')

In this small snippet of code, imagine that pin_number references a push button switch in a pull-down configuration (normally low). Normally this switch would be outputting low (0). Therefore this while loop will continually loop (and do nothing) as long as GPIO.input(pin_number)!=1 returns True. When the button is pushed...this comparison will fail and the while loop will be exited! How often will it check the pin? Very often. Probably ever 100 micro seconds or so....which in human time is really fast (in computers this is actually really slow, but that's ok.)

2.2.4) Storing Data

I've included a basic code to get you started! In the code I've initialized two empty lists, sound_delays and light_delays that you can feel free to use as places to store your recorded delays. Then at the end (or after a desired number of experiments), you can calculate some statistics on these two. You can add values as necessary to these lists using the .append method. For example:

new_value = some_calculation()
sound_delays.append(new_value)
#sound_delays now has the value new_value tacked onto its last (-1th) spot!

3) Get Some Data

This second half of the lab is neat because you're writing code and then carrying out experiments using what you wrote. This is a big part of doing research in the STEM fields. Even if you want to go into pure biology or pure physics, the ability to quickly write/design something (either in programming or wiring or something else) can really, really assist you

Carry out some experiments and see what the mean response time is for both stimuli.

Which one tends to be faster? Can you think of a biological reason for it? In the video embedded earlier in the lab, you'll see that both of my average values (small number of samples, though...) are the same for light and sound. Rarely does this happen...I think I was tired. See if you can beat my averages (not too difficult) and feel free to post questions

When you're done, please copy/paste your ENTIRE working code into this box below and submit it for my records and so I can see what you've been doing! Also share add a linkt to a working video of your system below. Your generation is raised with 60fps video games, so I expect you to have hair-trigger responses when it comes to stimuli. No doctoring the numbers, either. I want to see the code that generated your numbers. I will not believe you if you say that you have a 1 ms response time (on average)....that means your code has a bug...not that you are some super-fast person.

Enter the url for the video

Place all of your code into a folder and zip it up before uploading here:

 No file selected

Enter any comments you may want me to know about. For example, you can list non-obvious things you got working, things you're proud of, outstanding issues, etc.. Please hit submit on this question even if you don't have any comments.


 
Footnotes

1There are some caveats with this. The Pi 2 has a pretty bad PWM signal (lots of jitter since it is software based) which means the 800 Hz tone will sound like a sick person a bit. That's ok for our purposes, but don't rely on it to generate beautiful music (click to return to text)

2Python uses the Mersenne Twister for most of its pseudo-random numbers. (click to return to text)