SketchingWiki

In this section we will give a thorough overview of the different software libraries and techniques used before we will have a look at the implementation itself.

As already stated in previous sections, this project uses the Raspberry Pi 2 Model B as its footing. As several different versions of the Raspberry exist which vary widely with regard to hardware layout and available software libraries, it is important to use this exact model version to ensure the interconnection between hardware and software works flawlessly.

The Raspberry Pi 2 Model B is available with and without a pre-installed OS. We used the one having NOOBS pre-installed, a tiny distribution of Linux.

To get the Raspberry running, some certain settings are necessary. This section gives a brief overview.

On the first boot, the Raspberry will show its Raspberry Pi Software Configuration Tool. Select “3 Enable Boot to Desktop/Scratch” and in the following menu “Console Text console, requiring login (default)” to turn off the graphical user interface which slows down the boot process.

Select “Ok”, then “Finish” to save the settings and to reboot. On the next boot, the Raspberry will show the standard Linux terminal window . The default login is pi, the default password raspberry.

By default, the keyboard is set to the EN_US layout. Set the correct layout, e.g. DE_DE, using the following commands:

sudo nano /etc/default/keyboard
sudo reboot

Set the correct date and time using the following command. This is especially helpful when using GitHub, as it uses local timestamps and version control gets really messy if one of your machines running git has wrong time settings.

sudo date --set “2016-01-20 15:00:00”

As described in the Hardware section, a Wi-Fi stick is part of our hardware setup. It can be configured as follows. First, check whether the Wi-Fi adapter is present using ifconfig. It should show an adapter called “wlan0”.

To connect to a WPA-secured network, type sudo nano /etc/wpa_supplicant/wpa_supplicant.conf and specify the Wi-Fi network the Raspberry should connect to as follows. The key psk corresponds to the PSK passphrase as plaintext.

network={
	ssid=”HCI’s Network”
	psk=”foobazbar”
}

Run ifdown eth0, ifup wlan0 and ifconfig after each other to reload the configuration. Check whether the Wi-Fi is working using e.g. wget . Next, deactivate the interface eth0 using sudo nano /etc/network/interfaces and changing “dhcp” to “manual” in the following line:

iface eth0 inet dhcp

Afterwards, /etc/network/interfaces should look as follows. More info on how to setup Wi-Fi can be found here.

auto lo eth0
iface lo inet loopback
iface eth0 inet manual

allow-hotplug wlan0
iface wlan0 inet manual
wpa-roam /etc/wpa_supplicant/wpa_supplicant.conf

To be able to control the Raspberry Pi from remote, a SSH server can be setup using sudo raspi-config. Select “8 Advanced Options”, then “A4 SSH”, then “Enable”. You should now be able to connect to the Raspberry Pi from another machine in the same network using:

ssh pi@<IP-ADDRESS>

As described in the Hardware section, several different hardware components need to be accessed/controlled by the Raspberry Pi. In addition, different software-based processes are necessary. To allow the development of software which brings all these components together, a bunch of different software libraries are necessary. We will describe them in this section.

Our software is based on Python 2.7 which is shipped with NOOBS. Ensure that the python command runs this exact version and not any newer, e.g. using alias. Several development extensions of Python are necessary and can be be installed using:

sudo apt-get install python-dev-all python-pip python-dev python-setuptools enum34

For controlling the sound card, Pulseaudio is used. It can be installed as follows:

sudo apt-get install pulseaudio amixer

To control audio playback and recording, please install necessary libraries using:

sudo python-alsaaudio python-pyalsa libasound2-dev python-pyaudio lame

To check whether sound is working correctly, run the following commands:

amixer set PCM on
amixer set PCM 100%
wget http://any.tld/any.mp3
omxplayer any.mp3

As described in the Hardware section, a USB sound card is used to play and record sound. To set it as the first sound card, use the following commands or find more information here . Firstly, use nano /etc/modprobe.d/alsa-base.conf and comment the following line out before rebooting using sudo reboot.

# options snd-usb-audio index=-2

Audio recording can be tested following this blog post.

Pyglet is a “cross-platform windowing and multimedia library for Python” which allows both video and audio playback. It can be installed using:

sudo easy_install pyglet

Ensure that necessary dependencies are installed too:

sudo apt-get install libavbin-dev libavbin0

I2C is a serial bus we use to query the hardware slider component. Normally, the following setup steps will suffice, otherwise please follow the setup steps described here.

Use sudo raspi-config to bring up the Raspberry Pi configuration tool, select “8 Advanced Options”, then “A7 I2C” then two times “Yes”.

Install several necessary libraries as follows, see more details here and here.

sudo apt-get install python-smbus i2c-tools
sudo apt-get install build-essential libi2c-dev i2c-tools python-dev
sudo apt-get install libffi-dev
sudo easy_install cffi
sudo easy_install smbus-cffi

To check whether the I2C bus and the connected slider hardware control are working, see section Tests.

The RFID reader can be checked using the screen command which can be installed using sudo apt-get install screen. Type the following to display scanned RFIDs:

screen /dev/ttyUSB0 9600

To get the FadeCandy software running and to be able to control the different attached LEDs with it, this and this tutorial are good material to read. First of all, the FadeCandy source code needs to be downloaded and make has to be run.

git clone git://github.com/scanlime/fadecandy
cd fadecandy/server
make submodules
make
sudo mv fcserver /usr/local/bin

The FadeCandy server has to be configured editing /usr/local/bin/fcserver.json.

{
        "listen": [null, 7890],
        "verbose": true,

        "color": {
                "gamma": 2.5,
                "whitepoint": [0.7, 0.7, 0.7]
        },

        "devices": [
                {
                        "type": "fadecandy",
                        "serial": "OWYYEMYUHLUBCTQZ",
                        "map": [
                                [ 0,  0, 0, 64 ]
                                [ 0,  0, 0, 64 ]
                                [ 0,  0, 0, 64 ]
                                [ 0,  0, 0, 64 ]
                                [ 0,  0, 0, 64 ]
                                [ 0,  0, 0, 64 ]
                                [ 0,  0, 0, 64 ]
                                [ 0,  0, 0, 64 ]
                        ]
                }
        ]
}

To ensure that the different components are working on the Raspberry Pi, they can be tested separately using the following python test files provided in the raspberry directory. Ensure to cd to the directory before running the tests on the command line.

The dimmer/slider component uses the I2C bus to query analog input plugged at the SDC GPIO pin. Run sudo python dimmer-test.py to check whether the component works properly. The script outputs values between 0 and 255 on the command line. As the I2C bus needs access to main memory, the script needs to be run as root.

Using the GPIO test script allows to check whether different hardware buttons are working correctly and whether the wiring is correct. Run sudo python gpio-test.py to start the script. As it accesses main memory, running as root is necessary.

The script monitors digital input hardware / buttons at the GPIO pins 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26 and 27 (BCM GPIO layout). As soon as one of these channels changes its value (edge detection), the script prints both the corresponding button channel number and the new value to the command line.

To test the NeoPixels illumination, run the following command to start the FadeCandy server.

sudo fcserver &

Run one of the three test scripts neopixels-band-test.py, neopixels-channel-test.py or neopixels-test.py using:

sudo python <SCRIPT_NAME>

1. neopixels-test.py:
   Changes the color of one single NeoPixel on channel 0, index 0 of the FadeCandy board every one second from red to green to blue.
2. neopixels-band-test.py:
   Turns NeoPixels on channel 0, index 0 to 15 from black to green. One pixel per second.
3. neopixel-channel-test.py:
   Tests the real illumination setting: Single four NeoPixels on channel 0, Adafruit Flower rings on channels 2 and 4 and a NeoPixel strip on channel 7.

Initializes all NeoPixels with black, starts from channel 0 to 7 and sets all corresponding NeoPixels to green in increasing order, waiting 200ms in between turning two adjacent NeoPixels.

Run the following command to test audio recording. Ensure the microphone is plugged into the external USB sound card and that the sound card is connected too. The script records a sequence of 10 different MP3 files of length 1s each. The output MP3 files will be placed in the raspberry directory.

sudo python recording-test-pyaudio.py

The RFID reader can be tested using the following command. Ensure the hardware is plugged using USB and that it registers a serial interface on /dev/ttyUSB0. To check whether the serial connection is working, screen /dev/ttyUSB0 9600 can be used.

sudo python rfid_reader.py

To test higher level software, the following three helpers can be used.

1. SDC Helper (analog input)
2. GPIO Helper (digital input)
3. FadeCabdy Helper (illumination)

Use the following commands in the the directory raspberry/.. to run the tests.

sudo python gpio_sdc_helper.py 
sudo python gpio_helper.py 
sudo python illumination_neopixels.py 

This project’s challenging part was the necessity to bring several different hardware and software together and how to implement the communication between them. The main resulting software components are:

1. Main component
2. Child components
   1. GPIO Bench
   2. Illumination
   3. Playlist Directory
   4. RFID Reader
   5. Sound Player
   6. Sound Recorder

The main component (mp3player.py) controls the interaction between hardware and software components and assigns processing time to each child component. Therefore it uses the Observer Software Design Pattern to allow sending commands to each component and to allow each component to raise events handled by the main component.

The main idea behind this design pattern is that different components (so called observables/subjects) can work on their own independently and notify possible observers about certain events. Observers can subscribe to these events and perform certain actions as soon as they are risen. The main component is such an observer whereas the child components are observables.

Child components can be divided into (1) Non-threaded components and (2) Threaded components. Non-threaded components implement a unidirectional data flow: They perform a certain action, e.g. controlling an output device, but they don’t raise events the main component would need to handle. Therefore they offer a bunch of methods the main component can invoke. In contrast, threaded components implement a bidirectional data flow: They offer methods the main component can invoke too, but they are also able to raise events the main component needs to handle, e.g. if a hardware button is pressed. In the following section, an overview about the purposes of the different child components is given.

The GPIO Bench (gpio_bench.py) watches for digital and analog input changes on both the GPIO (buttons) and SDC (slider) pin(s). Therefore it stores the initial values of each input device and looks for changes in a certain interval. If an input device changes its value, the GPIO Bench raises one of the following events:

• PLAY_PAUSE: The play/pause button was pressed.
• RECORD_START_END: The record button was pressed.
• PREV: The previous button was pressed shorter than the wind threshold indicating that the user wants to go one track backwards.
• WIND_BACKWARDS: The previous button was pressed longer than the wind threshold indicating that the user wants to wind backwards.
• WIND_BACKWARDS_RELEASE: The previous button was released after winding was active.
• NEXT: The next button was pressed shorter than the wind threshold indicating that the user wants to go one track forwards.
• WIND_FORWARDS: The next button was pressed longer than the wind threshold indicating that the user wants to wind forwards.
• WIND_BACKWARDS_RELEASE: The next button was released after winding was active.
• VOLUME_UP: The volume slider increased its value.
• VOLUME_DOWN: The volume slider decreased its value.
• SHUTDOWN: The shutdown/boot button was pressed.

As previous/wind backwards and next/wind forwards are bound to the same button, it’s necessary to detect which action the user wants to perform. Therefore a time threshold of 800ms distinguishes between whether to go to the previous/next track or whether to perform winding. As soon as the user pressed the previous/next button for at least 800ms, the winding functionality in the corresponding direction is activated.

The Illumination component (illumination.py) controls the colors all connected NeoPixel LEDs currently have and provides several methods to change those colors. These include:

• rfid_recognition(valid_rfid): Indicates that a RFID was recognized. The valid_rfid flag indicates, whether it was a valid one (contained in the Playlist Directory).
• update_btn_prev(is_disabled, is_winding, is_prev=False): Updates the illumination of the previous button according to its state indicated by the flags is_disabled, is_winding and is_prev.
• update_btn_next(is_disabled, is_winding, is_next=False): Updates the illumination of the next button according to its state indicated by the flags is_disabled, is_winding and is_next.
• update_btn_play_pause(is_disabled, is_playing): Updates the illumination of the play/pause button according to its state indicated by the flags is_disabled and is_playing.
• update_btn_recording(is_disabled, is_recording): Updates the illumination of the record button according to its state indicated by the flags is_disabled and is_recording.
• update_track(number): Updates the illumination of the track indicators (NeoPixel rings around previous and next buttons) according to number [0.00, 1.00].
• update_position(position): Updates the illumination of the position indicator (NeoPixel ring around play/pause button) according to position [0.00, 1.00].
• update_volume(volume): Updates the illumination of the volume indicator (NeoPixel strip) according to volume [0.00, 1.00].
• shutdown(): Indicates that the MP3 Player is going to shut down.

The main component is able to change the illumination of PiMu using some of these methods.

The Playlist Directory (playlist_directory.py) keeps a mapping between certain RFID tags and corresponding playlists. Therefore within the mp3/ directory, for each RFID tag a directory named accordingly is placed containing the mapped MP3 files. This component provides the following methods:

• set_root(root_path): Sets the root_path the Playlist Directory expects RFID folders to be placed below.
• exists(rfid): Checks whether a directory and thus a playlist with the given rfid exists.
• get_playlists(): Returns a list of names of existing playlists (=rfids)
• load(rfid): Loads a playlist with the given rfid as a name.
• get_mp3s(): Returns the file paths of all MP3s present in the currently loaded playlist.
• num_mp3s(): Returns the number of MP3s present in the currently loaded playlist.

The RFID Reader (rfid_reader.py) watches for valid detected RFIDs at the serial terminal the hardware RFID component is connected to. It provides the following methods:

• set_port(port): Sets the serial terminal port the RFID reader is connected to (e.g. COM9 or USB0).

In addition, it is able to raise the following events:

• RFID_INSERTED: A RFID tag was detected.

The Sound Player (sound_player.py) component allows the playback of sound files. It provides the following methods:

• get_volume(): Returns the current volume [0.00, 1.00].
• set_volume(volume): Sets the current volume [0.00, 1.00].
• volume_up(): Increases the current volume by 5%.
• volume_down(): Decreases the current volume by 5%.
• load(mp3_list, index=0, offset=None): Loads a list of MP3 files (mp3_list) and starts playing the nth file specified by index at the position specified by offset.
• stop(): Stops the playback and unloads the current list of MP3 files.
• pause_unpause(): Pauses or unpauses the playback.
• next(): Starts the playback of the next MP3 file.
• prev(offset=None): Starts the playback of the previous MP3 file at the given offset
• wind_forwards(): Winds the currently playing MP3 file forwards by 10 seconds.
• wind_backwards(): Winds the currently playing MP3 file backwards by 10 seconds.
• is_playing(): Returns whether a MP3 file is currently loaded and playing.
• is_loaded(): Returns whether a list of MP3 files is currently loaded.
• get_current_track_number(): Returns the current track number as a tuple (track_number, total_number_of_tracks).
• get_current_track_position(): Returns the current track position as a tuple (track_position, total_track_length)

In addition, it’s able to raise the following events:

• SONG_ADVANCED: The song has advanced by 2.5 seconds (10 seconds respectively, if song advanced because of winding).

The Sound Recorder (sound_recorder.py) allows to record a sound file in WAV or MP3 format. It provides the following methods. Although it is a threaded component, it does not raise any events.

• start_recording(): Starts a new recording.
• stop_recording(): Ends the current recording, if any.
• is_recording(): Returns whether the recorder is currently recording.

Each threaded component has its own thread assigned to be able to perform certain actions. Some examples include:

1. The RFID reader continuatively listens for RFIDs on its serial port.
2. The Sound Player plays a MP3 file and listens for control commands.
3. The Sound Recorder continuatively records sound chunks of 1024 bit size.
4. The GPIO Bench watches for changes of input devices.

Although the Raspberry Pi supports multi-threading , it only supports it on software level (single core, switching between threads, instructions of one thread processed at a time) and does not provide real multi-threading (e.g. parallel processing using multiple cores).

This results in certain limitations, these include that a while(True) loop in multiple threads is not a good thing to use as the software scheduler simply doesn’t know which thread to assign processing time at what time as to him it looks like every thread is in need of 100% processing time. For example, this would result in GPIO events not being detected or a delayed playback of sound.

To avoid this issue, all threaded components (namely GPIO Bench, RFID Reader, Sound Player and Sound Recorder) use sleeps of 10ms in their run loop to allow the scheduler to assign enough processing time to each threaded component.

To run the PiMu software, type sudo PYGLET_SHADOW_WINDOW=false python mp3player.py. The PYGLET_SHADOW_WINDOW variable is necessary, as the video playback function of Pyglet is based on Open GL which is not supported on the Raspberry Pi. Without specifying it, Pyglet would start in video mode which would result in an error. By setting it to False, Pyglet starts in Audio-only mode and runs without any issues.

To automate running the PiMu software, an alias can be assigned as follows:

cd
nano .bashrc
alias pimu=’sudo PYGLET_SHADOW_WINDOW=false python ~/physicalcomputing/mp3player.py’

In addition, a startup script allows to run the software on each reboot of the Raspberry Pi. Place the following commands in /etc/rc.local using sudo nano /etc/rc.local.

sudo fcserver /usr/local/bin/fcserver.json &
sudo PYGLET_SHADOW_WINDOW=false python /home/pi/physicalcomputing/mp3player.py &

The source code is available on GitHub. It currently requires repository access, as it contains MP3 files which are not freely available. See the Team section to contact us for repository access.

Feel free to ask about any details (prename.name@uni-konstanz.de).