The SmartTab, a robot – tablet combination, aiming to simplify concurrent handling of multiple tablets, can create multi-device-displays and adjust his position dynamically to multiple situations. The SmartTab is also a platform to evaluate interaction with self-actuated displays and their benefit to the user.
However, there is still room for improvement. Therefor, we decided to develop OptiSense. OptiSense is a combination of several addons, aiming to help the SmartTab getting more autonomously.
In the following, you can read about our ideas for improvement, the technological concepts behind them, how we integrated them, some troubles we were facing during this process and the solutions on how we managed to get rid of those problems.

During the workshop we thought about different use cases for the SmartTab. We discovered lots of interesting and novel enhancements for the little robot of which the best idea was a small servant, carrying around your tablet and driving safely over a table without invading your predefined personal space.

The SmartTab is designed as an office robot driving on tables and showing nearby people some information on a tablet mounted on the top.
To protect the robot from falling down the table and potentially damaging itself or the tablet, it is essential that edges of the table are recognized and the robot immediately stops by itself.
Another characteristic of office tables is, that plenty of objects are spread all over the place most of the time. If for example there is a glass full of water, the robot could knock it down. The best case scenario would be that this just causes a big mess, but in the worst case it could destroy the robot and tablet. To avoid both, the SmartTab should be able to recognize obstacles and drive autonomously around them. Furthermore, the users should be able to mark some areas the SmartTab is not allowed to pass by using colored tape. In this area they could place for example things they need like water, a keyboard or some notes. This would even allow the users to place things on the table the robot would not be able to detect, like single sheets of paper, without him driving over them.
Additionally, as office tables can be really big, it would be useful if the robot could determine its' position on the table. This would allow a planned movement and orientation towards the users, as in contrast to a random movement. Moreover, this would facilitate the use of special areas like a recharging station where the SmartTab could drive to after the battery falls below a certain threshold.
As far as the robot goes, it was really important that the used hardware is small in size, because, as the SmartTab was not designed for further addons, there was only very limited space. Another, but not minor limitation, was, that there is only a certain amount of power available, of which the most is used by the engine, so it was really important to watch at the power demand of the hardware we wanted to include.
In the next topic you will read about the different ideas for the solution of our features and reasons why we used or did not use certain technologies.

From the Domain Characteristics, we gathered some goals that are useful for our robot. The more important goals are higher in the list:

• Edge recognition
• Position tracking
• Microcontroller
• Obstacle recognition
• Avoiding previously marked areas
• Special areas

As this feature directly affects the safety of robot and tablet, this was our priority.
The SmartTab is capable of driving omnidirectional, meaning that it can drive in any direction at any time, so it was important to make sure that none of the four wheels was able to drive over an edge. Therefor we needed four measure points, one in each corner of the case near the wheels. As there were only some millimeters of usable space, the only possible solution were infrared proximity sensors. As sensor we used the QRE1113 . This sensor has the advantages that its analog, making it possible to detect even small changes in the surface. Another advantage is that it runs with both, 3.3V and 5V, so it can be powered with ease from different sources.

We considered many possible techniques for the position tracking.
The first idea was using Bluetooth. Therefor we thought about placing four Bluetooth beacons in the corners of the table and use the tablet to calculate its' position by triangulating the strength of the signals. But on the one hand, the signals fluctuate a lot, making it nearly impossible to calculate the position with a decent accuracy. On the other hand, this would require an external setup, making the use of the robot unnecessarily complex. So we needed to think for other possibilities.
Then we came up with ultrasound. This would work by placing an ultrasonic emitter on the robot, and four receivers near the edges of the table. Through the traveling times of the sound waves from the emitter to the different receivers, the relative position of the robot could be calculated. In a silent environment this would enable an accuracy of 4 to 8 centimeters and in a noisy one still 10cm. However, this solution has three major disadvantages. For one, the emitter would have to be placed on top of the case. If a tablet is mounted on the robot, this would lead to a blocking of the radio signal and a distorted result of the position tracking. Another problem is that the emitter would need to be rotated all the time using a motor, what would drain the power sources even more than it already does. But the biggest problem with this solution is the the complexity of the setup of the hardware. All four receivers would need to be connected to a circuit, requiring a lot of cable and time for setup. Also the robot would have to communicate with the receiver circuit, making it even more complex and energy consuming. So we discarded this solution as well.
Our third idea was a tablecloth filled with RFID-tags combined with a RFID-reader mounted at the bottom of the robot. Whenever the SmartTab would drive over such a tag it would get some info on where on the table it is located. But this solution has several drawbacks, too. First of all, if we want to get a decent accuracy we would need to use lots of RFID-tags. And the bigger the table, the more tags would be needed. Another problem with this solution would be that its working well with tables that are the same size as the tablecloth or smaller, but on bigger tables, or tables with other shapes, it would not cover the whole table. The third problem is, that it occasionally happens that a glass of water gets knocked down. With our tablecloth, this would damage the RFID-tags, making it necessary to replace them. As this would be very annoying every time it happens, this solution is not an option, too.
Our last and ultimately used idea was simple odometry. Therefor we measure the direct movement and rotation on the robot itself. To get those data we use two optical mouse sensors, one for the movement direction and one for the rotation. The accuracy of those measurements depends heavily on the used sensor. The best results are gained by using laser mice with as much dpi as possible. For this reason, we are using two times the Magic Mouse by Apple. Besides the great accuracy those mice offer, they are small enough to fit under the robot (once the hull of the mice is removed). Another advantage of them is that they are connected via Bluetooth, making it easy to read their data without hacking the mouse itself. However, we faced the problem that we could connect the mice with the Edison, but they did not register properly in its' OS, making it impossible for us to read their data.

For the microcontroller, the biggest affordance was its size. As there is only very limited space in the robot, it needed to be as small as possible while still offering the needed functionality. This already limited the choice to either the Arduino Nano or the Intel Edison Mini Breakout Board. Both have advantages as well as disadvantages.
One of the two benefits of the Arduino for our purpose is that it has 8 pins for analog input. This makes it very easy to connect to the four analog proximity sensors mentioned above. The other advantage is that the board runs on 5V, enabling it to supply external sensors with 3.3V or 5V, what makes the use of the proximity sensors even more easy. However, the big drawback of the Arduino Nano is that it does not have Bluetooth. As we want to communicate with the magic mice via Bluetooth, this is the reason we decided to use the other microcontroller.
The main benefit of the Intel Edison Mini Breakout Board in contrast to the Arduino Nano is the availability of Bluetooth. This, in combination with yocto Linux, the operating system of the Intel Edison, simplifies the connection to the magic mice. That is the main reason we chose the Edison over the Arduino. Nevertheless, one of the bigger drawbacks is the lack of analog pins, but therefor it has lots of GPIO-pins. If you have analog input you can simply bypass the missing analog pins by using a capacitor in your circuit and measure its' charging time. This way, you can „convert“ an analog signal into a digital one. A simple circuit using this easy method will be explained later on. The second major disadvantage of the Edison is that the board runs at 1.8V. In situations like ours where the sensors need at least 3.3V, you can bypass that problem, too. This happens with the usage of NPN-transistors. Like for the first problem, an example of the corresponding circuit will follow down below.

There were two different possibilities for the recognition of obstacles.
Our first idea was to just use some proximity sensors like we did for the edge recognition. Due to their low sensing radius, we would need approximately 4 sensors per side, making it 16 in total. This means that we would need 32 GPIO-pins, and again as many for the voltage and the grounding together, which would not just cause a big mess of cables inside the robot, but also a big problem connecting all those pins to the Edison. As we already came to the limit for GPIO-pins with the 4 sensors for the edge recognition, it would basically just be impossible to connect 16 more sensors to the microcontroller without the help of an big and complex circuit that would never fit inside the SmartTab. So we came up with another idea.
We also considered using ultrasound. This time, in contrast to the idea for the position tracking, the emitter and receiver both would have to be mounted on the top of the case. Now they would work similar like a sonar in ships and submarines, detecting obstacles by the echo of sound waves they produce. The biggest problem of our first idea with the ultrasound, the complex setup, is now gone. However, the tablet would still reflect the signal and thus make it hard to use it in any direction. Also, since the receiver is now mounted on the robot, too, this would increase the already high power demand even more. As the robot has only limited power supply, we needed to discard this idea too, and because we could not come up with any other way to solve the problem with the obstacle recognition, we decided to not include it into the SmartTab.

For this feature we were able to use the sensors we already implemented for the edge recognition. As the QRE1113 uses infrared, it does not just determine if any object is near, but it is also able to detect the color of it. As darker colors absorb more light, less reflections of the sent infrared signal reach the receiver of the sensor, thus making it believe the object is further away than it actually is. If, for example, there are two different objects at the exact same distance to the sensor, the darker object will be interpreted as farther away. This means that we are able to distinguish different surface colors by just observing the measured value from the sensor. So, if the SmartTab drives on a white table, you can place black tape on it. Once the robot reaches this tape, he will misinterpret it as an edge and simply drive another way.

For this feature, we immediately thought about using RFID. The idea was, that the user can place a RFID-tag somewhere on the table near a special place, for example a recharging station. When the SmartTab drives over this tag, it knows that there is something special. Through the position tracking mentioned above, the robot would also roughly know the position of this tag, and would be able to drive autonomously back there after certain events like a low battery for example. The problem is, that the Edison simply does not supply enough power. Normally an RFID-reader needs 5V, but you can reduce it to 3.3V by lowering its range. However, the Edison just supplies 1.8V. As the RFID-reader is a digital device it requires a sophisticated circuit to elevate the voltage to the required level. Without the proper electrical parts we had to abandon this idea, but with the right parts it would be pretty much plug and play and really easy to add to the SmartTab.

jinput: getting started
java.library.path

Intel Edison Mini Breakout Board
Sensors
Board
Resistance 220 Ohm
Resistance 100 Ohm
Capacitor 470pF

The first feature we were working at was the edge recognition. Therefor, we built a circuit to use the sensors we mentioned in the previous topic.
As already mentioned before, the sensors are analog whereas the Edison just has digital pins. For this reason we had to „convert“ the analog signal into a digital one. This happens with the help of a capacitor. When infrared light reaches the receiver of our sensor, the capacitor charges. The more light it detects, the faster the capacitor charges. Once the capacitor is fully charged, it discharges itself automatically. When you measure the time it takes to charge the capacitor, you now have the previously analog signal changed into a digital one, making it easy to use for the Edison.
The next step we did, was converting this circuit layout into an actual electrical circuit to test our sensors. The result looked like this:
After this step, and some testing on how to get the data with the Intel Edison, we were ready to build our final electrical circuit. This is basically one circuit explained above for every sensor, making it 4 in total. However we were able to combine all the power and ground connections to one each, so we only needed 10 pins on the Edison instead of 16.
As you could already see in the picture above, at this point there was a lot of soldering involved. You not just have to solder the pins to the Intel Edison Mini Breakout Board, but also to the plate itself. To make sure that everything is connected what needs to be, we soldered some paths at the bottom.
Now, that our electrical setup was finished, we put all the pins in the corresponding connectors. After finding an appropriate place for this hardware inside the robot, we taped the plate to the case, and used some hot glue to fixate the proximity sensors inside the edges of the case and the cables to a secure place. Finally, the case with all the electrical hardware inside looked like this:
The last step before everything for the edge recognition was working, was supplying the Intel Edison with power. Therefor, we connected two cables, one for power and one for grounding, to the battery case of the SmartTab, supplying 9V. As the Intel Edison has two pins for input voltage, one for a 3.3V battery and one for 7-15VDC, it is crucial to connect the power supply to the right pin. For our case, that is J21, the pin below the two USB-ports. To be able to turn the Edison on and off, we put a switch between the battery case and the board, simply cutting the Edison off of energy. In the end, the robot looked like this:

As mentioned in the Technological concept, we decided to use odometry for position tracking. Therefor, we were searching for mice that are small enough to fit under the robot as well as precise enough for a decent accuracy. The solution was the magic mouse by Apple, as already stated a bit above. However, as they were still to big, we needed to get rid of the case of the mouse. To achieve this, we build a do-it-yourself heating oven, with the purpose to heat the glue so that you can just separate the different parts of the mouse.
After the glue became warm enough, you could, with the help of a credit card or something similar, separate the different parts of the case. If the glue is still to strong, you can just warm it up more.
After this is done, the mouse was small enough to fit under the robot.T he rest of the magic mouse looked like this:
The next step is to connect the two mice via Bluetooth with the Edison. This works with the bluetoothctl interface of Linux. Here you can find all the needed information for it. Unfortunately, we ran into some trouble while trying to read the data of the connected mice with the OS of the Edison. We will explain more details on this problem later on.

We were using analog sensors, but the Edison only supports digital pins. The solution for this problem, as already explained above, was to „convert“ the analog signal into a digital one. This works by using a capacitor. The circuit layout shown before should show it pretty well.

This was one of the first and most annoying problems we were facing. Basically, the main problem is, that the naming of the pins on the Mini Breakout Board differs from the one on the „normal“ board, the Intel Edison Ardiuno Board. If you want to use GPIO-pin 15 on the Arduino board, you can simply address it. However, for the Mini Breakout Board, this works completely different. For example, GPIO-pin 165 on the mini board itself is called J18-2, but neither of those names you can use to address it. If you want to address it in your code, you need to address GPIO-pin 15. As most of the websites in the internet refer to the Arduino board, it is tricky to find out which pin you really need to address. However, this website helps in finding the right names.

Another problem we faced was that the Edison only supports 1.8V. For analog sensors requiring 3.3V you can simply amplify the voltage by using a NPN-transistor, like we explained above. However, you cannot use digital sensors, like RFID-readers, because there is no possibility to supply it with enough power by the Edison.

As mentioned at the end of the last big chapter, connecting the mice with the Edison was no problem at all. With the commands found here it is really easy to connect the mice. They even get shown as „Human Interface-device“. As we already had some java code to read the data from the mice that worked fine with Ubuntu 15.10, we were confident that it also works with the Edison. However, we were not able to get those data with yocto Linux. As they were connected as mice, they should show events in a folder of the OS, and we wanted to read those events. But somehow, none of the events occurred making it impossible for us to use the mouse data.

The last problem we faced was the biggest one. As we did not want to connect the Edison via USB with a tablet mounted on top of the robot or even a laptop, we needed to supply it somehow with power. The engine of the robot is powered with 12 batteries, supplying it with in total 9V. Since the Edison supports input voltage with up to 15V, we decided to supply it with the power from the robot. So far so good. But the problem, or more like the danger is, that the Edison has two different pins for input voltage, one for a 3.3V battery and one for 7-15VDC input. The big danger is that both those pins look similar. While it is totally safe to put 3.3V into the pin for the 7-15VDC, it is not recommended to do it the other way round. If it happens nevertheless, like it happened with us by accident, the board will not survive it. If you want to power your Edison without using USB, make sure to check double if you are using the right pin.

A very good guide for the setup of the Intel Edison Mini Breakout Board can be found here. If your OS is Ubuntu 15.10 and the installer says it is not available for this OS, you can just follow these steps:

1. edit pset/install.cab using vim
2. make sure to launch in binary mode: vim -b install.cab
3. replace every 15.04 with 15.10

Once the board is running, you need to login as „root“. For the first login, there is no password set yet. We changed the password of our Edison to „physcomp“. After that you need to connect to WiFi using the tutorial linked above. After this is set up, you can connect to your Edison using ssh.

Interfacing with the Intel edison is mainly done via ssh. In order for that to work you need an ip-connection. Initially this is done easily with usb-cables and these instructions: https://software.intel.com/en-us/forums/internet-of-things/topic/533839 After that you can configure the Intel Edison to connect to your home network and use a wireless connection.

Google-Gson library with Eclipse
The .jar files and Java shared libraries needed on runtime for using MRAA & UPM is /usr/lib/java.