PandaBoard- All thanks to Texas Instruments

Yes!!! Now we have with us the power of PandaBoard and all thanks to Texas Instruments for providing us the board for our project and guess what..it's sponsored by them.

After half an hour long conference with Vikas Joshi representing TI, he sent to us PandaBoard which will give a special edge to our project. We have already installed Ubuntu OS image on it and we are heading towards our next step to complete Phase I of the project.

Thank You Texas Instruments!!!

Kinectised the BeagleBoard

Finally, we have "Kinectised" the beagleboard. After working for more than a week to install OpenNI and Kinect sensor on the beagleboard things have finally worked out for us.

OpenNI though initially  showed some errors while installing but it was done in a couple of days.

Then we tried the methods given on N number of sites on the internet, still we were not able to install Kinect Sensor. Ultimately, we had to use our brain and employ some tricks to get that working. For this we understood it first by installing it on Linux installed on our laptop and then going through the files and the functions handled by them. Then we moved on to beagle board to resolve the error. Detailed process will be posted after we complete the next phase.

So, now the game is ON and we are just waiting for the delivery of Logic level converters and vibrations motors which we have ordered from Sparkfun. Stay tuned for the next updates!!

A new ray of hope amidst Diwali festivities

Huh! After days of searching for the PandaBoard, we finally have found a ray of hope. As mentioned in the previous post we had planned to buy PandaBoard for the project. However, non-availability of it in the market gave us a setback. In order to meet the deadlines we had to start our project with BeagleBoard till the time we found a PandaBoard for our project. We were sure that the unmatched feature set and ease of usage of the PandaBoard would help us in developing a good product quickly. Due to these reasons we kept searching for all the possibilities of getting a chance to work on the PandaBoard. Hurray! We finally have got a new hope from Texas Instruments who have given a positive signal for lending a PandaBoard to us atleast till the end of our ELM. We are excited to try our hands on the same. We wish Diwali brings good news from across the corners for you all too. Happy Diwali Guys and Gals :-) .

Serially the way out

In order to transmit the data from BeagleBoard to Mbed microcontroller to actuate the vibration motors we need to transmit data serially. Hence we successfully tested serially full duplex communication from Linux (x86 processor) to other PC terminal. Next target is running the Kinect and OpenNi on Ubuntu (we have already succesfully checked the same on Windows7).

Down to the basics

Next, we succesfully tested serial transmission and reception from mbed(through 9th & 10th serial port pins). 

For testing data transmission, we made use of ebox (as we didn't had PC) that was already available with us. To convert the voltages( ttl to RS-232), we had to make use of MAX 232. The circuit looked as follows:

It must be noted that the PCB shown in the diagram on the right is used only for the usage of MAX232 IC in that circuit(since we didn't have spare IC).

Parallel Action on mbed

While other stuff was being done, we registered mbed online and started exploring it using its online available compiler. We tried simple LED programs and PWM and they worked without any problem.

Getting to the technicalities

As mentioned in the previous post, we faced certain issues with Ubuntu. We had different concerns for Natty and Maverick. In case of Natty, our main concern was the extra amount of space being taken up by the OS that we had to eventually buy another 8GB SD Card. Thus, as we figured out the main problem with installing Natty is the space constraint.

In case of Maverick, even after strictly following the steps given in the wiki link:https://wiki.ubuntu.com/ARM/OMAPMaverickInstall

We were not able to successfully install it. We were following the installing procedures for BeagleBoard xM ver A3 and later. However, later we discovered that although the procedure for all other versions of BeagleBoard xM after A3 is same, ver C is an expection to it. We figured out that since our BeagleBoard xM was Rev C, we had to adopt different procedure instead of the normal procedure (also mentioned in above link) as an exception.

Time to explore :-)

Since we had worked on Windows CE7 before, we thought of exploring other Operating Systems. Spending some time on this process was necessary as we want to test the pros and cons of various OS before choosing one.

We tried installing and booting the OS on BeagleBoard for testing purposes.

1. Angstorm: : We started with Angstorm with worked perfectly fine without creating any issues.

2. Ubuntu Flavors: Next we applied the same procedure of testing the image on Natty and Maverick separately. We came up with certain difficulties in this (discussed in the next post). However, we were able to resolve them soon.

3. Android: Considering the Android revolution going on at present with high graphic capabilities, we thought it also try our hands on it. This would help us later in integrating our product with State of the Art application, features.

Thus, we were finally able to successfully test install all main OS on BeagleBoard. During this process we learned that for an OS to successfully boot this process is needed-

1. Partitioning the SD card into two parts-

             SD card            : /dev/sdb  [4 GB]
______________________________________________
     Name                     Path              Space                 Type
______________________________________________
boot partition    |       /dev/sdb1     |    74 MB         |   FAT32
rootfs partition  |       /dev/sdb2     | Rest Space      |   EXT3
______________________________________________

2. Copying appropriate files into these partitions-

__________________________________________________
     Name                     Path                           Files
__________________________________________________
boot partition    |       /dev/sdb1     |   MLO, u-boot.bin.ift, uImage
rootfs partition  |       /dev/sdb2     |   vmlinuz
__________________________________________________


These four files are the main files for BB-xm we need 'boot.scr' too.

What's in the name ...

After considering various options we had earlier come out with 'viSparsh' as the name of our project. This word has been coined by our team. It is a conglomeration of two words namely vision (Vis) & Touch (Sparsh). We felt the name ViSparsh truly depicted the purpose of our project which was to aid the visually impaired people through the sense of Touch. We also thought this as an apt name for our team too whereby the same word would signify our team's aim of touching the lives of several people through our vision. 

Since we have also built a small lab for the first time at YIF we thought of giving it a name too. One of our fellows (Mr. Balaji) helped us in searching a name and we all finally agreed on 'e-Capsule Lab' as the name of our laboratory. The reason for this were manifolds. Firstly, the size of the lab was that of a capsule. Secondly, 'e' would signify electronics in the word 'e-capsule'. Thirdly, we aim to make electronic solutions for the society with the same aim as capsules are made in the medicine field for the betterment of mankind. Thus, we thought this name to be apt for our lab. We hope that future students of YIF would carry forward the name and build great solutions under this lab that would help our society in path-breaking ways.

The Road Ahead

Some of the other things that we also have started working on are follows:

- We are exploring the options for connecting Internet to the BeagleBoard. Since it doesn't have an inbuilt WiFi, we have decided to go with LAN connection at present. However, since router is placed at some distance, we have to make 20m length cable for our usage.

- For a day or two, we are testing various OS platforms to search the best option suiting our requirements. We have already successfully tested on Angstrom which worked perfectly fine. Our next targets are of Ubuntu Maverick and Android which we hopefully aim to finish soon.

- We have started exploring Kinect to test its various parts. We even did Kinect tear-down in order to gain better understanding of the product.

Crossing Barriers

As was the case with the non-availability of PandaBoard so was with mbed. We were in a crisis situation as one of the main components (mbed) was again out of stock. However, we didn't lose hope and started figuring out other possibilities of getting a mbed to work on. While the search was still on, our teammate Tushar gave us a surprise that we had actually won a free mbed (form was secretly filled by him). We took a sigh of relief as we had crossed another barrier and saved some money :-) . As a back up we had earlier thought of buying 2 mbeds. Luckily we were fortunate enough and after few days mbed was again in stock. Since one of the required two mbeds was already won (:-) ), we thought of placing the order for the 2nd one for our safety.

After we received the mbed yesterday, one of our tasks was to get accustomed to the product as we all three were new to it. We successfully implemented some serial transmission codes through the mbed online compiler.

Slight modification in components

As mentioned in our previous blog, we had initially planned of buying PandaBoard considering the project's future advancements to make use of WiFi/Bluetooth and other upgraded hardware configuration requirements However, things never work as planned :-( and so was the case with us. There was no PandaBoard availability in India before 15th Nov. '11. Our time was passing by swiftly and surely we could not afford to wait so long. In order to keep our deadlines we had to resort to buying 1 BeagleBoard instantly for the initial work while waiting for the PandaBoard's availibility.

Requirement to change the Monitor

We had initially bought Benq 18.5" TFT (LED) Monitor which had only VGA port. However, when Beagleboard arrived we figured out that it would not be a feasible idea to convert HDMI to VGA (Beagleboard-xm had HDMI connectivity while our Benq monitor supported VGA). Fortunately, the vendor agreed to replace the Benq Monitor with Samsung LCD monitor which supported both DVI-D and VGA. In order to use BeagleBoard with this new monitor we simply had to use a HDMI-DVI-D converter. We even checked XBOX360 connectivity with the Monitor and it worked perfectly fine. Thus, finally we found the appropriate monitor for the project.

Parts List

Name of the main components required in the initial phase:

1. Microsoft X-Box 360 4 GB Kinect Bundle

2. PandaBoard

3. mbed - LPC1768 Development Board

4. Vibration Motors (ROB-08449)

5. Logic Level Converter (BOB-08745)

Get Set Go.....

After few weeks of hustle bustle, we were finally able to kick start our project formally on 10th October 2011 when we also began with our 4th term of the program. After the first skype session with our mentor some of our main priorities were figured out to be as follows:

- The first target was to get budget sanction for our project which took about 10 days of time as this was the first time an ELM collaboration was happening between UPenn and YIF. Our budget was finally sanctioned on 23rd Sept. 2011.

- Our second target was to set up a lab at YIF. Since YIFP is in the first year of it's launch, we had space constraint issue. After pondering over various possibilities instead of lingering on, we finally thought to make temporary arrangements for the lab in one of our teammates' hostel room.

- On a parallel track we prepared the part list, did some market survey, figured out the best buy prices and ordered them. As almost all the parts were needed to be imported (required shipping), it again took some of our time in starting the project. Since in between the festive season also began and there was holiday between 28th September- 9th October '11, a further delay happened in obtaining the components. Overall this part also took around 10 days.

- Till the time these components were arriving we went to the local market to buy the miscellaneous items like solder iron, wire, extension board, LAN cable etc required to set up our lab.

- Initially as per our market survey we had decided on buying an ASUS laptop so as to best suit our requirements. However since we had a Samsung laptop (being given to each of the fellows by YIF), for starting purposes we thought of buying Monitor, Keyboard and Mouse separately instead of a new Laptop( with Graphics Card option etc) which costed us less.

Use Case Scenario

In the typical use case, a visually impaired person would buckle the viSparsh belt around his waist. The belt would, then, guide him to navigate. Whenever an obstacle comes into the path, viSparsh would inform him for the presence of an obstacle through vibration motors. The viSparsh belt would utilize the RGB image and Infrared depth data captured by Kinect to decide the incoming obstacle and its direction. Suppose if there is an obstacle in the right front of viSparsh user the belt would generate vibrations in the right side. The user would move in the opposite direction to avoid the obstacle.

This typical use case scenario is represented below through a figure-

First meeting with Prof Mangharam :)

Skype Meeting with Prof. Rahul Mangharam 

12-Sept-2011 

6.30 PM IST (9.30 AM EST)

•  Prof. Magharam directed us to take our ELM project work at YIF as a challenge in the form of a start-up company. He thereby suggested not only to emphasize on the technical aspects of the project but also to get exposed to aspects such as Market Research, User Experience, Case Studies etc.
• We were asked to immediately set up a lab at YIF to start our project work. For this we were asked to make and share a part list required for starting this project at the earliest.
• It was mentioned that after the completion of this initial task we would be required to write a second summary of how this product could be made viable by making it cost effective, working on the slim look, integrating with other solutions etc. Prof. asked us to give importance to the usability aspects of the product also as there is no point making a technological product unless it fulfils the demand of the users.

v

Future Vision & Impact

We have been allotted the time span of four weeks to build this prototype inclusive of simulation and real time testing of the prototype. This would be submitted to Prof. Mangharam by a video demonstration.

Then we would be focusing on making it a more viable productized version. This includes developing customized version depending on the user requirements. For example, developing low cost solution with only necessary features would serve the people living in rural areas. And designing relatively high cost product with various additional features like GPS tracking, education and entertainment related application, etc. for the user who can afford to pay for more advanced version.

We firmly believe that the final product will certainly add a social value which would help the visually impaired to move from the shackles of their dependencies. It would create a sense of freedom in them and give them zeal towards living a happier life. It would further make them believe that too have equal opportunity as a normal person to grow and achieve. Thus, overall this product would create a high impact on the society. 

Initial Findings

On being given the initial idea, we started to find more about the latest inventions going on these fields. After some study we found that all the big universities across the world have a haptics division which are pursuing research in the domain of our present interest including MIT Media Labs, Stanford etc. Further, since the release of XBOX360 Kinect in November last year, almost all these haptic departments have started projects for leveraging Kinect's capability for making various haptic interfaces. Even though the release of Kinect has still not completed one year, vast applications have already been developed centered around it. Even the present study of work given to us belongs to that category. Even various applications are already existing in the labs/markets which make use Kinect for making assistance technological devices for the visually impaired people. 

Keeping in mind the above stated facts, we are now fully aware that our task would not be to simply emulate what already has been done at various places but to go few steps further to make an innovative product which helps the society to grow.

All excited for the first meeting with Prof. Rahul (Our mentor)

Few days back we got the news that Prof. Rahul Mangharam from UPenn will be mentoring us as part of our ELM@YIF. Since then we have been very excited to meet Prof. Mangharam . Prof. has come up with an initial idea to assuage our thirst for doing research in the areas of HCI, embedded systems and robotics while adding value to the society. The initial idea was as described in the blog: http://kinecthesia.blogspot.com/ . This was a 3.5 months project taken up by undergraduate students at UPenn early this year.

Since our ELM is of 8 months, the task given is to deliver a viable product taking 'Kinecthesia' as our premises. The idea statement has been stated as follows by the Prof. - "To develop a Haptic Belt with the XBOX360 Kinect for visually impaired persons" where the initial technical effort needs to be taken to develop some relatively low-cost technology pieces and integrating them into a socially useful application. Although this is just an initial idea, we hope to do effective brainstorming with the Prof. when we talk to him for the first time. We all are geared up to officially start the project from today. Yippeee :-)