I found that the ones I tried did result in a working version of Processing on the Raspberry Pi, however none would run the BrainGrapher sketch.

More searching revealed that the problem lay with the serial comms – the Processing distribution is not supplied with a library function that works on an ARM on which the Raspberry Pi is built.

There are a couple of techniques on the Internet that describe how to replace the serial comms file, again some of these didn’t fully work.

The version below works, it uses the latest version of Processing, at the time of writing that is 2.1.1 and Oracle Java version 1.7. I make no claims that any of the work in the steps below are mine and the original can be found here, I replicate these step here for convenience and due acknowledgement and credit given to the original author.

http://scruss.com/blog/2014/01/07/processing-2-1-oracle-java-raspberry-pi-serial-arduino-%E2%98%BA/

With Sun Oracle hardfloat Java now available, Processing now runs at a decent clip on the Raspberry Pi. My old instructions are now very obsolete. Here are current, tested instructions for installing it under Raspbian.

Update the system, always a good plan if you’re doing major upgrades:

sudo apt-get update

sudo apt-get dist-upgrade

Install Sun Oracle Java

sudo apt-get install oracle-java7-jdk

Check if the right version is installed as default:

java -version

This should give

java version “1.7.0_40″

Java(TM) SE Runtime Environment (build 1.7.0_40-b43)

Java HotSpot(TM) Client VM (build 24.0-b56, mixed mode)

If you get anything else, you need to make Sun Oracle’s version the default:

sudo update-alternatives –config java

Download & Install Processing

Go to Download \ Processing.org and get the Linux 32-bit version. It’s big; about 100 MB. I’m going to install it in my home directory, so the base path will be ~/processing-2.1. Extract it:

tar xvzf processing-2.1-linux32.tgz

Now you have to remove the included x86 Java runtime, and replace it with the Raspberry Pi’s armhf one:

rm -rf ~/processing-2.1/java

ln -s /usr/lib/jvm/jdk-7-oracle-armhf ~/processing-2.1/java

You should now have a Processing installation that will run, but there’s some more we need to get serial and Arduino support.

Install the java Simple Serial connector

Download jSSC-2.6.0-Release.zip and extract it:

unzip jSSC-2.6.0-Release.zip

Now overwrite the jssc.jar that ships with Processing with the one you just downloaded:

mv jSSC-2.6.0-Release/jssc.jar ~/processing-2.1/modes/java/libraries/serial/library/

You can remove the jSSC folder now:

rm -r jSSC-2.6.0-Release

Fire up Processing:

~/processing-2.1/processing

Open the sketch BrainGrapher and connect Arduino running Brain Master to USB turn on Mindflex headset.

This works however the window opened by this sketch is much bigger than the size available on my small screen, it is not auto-scaled. So a little more work required.

It was fairly easy to adjust screen size, at the beginning of the sketch in the setup method the window size is defined, the size values were altered to suit my screen.

The next thing I did was to add an icon to the desktop. This is not a easy to do as it is in Windows, a file has to be created and I gather the procedure differs between operating systems, I have Debian-Wheezy installed on my Rpi.

Before creating the file, you need to locate where the icon is for Processing, it is stored in the /processing-2.1.1/lib/icons folder, you will also need to know the location of the executable, this is in the /processing-2.1.1/ folder and is called processing. So create a new text file called processing-2.1.1.desktop in the /home/pi/Desktop/ directory.

It should contain the following:

[Desktop Entry]

Type=Application

Encoding=UTF-8

Name=Processing-2.1.1

Comment=Processing software

Icon=/home/pi/processing-2.1.1/lib/icons/pde-128.png

Exec=/home/pi/processing-2.1.1/processing

Terminal=false

Categories=None;

As soon as the file is saved and closed, the icon should appear on the desktop and clicking it should launch the processing application.

A good description can be found at https://learn.sparkfun.com/tutorials/hackers-in-residence—hacking-mindwave-mobile. I make no excuses in this section being my own work, it has been direct from the quoted source.

The 5 types of brainwaves measured and reported originate from different parts of the brain. They also have extremely low amplitudes and frequencies.

Gamma waves are oscillating waves with frequencies around 40 Hz, although they can be as high as 100 Hz and as low as 24 Hz. These originate from the thalamus (buried deep in the centre of the brain) and are responsible for states of high attention and concentration.

Beta waves are between 12 and 30 Hz and are the states associated with normal waking consciousness. These are emitted from the motor cortex, a region of the cerebral cortex, which is the outermost layer of tissue on the brain. Beta waves are split into three sections: Low Beta Waves (12.5-16 Hz, “Beta 1 power”); Beta Waves (16.5–20 Hz, “Beta 2 power”); and High Beta Waves (20.5-28 Hz, “Beta 3 power”).

Alpha waves originate at the Occipital lobe and have a frequency of 8-12Hz. These are most present when you are awake but are very drowsy or relaxed.

Theta waves are oscillating waves that are located in the Hippocampus and are associated with dreaming. They are in the 4-7Hz range.

Delta waves are associated with very deep, dreamless sleep cycles and are high amplitude waves, which have a 0 to 3Hz frequency. These waves emit from both the thalamus and the cortex.

The difference between these waves can be more easily understood when seen side-by-side.

This is my last post on the BCI for the time, the write-up is up to date with my experiments. Future plans are to get the visualiser working on my Raspberry Pi, apparently it can be done. I also want to see if I can take the Arduino out of the equation and have the data from the XRF wireless module connect to the Rpi and work. I would also like to make a visual display from an array of multi-coloured LED’s connected direct to the Arduino, dispensing with the need for a host. Then there is the more complicated and delectate modification to Mode 2, to get the raw eeg data. These are really on the visualisation side, returning to the original game, the output of the TGAM1 is used to control a the fan speed. I would want to look at various control options, what could be and how to, probably based entirely on the Arduino, with maybe option to see the brain electrical activity at the same time. I will write up progress as and when, I’m not sure in what order I will tackle these ideas.

A good explanation of the various values that are output from NeurSky’s TGAM1 module can be found in http://www.uninettunouniversity.net/allegati/1/CommonFiles/Eventi/it/30/705/Brain%20Computer%20Interface%20System.pdf , from page 20.

The TGAM1 module operates in one of two modes. Mode 1 operates at 9,600bps and gives out the following data:

Poor quality value 0 – 200

EEG bands value 3-byte integer x24 (see below)

Attention value 0 – 100

Meditation value 0 – 100

Mode 2, operates at 57,600 bps and in addition to the mode 1 outputs also provides a 10-bit raw eeg signal. Thus to get the raw eeg signal the baud rate must be changed to 57,600bps.

The TGAM1 breaks up an analyses the eeg signal in to eight frequency bands and outputs them as a sequence of unsigned integers (3 bytes) in order shown below, values are unit-less.

Delta    0.5 – 2.75 Hz

Theta    3.5 –6.75 Hz

Low alpha            7.5 – 9.25 Hz

High alpha            10 – 11.75 Hz

Low beta            13 – 16.75 Hz

High beta            18 – 29.75 Hz

Low gamma            31 – 39.75 Hz

Mid gamma            41 – 49.75 Hz

The Arduino UNO running the brain-master software takes the raw data via the wireless link and translates these in to a more useful format that in this case is used by the visualiser software to display the various bits of information. The rest of this post describes the software for visualising to brain wave data.

The Arduino needs to be plugged in to a host computer which for visualisation requires processing 2 software http://processing.org/ .

Also required is brain grapher https://github.com/kitschpatrol/BrainGrapher

You will probably need Java installed also, I’m not sure if this needs to be any particular version, Java VM exists on my laptop and at the end of these steps, it ran fine.

Download and unzip processing 2 and brain grapher.

Go to the processing 2 folder and launch ‘processing.exe’ after a while the screen will appear, on the file menu, select open and navigate to the folder containing brain grapher, you will see several files, select brain grapher, and open.

A new window will open, click the run button at the top left and shortly a chart display will appear. A useful feature is ‘Present’, clicking this runs the program (a sketch as it is called) in full screen mode like a presentation.

I had to change one in the program (sketch), the line that reads:

serial = new Serial(this, Serial.list()[0], 9600);

I needed to change the list entry number from [0] to [2], as this was the list number that COM5 appeared on my laptop.

I have not checked proper eeg, but holding the electrodes between fingers produced a graph, see below.

In my next post I will explain something about the brainwave signals that are given by the TGAM1.

The modifications on the Internet all make use of a wired connection between the headset, Arduino and host computer for visualisation, the one exception is http://www.instructables.com/id/Mindflex-Duel-Bluetooth-Arduino-FIO-Hack/

That uses a Bluetooth module and FTDI module with a small Arduino in the headset. Whilst, as demonstrated a working solution, the kit is over complicated and one of the board also needs modifying thus adding to the complexity of going wireless. It is not clear how well it all fits in to the space of the headset.

My solution is cheap (~£25) and elegant, it adds the minimum hardware to the Headset, inside the electronics pod. The proposed solution enables the outward appearance to be retained without any hacking of the case. Provides a totally wire-free link to the Arduino, thus avoiding all issues with connection of other apparatus compromising safety. It is powered from the 3.3V rail of the headset.

In searching for a suitable device I came across this http://shop.ciseco.co.uk/xrf-wireless-rf-radio-uart-serial-data-module-xbee-shaped/ The XRF is an RF data transceiver module, that appeared to be of the right size and according to claims works straight out of the box at 9600bps.

It seemed easy to interface to the TGAM1 module; only 4 wires (3 actually) are needed. In the headset only the transmit data line will be used, so only three connections needed. On the Arduino, again only three connections required, this time the receive line, power and ground. If I want to later, the speed can be changed, again matching that coming out of the TGAM1 module. I bought two, one to go in the headset, the other to attach to the Arduino.

Out of the box it works in the 868MHz region but can be changed to 915 or 432 bands. It works on 3.3V so will work directly in the Mindflex headset, will it work on the Arduino as it has a 3.3V power rail present on one of the pins (UNO), later versions of the XRF (v1.6 up) are 5V tolerant on the data lines.

Although there is lots of information on the XRF module available here http://openmicros.org/index.php/articles/247-xrf-document-index, finding the functions of the pins is difficult, below shows the basic pins needed, the XBEE is the same thing apparently.

I soldered some ribbon cable on to the appropriate pins of the XRF and temporarily joined them up to the wires on the Mindfelx headset. On the second XRF I connected power to 3v3 and ground of the Arduino Uno and the Rx pins together. Switching it all on it did not work, so I investigated the reason. Another data sheet http://www.hobbytronics.co.uk/datasheets/XRF-v1-6.pdf shows that Tx is Data out and Rx is Data in, so I connected them round the wrong way. Swapping the lines around resulted in a functional radio data link. Fortunately this ambiguity didn’t damage anything. So the Tx and Rx pins of the XRF module refer to the data port, not the RF side, that is XRF pin Tx is data out and pin Rx is data in. The photo below shows in close-up the XRF and soldered connections.

There is a little bit of space in the headset to accommodate the XRF module, however the enclosure can not be secured together, so I cut off the pins of the XRF module to give it a lower profile, then put some heat shrink sleeve over it to ensure no extraneous electrical contact could be made to the electronics within the headset. The photos below show the encapsulation and wiring in the headset.

Now for more permanent connections to the Arduino. The XRF has a pin spacing of 2mm so it won’t plug in to a standard breadboard that has 2.54mm (0.1”) spacing. Ciesco produce an Arduino shield for this module, it contains other stuff as well, they also provide kits for passive and active interfacing – break out boards http://shop.ciseco.co.uk/xbbo-break-out-board-for-xbee-shaped-modules/ It appears that Rapid Electronics supply a ‘bare bones’ shield for the XRF at a reduced price http://www.rapidonline.com/electronic-components/xbee-shield-v0-6-for-arduino-xino-xrf-xv-xbt-and-more-73-4602

I cut off one end of three jumper-wire pins, soldered the wire ends to the appropriate terminals on the XRF (3v3 = orange to pin 1, gnd = black to pin 10, Tx (Dout) = blue to pin 2) and then plugged these in to the appropriate sockets on my UNO (orange to 3v3, black to gnd and blue to Rx).

Next, the Uno was connected to my laptop via USB and Arduino PDE was launched followed by opening up the serial monitor (no sketch was loaded, it had previously been uploaded to the UNO). Turning on the Mindflex headset, after a second or so data started streaming via the RF link and Arduino to my laptop, see screenshot below.

It is possible to use Arduino DIO pins other than the one shared with the USB, if left connected to the Rx pin there might be a conflict when programming the Arduino necessitating the removal of the connection to the XRF, during tests I did experience trouble and had to disconnect also the USB from the UNO and re-launch the Arduino PDE. In another post I will explain how to move the XRF output to what is called a ‘soft serial’ connection.

That is it for this post, next post I will explain how to get the output from the Arduino working so that the brainwaves can be visualised.

I have for testing an Arduino UNO, and it connects to my laptop using a standard USB A-B cable.

You will need an environment for the software, to allow the programs to be created and uploaded to the Arduino http://arduino.cc/en/main/software Confusingly this is called Arduino, download and unzip the latest version.

Remember to install the drivers from the Arduino folder. Plug the Arduino UNO in to your computer using an A-B USB cable and follow instructions – found new hardware.

You will also require the brain-master library, so download and unzip the Arduino brain-master https://github.com/kitschpatrol/Brain This is software that makes the UNO work with the headset, or specifically the TGAM1 module.

You need to make a library folder in the Ardurino library structure on your computer. Navigate to the Arduino folder that was just unzipped and open the folder called “libraries”, create a new folder named “Brain”. Now go to the brain-master and select “Brain.cpp” and “Brain.h” and copy them to the newly created folder in the ‘libraries’ folder.

Back to the brain-master folder and open the examples folder and then open the ‘BrainSerialTest’ folder, if there are copies of Brain.cpp and Brain.h there, then delete them.

Now locate and run Arduino.exe from the Arduino folder, this is the Programming Development Environment (PDE). I made a shortcut to my desktop to launch this easily.

Within the PDE, load (open) the BrainSerialTest sketch.

Ensure that the PDE is set for the specific Arduino board you have and that the com port is correct (use device manager to find this) and then upload the program to the Arduino.

Disconnect the Arduino (turn off any power sources including removing USB), connect the output of the Mindflex to the Rx pin and ground to GND. Turn on Mindflex and connect Arduino.

My Arduino when installed comes up on COM5, so the port was changed in the PDE.

With all this done, you can test that you get data output from the Headset though the Arduino by opening up within the PDE the serial monitor, after a few seconds you should see lines of data streaming in.

All should be OK and now that the code has been loaded in to the UNO, this step does not have to be performed again, unless you re-program the UNO for some other application or if you make any changes to the brain-master code.

In the next instalment I will describe the radio link from the headset to the Arduino UNO.

Several sources on the Internet claim it is easy to hack, only two wires need to be soldered on to the output of the NeuroSky Chip/module and this will provide a continuous stream of eeg data that can be connected to an Arduino, leaving the game intact should anyone also want to play it.

There are concerns that with the electrodes making contact with the human body, especially on the head, in the brain region over safety with other electrical connections – Arduino and laptop/computer being the ultimate destination for the data.

There would appear to be a number of ways to improve on the safety aspect by providing isolation in the way of either an optical or radio link at some stage on the headset, of course retaining original battery power. Anyone wanting to provide optical isolation, look at 6N138 or 6N137, check speed. 470 resistor in series with T to cathode, anode to V. on other side vcc to pin 8, 1k pin 7 and 5, gnd to pin 5 out pin 6.Make sure to keep supply and ground rails separate to maintain isolation.

Looking at things in a block form then one easy way could be to use the wireless apparatus within the headset, picking up the output, the other side of the radio link within the console. This could mean a larger modification. It does not seem possible to use the in-built wireless module as the microprocessor within the headset does some unknown things with the data from the TGAM1 module.

There is also power consumption, if the standard modification is followed, then there is a lot of unused electronics that is still being powered. It seems all that needs powering is the NeuroSky chip.

It should be possible to de-solder the 9 pins and remove the Neurosky module and use it elsewhere providing it with a 3.3V supply. Alternately these pins could be de-soldered and the main PCB removed and discarded leaving the housing available to accommodate other electronics along with the module. I might explore more on this at a later date.

For initial modification, I soldered on a blue wire to T pin and pink wire to ground pin on the TGAM1 module and brought these wires out through the existing portal where the ear-clip cable exits. There is no need to drill a hole in the case of the headset as suggested by some Internet sources.

In the next part I will describe connecting up to the Arduino, the Arduino and visualiser software to get the system running as a first step in my experiments.

The duel comes with two headsets; only one is needed for the BCI, so there is a spare in case things get damaged.

Each head set comprises a headband with integral electrode that should, according to instructions be positioned over the left eye on the forehead. An ear clip completes the biological connections to an electronics pod that contains a special circuits and a 2.4GHz wireless transmitter to the console. There is also a battery pack (3x AAA cells) on the headband.

The forehead electrode is made from some unknown conductive fabric with unknown properties and it appears is placed over what feels like a piece of contoured felt. Some how electrical contact is made to a wire or cable that connects to the electronics. The ear electrode is actually two electrodes (more about this later) made of unknown material and connects via a cable to the electronics compartment. In clinical eeg the working electrode would be silver/silver-chloride (Ag/AgCl) and the ear clip might be of the same or stainless steel or conductive rubber. The forehead electrode would more than likely have some conductive gel to improve electrical conductivity to the skin. These connections are dry, so hopefully the input impedance of the electronics will be very high. Using dissimilar metals for electrodes will result in a small DC potential existing, this could influence the machine. Also questions of toxicity of the metals in contact with the skin should be considered. There are sources of conductive fabric and ear clip electrodes available on the internet and one, if so desired could fairly easily make up a comparable headband, how safe this would be will require more work. Ions can be transferred from conductive parts in to the body with only a minute amount of electrical current flowing and there are also local allergic reactions to be considered from the materials.

Opening up the electronics case (the other case contains only batteries) there are a few landmarks on the circuit board labelled G8336-2G, no info could be found on this. At top right is the on/off slide switch below this are some electrolytic capacitors and an SMD semiconductor labelled U5 and marked AA-17, this might be a regulator, no real data could be found, but it would make sense. On the underside of the main PCB is a LED that illuminates when on, it also acts as a low battery indicator and flashes when the battery is low and needs changing.

To the left of the RF module and under a quality control sticker is an integrated circuit; I could see no markings on the circular button encapsulation or any identification close by, I believe this to be a small microprocessor or microcontroller.

Below this IC is a transistor labelled Q13 and a 3-pin IC labelled U6, both are SMD and writing too small for me to read. There are also a couple more electrolytic capacitors in this area.

The main area of interest is below the slide switch – the NeuroSky module. The board is called TGAM1 and its specification can be found here http://wearcam.org/ece1766/neurosky_eeg_brainwave_chip_and_board_tgam1.pdf

At one edge there are five solder pads connected to wires to the electrodes. At the top a white wire connects to the pad labelled eeg, below this is another wire coloured black, going to the screen of the electrode cable. Below this is a green wire for the ground electrode on the ear, followed by black as the reference shield, then at the bottom a white wire connected to a pad labelled ref for the reference electrode, these three wires go to the ear clip cable. In some electrophysiological applications it is usual to have a reference and ground or common electrode, these are typically spaced some distance apart, interestingly both these are separated by the thickness of the ear lobe. The cable screens are more than likely connected to the ground, the ref could be a differential connection to the input amplifier on the NeuroSky Chip.

At the other end of the module and at the bottom corner there are four pins that solder to the main circuit board, these are arranged in a square formation. Reading top left and going right these are labelled as T for transmit data and R for receive data. The lower pair only the left is labelled as V. The lower pair are the power supply rails with the unmarked terminal being ground and V being the power line (3.3V).

As indicated in the above photo, there are three links, called B0, B1 and M. The links B0 and B1 are used to set the mode of the TGAM1 and default baud rate of the data. The M link is used to set the frequency of a notch filter to either 50 or 60Hz. Reading the specification sheet does not make it clear, however looking at other information on the Internet, it appears the links are in fact 10k resistors, http://darrenmothersele.com/blog/2013/09/21/brain-wave-visualisation/ depending on what way round the meter is connected the value in-circuit measures about 1k or 500 Ohms, all are connected to ground. Another source on Internet says the resistance is 1k.

The baud rate on this setting is 9600 bps (other options are 1200 and 57600), interpreting the data sheet and from information on the Internet, this apparently can be changed by sending some data on the R-pin. The default setting for the notch filter is 50Hz, there is no software command to alter this, it must be done as a hardware configuration. If required, it would be fairly easy to add switches to select the various options, so long as soldering could be acheieved.

It should be possible to de-solder the 9 pins and remove the Neurosky module and use it elsewhere providing it with a 3.3V supply. Alternately these pins could be de-soldered and the main PCB removed and discarded leaving the housing available to accommodate other electronics along with the module.

It does not seem possible to buy the TGAM1 module, at least as an individual wishing to experiment.

The battery current was measured as approximately 14.2mA, disconnecting the module from the remaining circuit would not be too easy, it could be done. The module, in fact all modules are powered by 3.3V, the module itself according to the specification sheet takes 15mA, so there might not be too much of an advantage in disconnecting the other components. With alkaline AAA battery (capacity ~1000mAh) the expected operational hours is therefore around 71 hours.

The next part of this series look at making the modifications and getting it working to display brainwave information.

It was released in 2009 http://en.wikipedia.org/wiki/Mindflex and uses a chip or module produced NeuroSky. This module is used in some other games and for BCI investigations along with other devices and techniques, these are compared here http://en.wikipedia.org/wiki/Comparison_of_consumer_brain-computer_interface_devices

A good starting point for background reading on the modifications required is http://frontiernerds.com/brain-hack This describes how to get it working with an Arduino that interfaces to a visualisation system, all links to software needed for the Arduino and the visualisation host are provided. There is other information out on the Internet, most refer to the same software and easy modifications.

There is one other source, that describes a more difficult modification that allows the raw brain data to be directly output http://darrenmothersele.com/blog/2013/10/07/mindflex-raw-eeg-data/ This requires special soldering skills on some of the micro-miniature components inside and has not been tackled by me at the moment, it might be at a later date. These series of blogs will refer only to my experiments concerning the first and easy modifications, but will however refer to the second, in-depth mods.

After all this reading, it was time for me to invest in one of these toys, I managed to secure a dual headset item (Mind Flex Duel) on Ebay for a very reasonable sum, when it arrived, it was almost brand new and with a full set of batteries.

The game comes with two headsets, a base console and many parts to allow you to create your own obstacle course for either one or two players to engage in six different games that involve using your mind to control a fan raising and lowering a sponge ball. I did of course try this out and had some fun with this before moving on to explore the inner workings of one of the headsets.

The next part will look at the headset in more detail and explain the innards of it, getting ready for the modifications.