| Abstract | Brain-Computer Interface (BCI) is about controlling devices directly with brain potentials, bypassing the external motor organs such as arms or legs. It is a part of Human-Computer Interaction (HCI) research, which itself is a part of computer science. Starting from punched cards, through keyboards, mouse, multimedia, a new possibility now is interaction through physiological signals, including brain signals. The human brain generates various types of potentials, depending on the task considered. So far, the mental states used in BCI are the state of relaxation (alpha rhythm), the state of mental task, such as calculation (beta rhythm), the state of response to stimuli (evoked potentials), the state of intention to move (mu rhythm), and the state of expectation (CNV potential). The CNV potential in a BCI paradigm was first introduced in this work. Experimental research (materials and methods) in this dissertation is carried out using a special experimental paradigm, which is called the CNV flip-flop paradigm. A subject hears two auditory stimuli, S1 (warning) and S2 (imperative, to be reacted on) stimulus. The brain develops expectation (S2/S1) on S2, given S1. When the expectation produces a certain level of CNV amplitude, the computer turns off the S2 signal. Since there is no S2, the CNV potential disappears. The computer turns on the signal S2 again, and after several trials the CNV reappears. The paradigm generates an oscillatory process in the brain, which produces series of appearances and disappearances of the CNV potential. The paradigm tackles a difficult problem in signal processing, namely dealing with a time varying potential. A neural network learning algorithm to deal with this problem was used. The BCI experimental research is based on controlling a robotic arm using the CNV flip-flop paradigm. The demonstration task is the Towers of Hanoi (TOH) problem, well known in computer science. A set of behaviors are preprogrammed to move one disk at a time toward the solution of the problem. For two-disk and three-disk TOH, three moves and seven moves are needed respectively. It was shown that using the CNV flip flop paradigm, a subject is able to generate series of CNV and noCNV events, that will reach the solution of the Towers of Hanoi problem with two and three disks.
The main contribution of this work is introducing the anticipatory brain potentials in the BCI research and achieving control of a robot using them. Also, it is shown how two robotic arms, working together on the same problem (namely the Towers of Hanoi with three disks), can be simultaneously controlled using this paradigm. This is the first time that such a thing has been achieved in the world. |
| Abstract (croatian) | Sučelje mozga s računalom (Brain-Computer Interface – BCI) je oblast istraživanja u kojoj se uređaji upravljaju izravno s pomoću potencijala mozga, zaobilazeći vanjske motoričke organe (ruke i noge). Ovo područje je dio šireg područja istraživanja interakcije čovjeka s računalom (Human-Computer Interaction – HCI), koje je samo po sebi dio računarskih i komunikacijskih tehnologija. Ljudski mozak generira različite vrste potencijala, ovisno o zadatku. Za sada, mentalna stanja koja se koriste u BCI-ju su stanje opuštenosti (alfa ritam), stanje mentalne angažiranosti (beta ritam), stanje odgovora na stimuluse (evocirani potencijali), stanje namjere za pokret (mu ritam) i stanje očekivanja (CNV potencijal). CNV potencijal u BCI paradigmi je predmet istraživanja prvi put uveden u ovom radu. Eksperimentalno istraživanje (materijali i metode) u ovoj disertaciji izvedeno je s pomoću eksperimentalne paradigme nazvane CNV flip-flop paradigma. Ispitanik prima dva zvučna podražaja S1 (upozoravajući) i S2 (imperativni, na koji treba reagirati). Mozak razvija potencijale očekivanja (S2/S1) na S2, nakon zadatog S1. Kada ti potencijali pređu određenu razinu CNV amplitude, računalo isključi stimulus S2. Budući da više nema S2, CNV potencijal nestaje. Računalo tada ponovo uključuje S2 i, nakon nekoliko pokusa, CNV se ponovo pojavljuje. Paradigma izaziva oscilatorni proces u mozgu, koji proizvodi niz pojavljivanja i gubljenja CNV potencijala. Paradigma se suočava s problemom obradbe vremenski promjenljivih signala. U radu je za obradbu signala kognitivne aktivnosti mozga korištena metoda zasnovana na neuronskim mrežama. Eksperimentalno istraživanje BCI je zasnovano na upravljanju robotske ruke pomoću CNV flip-flop paradigme. Prikazano je rješavanje problema Hanojskih tornjeva (Towers of Hanoi – TOH), dobro poznatog u računarstvu. Skup ponašanja robota je preprogramiran, kako bi se izvjela micanja diskova u tornjevima prema pravilima dolazeći tako do rješenja problema. Za probleme tornjeva sa dva i tri diska potrebna su odgovarajuće tri i sedam micanja. Pokazano je da je, pomoću CNV flip-flop paradigme, ispitanik u stanju proizvesti niz pojavljivanja i gubljenja CNV potencijala, koji donose rješenje problema. Glavni doprinos rada je uvođenje anticipacijskih potencijala mozga u BCI istraživanja i izvedbu upravljanja robota pomoću njih. Također je pokazano da je istim principom, s pomoću anticipacijskih potencijala mozga, moguće upravljati i s dvije robotske ruke (dva uređaja istovremeno). |
