Generiranje notnog zapisa na temelju video zapisa vremenskih tragova pritisaka tipki klavira

undergraduate thesis

• Title (english): Music score generation from video stream of piano keyboard time traces
• Author: Ana Bagić
• Mentor: Marko Čupić (mentor)
• Committee member: Marko Čupić (predsjednik povjerenstva)
• Committee member: Marko Barišić (član povjerenstva)
• Committee member: Vjeko Kužina (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-06-30, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U ovom radu proučavala sam problem generiranja notnog zapisa iz videozapisa koji padajućim linijama prikazuje u kojem trenutku je pritisnuta koja tipka klavijature. Analiziranjem svake slike videozapisa, dobivala sam informacije o pritisnutim tipkama, te sam ih koristila kako bih izgradila model pjesme. Korištenje aplikacije vrlo je jednostavno: korisnik odabere željeni videozapis, zada određene parametre pjesme (npr. raspon klavijature, mjera i okvirni tempo pjesme), i kao izlaz dobije datoteku u MusicXML formatu koja sadrži sve bitne informacije za generiranje notnog zapisa. Učitavanjem dobivene datoteke na jedan od mnogih alata dostupnih na Internetu, korisnik može dobiti notni zapis u PDF obliku. Zbog širokih mogućnosti glazbe, i slobode glazbenika, kod implementacije programa suočavala sam se s mnogim naizgled trivijalnim ali vrlo kompleksnim problemima. Korisnost ovog programa je vrlo velika čak i ako ne daje 100% točne rezultate, jer na sebe preuzima ogromnu količinu posla, a za glazbenika ostavlja samo male korekcije. U radu je objašnjena implementacija programskog rješenja, te su navedeni i analizirani dobiveni rezultati.

Abstract (english)

In this paper, I studied the problem of generating sheet music from a video that uses falling lines to show at what time which key on the piano keyboard was pressed. By analyzing each frame of the video, I received information about the keys pressed, and used it to build a song model. Using the application is very simple: the user selects the desired video, sets certain parameters of the song (e.g.piano keyboard range, measure and approximate tempo of the song), and as an output receives a file in MusicXML format that contains all the important information to generate the sheet music. By uploading the resulting file to one of the many tools available on the Internet, the user can get the sheet music in PDF format. Due to the wide possibilities of music, and the freedom of musicians, I faced many seemingly trivial but very complex problems when implementing the program. The usefulness of this program is large even if it does not give 100% accurate results, because it takes on a huge amount of work, and leaves the musician to deal with only small corrections. The implementation of the software solution is explained in this paper, and the obtained results are listed and analyzed.

• Keywords: generiranje notnog zapisa; glazba; računalni vid; detekcija objekata; obrada slike; obrada videozapisa
• Keywords (english): sheet music generation; music; computer vision; object detection; image processing; video processing
• Language: croatian
• URN:NBN: urn:nbn:hr:168:558041
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-14 20:01:31