Usporedba metoda za grupiranje tekstualnih dokumenata

undergraduate thesis

• Title (english): Comparison of Text Clustering Methods
• Author: Dario Bošnjak
• Mentor: Krešimir Pripužić (mentor)
• Committee member: Krešimir Pripužić (predsjednik povjerenstva)
• Committee member: Denis Salopek (član povjerenstva)
• Committee member: Marko Pavelić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2017-07-10, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Electrical Engineering

Abstract

Obrada i analiza dokumenata iz velikih kolekcija zahtijeva određenu organizaciju istih. Tomu služi grupiranje, postupak kojim će računalo, s obzirom na zadane parametre, odrediti grupu za dokumente iz kolekcije. Parametri se zadaju s obzirom na kolekciju i želje korisnika. Algoritam k-means jadan je od najpoznatijih algoritama, dok je poboljšanje dano algoritmom k-means++. Fuzzy k-means je meki algoritam grupiranja koji ovisno o zadanom parametru s određenim postotkom pripadnosti može svrstati dokumente u jednu ili više grupa. Algoritam multiple k-means++ svodi se na izvođenje algoritma k-means++ određeni broj puta. Navedeni algoritmi svrstavaju dokumente u zadani broj grupa, dok algoritam DBSCAN kao parametar ne prima željeni broj grupa, nego najmanji broj točaka u grupi i okolinu ɛ. Ako točka sadrži barem traženi broj točaka u svojoj okolini tada može postati jezgrom grupe, a u suprotnom će biti svrstana u neku grupu ili biti označena kao šum. Algoritam DBSCAN prilagodljiviji je od ostalih algoritama, ali je teško odrediti parametre s kojima bi se dobili dobri rezultati jer jako ovise o kolekciji.

Abstract (english)

Analysis of large collections can be a difficult task that can be simplified by dividing the collection into groups. Each group contains documents that are somehow similar. The process of dividing the collection can be done manually by a human, or automatically by a machine. Clustering is an automated process of dividing the collection into groups (clusters), with respect to parameters set by the user. K-means is a well known clustering algorithm, an improved version also exists, called k-means++. Both algorithms cluster a data point into one cluster. A fuzzy k-means algorithm is different, because it can cluster one data point into several clusters, depending on the user requirements. The DBSCAN algorithm, unlike aforementioned algorithms, does not require a fixed number of groups, but a minimum number of points in the group and ɛ neighborhood size. If a data point contains the minimum number of data points needed in it's neighborhood it can then become a cluster core, otherwise it will be grouped later in the process or labeled as noise. Both parameters are hard to set properly, because they severely depend on the collection.

• Keywords: prikupljanje informacija; indeks; grupiranje; strojno učenje; Apache Lucene; Apache Mahout
• Keywords (english): information retrieval; index; cluster; machine learning; Apache Lucene; Apache Mahout
• Language: croatian
• URN:NBN: urn:nbn:hr:168:179791
• Study programme: Title: Computing; Study programme type: university; Study level: undergraduate; Academic / professional title: sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva (sveučilišni/a prvostupnik/ prvostupnica (baccalaureus/baccalaurea) inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2019-03-13 19:14:05