Automatsko sustavno pretraživanje web prostora korištenjem Pythona

master's thesis

• Title (english): Crawling and Scraping using Python
• Author: Domagoj Marić
• Mentor: Miljenko Mikuc (mentor)
• Committee member: Miljenko Mikuc (predsjednik povjerenstva)
• Committee member: Marin Vuković (član povjerenstva)
• Committee member: Dragan Jevtić (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2019-07-09, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

Ovaj diplomski rad bavi se temom automatiziranog pretraživanja web prostora i ekstrakcije korisniku korisnog sadržaja iz web stranica. Obrađene su temeljne prakse, potrebna predznanja i demonstrirani napredniji Python alati za izradu programskih rješenja koja ispunjavaju ovu funkcionalnost. Opisane su prednosti i nedostatci svakog obrađenog alata te su dva najprikladnija alata iskorištena za izradu programskog rješenja za jednostavno i sveobuhvatno pretraživanje web stranica registriranih na www.hr, "početnoj stranici Hrvatske".

Abstract (english)

The main subject of this thesis is the automated web browsing and data extraction from the retrieved web sites. These goals are realized by using web crawling and web scraping tools, respectively. In the thesis, I mentioned the main practices of web crawling and web scraping, the basic knowledge prerequisites to start, and demonstrated the four most popular web scraping tools in Python. Considering the main advantages and disadvantages of the tools, I picked two for my main application - a web crawling/scraping application for simple and comprehensive browsing of the web site catalogue on www.hr, the "Croatian Homepage".

• Keywords: automatsko pretraživanje weba; crawling; scraping; beautifulsoup; lxml; scrapy; selenium
• Keywords (english): automated browsing; crawling; scraping; beautifulsoup; lxml; scrapy; selenium
• Language: croatian
• URN:NBN: urn:nbn:hr:168:287728
• Study programme: Title: Information and Communication Technology; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka informacijske i komunikacijske tehnologije (magistar/magistra inženjer/inženjerka informacijske i komunikacijske tehnologije)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2020-01-07 20:02:07