Biblioteka za dohvat novinskog članka s portala i pridruženih komentara na društvenim mrežama

master's thesis

• Title (english): Library for downloading news article on portals and associated comments on social networks
• Author: Šimun Matešić
• Mentor: Stjepan Groš (mentor)
• Committee member: Stjepan Groš (predsjednik povjerenstva)
• Committee member: Marin Golub (član povjerenstva)
• Committee member: Leonardo Jelenković (član povjerenstva)
• Granter: University of Zagreb; Faculty of Electrical Engineering and Computing; Zagreb
• Defense date and country: 2021-07-14, Croatia
• Scientific / art field, discipline and subdiscipline: TECHNICAL SCIENCES; Computing

Abstract

U sklopu ovog rada otkriven je unutarnji način rada nekolicine sustava za komentiranje na webu te je na osnovu tih saznanja implementiran sustav za dohvat članaka i njihovih komentara. Krajnji cilj rada je stvoriti web scraping sustav s elementima web pauka koji ispunjava svoju svrhu, a da je lako nadogradiv, robusan te lak za održavati. Odnosno, potrebno je osigurati da je implementacija provedena u skladu s dobrim praksama programskog inženjerstva. Kako bi se moglo reći da sustav ispunjava svoju svrhu, mora moći samostalno preuzimati članke i komentare u odnosu na definirana ograničenja te održavati rad bez obzira na vanjske promjene sustava komentiranja na webu. U radu je prikazan postupak reverznog inženjerstva sustava komentiranja, arhitektura razvijanog sustava te implementacijski detalji bitni za razumijevanje najvećih problema koje je ovaj rad riješio.

Abstract (english)

This paper serves to explain the inner-workings of comment systems used on the web, and based upon that theoretical background goes through the process of building a system which retrieves articles and associated comments from news sites. The end goal of this paper is to build an easily-upgradeable, robust and easy-to-maintain web scraping and crawling system which fulfills its purpose. In other words, good practices must be enforced during the whole implementation life-cycle. The criteria which tells us if the system has fulfilled it's purpose is when it can automatically download news articles and comments and keep working regardless of outside changes of comment systems it relies on. The paper contains the reverse engineering process required to build a base of knowledge of comment systems, the architecture of the built system, and most importantly the implementation details which are needed to fully understand the problems this paper solved.

• Keywords: reverzno inženjerstvo; web pauk; web scraper; programsko inženjerstvo; dobre prakse
• Keywords (english): reverse engineering; web crawler; web scraper; software engineering; bestpractices
• Language: croatian
• URN:NBN: urn:nbn:hr:168:633776
• Study programme: Title: Computing; Study programme type: university; Study level: graduate; Academic / professional title: magistar/magistra inženjer/inženjerka računarstva (magistar/magistra inženjer/inženjerka računarstva)
• Type of resource: Text
• File origin: Born digital
• Access conditions: Closed access
• Created on: 2023-01-13 13:03:42