Workshop on Evidence Collection and Analysis of Webpages

webscrabing, HTTP, HTTPS, decoding

Cybercrimes such as ransomware, cyberbullying, scam, illicit darknet activities, inappropriate sexual content distribution,  or even phishing have a very unstable nature when it comes to the collection of evidence. Webpages related to these crimes are usually available only for a couple of days, sometimes even hours. The workshop presents methods on how to effectively download, decode, parse and archive such webpages. It focuses on a safe and auditable collection of valuable (meta)data that can be later used as proof. The presentation outlines the theory behind modern web design (HTML, CSS, Java/TypeScript), well-known libraries for scraping, and decoding (e.g., Scrapy, Selenium) and current challenges (such as single-page applications, access to dynamic content, execution of JavaScript). The session includes demonstrations of the collection process and existing tools. Participants will receive our open-source tool, which easily archives a  given URL content together with a basic set of metadata.