https://measuretheweb.org/ 2026-04-21T10:19:27+00:00 https://measuretheweb.org/ https://measuretheweb.org/_media/wiki/dokuwiki.svg text/html 2026-03-27T07:52:01+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/start?rev=1774597921&do=diff Welcome to Measure The Web Empirical studies on the web require researchers to navigate a complex landscape of experimental design choices, ranging from selecting a representative sample of websites to choosing the appropriate crawling technology. Similarly, analyzing results involves critical decisions, such as website categorization and statistical methodology. Too often, these decisions are made based on limited guidance, informal advice, or trial and error, despite their profound impact on … text/html 2026-03-24T09:34:11+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/programming/similarweb?rev=1774344851&do=diff While official API use is paid, Similarweb offers a browser extension, which uses an internal API to retrieve data. Methodology for Code Reproduction To update the scraper logic when Similarweb changes their verification, follow these steps: * Install the extension in a Chromium-based browser. text/html 2025-10-09T07:18:14+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/privacy/requests?rev=1759994294&do=diff Classifying Web Requests A common task in web privacy measurements is to determine which web requests correspond to the benign loading of required web resources and which are used to track users. There are two main methods for such classification: matching requests against crowd-sourced lists (typically used in ad-blocking or tracking protection extensions) or using machine learning ( text/html 2025-06-23T08:50:44+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/design/website_classification?rev=1750668644&do=diff Overview Similarly to Website Selection, also the classification of websites according to their topic, properties of their companies (industry type, employees count) is often important step in results processing of web measurement studies. Awareness of the available services, their capabilities but also limitations is essential for high-quality research. text/html 2025-03-19T10:36:10+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/research_journey?rev=1742380570&do=diff Research Journey At the moment, this page is WiP and only relevant to automated measurements. Maybe we have to create the same page for manual studies (I am not covering them at the wiki at all now). The plan is that each point links to some page. text/html 2025-03-19T10:27:48+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/writing/conferences?rev=1742380068&do=diff Conferences and Journals This page should outline list of publication venues suitable for web measurement studies, typically with focus on security or privacy. URLs to conferences were up to date in 2025, but year of conference is often embedded in the link and can be easily updated. text/html 2025-03-19T08:51:51+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/programming/interaction?rev=1742374311&do=diff Interaction with website This page only contains notes Topics to cover (in separate pages): * Input into website forms * Interacting with consent * Stateful and stateless crawling * Urban et al. studied crawling only index pages vs crawling multiple subpages of a website, showing significant increase in third-party requests and cookies when you browse also subpages. From 10 visited subpages is increase small. text/html 2025-03-19T08:37:03+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/literature/bibliography?rev=1742373423&do=diff text/html 2025-03-18T14:20:41+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/programming/stateful_stateless?rev=1742307641&do=diff Stateful and Stateless Crawling This page only contains notes Key message: * Majority of web measurements studies use stateless crawls, as it is easy to associate events with the single browsed visited website. Also, stateless crawls do not depend on crawling order and are easier to parallelize. text/html 2025-03-18T14:07:32+00:00 archives (archives@undisclosed.example.com) https://measuretheweb.org/design/archives?rev=1742306852&do=diff Use of web archives for research Various researchers have utilized web archives for longitudinal analyses of privacy and security features . Using archives for research have several advantages. Their use helps towards solving the issue of reproducibility of research, given that a static snapshot is used for each page visit. It also eliminates the need to write project-specific crawlers. However, there are a number of pitfalls associated with the use of archives. Additionally, there are numero… text/html 2025-03-18T14:06:10+00:00 archives (archives@undisclosed.example.com) https://measuretheweb.org/playground/playground?rev=1742306770&do=diff PlayGround text/html 2025-01-07T17:45:35+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/?image=privacy%3Aprivacy_cookies_cookieblock_vs_cookiepedia.png&ns=privacy&rev=1736271935&tab_details=history&media_do=diff&do=media <img src="https://measuretheweb.org/_media/privacy/privacy_cookies_cookieblock_vs_cookiepedia.png?w=500&amp;h=500&amp;tok=fc904a" alt="privacy_cookies_cookieblock_vs_cookiepedia.png" loading="lazy" width="500" height="500" /> text/html 2025-01-03T17:08:56+00:00 admin (admin@undisclosed.example.com) https://measuretheweb.org/?image=wiki%3Aprivacy%3Aprivacy_adgraph_performance_lists_iqbal2020_adgraph.png&ns=wiki%3Aprivacy&rev=1735924136&tab_details=history&media_do=diff&do=media <img src="https://measuretheweb.org/_media/wiki/privacy/privacy_adgraph_performance_lists_iqbal2020_adgraph.png?w=500&amp;h=500&amp;tok=8dc038" alt="privacy_adgraph_performance_lists_iqbal2020_adgraph.png" loading="lazy" width="500" height="500" /> text/html 2025-01-03T17:07:20+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/?image=privacy%3Aprivacy_adgraph_performance_lists_iqbal2020_adgraph.png&ns=privacy&rev=1735924040&tab_details=history&media_do=diff&do=media <img src="https://measuretheweb.org/_media/privacy/privacy_adgraph_performance_lists_iqbal2020_adgraph.png?w=500&amp;h=500&amp;tok=d3576d" alt="privacy_adgraph_performance_lists_iqbal2020_adgraph.png" loading="lazy" width="500" height="500" /> text/html 2025-01-02T17:13:57+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/?image=design%3Awebsite_ranking_list_popularity_scheitle2018.png&ns=design&rev=1735838037&tab_details=history&media_do=diff&do=media <img src="https://measuretheweb.org/_media/design/website_ranking_list_popularity_scheitle2018.png?w=500&amp;h=500&amp;tok=f313a7" alt="website_ranking_list_popularity_scheitle2018.png" loading="lazy" width="500" height="500" /> text/html 2025-01-02T17:13:56+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/?image=design%3Awebsite_ranking_list_popularity_xie2022.png&ns=design&rev=1735838036&tab_details=history&media_do=diff&do=media <img src="https://measuretheweb.org/_media/design/website_ranking_list_popularity_xie2022.png?w=500&amp;h=500&amp;tok=24e12f" alt="website_ranking_list_popularity_xie2022.png" loading="lazy" width="500" height="500" /> text/html 2025-01-02T17:13:50+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/?image=design%3Awebsite_classification_coverage_vallina2020.png&ns=design&rev=1735838030&tab_details=history&media_do=diff&do=media <img src="https://measuretheweb.org/_media/design/website_classification_coverage_vallina2020.png?w=500&amp;h=500&amp;tok=197176" alt="website_classification_coverage_vallina2020.png" loading="lazy" width="500" height="500" /> text/html 2025-01-02T17:13:50+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/?image=design%3Awebsite_classification_popularity_vallina2020.png&ns=design&rev=1735838030&tab_details=history&media_do=diff&do=media <img src="https://measuretheweb.org/_media/design/website_classification_popularity_vallina2020.png?w=500&amp;h=500&amp;tok=d062ce" alt="website_classification_popularity_vallina2020.png" loading="lazy" width="500" height="500" /> text/html 2025-01-02T17:13:49+00:00 karelkubicek (karelkubicek@undisclosed.example.com) https://measuretheweb.org/?image=design%3Awebsite_ranking_list_manipulations_lepochat2019.png&ns=design&rev=1735838029&tab_details=history&media_do=diff&do=media <img src="https://measuretheweb.org/_media/design/website_ranking_list_manipulations_lepochat2019.png?w=500&amp;h=500&amp;tok=c4b64e" alt="website_ranking_list_manipulations_lepochat2019.png" loading="lazy" width="500" height="500" />