This is an old revision of the document!
Table of Contents
Crawling Location
The vantage point of a measurement is the network position your traffic appears to originate from: its IP address, and everything a website can infer from it — country, city, network type, and whether it looks like a person or a datacenter. It is chosen at least implicitly by every study that touches the live web, and it is the design decision least often reported: of the crawling papers in our corpus of seven major security and privacy venues, 70.8% record a vantage point and never say where it was (see Use in Publications).
That silence matters because the vantage point changes three separable things, and only the first is obvious:
- Which law the website thinks applies to you. Consent banners are routinely geo-targeted, so a GDPR study run from the wrong country measures a different website.
- What content you are served at all. Geoblocking, CDN geo-routing, and localisation mean the page differs by country regardless of privacy law.
- How your IP is treated. Datacenter, university, Tor and residential addresses receive measurably different treatment from bot management and from trackers.
This page covers all three, how researchers have actually chosen vantage points, the practical options, and how to verify that the vantage point you think you have is the one you got. It pairs with IP classification (classifying other people's addresses), Website selection (which sites), and Crawler (which tool).
Why the Vantage Point Changes Your Results
Jurisdiction: geo-targeted consent
Consent Management Platforms let a website operator bind a banner configuration to a visitor's region, resolved from the IP address. This is a documented product feature, not an inference: Cookiebot offers displaying the banner to visitors from specific regions only, and Usercentrics documents a "Display CMP only to EU and EEA users" option which “will exclude users that are located outside of this region, such as users from UK and Switzerland”.
The consequence is sharp: the EU quick-select bundle in a CMP does not include the United Kingdom or Switzerland. A crawl from Zürich or London can legitimately see no banner at all on a site that shows one to every visitor from Frankfurt. If your paper says “we crawled from Europe”, a reader cannot tell whether you were inside the configuration that your research question is about.
The strongest empirical treatment of the question is van Eijk et al.'s ConPro 2019 study [1van Eijk, Rob; Asghari, Hadi; Winter, Philipp; Narayanan, Arvind (2019): "The Impact of User Location on Cookie Notices (Inside and Outside of the European Union)", in: Workshop on Technology and Consumer Protection (ConPro), IEEE Security and Privacy Workshops. (Link)], and its result is more nuanced than the mechanism above suggests. Crawling 1,500 sites from each of 18 countries, they found that a site's top-level domain explains much more of the variance in cookie-notice behaviour than the visitor's vantage point does — most websites apply one set of rules to everybody. The exception is precisely the interesting case: notices on .com domains do differ between visitors inside and outside the EU.
Read that as a two-part instruction rather than a licence to ignore location. Most sites will not change for you, so a single well-chosen vantage point is usually defensible — but the subset that does vary is the subset a compliance study is about, so the vantage point must be reported and justified either way. Ogut et al. [2Ogut, Aysun; Turanlioglu, Berke; Metiner, Doruk Can; Levi, Albert; Yilmaz, Cemal; Cetin, Orcun; Uluagac, Selcuk (2024): "Dissecting Privacy Perspectives of Websites Around the World: "Aceptar Todo, Alle Akzeptieren, Accept All..."", in: Proceedings of the USENIX Security Symposium. (Link)] take the design one step further, collecting each country both from inside it and while “simulat[ing] access from outside the country”, which makes the in/out contrast a measured variable rather than an assumption.
Content: geoblocking and geo-differentiation
Even setting privacy law aside, the web is not the same everywhere:
- Outright geoblocking. McDonald et al. [3McDonald, Allison; Bernhard, Matthew; Valenta, Luke; VanderSloot, Benjamin; Scott, Will; Sullivan, Nick; Halderman, J. Alex; Ensafi, Roya (2018): "403 Forbidden: A Global View of CDN Geoblocking", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] measured CDN-level geoblocking from 177 countries, finding server-side blocking of entire regions. A crawl from a blocked country records an absence that is a property of your vantage point, not of the site's tracking behaviour.
- Geo-differentiated content. Kumar et al. [4Kumar, Renuka; Virkud, Apurva; Sundara Raman, Ram; Prakash, Atul; Ensafi, Roya (2022): "A Large-scale Investigation into Geodifferences in Mobile Apps", in: Proceedings of the USENIX Security Symposium. (Link)] compared mobile apps across 26 countries chosen “to have reliable direct vantage points”, finding systematic differences in availability and behaviour.
- Personalisation. Kliman-Silver et al. [5Kliman-Silver, Chloe; Hannak, Aniko; Lazer, David; Wilson, Christo; Mislove, Alan (2015): "Location, Location, Location: The Impact of Geolocation on Web Search Personalization", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] showed that geolocation drives measurable web-search personalisation, so location is a confound in any study of ranked or targeted output.
Treatment: what your IP says about you
The single most useful paper for this page is Jueckstock et al. [6Jueckstock, Jordan; Sarker, Shaown; Snyder, Peter; Beggs, Aidan; Papadopoulos, Panagiotis; Varvello, Matteo; Livshits, Benjamin; Kapravelos, Alexandros (2021): "Towards Realistic and Reproducible Web Crawl Measurements", in: Proceedings of the ACM Web Conference. (DOI)], which treats the vantage point as an experimental variable and measures its effect directly. They crawled Tranco's top 25k from three deliberately contrasting positions and describe the design in exactly the terms a methods section should:
We collected data from three distinct, representative VPs: a major research university network, a nearby residential ISP network, and a popular cloud provider's network.
Their framing is worth adopting wholesale. The residential network is “the ostensible best-case in VP realism, as it is used exclusively for end-user activities”; the cloud network (Amazon AWS) is “an expected worst-case in VP realism as its typical use is for infrastructure rather than end-user network access”; and the university network “combines both end-user and infrastructure activities”, so “its realism is presumed to fall somewhere between the residential and cloud extremes”. That middle case is the one worth dwelling on, because a university network is the most common vantage point in the corpus — the field's default sits between the best and worst case and is rarely acknowledged as such.
They report that web privacy and security measurements are significantly affected by vantage point and browser configuration — the empirical basis for treating this as a first-class design choice rather than an implementation detail.
The mechanism behind the cloud penalty is commercial: bot-management and CDN vendors score datacenter address space as low-trust, so cloud-hosted crawlers are disproportionately challenged, rate-limited or served alternative content.1) The same logic in a more extreme form is documented for anonymity networks: Khattak et al. [7Khattak, Sheharbano; Fifield, David; Afroz, Sadia; Javed, Mobin; Sundaresan, Srikanth; McCoy, Damon; Paxson, Vern; Murdoch, Steven J. (2016): "Do You See What I See? Differential Treatment of Anonymous Users", in: Proceedings of the Network and Distributed System Security Symposium. (Link)] measured differential treatment of Tor users, so a Tor vantage point measures Tor's reception as much as the site's privacy behaviour.
Use in Publications
The figures below come from a structured extraction over 5,859 full-text papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. Unless stated otherwise the population is the 1,120 papers that ran a crawl, and sentinel values (not-stated) are counted as what they are rather than as answers. The 2025 and 2026 venue-years are provisional — CCS and IMC 2026 have not been held, and IEEE S&P and WWW 2026 abstracts are not yet in the selection source — so they are under-represented by construction. Methodology and limitations are at the end of this section.
Most papers do not say where they measured from
| Outcome | Papers | Share of 1,120 |
|---|---|---|
| No vantage point recorded at all | 31 | 2.8% |
| Vantage point recorded, location not stated | 793 | 70.8% |
| States at least one place | 296 | 26.4% |
Of those that do, most use a single vantage point
| Distinct places | Papers | Share of 296 |
|---|---|---|
| 1 | 174 | 58.8% |
| 2 | 45 | 15.2% |
| 3 | 30 | 10.1% |
| 4 or more | 47 | 15.9% |
Multi-vantage measurement is therefore 41.2% of papers that state a location, but only 10.9% of all crawling papers. Reporting it the second way is the honest framing: the denominator is every paper that could have said something.
Where from
| Place | Papers | Share of 296 stating |
|---|---|---|
| United States | 182 | 61.5% |
| Germany | 60 | 20.3% |
| Europe (no country given) | 42 | 14.2% |
| multi-country (e.g. “61 countries”) | 37 | 12.5% |
| United Kingdom | 26 | 8.8% |
| China | 23 | 7.8% |
| India | 23 | 7.8% |
| Singapore | 19 | 6.4% |
| Canada | 18 | 6.1% |
| Australia | 18 | 6.1% |
Aggregated: United States or North America 193 papers (65.2%), EU/EEA 124 (41.9%), both 74. Shares exceed 100% because a paper can name several places. Ninety-one distinct places appear in total, so the long tail is thin — the field measures the web overwhelmingly from the United States and from Germany.
From what kind of infrastructure
Only 445 of 1,120 (39.7%) state an infrastructure type at all.
| Infrastructure | Papers | Share of 445 stating |
|---|---|---|
| University network | 134 | 30.1% |
| Cloud provider | 123 | 27.6% |
| Research testbed2) | 122 | 27.4% |
| Commercial VPN | 48 | 10.8% |
| Volunteer devices | 29 | 6.5% |
| Residential | 19 | 4.3% |
| Proxy service | 18 | 4.0% |
| Tor | 15 | 3.4% |
| Mobile network | 9 | 2.0% |
Note how rare residential is (4.3% of the papers that say anything) against Jueckstock et al.'s finding that it is the realism best-case. The named providers tell the same story — of the 165 papers naming a platform that folds to a provider family, Amazon dominates:
| Provider family | Papers | Share of 165 naming | Spellings folded |
|---|---|---|---|
| Amazon AWS / EC2 | 68 | 41.2% | 16 |
| Commercial VPN (named) | 20 | 12.1% | 17 |
| Google Cloud | 11 | 6.7% | 6 |
| Tor | 11 | 6.7% | 2 |
| University network | 10 | 6.1% | 10 |
| DigitalOcean | 9 | 5.5% | 3 |
| Other research testbed | 9 | 5.5% | 9 |
| PlanetLab (discontinued) | 8 | 4.8% | 1 |
| Residential/datacenter proxy vendor | 8 | 4.8% | 4 |
| Alibaba / Aliyun / Baidu | 6 | 3.6% | 6 |
| ISP / mobile carrier | 6 | 3.6% | 6 |
| Linode / Vultr / OVH / Hetzner | 6 | 3.6% | 4 |
| Microsoft Azure | 5 | 3.0% | 1 |
| RIPE Atlas | 5 | 3.0% | 1 |
| CDN (Cloudflare / Akamai) | 4 | 2.4% | 3 |
| M-Lab / CAIDA Ark | 2 | 1.2% | 2 |
A service string can name more than one platform (“Amazon EC2 and Aliyun”), so it counts in each family and the shares exceed 100%. 186 papers name a platform in total; the 21 not in the table named something that is not a vantage point at all — vantage.serviceName also catches services a paper queried (VirusTotal, Google Translate, Safe Browsing). The full residue is on the provenance page.
Reporting is improving, slowly
| Indicator | 2010–2013 | 2014–2017 | 2018–2021 | 2022–2024 | 2025–2026 (provisional) |
|---|---|---|---|---|---|
| Papers that crawled | 102 | 167 | 308 | 345 | 198 |
| States a place | 20.6% | 24.0% | 26.9% | 27.5% | 28.8% |
| More than one place (of those stating) | 52.4% | 30.0% | 34.9% | 45.3% | 47.4% |
| EU/EEA vantage (of those stating) | 19.0% | 25.0% | 38.6% | 46.3% | 59.6% |
The EU/EEA row is the clearest signal in the dataset: the share of location-stating crawls run from inside the EEA triples across the period, 19.0% to 59.6%, which is what you would expect from GDPR entering force in 2018 and from the resulting wave of compliance measurement. Location reporting itself improves far less — from 20.6% to 28.8% over sixteen years. Read the last column with care: CCS and IMC 2026 have not been held and two more 2026 venue-years are incompletely selected, so it rests on fewer papers than the 198 count suggests it should.
Legal framing does predict the vantage point
Grouping the 1,120 crawling papers by which law, if any, they assessed:
| Crawling papers that… | N | State a place | EU/EEA vantage | US vantage |
|---|---|---|---|---|
| assessed an EU law (GDPR / ePrivacy / DSA) | 93 | 55.9% | 48.4% | 26.9% |
| assessed a US law (CCPA / COPPA / …) | 43 | 55.8% | 25.6% | 51.2% |
| assessed no law | 989 | 22.9% | 7.7% | 15.4% |
Papers with a legal question are more than twice as likely to say where they measured from, and they line their vantage point up with the jurisdiction they are assessing. Of the 93 crawling papers assessing an EU law, 52 state a place and 45 of those (86.5%) measured from the EU/EEA — good practice, clearly visible in the data. The residue is the interesting part: 7 papers assessed an EU law from outside the EU/EEA, and 41 assessing an EU law never said where they were.
The gap is worse where it matters most. Of the 349 papers that state what their crawler did with the consent banner — accept, reject, a CMP-specific choice, or explicitly no interaction — only 68 (19.5%) state an EU/EEA vantage, and 210 give no location at all. Consent behaviour is the single most geo-dependent thing on the web, and the majority of papers interacting with it do not report the jurisdiction they observed it from.
Both figures come from the same per-paper record, so the 349 are a subset of the 1,120 crawling papers rather than a separately sampled group: the consent action and the vantage point were extracted in one pass from the same full text, and a paper counts here only if it stated its consent action explicitly.
Good examples of the reporting standard this page argues for, all from the corpus:
- “This crawl was made from France on September 20th and 21st 2019.” [8Matte, Célestin; Bielova, Nataliia; Santos, Cristiana Teixeira (2020): "Do Cookie Banners Respect my Choice? Measuring Legal Compliance of Banners from IAB Europe's Transparency and Consent Framework", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)]
- “We crawl websites using 30 German datacenter IP addresses provided by The Bright Initiative from Bright Data.” [9Bouhoula, Ahmed; Kubicek, Karel; Zac, Amit; Cotrini, Carlos; Basin, David (2024): "Automated Large-Scale Analysis of Cookie Notice Compliance", in: 33rd USENIX Security Symposium (USENIX Security 24), pp. 1723-1739. USENIX Association, Philadelphia, PA. (Link)]
- “all but one use an IP address associated with a European server (Frankfurt am Main; DEU) … the other is run from a US IP address (Council Bluffs, IA; USA).” [10Demir, Nurullah; Urban, Tobias; Pohlmann, Norbert; Wressnegger, Christian (2024): "A Large-Scale Study of Cookie Banner Interaction Tools and their Impact on Users' Privacy", in: Proceedings on Privacy Enhancing Technologies, pp. 5-20. (DOI)]
- “We choose three geolocations for our measurement: (1) Germany (EU), (2) Japan (AS), and (3) the United States (NA).” — via ProtonVPN [11Demir, Nurullah; Große-Kampmann, Matteo; Urban, Tobias; Wressnegger, Christian; Holz, Thorsten; Pohlmann, Norbert (2022): "Reproducibility and Replicability of Web Measurement Studies", in: Proceedings of the ACM Web Conference. (DOI)]
Each gives country, and where relevant city, infrastructure and date. That is enough for a reader to reason about jurisdiction and for a replication to match it.
Methodology and limitations of these figures
- How they were produced. One structured record per paper was extracted from full text, each tuple carrying a verbatim evidence quote and its section, so any figure here can be traced to the sentence that supports it. Locations are free text (“Frankfurt, Germany”, “US-East”, “61 countries across all world regions”) and were normalised to countries or regions before counting by
scripts/geo.mjs; 15 distinct strings across 13 papers could not be mapped and are excluded, and 5 of those papers are left with no place at all. The full residue list is on the provenance page. Papers, never tuples, are counted. - Silence is not absence. “Does not state a location” means the paper did not say, not that the authors did not know. These are reporting figures.
- Venue coverage. Seven venues only, and 2025–2026 are incomplete for reasons of calendar and indexing rather than relevance, so per-year trends should be read as ending in 2024. Which venues, which years and what each stage of the selection funnel costs are on corpus.
- Field stability. The stated/not-stated distinction and the infrastructure enum are reproducible to within a few points on a repeat extraction. Free-text service names are much less stable and are reported folded into families and as rankings, not as precise figures. The
Amazon AWS / EC2row, for instance, merges at least five spellings. crawledis defined as a paper whose crawl configuration was recorded or whose study types include an automated web crawl (1,120 papers, 19.1% of the corpus). This corpus is seven broad security venues, not a web-measurement corpus, so shares of all 5,859 papers would be meaningless here.- Every query behind this section, its report script and its unedited output are on crawling_location; corpus-level caveats are on corpus.
Choosing a Vantage Point
| Option | Realism | Geographic reach | Reproducible | Ethics burden | Notes |
|---|---|---|---|---|---|
| University network | Medium — real users, but recognisable address space | One site | High | Low | The corpus's most common choice. Check your AUP before crawling. |
| Cloud provider | Low — datacenter address space | Wide (30–40+ regions) | High | Low | Cheap, scriptable, and the most likely to be blocked or challenged. |
| Commercial VPN | Medium | Wide, but labels are often wrong | Medium | Low | Verify every exit; see Verify the Vantage Point. |
| Residential proxy | High | Very wide, city-level | Low — pool churns | High | Read the warning below before using one. |
| Volunteer devices / browser extension | Highest | Wherever your volunteers are | Low | High — human subjects | Used by [12Zeber, David; Bird, Sarah; Oliveira, Camila; Rudametkin, Walter; Segall, Ilana; Wolls´en, Fredrik; Lopatka, Martin (2020): "The Representativeness of Automated Web Crawls as a Surrogate for Human Browsing", in: Proceedings of The Web Conference 2020, pp. 167–178. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] and [13Iordanou, Costas; Smaragdakis, Georgios; Poese, Ingmar; Laoutaris, Nikolaos (2018): "Tracing Cross Border Web Tracking", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]. |
| Research platform (RIPE Atlas, M-Lab, CAIDA Ark) | Medium | Very wide | High | Low–medium | Network-layer measurement; mostly not a browser. |
University and cloud
Together these are the default, and between them they account for most of what the corpus reports. The trade-off is exactly the one Jueckstock et al. measured: cloud gives you geographic choice and reproducibility at the cost of realism, and it is the configuration most likely to be treated as a bot. If you crawl from a cloud region, say so, and treat unexplained blocks as a possible artefact of the vantage point rather than a finding.
Commercial VPN
The practical requirement is a scriptable interface. Mullvad documents a SOCKS5 proxy on 10.64.0.1:1080 reachable while connected, with per-server hostnames, which makes per-context exit selection straightforward from Playwright or Selenium; IVPN documents an equivalent SOCKS5 setup. Others expose WireGuard configuration files that can be scripted with more work. ProtonVPN — used by [11Demir, Nurullah; Große-Kampmann, Matteo; Urban, Tobias; Wressnegger, Christian; Holz, Thorsten; Pohlmann, Norbert (2022): "Reproducibility and Replicability of Web Measurement Studies", in: Proceedings of the ACM Web Conference. (DOI)] — ships an official Linux CLI.
Residential and mobile proxies
Treat residential proxies as a human-subjects and legal question, not a procurement one. These networks resolve your traffic through other people's devices. The academic literature on how the pools are built is unflattering: Mi et al. [14Mi, Xianghang; Feng, Xuan; Liao, Xiaojing; Liu, Baojun; Wang, XiaoFeng; Qian, Feng; Li, Zhou; Alrwais, Sumayah A.; Sun, Limin; Liu, Ying (2019): "Resident Evil: Understanding Residential IP Proxy as a Dark Service", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] characterised residential IP proxy networks as a “dark service”, finding hosts enrolled without meaningful consent.
The risk is current, not historical. On 2 July 2026 the FBI, with the IRS Criminal Investigation division, Google's Threat Intelligence Group and Lumen, seized hundreds of domains belonging to NetNut, a major residential proxy provider operated by Alarum Technologies, over an alleged overlap between its exit-node pool and the “Popa” botnet of at least two million compromised devices. Alarum disputes the allegations, stated it had not been formally contacted by the FBI as of 3 July 2026, and paused parts of the network. Treat the case as unresolved — but note that a researcher who had bought bandwidth from that pool would now be explaining it to their ethics board.
If you use one, we suggest: name the provider in the paper, state what the provider claims about consent, say whether your IRB or ethics board reviewed that claim specifically, and prefer providers used by prior peer-reviewed work. Bright Data's research arm (“The Bright Initiative”) is the route taken by [9Bouhoula, Ahmed; Kubicek, Karel; Zac, Amit; Cotrini, Carlos; Basin, David (2024): "Automated Large-Scale Analysis of Cookie Notice Compliance", in: 33rd USENIX Security Symposium (USENIX Security 24), pp. 1723-1739. USENIX Association, Philadelphia, PA. (Link)] — and note that they used datacenter IPs from it, not residential ones. See Ethics.
Research measurement platforms
RIPE Atlas (probes and anchors worldwide, credit-based), M-Lab and CAIDA Ark give wide, citable, reproducible geographic coverage. The catch is layer: they are built for network measurement, not for driving a browser, so they suit DNS, reachability and latency questions rather than tracking or consent. PlanetLab is discontinued — it appears in 8 papers in the corpus and is not an option for new work; the EdgeNet project is the nearest successor.
Verify the Vantage Point
Two independent reasons not to trust the label on your vantage point:
- VPN providers mislabel their own servers. IPinfo's December 2025 measurement study reports that 17 of 20 VPN providers had location mismatches, with 38 countries where a provider advertised presence and had no physical infrastructure. This is a vendor study rather than peer-reviewed work, but it matches the academic finding: Weinberg et al. [15Weinberg, Zachary; Cho, Shinyoung; Christin, Nicolas; Sekar, Vyas; Gill, Phillipa (2018): "How to Catch when Proxies Lie: Verifying the Physical Locations of Network Proxies with Active Geolocation", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] built active-geolocation techniques specifically to catch proxies lying about their physical location.
- Geolocation databases disagree with each other. They agree well at country level and poorly below it, which is the subject of IP classification.
The following script checks both. It queries several free services for your egress IP, reports whether they agree, flags datacenter/VPN/proxy addresses, and exits non-zero if the country is not the one you expected — so it can gate a crawl rather than merely inform you. Free tiers are rate-limited, so call it once when a vantage point comes up and again when it goes down, not per request. Adding a –json branch that dumps the same report as a dict is a two-line change if you want to log it alongside the crawl.
- verify_vantage.py
#!/usr/bin/env python3 """Verify where your crawler actually appears to be, before you trust a measurement. Exits non-zero if the observed country differs from --expect: python3 verify_vantage.py --expect DE || exit 1 """ import argparse, json, sys, urllib.error, urllib.request from collections import Counter TIMEOUT, UA = 12, "measuretheweb-vantage-check/1.0" # Each entry maps a service's response onto (country_iso, city, network). SERVICES = { "ipinfo.io": ("https://ipinfo.io/json", lambda d: (d["country"], d.get("city"), d.get("org"))), "ip-api.com": ("http://ip-api.com/json", lambda d: (d["countryCode"], d.get("city"), d.get("as"))), "ifconfig.co": ("https://ifconfig.co/json", lambda d: (d["country_iso"], d.get("city"), d.get("asn_org"))), "ipwho.is": ("https://ipwho.is", lambda d: (d["country_code"], d.get("city"), d.get("connection", {}).get("isp"))), "ipapi.is": ("https://api.ipapi.is", lambda d: (d["location"]["country_code"], d["location"].get("city"), d.get("company", {}).get("name"))), } # ipapi.is is the only one of the five shipping proxy/datacenter flags for free. FLAGS = ["is_datacenter", "is_vpn", "is_proxy", "is_tor", "is_crawler", "is_mobile"] # The EEA, which is what GDPR/ePrivacy jurisdiction follows. # Note the traps: Switzerland and the UK are NOT in it. EEA = {"AT","BE","BG","HR","CY","CZ","DK","EE","FI","FR","DE","GR","HU","IE","IT","LV", "LT","LU","MT","NL","PL","PT","RO","SK","SI","ES","SE","IS","LI","NO"} def fetch(url): req = urllib.request.Request(url, headers={"User-Agent": UA, "Accept": "application/json"}) with urllib.request.urlopen(req, timeout=TIMEOUT) as resp: return json.loads(resp.read().decode("utf-8")) def main(): ap = argparse.ArgumentParser() ap.add_argument("--expect", metavar="ISO2", help="country the vantage point should be in") args = ap.parse_args() results, raw, errors = {}, {}, {} for name, (url, extract) in SERVICES.items(): try: data = fetch(url) except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError) as exc: errors[name] = f"{type(exc).__name__}: {exc}" # rate-limited service: not fatal continue if data.get("error") or data.get("success") is False: errors[name] = f"service error: {data.get('reason') or data.get('message')}" continue raw[name], results[name] = data, extract(data) if len(results) < 2: print(f"FAIL: only {len(results)} service(s) answered; cannot cross-check", file=sys.stderr) return 2 countries = Counter(c for c, _, _ in results.values()) cities = Counter(city for _, city, _ in results.values() if city) country, votes = countries.most_common(1)[0] flags = {f: raw["ipapi.is"].get(f) for f in FLAGS} if "ipapi.is" in raw else {} ip = next(iter(raw.values())).get("ip") or next(iter(raw.values())).get("query") print(f"egress IP {ip}") print(f"country {country} (agreement {votes}/{len(results)}" f"{', UNANIMOUS' if len(countries) == 1 else ', DISPUTED'})") print(f"in EEA {country in EEA}") print(f"city {' / '.join(f'{c} x{n}' for c, n in cities.most_common())}" f"{'' if len(cities) <= 1 else ' <-- services disagree'}") print() for name, (c, city, net) in sorted(results.items()): print(f" {name:<14} {c} {str(city):<14} {net}") if flags: on = [f for f, v in flags.items() if v] print(f"\n ipapi.is flags {', '.join(on) if on else 'none set'}") for name, err in errors.items(): print(f" {name:<14} unavailable: {err}") code = 0 if args.expect and country.upper() != args.expect.upper(): print(f"\nFAIL: expected {args.expect.upper()}, observed {country}", file=sys.stderr) code = 1 if flags.get("is_datacenter"): print("\nWARNING: datacenter IP. Sites and CDNs treat this differently " "from residential traffic.", file=sys.stderr) return code if __name__ == "__main__": sys.exit(main())
Real output, run from a Swiss residential connection on 2026-08-05:
egress IP 82.220.84.43 country CH (agreement 5/5, UNANIMOUS) in EEA False city Zurich x2 / Zürich x1 / Samnaun x1 / Lausanne x1 <-- services disagree ifconfig.co CH Zurich BSE Software GmbH ip-api.com CH Samnaun AS9044 BSE Software GmbH ipapi.is CH Lausanne BSE Software GmbH / SolNet ISP ipinfo.io CH Zürich BSE Software GmbH ipwho.is CH Zurich BSE Software GmbH ipapi.is flags none set
Two lessons in one run. Country is reliable, city is not — all five services agree on Switzerland, and their city guesses span Zurich, Samnaun and Lausanne, points more than 200 km apart. Do not build a city-level claim on a single geolocation service. And in EEA False: Switzerland is not in the EEA, so this is not a valid vantage point for observing an EEA-targeted consent banner, however European it feels.
What to Report
For the vantage point to be reusable by a reader, a methods section needs:
- Country, and city if the claim depends on it.
- Infrastructure type: university, cloud region, residential ISP, VPN, residential proxy, volunteer devices.
- Provider name, and for cloud the region identifier (
eu-central-1is more useful than “AWS”). - Dates of the crawl — geo-targeting configurations change.
- Whether the location was verified, and how.
- For multi-vantage designs, whether crawls were simultaneous. Sequential crawls from different countries confound location with time.
- For residential proxies, the provider's consent claim and your ethics review of it.
Open Questions
- No peer-reviewed cross-vendor measurement of how much datacenter IP address space is penalised by bot-management vendors. Jueckstock et al. [6Jueckstock, Jordan; Sarker, Shaown; Snyder, Peter; Beggs, Aidan; Papadopoulos, Panagiotis; Varvello, Matteo; Livshits, Benjamin; Kapravelos, Alexandros (2021): "Towards Realistic and Reproducible Web Crawl Measurements", in: Proceedings of the ACM Web Conference. (DOI)] measure the effect on privacy metrics but not the mechanism per vendor.
- How stable is CMP geo-targeting configuration over time? All the industry documentation describes the feature; nobody appears to have measured how often operators change the region-to-template binding.
- Whether commercial VPN mislabelling has been re-measured academically since [15Weinberg, Zachary; Cho, Shinyoung; Christin, Nicolas; Sekar, Vyas; Gill, Phillipa (2018): "How to Catch when Proxies Lie: Verifying the Physical Locations of Network Proxies with Active Geolocation", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] (2018). The only recent figures we found are a vendor study.
- Residential proxy pool overlap with known botnets, post-NetNut. Spur, Synthient and Nokia Deepfield published attributions in June 2026; a systematic academic treatment would be valuable for ethics review.
Related Pages
- IP classification — classifying the addresses you observe, and the geolocation databases this page's script exercises.
- Website selection — country-specific top lists (CrUX has country breakdowns; SecRank is Chinese-DNS-based) interact with vantage choice.
- Archives — web archives sidestep the vantage point and introduce their own biases.
- Crawler — proxy and per-context network configuration per crawling library.
- Stateful stateless — the other axis Jueckstock et al. [6Jueckstock, Jordan; Sarker, Shaown; Snyder, Peter; Beggs, Aidan; Papadopoulos, Panagiotis; Varvello, Matteo; Livshits, Benjamin; Kapravelos, Alexandros (2021): "Towards Realistic and Reproducible Web Crawl Measurements", in: Proceedings of the ACM Web Conference. (DOI)] vary.
- Consent — what to do with the banner once you are in the right jurisdiction.
- Ethics — residential proxies, volunteer devices, and acceptable-use policies.
References
- [1]
- van Eijk, Rob; Asghari, Hadi; Winter, Philipp; Narayanan, Arvind (2019): "The Impact of User Location on Cookie Notices (Inside and Outside of the European Union)", in: Workshop on Technology and Consumer Protection (ConPro), IEEE Security and Privacy Workshops. (Link)
- [2]
- Ogut, Aysun; Turanlioglu, Berke; Metiner, Doruk Can; Levi, Albert; Yilmaz, Cemal; Cetin, Orcun; Uluagac, Selcuk (2024): "Dissecting Privacy Perspectives of Websites Around the World: "Aceptar Todo, Alle Akzeptieren, Accept All..."", in: Proceedings of the USENIX Security Symposium. (Link)
- [3]
- McDonald, Allison; Bernhard, Matthew; Valenta, Luke; VanderSloot, Benjamin; Scott, Will; Sullivan, Nick; Halderman, J. Alex; Ensafi, Roya (2018): "403 Forbidden: A Global View of CDN Geoblocking", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
- [4]
- Kumar, Renuka; Virkud, Apurva; Sundara Raman, Ram; Prakash, Atul; Ensafi, Roya (2022): "A Large-scale Investigation into Geodifferences in Mobile Apps", in: Proceedings of the USENIX Security Symposium. (Link)
- [5]
- Kliman-Silver, Chloe; Hannak, Aniko; Lazer, David; Wilson, Christo; Mislove, Alan (2015): "Location, Location, Location: The Impact of Geolocation on Web Search Personalization", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
- [6]
- Jueckstock, Jordan; Sarker, Shaown; Snyder, Peter; Beggs, Aidan; Papadopoulos, Panagiotis; Varvello, Matteo; Livshits, Benjamin; Kapravelos, Alexandros (2021): "Towards Realistic and Reproducible Web Crawl Measurements", in: Proceedings of the ACM Web Conference. (DOI)
- [7]
- Khattak, Sheharbano; Fifield, David; Afroz, Sadia; Javed, Mobin; Sundaresan, Srikanth; McCoy, Damon; Paxson, Vern; Murdoch, Steven J. (2016): "Do You See What I See? Differential Treatment of Anonymous Users", in: Proceedings of the Network and Distributed System Security Symposium. (Link)
- [8]
- Matte, Célestin; Bielova, Nataliia; Santos, Cristiana Teixeira (2020): "Do Cookie Banners Respect my Choice? Measuring Legal Compliance of Banners from IAB Europe's Transparency and Consent Framework", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
- [9]
- Bouhoula, Ahmed; Kubicek, Karel; Zac, Amit; Cotrini, Carlos; Basin, David (2024): "Automated Large-Scale Analysis of Cookie Notice Compliance", in: 33rd USENIX Security Symposium (USENIX Security 24), pp. 1723-1739. USENIX Association, Philadelphia, PA. (Link)
- [10]
- Demir, Nurullah; Urban, Tobias; Pohlmann, Norbert; Wressnegger, Christian (2024): "A Large-Scale Study of Cookie Banner Interaction Tools and their Impact on Users' Privacy", in: Proceedings on Privacy Enhancing Technologies, pp. 5-20. (DOI)
- [11]
- Demir, Nurullah; Große-Kampmann, Matteo; Urban, Tobias; Wressnegger, Christian; Holz, Thorsten; Pohlmann, Norbert (2022): "Reproducibility and Replicability of Web Measurement Studies", in: Proceedings of the ACM Web Conference. (DOI)
- [12]
- Zeber, David; Bird, Sarah; Oliveira, Camila; Rudametkin, Walter; Segall, Ilana; Wolls´en, Fredrik; Lopatka, Martin (2020): "The Representativeness of Automated Web Crawls as a Surrogate for Human Browsing", in: Proceedings of The Web Conference 2020, pp. 167–178. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)
- [13]
- Iordanou, Costas; Smaragdakis, Georgios; Poese, Ingmar; Laoutaris, Nikolaos (2018): "Tracing Cross Border Web Tracking", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
- [14]
- Mi, Xianghang; Feng, Xuan; Liao, Xiaojing; Liu, Baojun; Wang, XiaoFeng; Qian, Feng; Li, Zhou; Alrwais, Sumayah A.; Sun, Limin; Liu, Ying (2019): "Resident Evil: Understanding Residential IP Proxy as a Dark Service", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
- [15]
- Weinberg, Zachary; Cho, Shinyoung; Christin, Nicolas; Sekar, Vyas; Gill, Phillipa (2018): "How to Catch when Proxies Lie: Verifying the Physical Locations of Network Proxies with Active Geolocation", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
