Differences

This shows you the differences between two versions of the page.

--- design:mobile_and_app_measurement [2026/08/27 07:16] – External-currency review: fix nine release dates that an off-by-one releases.atom parse had paired with the previous release (TrollStore 2.1.1 is 2026-04-01, not 2024-09-02); LSPosed default branch stopped 2024-01-07; Xposed archive vs last-push distincti karel.kubicek.claude
+++ design:mobile_and_app_measurement [2026/08/27 07:40] (current) – Generic review: add 'What to Read First'; scope the pinning walkthrough as Android and give the iOS route; add on-device VPN capture, per-app traffic attribution and QUIC; name the reusable datasets; replace an un-intersected inference with the computed 6 karel.kubicek.claude
@@ Line 13: / Line 13: @@
 Everything on this page about "the literature" is a claim about **seven venues** — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026, 5,859 papers with extracted full text. Mobile-app work is also published at mobile-systems and software-engineering venues (MobiSys, MobiCom, MSR, ICSE, ASE) and at usable-security venues (SOUPS), none of which are in the corpus, so treat the counts as "how these seven security and privacy venues do mobile", not as the state of mobile research.((The absent-venue list is a judgement about where this literature publishes, not a measurement — the corpus cannot see venues it does not contain. Two of the outside-corpus papers cited on this page are from MSR and one from ESEC/FSE, which is the shape of the gap.)) See [[Literature:Corpus]].
+===== What to Read First =====
+Five papers will orient you faster than anything else on this page, and each is here for a methodological reason rather than a topical one.
+  - {[pradeep2022_comparative]} — **the certificate-pinning measurement.** The only paper we found that separates //potential// from //actual// pinning and reports what fraction of pinned destinations its bypass toolchain failed on. Read the methodology, not just the results.
+  - {[bonett2018_discovering]} — **why your static analysis is wrong.** Injects executable leaks into real apps and measures what three well-known detectors miss. Read it before you write "we used FlowDroid". {[pauck2018_promises]} is the independent second opinion.
+  - {[koch2023_enough]} — **the attrition model.** 4,779 apps downloaded, 3,654 analysed, every app-level percentage over the second number. Copy this reporting discipline exactly.
+  - {[kollnig2022_iphones]} — **how to compare two platforms without cheating.** Random samples per platform, cross-platform app identity defined explicitly, code analysis and traffic analysis reported separately.
+  - {[kumar2022_investigation]} — **why your storefront matters.** Geodifferences in mobile apps, measured.
+Two systematic reviews give the taxonomy of approaches, both from outside these seven venues and both old enough to predate Frida's adoption and every LLM-based method: {[kong2019_testing]} on automated Android testing and {[li2017_static]} on static analysis. Read them for the map, not the tool list.
 ===== Which Population Is Yours =====
@@ Line 28: / Line 40: @@
 Two more signals overlap it heavily but not completely: 364 papers name a tool the extraction classified as ''mobile-instrumentation'' (32 of them outside both slices above), and 502 papers draw a population whose unit is ''mobile-apps'' (24 outside both). If you are searching the literature yourself, none of these three queries alone finds everything.
-Where the work is published, as a share of each venue's own output: PETS 12.9% (66 of 510), NDSS 11.7% (82 of 701), CCS 10.7% (106 of 990), USENIX Security 10.1% (143 of 1,410), IEEE S&P 8.2% (63 of 767), IMC 5.0% (32 of 638), TheWebConf 4.4% (37 of 843). **PETS and NDSS are where a mobile privacy measurement most often lands**; IMC and TheWebConf, the two most web-flavoured venues here, publish the least of it.
+Where the work is published, as a share of each venue's own output: PETS 12.9% (66 of 510), NDSS 11.7% (82 of 701), CCS 10.7% (106 of 990), USENIX Security 10.1% (143 of 1,410), IEEE S&P 8.2% (63 of 767), IMC 5.0% (32 of 638), TheWebConf 4.4% (37 of 843). As a //share of venue output// PETS and NDSS lead, but only narrowly and over a two-point spread; **by volume USENIX Security (143 papers) and CCS (106) publish the most of it**. IMC and TheWebConf, the two most web-flavoured venues here, publish the least either way. Note also that ''appAnalysis'' includes malware and vulnerability work, not only privacy measurement, so none of these shares is a share of "mobile privacy papers".
 ===== Getting the Apps =====
@@ Line 42: / Line 54: @@
 | **F-Droid** | fully open index, ''index-v2.json'' with signature verification((F-Droid, [[https://f-droid.org/docs/All_our_APIs/|All our APIs]]. Checked 2026-08-27.)) | ~thousands of FOSS apps. A fine population for a study //about FOSS//; a badly biased one for a study about apps users have |
-**The official Google Play Developer (Android Publisher) API cannot fetch other developers' APKs.** It is scoped to apps you publish. There is no sanctioned bulk-download path, which is why every large Android study in the corpus uses AndroZoo, an unofficial scraper, or a mirror.
+**The official Google Play Developer (Android Publisher) API cannot fetch other developers' APKs.** It is scoped to apps you publish. There is no sanctioned bulk-download path, which is why the corpus's large Android studies rely on AndroZoo, an unofficial scraper, a mirror, a dataset someone else already collected, or industry telemetry — never on a first-party API. Of the 96 ''appAnalysis'' papers with a population of 100,000 apps or more, the sources include vendor product telemetry (NortonLifeLock, Palo Alto Networks), volunteer-device capture (Lumen), and inherited research datasets (Drebin, Andrubis) alongside the three routes above.
 Scraper currency, checked 2026-08-27:
@@ Line 161: / Line 173: @@
 | ''AndroidViewClient'' | 6 (1.1%) | |
-Only **48 of 529** papers report any coverage measure, and **30 (5.7%)** combine a UI driver with traffic interception — the "drive it and watch the wire" pipeline that a privacy measurement actually needs. When coverage //is// measured, the gap is large: {[bonett2018_discovering]} could only confirm 2,026 of 7,584 injected leaks as executable using GUI exploration, and {[lin2026_chameleoscan]} needed an LLM to get through the UI at all — it reports resolving 85.96% of intrusive ads and 95.56% of pop-ups blocking its exploration, and found distracting elements on 19.21% of the pages it visited.
+Only **48 of 529** papers report any coverage measure, and **30 (5.7%)** combine a UI driver with traffic interception — the "drive it and watch the wire" pipeline that a privacy measurement actually needs. When coverage //is// measured, the gap is large: {[bonett2018_discovering]} could only confirm 2,026 of 7,584 injected leaks as executable using GUI exploration, and {[lin2026_chameleoscan]} used an LLM to get through the UI — it reports resolving 85.96% of intrusive ads and 95.56% of pop-ups blocking its exploration, and found distracting elements on 19.21% of the pages it visited.
 The two systematic reviews to read before building a driver are {[kong2019_testing]} on automated Android testing (2019) and {[li2017_static]} on static analysis (2017). Both are outside these seven venues, and both are old enough that they predate Frida's adoption and every LLM-based method — read them for the taxonomy of approaches, not for the tool list.
 <WRAP important>
-**Attrition is a number you must report, and almost nobody does.** {[koch2023_enough]} downloaded 4,779 apps and successfully analysed **3,654 (76.5%)** — 71.0% on Android, 85.7% on iOS. Every percentage in that paper is over 3,654, not 4,779, and the paper says so. That is the model to copy: the app set you //analysed// is not the app set you //sampled//, and the difference is not random.
+**Attrition is a number you must report, and almost nobody does.** {[koch2023_enough]} downloaded 4,779 apps and successfully analysed **3,654 (76.5%)** — 71.0% on Android, 85.7% on iOS. Its app-level results are all over 3,654, not 4,779, and the paper says so. That is the model to copy: the app set you //analysed// is not the app set you //sampled//, and the difference is not random.
 </WRAP>
@@ Line 172: / Line 184: @@
 To see an app's traffic you terminate its TLS at a proxy you control. Four things stand between you and that, in order.
+<WRAP info>
+**Steps 1–3 below are Android.** iOS interception is a different problem and the page treats it as one: on iOS you install your CA as a configuration profile //and// enable full trust for it in Settings, which gets you unpinned traffic; for pinned apps you need code injection, which on a store-signed binary means a jailbroken device — so the iOS route runs through the jailbreak tooling in [[#iOS]] above, and your device and firmware are part of your sample description. {[pradeep2022_comparative]} did exactly this (Checkra1n plus Frida) and is the worked example. The one iOS-specific pinning-bypass tool the corpus names, SSL Kill Switch 2, is unmaintained and its README caps support at iOS 14.2 — there is no maintained drop-in replacement, so expect to write Frida hooks yourself.
+</WRAP>
 **1. The app must trust your CA.** Since **Android 7.0 (API level 24)** — 2016 — the default trust anchors for an app are system CAs only. Android's own documentation gives the defaults verbatim: apps targeting "Android 6.0 (API level 23) and lower" get %%<certificates src="system" />%% //and// %%<certificates src="user" />%%; apps targeting "Android 7.0 (API level 24) to Android 8.1 (API level 27)" get system only.((Google, [[https://developer.android.com/privacy-and-security/security-config|Network security configuration]]. Checked 2026-08-27.)) Installing your root as a //user// certificate therefore does nothing for any modern app. Your options are to install it as a **system** CA (root plus Magisk, or a modified system image), or to repackage the app with a Network Security Configuration that trusts user CAs, or to hook the validation.
@@ Line 197: / Line 213: @@
 ^ Tool ^ Latest ^ Verdict ^
 | **Frida** | 17.17.0, 2026-08-05((GitHub, [[https://github.com/frida/frida|frida/frida]]. Checked 2026-08-27.)) | **Current, and the field's default.** 0% of 2010–2015 ''appAnalysis'' papers, **33.3% of 2025–2026** |
-| **objection** | 1.12.5, 2026-06-02((GitHub, [[https://github.com/sensepost/objection|sensepost/objection]]. Checked 2026-08-27.)) | **Current.** A Frida front-end; 8 of its 10 uses in this corpus are pinning bypass |
+| **objection** | 1.12.5, 2026-06-02((GitHub, [[https://github.com/sensepost/objection|sensepost/objection]]. Checked 2026-08-27.)) | **Current.** A Frida front-end; 7 of its 9 uses among the 529 are pinning bypass |
 | **Magisk** | v30.7, 2026-02-23((GitHub, [[https://github.com/topjohnwu/Magisk|topjohnwu/Magisk]]. Checked 2026-08-27.)) | **Current.** How you get a system CA onto a handset |
 | Xposed | archived; last release v89 (2017-12-17), last push 2020-06-08((GitHub, [[https://github.com/rovo89/Xposed|rovo89/Xposed]] — repository flag ''archived: true''. Checked 2026-08-27.)) | **Dead.** Peaked at 10.2% of 2016–2020 papers, 2.2% in 2025–2026 |
@@ Line 205: / Line 221: @@
 | TrustMeAlready | archived 2019-09-23((GitHub, [[https://github.com/ViRb3/TrustMeAlready|ViRb3/TrustMeAlready]]. Checked 2026-08-27.)) | **Dead** |
-**4. You have to report what you lost.** Only **13 of 529** papers mention the Network Security Configuration at all; **71 (13.4%)** mention pinning; **33** describe bypassing it. Given that traffic interception appears in 110 papers, **most papers that intercepted traffic never wrote down what they could not intercept**.
+**4. You have to report what you lost.** Intersected properly, over the full text of the **110 ''appAnalysis'' papers that named a traffic-interception tool**: **44 (40.0%)** mention pinning at all, **22 (20.0%)** describe bypassing it, **6 (5.5%)** mention the Network Security Configuration, and **65 (59.1%) mention none of the three**. So a clear majority of the papers that intercepted app traffic never wrote down what they could not intercept — and the ones that did are concentrated in the last few years.
+==== Three things that will bite you that no paper warns you about ====
+  * **A proxy sees the whole device, not your app.** Play Services, the OS itself and every other installed app also talk. A per-app claim needs per-app flow attribution — the UID behind the socket, a Frida stack trace, or one app installed at a time on a clean profile. A narrow probe for a paper //describing// that step finds **1 of 529**, which is more a comment on how rarely it is written down than on how often it is done. {[rodriguez2025_settings]} attributes connections to SDKs via Frida stack traces precisely because the proxy log alone cannot.
+  * **QUIC bypasses a TCP proxy.** An app speaking HTTP/3 over UDP will simply not appear in mitmproxy unless you block UDP 443 to force TCP fallback, or terminate QUIC. A traffic measurement that does neither has a silent hole whose size it cannot report. **14 of the 529 papers mention QUIC at all.**
+  * **There is a no-root route, and the corpus mostly forgot it.** An on-device ''VpnService'' app captures traffic without rooting, which is how the Lumen/ReCon/AntMonitor line of work collected from real users' devices. **39 of the 529 papers name such a tool or the API; 8 describe the technique.** It cannot break pinning, and it changes your population from "apps you chose" to "apps your volunteers had" — but for a field study, or for any measurement where rooting the handset is not an option, it is the technique to reach for. Lumen itself is gone (see below); the approach is not.
 <WRAP tip>
-A defensible pipeline in 2026, and the one the corpus's recent traffic-measurement papers converge on: **Magisk-rooted physical handset → your root as a system CA → mitmproxy → Frida/objection to unhook pinning where needed → Appium or UiAutomator to drive**, with a per-app record of //downloaded, installed, launched, traffic captured, traffic decrypted, pinning detected//. ''mitmproxy'' is in 52 papers (9.8%) and rising; that record is what turns "we intercepted traffic" into a number.
+A defensible pipeline in 2026, built from the components recent papers most often combine — Frida and mitmproxy co-occur in 16 of the 191 ''appAnalysis'' papers since 2023, and the full named stack in 3, so this is a recommendation assembled from practice rather than a practice the field has converged on: **Magisk-rooted physical handset → your root as a system CA → mitmproxy → Frida/objection to unhook pinning where needed → Appium or UiAutomator to drive**, with a per-app record of //downloaded, installed, launched, traffic captured, traffic decrypted, pinning detected//. ''mitmproxy'' is in 52 papers (9.8%) and rising; that record is what turns "we intercepted traffic" into a number.
 </WRAP>
@@ Line 216: / Line 238: @@
 ^ Target ^ Papers ^ Share ^
-| ''other'' (outside the enum) | 234 | 44.2% |
 | ''mobile-app'' | 198 | 37.4% |
 | ''vulnerability'' | 113 | 21.4% |
@@ Line 280: / Line 301: @@
 | Apple App Store (largest single spelling) | 11 | 2.1% |
 | F-Droid | 7 | 1.3% |
+**Datasets somebody else already collected.** ''pre-existing-dataset'' is the sampling method for 71 of those 400 papers (17.8%), and the page would be failing its own purpose if it did not name them. Beyond AndroZoo, the sources the corpus's own large populations draw on include **Drebin** and **Andrubis** (malware), the **Gamba et al. preinstalled-app dataset** and **Firmware Scanner** corpus, **Koodous**, **VirusShare** and the **AMD** dataset, **PrivacyGrade**, and the **Lumen** volunteer captures. Two of these are worth a warning: a malware dataset is not a sample of apps users have, and a 2014-era dataset used in a 2026 paper is a claim about 2014. Check the collection date before you inherit one.
 **How the sample was drawn**, for the 400 papers with a mobile-apps population: ''purposive'' 196 (49.0%), ''top-n'' 140 (35.0%), ''random'' 91 (22.8%), ''pre-existing-dataset'' 71 (17.8%), ''exhaustive'' 51 (12.8%), ''seed-and-crawl'' 38 (9.5%), ''convenience'' 35 (8.8%), ''stratified'' 28 (7.0%). Multi-valued, so shares do not sum to 100%. **"Top-//n// of a store chart" is the mobile equivalent of "Alexa top 1M"** and inherits every objection on [[Design:Website selection]] plus one more: the chart is per-country and per-day and there is no Tranco for it.
@@ Line 316: / Line 339: @@
 | mentions an LLM or GPT (case-sensitive) | 42 |
 | mentions store terms of service near scraping/downloading | 8 |
+| mentions QUIC | 14 |
+| names an on-device-VPN capture tool or the ''VpnService'' API | 39 |
+| describes the on-device-VPN capture technique | 8 |
+| describes attributing traffic to a specific app or library | 1 |
 ==== Methodology and limitations of these figures ====
@@ Line 347: / Line 374: @@
 | **Random UI events (Monkey)** as the exploration strategy | **The floor.** Still shipped, still used, still shallow. | 6.4% of papers; {[kong2019_testing]} is the survey of what else exists |
 | **LLMs for classification** (policies, UI text, labels) | **New in 2025–2026 and moving fast.** | ''classification.method = llm'' among ''appAnalysis'' papers goes 0% (pre-2021) → 0.5% (2021–2024) → **15.6%** (2025–2026). 90-paper denominator; do not over-read the level, but the direction is unambiguous |
-| **LLM-driven UI exploration** | **New, promising, one data point.** | {[lin2026_chameleoscan]} is the only paper in the corpus using an LLM to //get through the UI// rather than to classify text. One paper is not a practice |
+| **LLM-driven UI exploration** | **New, promising, one data point.** | {[lin2026_chameleoscan]} is the only paper we found in these seven venues using an LLM to //get through the UI// rather than to classify text. One paper is not a practice |
 | **''gplaycli''** to fetch APKs | **Abandoned.** | Last release 2020-03-10, last commit 2021-10-02 |
 | **''PlaystoreDownloader''** | **Archived 2024-07-27.** | Read-only repository |
@@ Line 379: / Line 406: @@
 <WRAP todo>
-  * **Nobody has re-measured the pinning-bypass success rate since 2022.** {[pradeep2022_comparative]} gives ≈51.5% of pinned Android destinations circumvented and ≈66.2% on iOS. Frida is now at 17.17.0. Repeating that single measurement would be cheap and would be cited by every mobile traffic paper thereafter.
+  * **We found no re-measurement of the pinning-bypass success rate since 2022 in these seven venues.** {[pradeep2022_comparative]} gives ≈51.5% of pinned Android destinations circumvented and ≈66.2% on iOS. Frida is now at 17.17.0. Repeating that single measurement would be cheap and would be cited by every mobile traffic paper thereafter.
   * **There is no maintained, validated third-party-SDK detector.** LibRadar and LibScout are the field's workhorses and both stopped in the 2010s. A maintained signature set with a published validation would immediately improve a large slice of the mobile privacy literature. {[specter2025_fingerprinting]} shows the scale of what is missing.
   * **How much does a mirror change the binary?** We found no study in these seven venues that systematically compares APKs from APKPure, APKMirror or Aptoide against the Play original by signature and content. 10 papers in ''appAnalysis'' name a mirror among the tools they used, and 46 mention some third-party APK/IPA mirror or downloader somewhere in the text, so this is not a hypothetical gap.