Skip to content

prio: order tied URLs per network and day (needs decision) - #1295

Draft
aagbsn wants to merge 3 commits into
masterfrom
prio_random_tiebreak
Draft

aagbsn wants to merge 3 commits into
masterfrom
prio_random_tiebreak

Conversation

@aagbsn

@aagbsn aagbsn commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Needs decision, optional. This is one way to break ties between URLs of equal weight. We haven't chosen between it, a random tie-break, none at all, or an explicit confirmation target in the counting formula.
#1294 doesn't depend on it.

compute_priorities sorts URLs by weight alone, so URLs with the same weight come out in the same order for every probe in a country and network. Most URLs tie while the measurement counts read as zero. That's always the case for the OONI Probe app until #1294, and it happens for other probes whenever merges reset counters_asn_test_list. Background runs test only the top 20 URLs (100 when charging), so probes on a network keep testing the same URLs.

This breaks ties with a shuffle seeded by the probe's country, network and UTC day. The shuffle runs before the stable sort by weight, so the ranking is unchanged.

Weights are recomputed from the counts on every request, so a URL leaves the tie once its measurements are counted. The seed only orders the URLs that still tie:

  • Probes that check in before each other's results are counted get the same tied URLs, so they repeat each other's measurements. A fully random tie-break would spread them apart.
  • Later probes on the same network carry on down the tied URLs in the same order, rather than each picking at random.
  • Other networks in the country use a different order.
  • While the counts don't change, as for the app today or after a merge resets them, the order stays the same for the whole day and changes the next day. TIE_BREAK_SLOT_SECONDS sets that slot.

Implementation details:

  • The entries come from a query without ORDER BY, so they are sorted by URL before the seeded shuffle. Otherwise the same seed could give different orders.
  • Without a seed, compute_priorities keeps the entries' order, which is how show_countries_prioritization uses it.

The first reactive test list broke ties with RANDOM() (080ac01); c5f1623 dropped it when it introduced the weight formula.

Tests check that:

  • the same seed gives the same order whatever order the entries arrive in (this fails without the sort);
  • other networks and other days give other orders;
  • the highest weight stays first;
  • tie_break_seed changes per network and per day.
aagbsn added 2 commits October 7, 2026 09:34
compute_priorities sorted URLs by weight alone, so URLs with the same
weight came out in the same order for every probe in a country and
network. While measurement counts in counters_asn_test_list read as
zero, most URLs tie, and background runs of the OONI Probe app test only
the top 20 URLs (100 when charging): every app on a network tested the
same URLs. Last week the app's top 20s across about 130 countries held
84-107 distinct URLs a day.

Shuffle before the stable sort, so ties come out in a random order and
the ranking is unchanged. The first reactive test list broke ties with
RANDOM() (080ac01); c5f1623 dropped it with the weight formula.
A tie-break that differs on every request spreads the probes on a
network over different URLs, so fewer of them measure the same URL and
can confirm each other's results. Last week, in most of the 30 most
measured countries, 35-81% of the URLs a network measured on a day were
measured by two or more runs that day.

Seed the shuffle with the probe's country, network and UTC day instead:
probes on the same network get tied URLs in the same order during a
day, and in a new order the next day, so over several days the network
still works through more of its list, and other networks in the country
start elsewhere in it. TIE_BREAK_SLOT_SECONDS sets the slot; a shorter
one spreads measurements over the list sooner, which matters for
noticing new blocking quickly, with fewer probes per URL in each slot.

The entries come from a query without ORDER BY, so they are sorted by
URL before the seeded shuffle, or the same seed could give different
orders. compute_priorities without a seed keeps the entries' order, for
show_countries_prioritization.
@aagbsn
aagbsn marked this pull request as draft October 7, 2026 09:56
@aagbsn aagbsn changed the title prio: break ties between equal weights at random Oct 7, 2026
The comment said probes on a network measure the same URLs for a whole
slot. Weights are recomputed from the counts on every request, so a URL
leaves a tie once its measurements are counted: the seed only orders the
URLs that still tie. Probes that check in before each other's results
are counted get the same URLs, later ones carry on down the tied URLs in
the same order, and the slot only matters while counts don't change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant