CI fails with error: ValueError: time data 'latest' does not match format '%Y%m%d' #164

Activity

1. aagbsn commented on Jun 29, 2026

   

   aagbsn

   on Jun 29, 2026

   ContributorAuthor

   More actions

   per @LDiazN

   data/oonipipeline/src/oonipipeline/tasks/updaters/asnmeta_updater.py

   Line 18 in 17a4258

   AS_ORG_MAP_URL = "https://archive.org/download/ip2country-as/all_as_org_map.json"

   the dataset: https://archive.org/download/ip2country-as/all_as_org_map.json
   seems to have changed
2. LDiazN commented on Jun 29, 2026

   

   LDiazN

   on Jun 29, 2026

   Last edited by LDiazN

   Contributor

   More actions

   Here's a sample of the json downloaded from that URL:

   {  "1": [
       [
         "Level 3 Communications, Inc.",
         "US",
         "20040604",
         "LVLT-1",
         "ARIN"
       ],
       [
         "Level 3 Parent, LLC",
         "US",
         "20180220",
         "LVLT-1",
         "ARIN"
       ]
     ],
   ...

   And this is the expected format:

   data/oonipipeline/src/oonipipeline/tasks/updaters/asnmeta_updater.py

   Lines 43 to 51 in 17a4258

   	rows.append(
   	{
   	"asn": asn,
   	"org_name": v[0],
   	"cc": v[1],
   	"changed": changed,
   	"aut_name": v[3],
   	"source": v[4],
   	}

   The conflicting entry looks like this:

   "3288": [
       [
         "MACAM-MOFET Institute, Israel",
         "",
         "20000101",
         "MOFET-AS",
         "RIPE"
       ],
       [
         "Mofet Institute",
         "",
         "20000101",
         "MOFET-AS",
         "RIPE"
       ],
       [
         "Mofet Institute",
         "IL",
         "20000101",
         "MOFET-AS",
         "RIPE"
       ],
       [
         "Zipcom Communications LTD",
         "IL",
         "20260601",
         "MOFET-AS",
         "RIPE"
       ],
       [
         "Mofet Institute",
         "IL",
         "latest",
         "MOFET-AS",
         "RIPE"
       ]
     ],

   Seems like the most recent one is expressed with a "latest" in some cases, breaking this line:

   data/oonipipeline/src/oonipipeline/tasks/updaters/asnmeta_updater.py

   Line 42 in 17a4258

   changed = datetime.strptime(v[2], "%Y%m%d").date()
3. aagbsn commented on Jun 29, 2026

   

   aagbsn

   on Jun 29, 2026

   ContributorAuthor

   More actions

   'latest' isn't very descriptive, but maybe we can guess it from the current release version
4. aagbsn

   added a commit that references this issue on Jun 30, 2026

   add reference to #164

   447bf35