Repository navigation
Add support for annotating targets and hostnames #75
Description
It would be nice to add annotations that we can update over time related to hostnames.
Things that I would like to be able to say are flags like if the hostname hosts a domain name parking page, if the hostname is expired or any other feature that's relevant to the analysis.
See also:
- testlists: semi-automatically detect and remove domains that are now for sale (aka parked domains) probe#1826
- process: test lists maintenance probe#1745
Relevant issues:
- Certificate transparency data as baseline backend#177
- Detect when a site is behind cloudflare backend#670
- Differentiate between seized domains and block pages backend#506
- Improve pipeline level heuristics for detecting blocking backend#570
- api: add support for returning category_code and canonical_domain for a given domain backend#609
- test-lists: add checks for domain and URL category code inconsistencies backend#611
Metadata
Metadata
Assignees
Labels
No labels
Activity
[-]Add support for annotating targets or hostnames[/-]hellais commentedon May 6, 2024
Sharing additional notes from slack:
On the topic of parked domains, there is this paper that documents a methodology for automatically detecting such domains: https://www.ndss-symposium.org/ndss2015/ndss-2015-programme/parking-sensors-analyzing-and-detecting-parked-domains/.
I think for a basic first pass we don’t even have to implement their ML based approach, but just use the list of the most common nameservers used by domain parking services and filter them out based on that (there is such a list in the paper).
hellais commentedon Jan 7, 2026
We might want to look into ingesting data from the internet yellow pages for this: https://iyp.iijlab.net/