various paging patterns:

• using query parameters
  • limit & offset - specify no. desired, and which record to start at
  • per_page & page - specify no. desired, and which page to return (page size variable)
• using link headers
  • headers return some combination of next, first, last, or prev links - if these provided, in all cases should probably use these - Github recommends that at least

examples

• github:
  • params: page, per_page
  • strategy: GH suggests using link headers to do paging b/c sometimes paging is based on SHA's instead of page numbers
• GBIF
  • params: limit, offset
  • strategy: page via query parameters
• Crossref
  • params: rows, offset
• idigbio
  • params: limit, offset (Elasticsearch backed, but not exposed directly)
• vertnet
  • params: limit, cursor
• Tropicos
  • params: pagesize, startrow
• NOAA NCDC v2
  • params: limit, offset
• CKAN API
  • params: limit, offset
• Berkeley Ecoengine
  • params: page_size, page
• iNaturalist
  • params: per_page, page
• DataCite
  • params: rows, start (solr backed)
• PLOS Search API
  • params: rows, start (solr backed)
• Europeana API
  • params: rows, start (solr backed)
• ORCID API
  • params: rows, start (solr backed)
• DPLA API
  • params: page_size, page (Elasticsearch backed)
• Twitter search API
  • params: count (that's it, AFAICT, not sure this is accurate)
• Enigma
  • params: limit, page

approach

Automagically figure out what params to pass and their values given user input. We'll need some user input:

• name of query parameters
• how many records they want
• what record to start at
• maximum records allowed (if known)

Things we can figure out automatically

• whether API uses link headers or not (just look for link headers)
• if a cursor is used (e.g., Vertnet API) look for a cursor