Using an API With Paging
Last updated
Last updated
Certain APIs will split up their results across multiple "pages" if the list of results is too large. For instance, consider the Google Calendar API. If an API request to "list all events" returns 100 events, the Google Calendar API might choose to break the response up into two responses of 50 events each. The first response would contain the first 50 events, as well as a token that must be used to request the second set of 50 events.
The HTTP Request Data Operation has several fields that handle the logic for paging.
In order to access the response of the HTTP request after it is repeated, use the paginate namespace to access data in the response.
For example, if the url for the next page of responses is in the result of the response, in the URL of the HTTP request the expression would look something like:
This box should be checked if the API endpoint supports paging. If checked, the following 3 fields will appear.
####Next Page Check
The API will indicate in each response whether there is another page of data that must be queried. This field must parse that response and return a Boolean value. If Next Page Check resolves to TRUE, the whole HTTP Request Operation will be repeated.
For instance, if the Google Calendar API has another page of data, its response will include a nextPageToken. In this case, a Next Page Check value of result.nextPageToken != NULL would be appropriate.
####Next Page Body
Determines the data that should be extracted from each page. The data from each page will end up in the operation's Output as an aggregated list.
For instance, the Google Calendar API returns an object like:
In this case, a Next Page Body of result.items would be appropriate.
####Max Iterations
Determines the maximum number of pages the HTTP request can query before finishing. Note that there is a built-in maximum of 25 pages that cannot be exceeded.
