API Pagination Collector

Start from brute force

The first instinct is to ignore the cursor and concatenate the pages in list order. That is one pass, O(N) time and O(N) space for N items, and it is wrong: the order and the set of pages come from next, so an unreachable page leaks in and a chain like 0 → 2 → 1 comes out scrambled. The second instinct follows next with while i is not None and nothing else. On a healthy chain that is correct at the same cost. On a malformed API whose cursor points back at an earlier page it never terminates: it re-reads the same pages forever and out grows until memory runs out.

The approach

Treat the pages as a linked list: index 0 is the head and next is the pointer. The reference collect keeps three pieces of state: i, the page about to be read (starting at 0); out, the items gathered so far; and seen, the set of page indices already read. The loop runs while i is not None and i not in seen. Each pass adds i to seen, extends out with pages[i]["items"], and moves to pages[i]["next"].

The invariant is that out holds exactly the items of the pages in seen, in the order they were reached. Every pass adds a new index to seen, so the loop runs at most once per page and terminates on any input, including a page that points at itself. The explicit is not None test matters too: page 0 is a valid cursor, and a bare while i: would stop before reading anything. The TypeScript version is the same loop with i !== null and out.push(...pages[i].items).

Walk through an example

On the first example the loop reads three pages, then stops when next comes back None.

Complexity

O(P + N) time for P pages reached and N items copied: each page is read once behind an average-O(1) set check, and each item is copied once. O(N + P) space for out and seen. The unguarded loop costs the same on a healthy chain but has no bound on a cycle; list-order concatenation is O(N) and simply returns the wrong list.

Edge cases to test

• [{"items":[1],"next":0}] → [1]. A page that points at itself; without seen the loop never ends.
• [{"items":[1],"next":1},{"items":[2],"next":0}] → [1, 2]. A two-page cycle: stop on the revisit and return what was gathered, with no duplicates.
• [{"items":[],"next":1},{"items":[7],"next":null}] → [7]. An empty page is not the last page; stopping on empty items drops data.
• [{"items":[1],"next":2},{"items":[99],"next":null},{"items":[2],"next":null}] → [1, 2]. Page 1 is unreachable; list-order concatenation wrongly includes 99.
• [{"items":[9],"next":null}] → [9]. The start cursor is 0, which is falsy, so while i: returns [] here and on every other input.
• [{"items":[],"next":1},{"items":[],"next":null}] → []. Every page is empty: return an empty list, not None.

Common mistakes

• An infinite loop if the next token never clears — guard with a max-pages cap or a seen-cursor check.
• Dropping the final page’s items by stopping one iteration too early.
• Assuming a fixed page count instead of following the cursor until it ends.

Follow-ups an interviewer asks next

The cursor is an opaque string from a real HTTP API.

The loop does not change. Request the endpoint with the current cursor, extend the output with the response items, and read the next cursor from the body; seen becomes a set of cursor strings. Then add what a network brings: retry 429 and 5xx responses with backoff, honor a Retry-After header when the API sends one, and cap the total pages so a server bug cannot run the job forever.

The full result is too large to hold in memory.

Make the collector a generator that yields each page’s items as they arrive, so memory is one page plus the seen cursors instead of the whole data set. After each page is fully processed, persist its cursor as a checkpoint; if the job crashes it resumes from there instead of starting over and re-sending everything downstream.

Can you fetch the pages in parallel?

Not along one cursor chain: each response is the only place the next cursor appears, so the requests are sequential by construction. Offset pagination lets you compute every page URL up front and fetch them concurrently, but inserts and deletes during the sync shift the offsets and cause skipped or repeated rows. To parallelize safely, split the data into independent ranges, such as creation-date windows, and follow one cursor per range.

Should a repeated cursor return partial data or fail?

This problem asks you to stop, which is the safe minimum, but stopping quietly makes a broken sync look successful while rows are missing. In production, treat a repeated cursor as a server bug: raise or log an error that names the cursor and the page count, so the run is marked failed and can be retried or reported to the API owner.

Where this shows up

This is the loop behind every “sync everything” integration: the customer’s API returns 100 rows at a time behind a cursor and you need all of them. Cursor-following is the robust pagination pattern — unlike offset pagination, it does not drift or skip rows when the data changes underneath you mid-sync.

Reference solution

def collect(pages):
    out = []
    i = 0
    seen = set()
    while i is not None and i not in seen:
        seen.add(i)
        out.extend(pages[i]["items"])
        i = pages[i]["next"]
    return out


print(collect([{"items":[1,2],"next":1},{"items":[3],"next":2},{"items":[4,5],"next":None}]))
# -> [1, 2, 3, 4, 5]

function collect(pages: any[]): any[] {
  const out: any[] = [];
  let i: number | null = 0;
  const seen = new Set<number>();
  while (i !== null && !seen.has(i)) {
    seen.add(i);
    out.push(...pages[i].items);
    i = pages[i].next;
  }
  return out;
}

console.log(JSON.stringify(collect([{items:[1,2],next:1},{items:[3],next:2},{items:[4,5],next:null}])));
// -> [1,2,3,4,5]