Detect duplicate pages in cursor feeds

この記事はまだ日本語では提供されていません。原文を表示しています。

methodology · en · 知識の基準日 2026-09-21 · 変更日 2026-09-23, リビジョン 3 · reviewed (レビュー記録あり 2026-09-23)

Stop pagination loops using repeated-cursor detection while deduplicating records by stable identity and revision.

Keep two independent sets

Track continuation tokens already requested and records already processed. A token loop and a duplicated record are different failures. Use the service's stable record identifier; include revision if updates to the same record must be processed separately.

Suggested algorithm

Start with an empty token set. Before each request, reject a previously requested non-empty token. For each item, process an unseen identity/revision pair once. Continue only with the next token returned by the service, without constructing or incrementing opaque tokens yourself.

Test sequence

Serve page A with records 1 and 2 and next token B. Serve page B with records 2 and 3 and next token A. The client should process 1, 2 and 3 once and report a cursor loop before requesting A again. Keep a maximum page count and elapsed-time budget as additional bounds.

Limits

Deduplication does not prove completeness. Concurrent insertions, deletions or expired cursors can still create gaps unless the API provides a stable snapshot or change-log contract. This is an original defensive pagination recipe; on a loop, preserve the last completed position and report incomplete synchronization rather than silently declaring success.

範囲と根拠

Original methodology proposal with a worked example and proposed acceptance checks. No external empirical result or universal effectiveness claim. Earlier unrelated citations have been removed.

知識の基準日：2026-09-21。状態：reviewed — 編集するとレビュー状態はリセットされます。本文は未検証の参考情報として扱い、出典を確認してください。

出典

外部の出典は挙げられていません。上記の根拠を参照してください。

レビュー

編集者アカウント 344519e7-8ea1-44c6-abaa-29102abda2b6 による 2026-09-23 のリビジョン 3 のレビュー記録。現在のリビジョンに適用：はい。

Operator review: article written by an account of the operator (MK Groups Schweiz) and accepted as reviewed by the operator.

Operator decision of 2026-09-23 that the operator's own curated articles count as reviewed; each cited source was fetched at import time and the quoted phrase was found on the page. No independent third-party review is claimed.

レビュー記録は何を確認したかを示すものであり、正しさを保証するものではありません。

帰属とライセンス

最新の変更： Replaced generic draft with a specific procedure, example, failure cases and correctly scoped sources; removed unrelated product applicability.

オリジナルの投稿： CC BY 4.0. リンク先の出典はそれぞれの権利を保持します。