Fix pagination: sync all 102 followed users instead of just 30#33
Merged
Conversation
Fix issue where only 30 followed users were synced instead of all 102. GitHub API returns 30 results per page by default, but we were only fetching the first page. Changes: - Add getPaginated() helper that fetches all pages (100 items/page) - Update all list methods to use pagination: * GetFollowedUsers * GetFollowedUsersByUsername * GetStarredRepos * GetStarredReposByUsername * GetOwnedRepos * GetOwnedReposByUsername * GetRecentEvents * GetReceivedEvents - Add comprehensive pagination tests This ensures users like 'expede' who were beyond the first 30 results will now appear in the feed. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
justinabrahms
added a commit
that referenced
this pull request
Jan 22, 2026
Investigation reveals three issues causing slowdown: 1. Cache invalidation (one-time): URL changes invalidated ETags 2. Extra API calls for exactly 100 items: Main issue causing 23-50% slowdown for affected users 3. No multi-page caching: Minor cumulative impact The main culprit is issue #2: current logic makes an extra API call when result sets have exactly 100 items because it can't distinguish between 'exactly 100 total' and '100 with more pages'. Solution: Use GitHub's Link header to detect pagination instead of guessing based on result count. Files: - PAGINATION_ANALYSIS.md: Complete technical analysis with impact tables - PAGINATION_FIX_POC.md: Implementation guide showing how to fix issue #2 For users following 100 people where 30% have exactly 100 items: - Current: ~391 API calls - After fix: ~301 API calls (23% reduction)
justinabrahms
added a commit
that referenced
this pull request
Jan 22, 2026
Investigation reveals three issues causing slowdown: 1. Cache invalidation (one-time): URL changes invalidated ETags 2. Extra API calls for exactly 100 items: Main issue causing 23-50% slowdown for affected users 3. No multi-page caching: Minor cumulative impact The main culprit is issue #2: current logic makes an extra API call when result sets have exactly 100 items because it can't distinguish between 'exactly 100 total' and '100 with more pages'. Solution: Use GitHub's Link header to detect pagination instead of guessing based on result count. Files: - PAGINATION_ANALYSIS.md: Complete technical analysis with impact tables - PAGINATION_FIX_POC.md: Implementation guide showing how to fix issue #2 For users following 100 people where 30% have exactly 100 items: - Current: ~391 API calls - After fix: ~301 API calls (23% reduction)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the issue where gitstreams only syncs ~30 followed users instead of all 102.
The root cause was that GitHub's API returns 30 results per page by default, but we were only fetching the first page of results. This meant users beyond position 30 (like 'expede') were missing from the feed.
Changes
Testing
Impact
Users will now see activity from all their followed users, not just the first 30. This ensures a complete and accurate feed.