Skip to content

Releases: Automattic/stream-builder

CursorlessFilteredStream allows more retries when whole pages are filtered

07 Apr 10:56
f7de47f

Choose a tag to compare

We have updated CursorlessFilteredStream

The problem:

When a fetched batch contains only filtered-out elements, CursorlessFilteredStream counts it against the retry budget. After retry_count retries, pagination stops with an empty result and no cursor — even though the inner stream has more content.

The solution:

Preserve the retry budget when a page yields zero retained results. Retries are only consumed when partial progress is made. The inner stream's natural exhaustion still guarantees termination.

This solution could result in deeper recursion, and resulting in memory issues. This could happen if we have a huge block of filtered elements.

Lets say that we paginate by 10 elements, and we have 30 consecutive elements being filtered. This change addresses that issue by not counting "depth" for empty pages.

Previous version 1.0
page 1 => 8 elements - depth 2
page 2 => 10 filtered elements - depth 1

Current version 1.1
page 1 => 8 elements - depth 2
page 2 => 10 filtered elements - depth 1
page 3 => 10 filtered elements - depth 1
page 3 => 10 filtered elements - depth 1
page 4 => 5 elements - depth 0

If we make little progress by finding some elements in subsequent pages (inner enumeration), we will decrement retry count (depth). but when we have a big chunk of filtered elements, we will be able to skip the whole block and fetch a few more elements from a subsequent page.

How to deactivate this functionality

If you notice that this goes too deep and causes memory issues, you can turn it off by setting skip_empty_pages optional property to false. It has been made true as default because we consider this change to be a bug fix.

PHP 8.4 compatibility

27 Oct 15:24
e3fc008

Choose a tag to compare

What's Changed

  • update CappedPostRanker constructor args to pass optional params last by @lucila in #42

Full Changelog: 0.12.1...1.0.0

Improve FilteredStream by reducing premature exhaustion when retries are enabled

09 Jun 19:27
c77da04

Choose a tag to compare

What's Changed

  • Propagate inner stream is_exhaustive through recursive calls by @lengare in #37

New Contributors

Full Changelog: 0.12.0...0.12.1

0.12.0

20 May 14:04
6ae828c

Choose a tag to compare

What's Changed

  • Streams relying on inner streams override can_enumerate, relying on the inner's stream implementation of can_enumerate. by @lucila in #38

Full Changelog: 0.11.2...0.12.0

0.11.2

19 Dec 13:17
a91ac23

Choose a tag to compare

What's Changed

Full Changelog: 0.11.1...0.11.2

0.11.1

07 Feb 20:22
210b735

Choose a tag to compare

What's Changed

  • add component name for TemplateProvider::get_template error message by @lucila in #34

Full Changelog: 0.11.0...0.11.1

0.11.0

29 Jan 20:20
f1e4063

Choose a tag to compare

What's Changed

  • Update ChronologicalStreamMixer so that the "is exhausted" flag uses <= instead of < when comparing the expected count and the actual count. by @lucila in #9

Full Changelog: 0.10.1...0.11.0

0.10.1

27 Jan 15:35
f995890

Choose a tag to compare

What's Changed

New Contributors

Full Changelog: 0.10.0...0.10.1

0.10.0

17 Jan 13:51
4d647e0

Choose a tag to compare

What's Changed

  • Test on PHP 8.3 by @sanmai in #29
  • implement release_all_elements method on StreamTracer. Make StreamFilter trace an event when all the elements have been released by the filter. by @lucila in #30
  • add release ratio to meta field on the end_filter tracer method. Update StreamFilter so that we can trace the release ratio of the filter step. by @lucila in #31

Full Changelog: 0.9.2...0.10.0

0.9.2

02 Nov 00:16
03750e8

Choose a tag to compare

What's Changed

  • Initial beginner's guide docs for stream caching by @markbiek in #27
  • PHP 8.1 compatibility updates by @markbiek in #28

Full Changelog: 0.9.1...0.9.2