Separating the function to loop over data from the one to create batches for training

[anagainaru]

Summary

Current behavior is to have a function get_cur_data_loaders that returns the dataloaders that is used for both looping for inference and for creating batches to train when doing continual learning. The current PR separates them into two separate functions.

Motivation

Having two functions allows us to control the granularity of the drift detectors during streaming (e.g. looking at element by element) without impacting the training (which should use the same batch size as what was used for the original training)

API / CLI Changes

!! This PR changes out current API !!

diff --git a/.claude/skills/new-harness/SKILL.md b/.claude/skills/new-harness/SKILL.md
index 0940025..1a4b9c9 100644
@@ -64,11 +64,14 @@ class <Name>Harness(BaseModelHarness):
         # Rebuild data loaders with new transforms
         # Track augmentation history for replay

-    def get_cur_data_loaders(self) -> Tuple[DataLoader, DataLoader]:
+    def get_stream_dataloader(self) -> DataLoader:
+        # Return data_loader for current data
+
+    def get_train_dataloaders(self) -> Tuple[DataLoader, DataLoader]:
         # Return (train_loader, val_loader) for current data
         # Call _dispose_current_loaders() first if loaders exist

-    def get_hist_data_loaders(self) -> Tuple[Optional[DataLoader], Optional[DataLoader]]:
+    def get_hist_dataloaders(self) -> Tuple[Optional[DataLoader], Optional[DataLoader]]:
         # Return historical data loaders for CL replay
         # Return (None, None) when task_counter == 1

Introduced a new function def get_stream_dataloader(self) that return the data loader over the stream of data using a different batch size that the training (the model harnesses could use the data side of the configuration, by default this is set to 1, each data in the stream should be analyzed in isolation).

The self.get_cur_data_loaders() is renamed to def get_train_dataloaders(self) and returns as before the training and validation used by the continual learning. It uses the batch_size specified in the training side of the configuration.

The get_hist_data_loaders is exactly the same but it's renamed to get_hist_dataloaders to be consistent with the others.

Usage

The MNIST example has been updated to show how to use the new function.

Imagenet and cifar have only minimal changes to update to the new API but the functionality did not change (the training and streaming loaders are exactly the same).

[ScSteffen]

I think this is a neccessary change - but very intrusive in the sense that all examples needs to be adapted.

Do you have an example that runs currently, so I can validate the changes locally?

[ScSteffen]

I think with this change we can further simplify:

==> the loop loader only needs a val_loader, since we are not training on this data

[anagainaru]

I think this is a neccessary change - but very intrusive in the sense that all examples needs to be adapted.

Do you have an example that runs currently, so I can validate the changes locally?

You do not need to change the examples, if this second function is not implemented we use the current behavior with looping over data using the training batch size.

[anagainaru]

I think with this change we can further simplify:

==> the loop loader only needs a val_loader, since we are not training on this data

Agree, I will make the change and update mnist so you can run using different batches if you specify in toml a data batch size.

[ScSteffen]

Functionality checked manually. Changes look good. Approved