Skip to content

Using incorrectly formatted auxilliary data file makes pyopia skip image with non-useful log message #350

@nordam

Description

@nordam

Describe the bug
Through copying a project folder from an older version, I ended up with an incorrectly formatted auxiliary data file. This resulted in pyopia skipping at the output step. The rois were written, but the stats.nc file were not written. The log message was

2025-08-07 14:22:59 INFO MainProcess [pipeline.run] Running pipeline step: load
2025-08-07 14:22:59 INFO MainProcess [pipeline.run] Running pipeline step: correctbackground
2025-08-07 14:23:00 INFO MainProcess [pipeline.run] Running pipeline step: imageprep
2025-08-07 14:23:00 INFO MainProcess [pipeline.run] Running pipeline step: segmentation
2025-08-07 14:23:00 INFO MainProcess [process.segment] segment
2025-08-07 14:23:00 INFO MainProcess [process.segment] clean
2025-08-07 14:23:01 INFO MainProcess [pipeline.run] Running pipeline step: statextract
2025-08-07 14:23:01 INFO MainProcess [process.__call__] statextract
2025-08-07 14:23:01 INFO MainProcess [process.concentration_check] 1.0% saturation
2025-08-07 14:23:01 INFO MainProcess [process.statextract] measure
2025-08-07 14:23:01 INFO MainProcess [process.measure_particles]   367 particles found
2025-08-07 14:23:03 INFO MainProcess [process.extract_particles] EXTRACTING 366 IMAGES from 367
2025-08-07 14:23:03 INFO MainProcess [process.__call__] CalculateImageStats
2025-08-07 14:23:03 INFO MainProcess [pipeline.run] Running pipeline step: output
2025-08-07 14:23:03 WARNING MainProcess [cli.process_file_list] [red]An error occured in processing, skipping rest of pipeline and moving to next image.(chunk 0)
2025-08-07 14:23:03 ERROR MainProcess [cli.process_file_list] 'time'

The last line is from cli.process_file_list printing the error passed down from pipeline.run, which in this case has something to do with the malformed auxiliary data file.

It would be useful if the error message gave an indication of where the problem might be found.

In the interest of early discovery, it would also be useful if an error in the auxiliary data would cause the processing to stop. Since the auxiliary data file is the same for all images, it is not necessary to keep trying again at every image, never producing any output.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions