Right now, the workflow is two processes:
- A monolithic counting process in Python
- Mageck
The problem with this is that the monolith is where most of the time is spent and despite having distinct sub-processing hiding within it does not take advantage of Nextflow as a wrapper to manage caching intermediate results.
Taken to the extreme, we could produce a single process per sgRNA sequence which manages its own count and pushes that forward. This would be too many small processes with far too limited scope to be useful, but the ideal/optimal balance is between that extreme and the current monolithic extreme.
Right now, the workflow is two processes:
The problem with this is that the monolith is where most of the time is spent and despite having distinct sub-processing hiding within it does not take advantage of Nextflow as a wrapper to manage caching intermediate results.
Taken to the extreme, we could produce a single process per sgRNA sequence which manages its own count and pushes that forward. This would be too many small processes with far too limited scope to be useful, but the ideal/optimal balance is between that extreme and the current monolithic extreme.