Multi GPU support

[dsatyam09]

Hey, does the cellsam inference support using multiple gpu on the same system, just for an example I ran through the cellsam code and checked my nvidia-smi and it showed me the utilisation of only one GPU although we have two of them.

[rossbar]

There is no native support for hardware management in the library. Though it's certainly possible, it requires some engineering on the user's part. Generally speaking, the model itself has a relatively low memory footprint (for inference), typically around 4-5GB, so an effective strategy would be to spawn multiple "worker" instances of the model (even on a single GPU, depending on how much memory you have) and feed tiles to the worker - at least for workflows involving large (i.e. whole-slide, or >3k x 3k) images. For different workflows (i.e. many different small images) a different parallelization scheme might make more sense.

Ultimately we leave this up to the user - if you're interested in exploring I'd start by checking out dask cluster interface for highly-tiled workflows. If you find a recipe that works for you, feel free to open a pull request to share!