Skip to content

handle CPU or GPU via flag? right now using Delta on CPU breaks if CuPy is installed #41

@lastephey

Description

@lastephey

Delta right now imports CuPy whether it's running on the CPU or GPU. This is a problem when running on the CPU but in the Delta container where CuPy is installed:

stephey@nid00218:/global/cscratch1/sd/stephey/delta/delta> OMP_NUM_THREADS=16 srun -n 16 -c 2 --cpu-bind=cores shifter python3 -m mpi4py.futures processor.py --config configs/hackathon_test.json --transport transport_tx  --num_ranks_preprocess=1 --num_ranks_analysis=16 --num_queue_threads=1 --run_id=test_25259_GT
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/cupy/__init__.py", line 20, in <module>
    from cupy import core  # NOQA
  File "/usr/local/lib/python3.8/dist-packages/cupy/core/__init__.py", line 1, in <module>
    from cupy.core import core  # NOQA
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.8/dist-packages/mpi4py/futures/__main__.py", line 72, in <module>
    main()
  File "/usr/local/lib/python3.8/dist-packages/mpi4py/futures/__main__.py", line 60, in main
    run_command_line()
  File "/usr/local/lib/python3.8/dist-packages/mpi4py/run.py", line 47, in run_command_line
    run_path(sys.argv[0], run_name='__main__')
  File "/usr/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "processor.py", line 28, in <module>
    from analysis.task_list import tasklist
  File "/global/cscratch1/sd/stephey/delta/delta/analysis/task_list.py", line 6, in <module>
    from analysis.helpers import get_analysis_task
  File "/global/cscratch1/sd/stephey/delta/delta/analysis/helpers.py", line 10, in <module>
    from analysis.task_spectral_cu import task_coherence_cu, task_crosscorr_cu, task_crossphase_cu, task_crosspower_cu
  File "/global/cscratch1/sd/stephey/delta/delta/analysis/task_spectral_cu.py", line 4, in <module>
    from analysis.kernels_spectral_cu import kernel_crosscorr_cu, kernel_coherence_cu, kernel_crosspower_cu, kernel_crossphase_cu
  File "/global/cscratch1/sd/stephey/delta/delta/analysis/kernels_spectral_cu.py", line 7, in <module>
    import cupy as cp
  File "/usr/local/lib/python3.8/dist-packages/cupy/__init__.py", line 41, in <module>
    raise ImportError(_msg) from e
ImportError: CuPy is not correctly installed.

I suggest that Delta have an --arch or --mode flag that a user can set to CPU or GPU depending on what they need. Then Delta could handle the imports accordingly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions