canopyRT

A sandbox for exploring leaf/canopy RT representations in LSMs

SMCE pcluster deployment

---

For questions, email Colin Quinn (cquinn8@um.edu) - Updated April 2026

---

A small example to run the canopyRT inversion is located in the pcluster/ folder. The only script that requires user-level arguments is submit_inversion_distributed.sh and denoted by USER INPUTS and described usage in the header. This demo is currently setup to simply distribute n_rows of the input spectral dataframe per SLURM array task. The number of tasks (aka nodes) that will be run is n_rows / total_rows.

Running the inversion

In order to run the canopyRT code, a pixi environment must be properly connected to your working setup:

1. ssh pcluster
2. cd /efs/shared/users/radiative_transfer/canopyRT/pcluster
3. Test the bash script: bash test_run.sh

If you see results in the results/ dir and the .err and .out files in logs/ look okay you can move on to the slurm script. If you see issues try debugging tips.

Assuming you have pixi working and can therefore load the required R libraries: 4) Customize parameters in pcluster/run_inversion_distributed.sh 5) Test run: bash submit_inversion_distributed.sh --test 6) Full run: bash submit_inversion_distributed.sh

>>> Config written to: /efs/shared/users/radiative_transfer/canopyRT/results/job_config-testrun.sh
# Auto-generated by submit_prospect_inversion.sh
# Fri Apr 17 21:23:54 UTC 2026
PROJECT_ROOT="/efs/shared/users/radiative_transfer/canopyRT"
RSCRIPT="/efs/shared/users/radiative_transfer/canopyRT/pcluster/invert_leaf_refl_spectra_PROSPECT-pcluster-parallel.R"
DATA_DIR="/efs/shared/users/radiative_transfer/canopyRT/data/compiled_data"
OUTPUT_DIR="/efs/shared/users/radiative_transfer/canopyRT/results"
LOG_DIR="/efs/shared/users/radiative_transfer/canopyRT/results/logs"
TOTAL_ROWS=50
CHUNK_SIZE=25
CPUS=4
MEM="8G"

    FULL RUN: submitting 2 tasks
    TOTAL_ROWS : 50
    CHUNK_SIZE : 25
    ARRAY      : 1-2
    OUTPUT_DIR : /efs/shared/users/radiative_transfer/canopyRT/results
    LOG_DIR    : /efs/shared/users/radiative_transfer/canopyRT/results/logs

Submitted batch job 8328 

7. While these are running check the SLURM queue. Each task receives a full node/worker (8328_1 and 8328_2), this script is currently "embarrassingly parallel" as in uses no parallelization, and subtasks do not communicate with each other. Therefore, use the smallest partition available (demand-8cpu currently), parallelization will be setup in the Rscript in the future.

squeue -l -u <username>
Fri Apr 17 21:24:55 2026
             JOBID PARTITION     NAME     USER    STATE       TIME TIME_LIMI  NODES NODELIST(REASON)
            8328_1 demand-8c prospect   cquinn  RUNNING       0:23     10:00      1 demand-8cpu-dy-demand-8cpu-nodes-1
            8328_2 demand-8c prospect   cquinn  RUNNING       0:23     10:00      1 demand-8cpu-dy-demand-8cpu-nodes-1

Debugging tips

Pixi env setup:

1. cd /efs/shared/users/radiative_transfer/canopyRT
2. Run source activate-rt_tools_env.sh

If there is no pixi installed to your user on the pcluster, install pixi following the env/README.md.

Pixi env config:

You may need to set the pixi config to point to the custom pcluster location. On the pcluster run:

# these will update pixi to use the pre-built env in canopyRT on the SMCE
$ /efs/home/$(id -un)/.pixi/bin/pixi config set detached-environments
$ /efs/shared/users/radiative_transfer/canopyRT/env/builds/ pcluster --global

# test loading the pixi env again from the basedir. You should see this output...
$ bash activate-rt_tools_env.sh
Usage 'source activate-rt_tools_env.sh' not 'bash activate-rt_tools_env.sh'
Activating RT Tools R Environment...
  R version: R version 4.5.3 (2026-03-11) -- "Reassured Reassurer"
  env       : /efs/shared/users/radiative_transfer/canopyRT/env/pcluster