[8.0] fix (resources): condor commandline call compatible with v24

[afalabel]

BEGINRELEASENOTES
*Resources
FIX: Condor command line call compatible with v24
ENDRELEASENOTES

[aldbr]

A few comments:

• https://github.com/DIRACGrid/DIRAC/actions/runs/15633872299/job/44045237809?pr=8225 is failing because the commit message should have the following form: "<type>(<scope>): <subject>". So something like: fix(resources): command line call compatible with v24 would work.

• https://github.com/DIRACGrid/DIRAC/actions/runs/15633872302/job/44045237775?pr=8225 is failing because you did not install pre-commit locally. In your mamba environment, if you just run pre-commit install, then the files you change should be automatically formatted correctly and the CI pipeline should pass.

• You should probably target the v8r0 branch: once merged, the fix will be automatically applied to integration (v9).

• Is getCEStatus() really working?
  I have the feeling that there is no I or R to parse in the output.
  Shouldn't we just try to parse the Total for query line?

Please let me know if I can help! 🙂

[fstagni]

Can the -json option be used here? https://htcondor.readthedocs.io/en/latest/man-pages/condor_q.html#output

It looks like in theory it would be less error-prone than parsing the output.

[afalabel]

Yes I have the code ready for that, I am waiting from @aldbr to test verify it and I'll push it.

[aldbr]

Thanks for your work @afalabel
I've adjusted a few things based on what you have done:

• get both the ClusterId and the ProcId to retrieve the job ID (<ClusterId>.<ProcId>)
• get HoldReason/Code/SubCode: they only appear in the json if the JobStatus == 5, this is why you did not see the fields (I manually held one of the job to check).
• increment waiting jobs if we find both JobStatus == 5 and HoldReasonCode == 16 (held jobs are not all waiting jobs)

And then because we changed parseCondorStatus (now named getCondorStatus), I needed to adapt HTCondorComputingElement too.
Because of that I changed the target to v9, just in case I am breaking something.

Now let me see if I can add a unit test or adapt the existing ones, and test the code with a few instances, just to make sure it works correctly.

Note: I also check if I could get the CE status for HTCondorComputingElement too, but this is still not possible (condor_q -totals returns the jobs of everyone when we use remote schedds, we would need to come with an owner name I think).

[aldbr]

I've tried to use the -spool option and conda_transfer_data command to avoid getting all the pilot outputs in the BatchOutput directory but it does not work as I expect (I don't manage to get any output at all).

As we have to clean up the files (not only the outputs but also the pilot wrappers) from time to time anyway, I think I will just add a cleanup method in the SSHComputingElement.
Probably a good opportunity to finish #7703

[andresailer]

Minor comments.
Looks good to me otherwise

[aldbr]

Thanks @andresailer, I've applied your suggestions.
I also added a commit to prevent pilots from being rescheduled (when there is no constraint from the admin side).
We tested it locally, it seems to work.

[DIRACGridBot]

Sweep summary

Sweep ran in https://github.com/DIRACGrid/DIRAC/actions/runs/15977484464

Failed:

• integration
  cherry-pick b1686ff into integration failed
  check merge conflicts on a local copy of this repository

  git fetch upstream
  git checkout upstream/integration -b cherry-pick-2-b1686ff54-integration
  git cherry-pick -x -m 1 b1686ff54
  # Fix the conflicts
  git cherry-pick --continue
  git commit --amend -m 'sweep: #8225 fix (resources): condor commandline call compatible with v24' --author=''
  git push -u origin cherry-pick-2-b1686ff54-integration
  
  # If you have the GitHub CLI installed the PR can be made with
  gh pr create \
       --label 'sweep:from integration' \
       --base integration \
       --repo DIRACGrid/DIRAC \
       --title '[sweep:integration] fix (resources): condor commandline call compatible with v24' \
       --body 'Sweep #8225 `fix (resources): condor commandline call compatible with v24` to `integration`.
  
  Adding original author @afalabel as watcher.
  
  BEGINRELEASENOTES
  *Resources
  FIX: Condor command line call compatible with v24
  ENDRELEASENOTES
  Closes #8238'