installation and deployment pain points

[joanetLiew]

Hi @xtanh ,

Thank you for making unistab available to the community.

However, I had to go through 'compiler hell', 'dependency puzzles' and model architecture traps to finally have it installed.

The major pain points are:

• The current dependency stack (Python 3.7 / PT 1.12) creates significant friction on modern RTX 40-series hardware;
• The mandatory compilation of OpenFold CUDA kernels is brittle;
• the software should provide a seamless fallback to native PyTorch operators if C++ extensions fail to compile;
• Furthermore, the strict key-checking in the model loader makes it nearly impossible to perform lightweight environment fixes without triggering a RuntimeError

To enhance the accessibility of UniStab for the structural biology community, I highly recommend transitioning to a Container-first distribution (Docker/Singularity). Additionally, implementing a native PyTorch fallback for the OpenFold kernels would significantly reduce the high entry barrier caused by C++ compilation errors on modern HPC environments.

Best -- Joanet

[xtanh]

Hi Joanet,

Thank you for your interest in UniStab and for taking the time to share this detailed and valuable feedback.

Actually, the dependency pain points and restrictions you pointed out (including the mandatory use of Python 3.7 and older PyTorch versions) are primarily caused by the underlying requirements of ESMFold. This indeed creates significant friction for environment setup and hardware compatibility.

Your suggestions regarding the friction on modern RTX 40-series hardware and the brittle nature of the OpenFold kernels are spot on. I will definitely adopt your recommendation and provide an official Docker version in the future (this month!!! ). Distributing the environment via containers will drastically lower the entry barrier caused by these C++ compilation errors.

In the meantime, since the current dependency stack is very strict, I have put together a detailed temporary workaround specifically addressing the version mismatches and the RTX 40-series compilation trap. I hope this helps you and other guys get it running smoothly:

1. Create the Conda Environment

Save the following as env.yaml to ensure strictly matched base packages (requires Python 3.7, CUDA 11.3, GCC/G++ <= 10.0.0):

name: unistab_env
channels:
  - conda-forge
  - bioconda
  - pytorch
dependencies:
  - conda-forge::python=3.7
  - conda-forge::setuptools=59.5.0
  - conda-forge::pip
  - conda-forge::openmm=7.5.1
  - conda-forge::pdbfixer
  - conda-forge::cudatoolkit==11.3.*
  - conda-forge::cudatoolkit-dev==11.3.*
  - conda-forge::einops==0.6.1
  - conda-forge::fairscale
  - conda-forge::omegaconf
  - conda-forge::hydra-core
  - conda-forge::pandas
  - conda-forge::pytest
  - bioconda::hmmer==3.3.2
  - bioconda::hhsuite==3.3.0
  - bioconda::kalign2==2.04
  - pytorch::pytorch=1.12.*
  - ehmoussi::gxx_linux-64

Then run:

conda env create -f env.yaml
conda activate unistab_env

2. Install Python Requirements & PyTorch

Save the required dependencies as requirements.txt:

biopython==1.79
deepspeed==0.5.9
dm-tree==0.1.6
ml-collections==0.1.0
numpy==1.21.2
PyYAML==5.4.1
requests==2.26.0
scipy==1.7.1
tqdm==4.62.2
typing-extensions==3.10.0.2
pytorch_lightning==1.5.10
wandb==0.12.21

Run the following commands:

pip install -r requirements.txt
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 -f https://mirror.sjtu.edu.cn/pytorch-wheels/torch_stable.html

3. Install DLLogger

git clone https://github.com/NVIDIA/dllogger.git
cd dllogger/
pip install -e .
cd ..

4. Install OpenFold 1.0.0 (Crucial Step)

If you install OpenFold directly using pip or the author's default command, it will install v2.0.0, which breaks the program. You must check out this specific commit:

git clone --filter=blob:none --quiet https://github.com/aqlaboratory/openfold.git ./openfold
cd ./openfold/
git rev-parse -q --verify 'sha^4b41059694619831a7db195b7e0988fc4ff3a307'
git fetch -q https://github.com/aqlaboratory/openfold.git 4b41059694619831a7db195b7e0988fc4ff3a307
git checkout -q 4b41059694619831a7db195b7e0988fc4ff3a307

5. Fix RTX 40-Series Compilation Issue

If you encounter the nvcc fatal : Unsupported gpu architecture 'compute_89' error when installing OpenFold, please modify the setup.py file in the openfold directory before running the installation:
Find this line: f'arch=compute_{major}{minor},code=sm_{major}{minor}'
Change it to: f'arch=compute_86,code=sm_86' (Note: You can run nvcc --list-gpu-arch to check your directly supported architecture and replace the number accordingly).

Finally, complete the installation:

pip install -e .
cd ..
pip install "fair-esm[esmfold]"

Thank you again for your help in improving the project's usability. I appreciate your patience while waiting for the Docker release!

Best regards,
Hong