Hardcoded PeptideL causes pipeline failure on Protein-DNA complexes (e.g. 3K5M)

[xinyuren-bio]

Hi team,

I encountered an issue when processing Protein-DNA complexes (eg: 3K5M) and traced the root cause to the hardcoded polymer_type in the parsing stage.

Here is the detailed flow of how this causes the pipeline to fail:

1. Parsing Stage (parse_mmcif): In the loop processing chains, polymer_type is hardcoded to gemmi.PolymerType.PeptideL. Inside parse_polymer, this forces chain_type to be set to PROTEIN (around line 746).

Result: For 3K5M, the DNA chains (Chain B, C) are incorrectly labeled as PROTEIN.

2. Tokenization Stage (boltz_protein.py): The code attempts to filter out non-protein chains:

if chain["mol_type"] != const.chain_type_ids["PROTEIN"]:
continue # Skip non-protein chains
Result: Since the DNA chains were mislabeled as PROTEIN in step 1, they are not skipped. The DNA residues (DA, DC, DG, DT) are then tokenized.

3. Training/Extraction Stage (extract_sequence_from_tokens): The code tries to map residues to standard amino acids. Since DNA residues (DA, DC...) are not in restype_3to1 (which only contains the 20 standard amino acids), they are identified as non-protein residues and filtered out.

Final Error: This leads to the error: "No protein sequence in 3k5m. Skipping." because the logic gets confused by the mislabeled chains.

Evidence from 3K5M.cif:

Entity 1: polypeptide(L) -> Chain A (Protein)

Entity 2: polydeoxyribonucleotide -> Chain B (DNA)

Entity 3: polydeoxyribonucleotide -> Chain C (DNA)

Is this intended behavior? I am not entirely sure if this hardcoding is a deliberate design choice (i.e., the current model is strictly intended for pure proteins only, and I shouldn't be feeding it complexes), or if this is an oversight in the parsing logic.

Thanks!

[yuyangw]

Hi @xinyuren-bio, thanks for your interest in SimpleFold. Yes, for current version of SimpleFold, it is for protein monomer structure generations only. You'll need to further train the model with non-protein tokens if protein complexes are parsed.

[xinyuren-bio]

download_pdb.py

700M_PDB_SP.txt

Hi @yuyangw,Thanks for your reply.
I am having trouble reproducing the results for SimpleFold. My current metrics show a gap compared to your reported values, and I suspect the issue might stem from my dataset or training setup.

Dataset Doubts My PDB dataset contains approximately 15k entries, which includes complexes such as Protein-DNA structures. Based on the parsing issues I encountered with 3K5M, should I be strictly filtering the data to only include single-chain proteins or monomers? I am wondering if the inclusion of these complexes is the primary reason for the performance drop.

Training Configuration I am training the model on 8x H100 GPUs with a gradient accumulation of 4. I have attached a screenshot of my current configuration and model output. Could there be any issues with my parameter settings for this specific hardware that might affect the convergence or final metrics?
I have attached my PDB download script for your review. Any guidance would be greatly appreciated.
Thank you!

[xinyuren-bio]

I’ve been training the 700M model on the PDB and SP datasets, and I wanted to double-check if my loss curves look right compared to yours.

Attached are the curves from my run. Do these numerical values look normal to you? It would be super helpful to know if we're on the right track as we try to wrap this up.

Thanks a lot for the help! It really means a lot to us.

[yuyangw]

Hi @xinyuren-bio, thanks for your followup. Both the training configuration and loss curve look reasonable to me.

In preparing training data, we include PDB structures besides monomers but only keep polypeptide chains from complex structures. So the number of entries should be larger than 120K as shown in your figure.

Also, I notice in your inference configuration $\tau$ is set to 0.3 while in our case we use $\tau=0.01$. This could be another factor resulting in the performance gap.

Hope this helps!

[xinyuren-bio]

Hi @yuyangw,

Thanks again for the previous advice! Setting τ=0.01 during inference definitely improved the metrics, though there is still a significant gap compared to the results reported in the paper.

I’ve attached a 3D structure predictions from my current checkpoint (230k steps). While the model seems to capture the general backbone topology correctly, the sidechains are showing significant steric clashes and unphysical orientations.

A few questions:

Is it normal to see poor sidechain refinement after 230k steps? Around which step count do you usually see the sidechains start to converge?

I’ve also attached my loss curves up to 240k steps and the GDT-TS curves. Based on your experience, is it likely that GDT-TS will continue to climb toward ~0.75 with further training, or does this look like a plateau?

I'd appreciate any insights you have on whether I should keep the training running or if there might be another bottleneck.

[xinyuren-bio]

Hi @yuyangw,

I’m providing an update as I’ve looked deeper into the structural outputs.

Initially, I thought the steric clashes were limited to the test set. However, after verifying samples from the training set, I found that they exhibit the same issues: the backbone topology is generally correct, but the sidechains are clashing and physically unreasonable.

This is the state of the model at the 230k-step checkpoint for the 700M version. I’ve attached comparative images and the PDB/CIF files from both sets to illustrate this.

Given this, I have a few questions:

Is it normal for a 700M model to still show severe sidechain clashes across all datasets at 230k steps?

In your experience, around which step count does the model typically begin to refine these sidechain geometries?

I’d appreciate your perspective on whether I should keep the training running or if there’s a bottleneck I should address.

Predicted Structure：

The original PDB structure:

Aligned View:

I’ve attached the predicted and original PDB files, but I renamed them to .txt to bypass GitHub’s upload restrictions.

0_sampled_0.txt

AF-P92668-F1-model_v4.txt

[yuyangw]

Hi @xinyuren-bio, thanks for the feedback. For 700M model training over 200K iterations, the generated structures should have plausible side chains. Based on my experience, such scenario can happen in early training stage but should be significantly mitigated after 100K iterations.

What is the batch size used in your training? I wonder if you're using a smaller batch size which leads to slow convergence.

[xinyuren-bio]

Hi @yuyangw, thank you so much for your detailed explanation and for being so patient with my follow-up questions.

Following your suggestion, I did a deeper comparison of the metrics. I noticed a very consistent 4x discrepancy between my results and yours. Specifically, my GDT-TS values at 40k and 160k steps match almost perfectly with what you reported at 10k and 40k iterations.

This leads me to believe that there might be a difference in how "steps" are being counted in our respective environments. To help clarify my setup, I previously uploaded the 700M_PDB_SP.txt file in an earlier comment. That log contains the full terminal output from the start of my training run, confirming the parameters:
Hardware: 8x H100 GPUs
Batch Size: 1 (per GPU)
Accumulate Grad Batches: 4

I have double-checked my code and TensorBoard, and the accumulate_grad_batches parameter seems to be passed correctly. However, the 4x lag in performance suggests that my model might simply be "earlier" in its training than the step count implies.

I would be very grateful if you could share a reference training loss curve for your 700M model. I’d like to see if my loss at 100k steps aligns with your loss at 25k iterations. If this 4x relationship holds, it would mean my current 230k steps are effectively only ~57k of yours, which would explain why the sidechain refinement hasn't matured yet.

Thank you again for your time and for all the help you've provided so far!

[yuyangw]

Hi @xinyuren-bio, I can provide some data points in training for your reference. Smooth LDDT loss is ~0.29 at 50K steps, ~0.27 at 100K steps, and ~0.253 at 200K steps. My experience is smooth LDDT loss is usually a good indicator for model performance. Having said that, 8 GPUs with gradient accumulation should be the same setting as our experiments. Could it because of the step logging issue when using gradient accumulation? Also, did you follow the same batch size per GPU which is 16 in our case? Hope this helps!