Skip to content

question about the training setup. #13

@CROVO1026

Description

@CROVO1026

If the goal is to compare LoRA-style methods fairly, why not fix one existing open-source MLLM checkpoint that already has a trained projector/connector (i.e., already aligned), and then apply LoRA vs MokA under the same SFT data/hparams/decoding? Why do you re-run the alignment pretraining stage?

Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions