domains / families / science / bionemo-kermt-pretrain-scratch

bionemo-kermt-pretrain-scratch

separate

Pretrain a fresh KERMT model from scratch on a user-provided corpus. Builds a new vocabulary from the corpus, instantiates the model architecture from defaults, and launches pretrain_ddp.py inside the kermt container (detached for long runs). Unlike kermt-continue-pretrain, no starting checkpoint is loaded — the model is randomly initialized.

When to use

• train a new model on a custom corpus
• reproduce a pretrain config end-to-end
• retrain a model on a custom chemistry domain

When not to use

• continuing an existing released ckpt
• adding a cMIM decoder to an encoder-only grover_base ckpt

Skill page

Source