About this role
The Member of Technical Staff, Pretraining will work at the core of LambdaQ Labs to train foundation models, managing the pretraining pipeline from data curation to large-scale distributed training on multi-node GPU clusters.
Skills for this role
PyTorchDistributed TrainingFSDPDeepSpeedMegatronTransformer internalsScaling lawsData curationTokenization