About this role
Design and build distributed training toolchains and optimize distributed training performance across computation, communication, and storage layers for ultra-large-scale model training. Focus on improving training stability and scalability and conducting frontier research into distributed training technologies for high-fidelity portrait and video generation.
Skills for this role
Distributed trainingData ParallelismPipeline ParallelismTensor ParallelismExpert ParallelismPyTorchDeepSpeedMegatron-LMGPU hardware architectureCUDA programmingCUDA kernel developmentNCCLcuDNNVideo pre-training methodologiesTransformer architecturesDiffusion modelsStable DiffusionFluxToolchain developmentSystem optimizationTraining stabilityScalabilityProduction-grade implementationGenerative AIVisual synthesisModel-as-a-Service (MaaS)