About this role
The LocalAI team is seeking a Systems Software Engineer to build and optimize efficient on-device AI inference stacks for RTX and DGX-class systems. The role involves developing modern inference runtimes, performing end-to-end model optimization, and ensuring production readiness for AI workloads on resource-constrained platforms.
Skills for this role
C++Machine LearningDeep LearningLlama.cppvLLMPyTorchWinMLDXCGCTensorRTCUDAVulkanDirectXQuantizationPruningSparsityDistillation