SYSTEMS & INFRASTRUCTURE / MBZUAI
Distributed training
& profiling.
I built distributed training pipelines on NVIDIA and AMD ROCm clusters and worked on mixed precision and multi-GPU profiling.
WHAT I BUILT
Training infrastructure
Distributed training
Built training pipelines on NVIDIA and AMD ROCm clusters for large vision-transformer architectures.
Mixed-precision scaling
Worked with FP16 and BF16 scaling for large ViT training.
Multi-GPU profiling
Profiled multi-GPU workloads as part of the research infrastructure.