Modern large-scale deep learning workloads highlight the need for parallel execution across many devicesin order to fit model data into hardware accelerator memories. In these settings, array redistribution maybe required during a computation, but can also become a bottleneck if not done efficiently