fsdp

Training runs and infrastructure decisions involving Fully Sharded Data Parallelism, model sharding, checkpointing, and memory tradeoffs.