FSDPConfig

class composer.utils.FSDPConfig(activation_checkpointing=False, activation_checkpointing_reentrant=True, activation_cpu_offload=False, auto_wrap=True, te_checkpoint_wrapper=False, te_shard_fp8_weight=False, backward_prefetch='BACKWARD_POST', backward_prefetch_limit=1, cpu_offload=False, data_parallel_shard_degree=- 1, data_parallel_replicate_degree=None, device_mesh=None, forward_prefetch=False, forward_prefetch_limit=1, ignored_modules=None, keep_low_precision_grads=False, limit_all_gathers=True, load_monolith_rank0_only=False, load_planner=None, mixed_precision='DEFAULT', process_group=None, save_planner=None, sharded_ckpt_prefix_dir='ep{epoch}-ba{batch}', sharding_strategy='FULL_SHARD', state_dict_type='full', sync_module_states=False, use_orig_params=True, verbose=False)

Configuration for Fully Sharded Data Parallelism (FSDP).