Contents Menu Expand Light mode Dark mode Auto light/dark mode
Composer
Light Logo Dark Logo
Star

Getting Started

  • ๐Ÿ’พ Installation
  • ๐Ÿš€ Quick Start
  • ๐ŸšŒ Welcome Tour

Tutorials

  • ๐Ÿ–ผ๏ธ Getting Started
  • ฦ’() Functional API
  • ๐Ÿฉบ Image Segmentation
  • ๐Ÿค– Custom Speedup Methods
  • ๐ŸŽ๏ธ FFCV DataLoaders
  • ๐Ÿค— Finetuning Hugging Face Models
  • ๐Ÿค— Pretraining and Finetuning with Hugging Face Models
  • โšก Migrating from PTL
  • ๐Ÿ›‘ Early Stopping
  • โ™ป๏ธ Auto Microbatching
  • โฏ๏ธ Autoresume Training
  • ๐Ÿฅก Exporting for Inference
  • ๐Ÿ”Œ Training with TPUs
  • ๐Ÿ•น๏ธ Distributed Training with Submitit
  • โ˜๏ธ Train ResNet-50 on AWS

Speedup Methods

  • ๐Ÿค– Algorithms
  • ฦ’() Functional

Trainer

  • โš™๏ธ Using the Trainer
  • ๐Ÿ›ป ComposerModel
  • ๐Ÿ’ฟ DataLoaders
  • ๐Ÿ“Š Evaluation
  • ๐Ÿ“‰ Schedulers
  • โŒ› Time
  • ๐Ÿ“† Events
  • โœ… Checkpointing
  • ๐Ÿชต Logging
  • ๐Ÿ›• File Uploading
  • โ˜Ž๏ธ Callbacks
  • ๐ŸŽ๏ธ Performance
    • โฑ๏ธ Performance Profiling
    • ๐Ÿฅฝ Analyzing Traces

Notes

  • ๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ Distributed Training
  • ๐Ÿ›‘ Early Stopping
  • ๐Ÿ”ข Numerics
  • โ™ป๏ธ Automatic Microbatching
  • โฏ Auto Resumption
  • ๐Ÿ“‹ Tensorboard Monitoring
  • ๐Ÿƒโ€โ™€๏ธ Run Name

Methods Library

  • ๐Ÿƒ Methods Overview
  • ๐Ÿฅธ ALiBi
  • ๐ŸŽจ AugMix
  • ๐ŸŠ BlurPool
  • ๐Ÿ“บ Channels Last
  • ๐Ÿ›๏ธ ColOut
  • โœ‚๏ธ CutMix
  • ๐ŸŽƒ Cutout
  • ๐Ÿ‹๏ธโ€โ™€๏ธ Decoupled Weight Decay
  • ๐Ÿšš EMA
  • โž— Factorize
  • ๐Ÿฐ Fused LayerNorm
  • โ›ฉ๏ธ Gated Linear Units
  • ๐Ÿ‘ป Ghost BatchNorm
  • ๐Ÿ“Ž Gradient Clipping
  • ๐Ÿฅ™ Gyro Dropout
  • ๐Ÿงˆ Label Smoothing
  • โ„๏ธ Layer Freezing
  • ๐Ÿง Low Precision LayerNorm
  • ๐Ÿฅฃ MixUp
  • ๐Ÿž๏ธ Progressive Image Resizing
  • ๐ŸŽฒ RandAugment
  • ๐Ÿ”๏ธ Sharpness Aware Minimization (SAM)
  • โš–๏ธ Scale Schedule
  • โฎ๏ธ Selective Backprop
  • ๐Ÿ”† Sequence Length Warmup
  • ๐Ÿซ€ Squeeze-and-Excitation
  • ๐ŸงŠ Stochastic Depth (Block)
  • ๐ŸŽฐ Stochastic Depth (Sample)
  • ๐Ÿงฉ Stochastic Weight Averaging
  • ๐„ท Weight Standardization

Model Library

  • ๐Ÿฆญ BERT
  • ๐Ÿ‘€ CIFAR ResNet
  • ๐Ÿคฟ DeepLabv3+
  • ๐Ÿ•ธ๏ธ EfficientNet
  • ๐Ÿ“š GPT-2
  • ๐Ÿ™๏ธ ResNet
  • โ†ฉ๏ธ UNet

API Reference

  • composer
    • Algorithm
    • Callback
    • ComposerModel
    • DataSpec
    • Engine
    • Evaluator
    • Event
    • Logger
    • State
    • Time
    • TimeUnit
    • Timestamp
    • Trainer
  • composer.algorithms
    • Alibi
    • AugMix
    • AugmentAndMixTransform
    • BlurPool
    • ChannelsLast
    • ColOut
    • ColOutTransform
    • CutMix
    • CutOut
    • EMA
    • Factorize
    • FusedLayerNorm
    • GatedLinearUnits
    • GhostBatchNorm
    • GradientClipping
    • GyroDropout
    • LabelSmoothing
    • LayerFreezing
    • LowPrecisionLayerNorm
    • MixUp
    • NoOpModel
    • ProgressiveResizing
    • RandAugment
    • RandAugmentTransform
    • SAM
    • SWA
    • SelectiveBackprop
    • SeqLengthWarmup
    • SqueezeExcite
    • SqueezeExcite2d
    • SqueezeExciteConv2d
    • StochasticDepth
    • WeightStandardization
  • composer.callbacks
    • CheckpointSaver
    • EarlyStopper
    • ExportForInferenceCallback
    • ImageVisualizer
    • LRMonitor
    • MLPerfCallback
    • MemoryMonitor
    • OptimizerMonitor
    • SpeedMonitor
    • ThresholdStopper
  • composer.core
    • ensure_data_spec
    • ensure_evaluator
    • ensure_time
    • get_precision_context
    • MemoryFormat
    • Precision
    • Serializable
    • Trace
    • TrainerMode
    • composer.core.BreakEpochException
  • composer.core.types
    • MemoryFormat
    • TrainerMode
    • composer.core.types.BreakEpochException
  • composer.datasets
    • build_ade20k_dataloader
    • build_cifar10_dataloader
    • build_ffcv_cifar10_dataloader
    • build_ffcv_imagenet_dataloader
    • build_imagenet_dataloader
    • build_lm_dataloader
    • build_mnist_dataloader
    • build_streaming_ade20k_dataloader
    • build_streaming_c4_dataloader
    • build_streaming_cifar10_dataloader
    • build_streaming_imagenet1k_dataloader
    • build_synthetic_ade20k_dataloader
    • build_synthetic_cifar10_dataloader
    • build_synthetic_imagenet_dataloader
    • build_synthetic_mnist_dataloader
    • ADE20k
    • PytTrain
    • PytVal
    • SyntheticBatchPairDataset
    • SyntheticDataLabelType
    • SyntheticDataType
    • SyntheticPILDataset
  • composer.devices
    • Device
    • DeviceCPU
    • DeviceGPU
    • DeviceMPS
    • DeviceTPU
  • composer.functional
    • apply_alibi
    • apply_blurpool
    • apply_channels_last
    • apply_factorization
    • apply_fused_layernorm
    • apply_gated_linear_units
    • apply_ghost_batchnorm
    • apply_gradient_clipping
    • apply_gyro_dropout
    • apply_low_precision_layernorm
    • apply_squeeze_excite
    • apply_stochastic_depth
    • apply_weight_standardization
    • augmix_image
    • colout_batch
    • compute_ema
    • cutmix_batch
    • cutout_batch
    • freeze_layers
    • mixup_batch
    • randaugment_image
    • resize_batch
    • select_using_loss
    • set_batch_sequence_length
    • should_selective_backprop
    • smooth_labels
  • composer.loggers
    • CometMLLogger
    • ConsoleLogger
    • FileLogger
    • InMemoryLogger
    • LoggerDestination
    • MLFlowLogger
    • ProgressBarLogger
    • RemoteUploaderDownloader
    • TensorboardLogger
    • WandBLogger
  • composer.loss
    • binary_cross_entropy_with_logits
    • soft_cross_entropy
    • DiceLoss
  • composer.metrics
    • BinaryF1Score
    • CrossEntropy
    • Dice
    • HFCrossEntropy
    • InContextLearningLMAccuracy
    • InContextLearningMetric
    • InContextLearningMultipleChoiceAccuracy
    • LanguageCrossEntropy
    • LossMetric
    • MAP
    • MIoU
    • MaskedAccuracy
    • Perplexity
  • composer.models
    • composer_deeplabv3
    • composer_efficientnetb0
    • composer_resnet
    • composer_resnet_cifar
    • composer_timm
    • create_bert_classification
    • create_bert_mlm
    • create_gpt2
    • mnist_model
    • vit_small_patch16
    • ComposerClassifier
    • HuggingFaceModel
    • Initializer
    • MMDetModel
    • UNet
  • composer.optim
    • compile_composer_scheduler
    • ComposerScheduler
    • ConstantScheduler
    • ConstantWithWarmupScheduler
    • CosineAnnealingScheduler
    • CosineAnnealingWarmRestartsScheduler
    • CosineAnnealingWithWarmupScheduler
    • DecoupledAdamW
    • DecoupledSGDW
    • ExponentialScheduler
    • LinearScheduler
    • LinearWithWarmupScheduler
    • MultiStepScheduler
    • MultiStepWithWarmupScheduler
    • PolynomialScheduler
    • PolynomialWithWarmupScheduler
    • StepScheduler
  • composer.profiler
    • cyclic_schedule
    • JSONTraceHandler
    • Marker
    • Profiler
    • ProfilerAction
    • SystemProfiler
    • TorchProfiler
    • TraceHandler
  • composer.utils
    • batch_get
    • batch_set
    • configure_excepthook
    • convert_flat_dict_to_nested_dict
    • convert_nested_dict_to_flat_dict
    • create_symlink_file
    • disable_env_report
    • enable_env_report
    • ensure_folder_has_no_conflicting_files
    • ensure_folder_is_empty
    • ensure_tuple
    • export_for_inference
    • export_with_logger
    • extract_hparams
    • format_name_with_dist
    • format_name_with_dist_and_time
    • get_composer_env_dict
    • get_device
    • get_file
    • get_free_tcp_port
    • import_object
    • is_model_deepspeed
    • is_model_fsdp
    • is_notebook
    • is_tar
    • is_tpu_installed
    • load_checkpoint
    • map_collection
    • maybe_create_object_store_from_uri
    • maybe_create_remote_uploader_downloader_from_uri
    • model_eval_mode
    • parse_uri
    • print_env
    • retry
    • save_checkpoint
    • ExportFormat
    • IteratorFileStream
    • LibcloudObjectStore
    • OCIObjectStore
    • ObjectStore
    • PartialFilePath
    • S3ObjectStore
    • SFTPObjectStore
    • StringEnum
    • composer.utils.MissingConditionalImportError
    • composer.utils.ObjectStoreTransientError
  • composer.utils.dist
    • all_gather
    • all_gather_object
    • all_reduce
    • barrier
    • broadcast
    • broadcast_object_list
    • get_global_rank
    • get_local_rank
    • get_local_world_size
    • get_node_rank
    • get_sampler
    • get_world_size
    • initialize_dist
    • is_available
    • is_initialized
  • composer.utils.reproducibility
    • configure_deterministic_mode
    • get_random_seed
    • get_rng_state
    • load_rng_state
    • seed_all
  v: latest
Versions
latest
stable
v0.12.0
v0.11.1
v0.11.0
v0.10.1
v0.10.0
v0.9.0
v0.8.2
v0.8.1
v0.8.0
v0.7.1
v0.7.0
v0.6.1
v0.6.0
v0.5.0
v0.4.0
v0.3.1
v0.3.0
v0.2.4
v0.2.3
Downloads
On Read the Docs
Project Home
Builds
Back to top
Edit this page

create_symlink_file#

composer.utils.create_symlink_file(existing_path, destination_filename)[source]#

Create a symlink file, which can be followed by get_file().

Unlike unix symlinks, symlink files can be created by this function are normal text files and can be uploaded to object stores via ObjectStore.upload_object() or loggers via Logger.upload_file() that otherwise would not support unix-style symlinks.

Parameters
  • existing_path (str) โ€“ The name of existing object that the symlink file should point to.

  • destination_filename (str | Path) โ€“ The filename to which to write the symlink. It must end in '.symlink'.

Next
disable_env_report
Previous
convert_nested_dict_to_flat_dict
Copyright © 2022, MosaicML, Inc.
Made with Sphinx and @pradyunsg's Furo