Contents Menu Expand Light mode Dark mode Auto light/dark mode
Composer
Light Logo Dark Logo
Star

Getting Started

  • ๐Ÿ’พ Installation
  • ๐Ÿš€ Quick Start
  • ๐ŸšŒ Welcome Tour

Tutorials

  • ๐Ÿ–ผ๏ธ Getting Started
  • ฦ’() Functional API
  • ๐Ÿค– Custom Speedup Methods
  • ๐Ÿค— Finetuning Hugging Face Models
  • ๐Ÿค— Pretraining and Finetuning with Hugging Face Models
  • โšก Migrating from PTL
  • ๐Ÿ›‘ Early Stopping
  • โ™ป๏ธ Auto Microbatching
  • โฏ๏ธ Autoresume Training
  • ๐Ÿฅก Exporting for Inference
  • ๐Ÿ”Œ Training with TPUs
  • ๐Ÿ•น๏ธ Distributed Training with Submitit
  • โ˜๏ธ Train ResNet-50 on AWS

Speedup Methods

  • ๐Ÿค– Algorithms
  • ฦ’() Functional

Trainer

  • โš™๏ธ Using the Trainer
  • ๐Ÿ›ป ComposerModel
  • ๐Ÿ’ฟ DataLoaders
  • ๐Ÿ“Š Evaluation
  • ๐Ÿ“‰ Schedulers
  • โŒ› Time
  • ๐Ÿ“† Events
  • โœ… Checkpointing
  • ๐Ÿชต Logging
  • ๐Ÿ›• File Uploading
  • โ˜Ž๏ธ Callbacks
  • ๐ŸŽ๏ธ Performance
    • โฑ๏ธ Performance Profiling
    • ๐Ÿฅฝ Analyzing Traces

Notes

  • ๐Ÿ‘จโ€๐Ÿ‘ฉโ€๐Ÿ‘งโ€๐Ÿ‘ฆ Distributed Training
  • ๐Ÿ›‘ Early Stopping
  • ๐Ÿ”ข Numerics
  • โ™ป๏ธ Automatic Microbatching
  • โฏ Auto Resumption
  • ๐Ÿ“‹ Tensorboard Monitoring
  • ๐Ÿƒโ€โ™€๏ธ Run Name

Methods Library

  • ๐Ÿƒ Methods Overview
  • ๐Ÿฅธ ALiBi
  • ๐ŸŽจ AugMix
  • ๐ŸŠ BlurPool
  • ๐Ÿ“บ Channels Last
  • ๐Ÿ›๏ธ ColOut
  • โœ‚๏ธ CutMix
  • ๐ŸŽƒ Cutout
  • ๐Ÿ‹๏ธโ€โ™€๏ธ Decoupled Weight Decay
  • ๐Ÿšš EMA
  • โž— Factorize
  • โ›ฉ๏ธ Gated Linear Units
  • ๐Ÿ‘ป Ghost BatchNorm
  • ๐Ÿ“Ž Gradient Clipping
  • ๐Ÿฅ™ Gyro Dropout
  • ๐Ÿงˆ Label Smoothing
  • โ„๏ธ Layer Freezing
  • ๐ŸŽ‚ Low Precision GroupNorm
  • ๐Ÿง Low Precision LayerNorm
  • ๐Ÿฅฃ MixUp
  • ๐Ÿž๏ธ Progressive Image Resizing
  • ๐ŸŽฒ RandAugment
  • ๐Ÿ”๏ธ Sharpness Aware Minimization (SAM)
  • โš–๏ธ Scale Schedule
  • โฎ๏ธ Selective Backprop
  • ๐Ÿ”† Sequence Length Warmup
  • ๐Ÿซ€ Squeeze-and-Excitation
  • ๐ŸงŠ Stochastic Depth (Block)
  • ๐ŸŽฐ Stochastic Depth (Sample)
  • ๐Ÿงฉ Stochastic Weight Averaging
  • ๐„ท Weight Standardization

API Reference

  • composer
    • Algorithm
    • Callback
    • ComposerModel
    • DataSpec
    • Engine
    • Evaluator
    • Event
    • Logger
    • State
    • Time
    • TimeUnit
    • Timestamp
    • Trainer
  • composer.algorithms
    • Alibi
    • AugMix
    • AugmentAndMixTransform
    • BlurPool
    • ChannelsLast
    • ColOut
    • ColOutTransform
    • CutMix
    • CutOut
    • EMA
    • Factorize
    • GatedLinearUnits
    • GhostBatchNorm
    • GradientClipping
    • GyroDropout
    • LabelSmoothing
    • LayerFreezing
    • LowPrecisionGroupNorm
    • LowPrecisionLayerNorm
    • MixUp
    • NoOpModel
    • ProgressiveResizing
    • RandAugment
    • RandAugmentTransform
    • SAM
    • SWA
    • SelectiveBackprop
    • SeqLengthWarmup
    • SqueezeExcite
    • SqueezeExcite2d
    • SqueezeExciteConv2d
    • StochasticDepth
    • WeightStandardization
  • composer.callbacks
    • ActivationMonitor
    • CheckpointSaver
    • EarlyStopper
    • EvalOutputLogging
    • ExportForInferenceCallback
    • FreeOutputs
    • Generate
    • ImageVisualizer
    • LRMonitor
    • MLPerfCallback
    • MemoryMonitor
    • MemorySnapshot
    • NaNMonitor
    • OOMObserver
    • OptimizerMonitor
    • RuntimeEstimator
    • SpeedMonitor
    • SystemMetricsMonitor
    • ThresholdStopper
  • composer.core
    • ensure_data_spec
    • ensure_evaluator
    • ensure_time
    • get_precision_context
    • MemoryFormat
    • Precision
    • Serializable
    • Trace
    • TrainerMode
  • composer.core.types
    • MemoryFormat
    • TrainerMode
  • composer.devices
    • Device
    • DeviceCPU
    • DeviceGPU
    • DeviceHPU
    • DeviceMPS
    • DeviceNeuron
    • DeviceTPU
  • composer.distributed
    • ddp_sync_context
    • fix_batch_precision_for_deepspeed
    • parse_deepspeed_config
    • prepare_ddp_module
    • prepare_fsdp_module
    • prepare_tp_module
    • DDPSyncStrategy
  • composer.functional
    • apply_alibi
    • apply_blurpool
    • apply_channels_last
    • apply_factorization
    • apply_gated_linear_units
    • apply_ghost_batchnorm
    • apply_gradient_clipping
    • apply_gyro_dropout
    • apply_low_precision_groupnorm
    • apply_low_precision_layernorm
    • apply_squeeze_excite
    • apply_stochastic_depth
    • apply_weight_standardization
    • augmix_image
    • colout_batch
    • compute_ema
    • cutmix_batch
    • cutout_batch
    • freeze_layers
    • mixup_batch
    • randaugment_image
    • resize_batch
    • select_using_loss
    • set_batch_sequence_length
    • should_selective_backprop
    • smooth_labels
  • composer.loggers
    • CometMLLogger
    • ConsoleLogger
    • FileLogger
    • InMemoryLogger
    • LoggerDestination
    • MLFlowLogger
    • MosaicMLLogger
    • NeptuneLogger
    • ProgressBarLogger
    • RemoteUploaderDownloader
    • SlackLogger
    • TensorboardLogger
    • WandBLogger
  • composer.loss
    • binary_cross_entropy_with_logits
    • soft_cross_entropy
    • DiceLoss
  • composer.metrics
    • BinaryF1Score
    • CrossEntropy
    • Dice
    • LanguageCrossEntropy
    • LanguagePerplexity
    • LossMetric
    • MAP
    • MIoU
    • MaskedAccuracy
  • composer.models
    • write_huggingface_pretrained_from_composer_checkpoint
    • ComposerClassifier
    • HuggingFaceModel
    • Initializer
  • composer.optim
    • compile_composer_scheduler
    • ComposerScheduler
    • ConstantScheduler
    • ConstantWithWarmupScheduler
    • CosineAnnealingScheduler
    • CosineAnnealingWarmRestartsScheduler
    • CosineAnnealingWithWarmupScheduler
    • DecoupledAdamW
    • DecoupledSGDW
    • ExponentialScheduler
    • LinearScheduler
    • LinearWithWarmupScheduler
    • MultiStepScheduler
    • MultiStepWithWarmupScheduler
    • PolynomialScheduler
    • PolynomialWithWarmupScheduler
    • StepScheduler
  • composer.profiler
    • cyclic_schedule
    • JSONTraceHandler
    • Marker
    • Profiler
    • ProfilerAction
    • SystemProfiler
    • TorchProfiler
    • TraceHandler
  • composer.utils
    • add_vision_dataset_transform
    • batch_get
    • batch_set
    • build_remote_backend
    • configure_excepthook
    • convert_flat_dict_to_nested_dict
    • convert_nested_dict_to_flat_dict
    • create_fsdp_config
    • create_interval_scheduler
    • create_symlink_file
    • disable_env_report
    • enable_env_report
    • ensure_folder_has_no_conflicting_files
    • ensure_folder_is_empty
    • ensure_tuple
    • export_for_inference
    • export_with_logger
    • extract_hparams
    • extract_path_from_symlink
    • format_name_with_dist
    • format_name_with_dist_and_time
    • get_composer_env_dict
    • get_compressor
    • get_device
    • get_file
    • get_free_tcp_port
    • get_save_filename
    • import_object
    • is_compressed_pt
    • is_hpu_installed
    • is_model_deepspeed
    • is_model_fsdp
    • is_notebook
    • is_tar
    • is_xla_installed
    • load_checkpoint
    • map_collection
    • maybe_create_object_store_from_uri
    • maybe_create_remote_uploader_downloader_from_uri
    • model_eval_mode
    • parse_uri
    • partial_format
    • print_env
    • retry
    • safe_torch_load
    • save_checkpoint
    • validate_credentials
    • CliCompressor
    • EvalClient
    • ExportFormat
    • FSDPConfig
    • GCSObjectStore
    • IteratorFileStream
    • LambdaEvalClient
    • LibcloudObjectStore
    • LocalEvalClient
    • MLFlowObjectStore
    • OCIObjectStore
    • ObjectStore
    • ParallelismConfig
    • ParallelismType
    • PartialFilePath
    • RemoteFilesExistingCheckStatus
    • RemoteUploader
    • S3ObjectStore
    • SFTPObjectStore
    • StringEnum
    • TPConfig
    • UCObjectStore
    • composer.utils.MissingConditionalImportError
    • composer.utils.ObjectStoreTransientError
    • composer.utils.VersionedDeprecationWarning
  • composer.utils.dist
    • all_gather
    • all_gather_object
    • all_reduce
    • barrier
    • broadcast
    • broadcast_object_list
    • get_global_rank
    • get_local_rank
    • get_local_world_size
    • get_node_rank
    • get_sampler
    • get_world_size
    • initialize_dist
    • is_available
    • is_initialized
  • composer.utils.reproducibility
    • configure_deterministic_mode
    • get_random_seed
    • get_rng_state
    • load_rng_state
    • seed_all
    • seed_context
Back to top
Edit this page

composer.devices#

Module for devices on which models run.

Classes

Device

Abstract class for a device on which a model runs.

DeviceCPU

An extension of Device for CPUs.

DeviceGPU

An extension of Device for GPUs.

DeviceHPU

An extension of Device for HPUs.

DeviceMPS

Device to support MPS, for training on Apple's M-series chips.

DeviceNeuron

An extension of Device for Neuron devices (Trn, Inf).

DeviceTPU

An extension of Device for TPUs.

Next
Device
Previous
TrainerMode
Copyright © 2022, MosaicML, Inc.
Made with Sphinx and @pradyunsg's Furo