- class composer.callbacks.MemoryMonitor#
Logs the memory usage of the model.
>>> from composer import Trainer >>> from composer.callbacks import MemoryMonitor >>> # constructing trainer object with this callback >>> trainer = Trainer( ... model=model, ... train_dataloader=train_dataloader, ... eval_dataloader=eval_dataloader, ... optimizers=optimizer, ... max_duration="1ep", ... callbacks=[MemoryMonitor()], ... )
The memory statistics are logged by the
Loggerto the following keys as described below.
Several memory usage statistics are logged on
The following statistics are recorded:
Number of memory allocation requests received by the memory allocator.
Number of memory free requests received by the memory allocator.
Amount of allocated memory in bytes.
Amount of active memory in bytes at the time of recording.
Amount of inactive, non-releaseable memory in bytes at the time of recording.
Amount of reserved memory in bytes at the time of recording.
Number of failed cudaMalloc calls that result in a cache flush and retry.
Memory usage monitoring is only supported for GPU devices.