GhostBatchNorm

class composer.algorithms.GhostBatchNorm(ghost_batch_size=32)

Replaces batch normalization modules with Ghost Batch Normalization modules that simulate the effect of using a smaller batch size.

Works by spliting input into chunks of ghost_batch_size samples and running batch normalization on each chunk separately. dim=0 is assumed to be the sample axis.

Runs on Event.INIT.

Parameters

ghost_batch_size (int, optional) – size of sub-batches to normalize over. Default: 32.