build_ffcv_cifar10_dataloader#

composer.datasets.build_ffcv_cifar10_dataloader(global_batch_size, is_train=True, download=True, drop_last=True, prefetch_factor=2, num_workers=8, ffcv_dir='/tmp', ffcv_dest='cifar_train.ffcv', ffcv_write_dataset=False, datadir=None)[source]#

Builds an FFCV CIFAR10 dataloader.

Parameters

  • global_batch_size (int) โ€“ Global batch size.

  • is_train (bool) โ€“ Whether to load the training data or validation data. Default: True.

  • download (bool, optional) โ€“ Whether to download the dataset, if needed. Default: True.

  • drop_last (bool) โ€“ Whether to drop last samples. Default: True.

  • prefetch_factor (int) โ€“ Number of batches to prefect. Default: 2.

  • ffcv_dir (str, optional) โ€“ A directory containing train/val <file>.ffcv files. If these files donโ€™t exist and ffcv_write_dataset is True, train/val <file>.ffcv files will be created in this dir. Default: "/tmp".

  • ffcv_dest (str, optional) โ€“ <file>.ffcv file that has dataset samples. Default: "cifar_train.ffcv".

  • ffcv_write_dataset (str | bool, optional) โ€“ Whether to create dataset in FFCV format (<file>.ffcv) if it doesnโ€™t exist. Default:

  • False. โ€“

  • datadir (str | None, optional) โ€“ Path to the non-FFCV data directory.