Hyperparameters for the Evaluator.


These classes are used with yahp for YAML-based configuration.


Params for the Evaluator.

class composer.datasets.evaluator_hparams.EvaluatorHparams(label, eval_dataset, eval_interval=None, subset_num_batches=None, metric_names=None)[source]#

Bases: yahp.hparams.Hparams

Params for the Evaluator.

Also see the documentation for the Evaluator.

  • label (str) โ€“ Name of the Evaluator. Used for logging/reporting metrics.

  • eval_interval (str, optional) โ€“ See Evaluator.

  • subset_num_batches (int, optional) โ€“ See Evaluator.

  • eval_dataset (DatasetHparams) โ€“ Evaluation dataset.

  • metrics (list, optional) โ€“ List of strings of names of the metrics for the evaluator. Can be a torchmetrics.Metric name or the class name of a metric returned by metrics() If None, uses all metrics in the model. Default: None.

initialize_object(model, batch_size, dataloader_hparams)[source]#

Initialize an Evaluator

If the Evaluator metric_names is empty or None is provided, the function returns a copy of all the modelโ€™s default evaluation metrics.

  • model (ComposerModel) โ€“ The model, which is used to retrieve metric names.

  • batch_size (int) โ€“ The device batch size to use for the evaluation dataset.

  • dataloader_hparams (DataLoaderHparams) โ€“ The hparams to use to construct a dataloader for the evaluation dataset.


Evaluator โ€“ The evaluator.