Tester

class neuralhydrology.evaluation.tester.BaseTester(cfg: Config, run_dir: Path, period: str = 'test', init_model: bool = True)

Bases: object

Base class to run inference on a model.

Use subclasses of this class to evaluate a trained model on its train, test, or validation period. For regression settings, RegressionTester is used; for uncertainty prediction, UncertaintyTester.

Parameters:

  • cfg (Config) – The run configuration.

  • run_dir (Path) – Path to the run directory.

  • period ({'train', 'validation', 'test'}, optional) – The period to evaluate, by default ‘test’.

  • init_model (bool, optional) – If True, the model weights will be initialized with the checkpoint from the last available epoch in run_dir.

evaluate(epoch: int = None, save_results: bool = True, save_all_output: bool = False, metrics: list | dict = [], model: Module = None, experiment_logger: Logger = None) → dict

Evaluate the model.

Parameters:

  • epoch (int, optional) – Define a specific epoch to evaluate. By default, the weights of the last epoch are used.

  • save_results (bool, optional) – If True, stores the evaluation results in the run directory. By default, True.

  • save_all_output (bool, optional) – If True, stores all of the model output in the run directory. By default, False.

  • metrics (Union[list, dict], optional) – List of metrics to compute during evaluation. Can also be a dict that specifies per-target metrics

  • model (torch.nn.Module, optional) – If a model is passed, this is used for validation.

  • experiment_logger (Logger, optional) – Logger can be passed during training to log metrics

Returns:

A dictionary containing one xarray per basin with the evaluation results.

Return type:

dict

class neuralhydrology.evaluation.tester.RegressionTester(cfg: Config, run_dir: Path, period: str = 'test', init_model: bool = True)

Bases: BaseTester

Tester class to run inference on a regression model.

Use the evaluate method of this class to evaluate a trained model on its train, test, or validation period.

Parameters:

  • cfg (Config) – The run configuration.

  • run_dir (Path) – Path to the run directory.

  • period ({'train', 'validation', 'test'}) – The period to evaluate.

  • init_model (bool, optional) – If True, the model weights will be initialized with the checkpoint from the last available epoch in run_dir.

class neuralhydrology.evaluation.tester.UncertaintyTester(cfg: Config, run_dir: Path, period: str = 'test', init_model: bool = True)

Bases: BaseTester

Tester class to run inference on an uncertainty model.

Use the evaluate method of this class to evaluate a trained model on its train, test, or validation period.

Parameters:

  • cfg (Config) – The run configuration.

  • run_dir (Path) – Path to the run directory.

  • period ({'train', 'validation', 'test'}) – The period to evaluate.

  • init_model (bool, optional) – If True, the model weights will be initialized with the checkpoint from the last available epoch in run_dir.