fastreid.evaluation.evaluator Namespace Reference

Classes
class	DatasetEvaluator

Functions
	inference_on_dataset (model, data_loader, evaluator)
	inference_context (model)

Function Documentation

inference_context()

fastreid.evaluation.evaluator.inference_context

(

model

)

A context where the model is temporarily changed to eval mode,
and restored to previous mode afterwards.
Args:
    model: a torch Module

Definition at line 156 of file evaluator.py.

def inference_context(model):
    """
    A context where the model is temporarily changed to eval mode,
    and restored to previous mode afterwards.
    Args:
        model: a torch Module
    """
    training_mode = model.training
    model.eval()
    yield
    model.train(training_mode)

inference_on_dataset()

fastreid.evaluation.evaluator.inference_on_dataset	(		model,
			data_loader,
			evaluator )

Run model on the data_loader and evaluate the metrics with evaluator.
The model will be used in eval mode.
Args:
    model (nn.Module): a module which accepts an object from
        `data_loader` and returns some outputs. It will be temporarily set to `eval` mode.
        If you wish to evaluate a model in `training` mode instead, you can
        wrap the given model and override its behavior of `.eval()` and `.train()`.
    data_loader: an iterable object with a length.
        The elements it generates will be the inputs to the model.
    evaluator (DatasetEvaluator): the evaluator to run. Use
        :class:`DatasetEvaluators([])` if you only want to benchmark, but
        don't want to do any evaluation.
Returns:
    The return value of `evaluator.evaluate()`

Definition at line 81 of file evaluator.py.

def inference_on_dataset(model, data_loader, evaluator):
    """
    Run model on the data_loader and evaluate the metrics with evaluator.
    The model will be used in eval mode.
    Args:
        model (nn.Module): a module which accepts an object from
            `data_loader` and returns some outputs. It will be temporarily set to `eval` mode.
            If you wish to evaluate a model in `training` mode instead, you can
            wrap the given model and override its behavior of `.eval()` and `.train()`.
        data_loader: an iterable object with a length.
            The elements it generates will be the inputs to the model.
        evaluator (DatasetEvaluator): the evaluator to run. Use
            :class:`DatasetEvaluators([])` if you only want to benchmark, but
            don't want to do any evaluation.
    Returns:
        The return value of `evaluator.evaluate()`
    """
    logger = logging.getLogger(__name__)
    logger.info("Start inference on {} images".format(len(data_loader.dataset)))
 
    total = len(data_loader)  # inference data loader must have a fixed length
    evaluator.reset()
 
    num_warmup = min(5, total - 1)
    start_time = time.perf_counter()
    total_compute_time = 0
    with inference_context(model), torch.no_grad():
        for idx, inputs in enumerate(data_loader):
            if idx == num_warmup:
                start_time = time.perf_counter()
                total_compute_time = 0
 
            start_compute_time = time.perf_counter()
            outputs = model(inputs)
            total_compute_time += time.perf_counter() - start_compute_time
            evaluator.process(inputs, outputs)
 
            idx += 1
            iters_after_start = idx + 1 - num_warmup * int(idx >= num_warmup)
            seconds_per_batch = total_compute_time / iters_after_start
            if idx >= num_warmup * 2 or seconds_per_batch > 30:
                total_seconds_per_img = (time.perf_counter() - start_time) / iters_after_start
                eta = datetime.timedelta(seconds=int(total_seconds_per_img * (total - idx - 1)))
                log_every_n_seconds(
                    logging.INFO,
                    "Inference done {}/{}. {:.4f} s / batch. ETA={}".format(
                        idx + 1, total, seconds_per_batch, str(eta)
                    ),
                    n=30,
                )
 
    # Measure the time only for this worker (before the synchronization barrier)
    total_time = time.perf_counter() - start_time
    total_time_str = str(datetime.timedelta(seconds=total_time))
    # NOTE this format is parsed by grep
    logger.info(
        "Total inference time: {} ({:.6f} s / batch per device)".format(
            total_time_str, total_time / (total - num_warmup)
        )
    )
    total_compute_time_str = str(datetime.timedelta(seconds=int(total_compute_time)))
    logger.info(
        "Total inference pure compute time: {} ({:.6f} s / batch per device)".format(
            total_compute_time_str, total_compute_time / (total - num_warmup)
        )
    )
    results = evaluator.evaluate()
    # An evaluator may return None when not in main process.
    # Replace it by an empty dict instead to make it easier for downstream code to handle
    if results is None:
        results = {}
    return results
 
 
@contextmanager