Inheritance diagram for fastreid.evaluation.evaluator.DatasetEvaluator:
Public Member Functions | |
| reset (self) | |
| preprocess_inputs (self, inputs) | |
| process (self, inputs, outputs) | |
| evaluate (self) | |
Detailed Description
Base class for a dataset evaluator. The function :func:`inference_on_dataset` runs the model over all samples in the dataset, and have a DatasetEvaluator to process the inputs/outputs. This class will accumulate information of the inputs/outputs (by :meth:`process`), and produce evaluation results in the end (by :meth:`evaluate`).
Definition at line 12 of file evaluator.py.
Member Function Documentation
◆ evaluate()
| fastreid.evaluation.evaluator.DatasetEvaluator.evaluate | ( | self | ) |
Evaluate/summarize the performance, after processing all input/output pairs.
Returns:
dict:
A new evaluator class can return a dict of arbitrary format
as long as the user can process the results.
In our train_net.py, we expect the following format:
* key: the name of the task (e.g., bbox)
* value: a dict of {metric name: score}, e.g.: {"AP50": 80}
Reimplemented in fastreid.evaluation.reid_evaluation.ReidEvaluator.
Definition at line 40 of file evaluator.py.
40 def evaluate(self):
41 """
42 Evaluate/summarize the performance, after processing all input/output pairs.
43 Returns:
44 dict:
45 A new evaluator class can return a dict of arbitrary format
46 as long as the user can process the results.
47 In our train_net.py, we expect the following format:
48 * key: the name of the task (e.g., bbox)
49 * value: a dict of {metric name: score}, e.g.: {"AP50": 80}
50 """
51 pass
52
53
54# class DatasetEvaluators(DatasetEvaluator):
55# def __init__(self, evaluators):
56# assert len(evaluators)
57# super().__init__()
58# self._evaluators = evaluators
59#
60# def reset(self):
61# for evaluator in self._evaluators:
62# evaluator.reset()
63#
64# def process(self, input, output):
65# for evaluator in self._evaluators:
66# evaluator.process(input, output)
67#
68# def evaluate(self):
69# results = OrderedDict()
70# for evaluator in self._evaluators:
71# result = evaluator.evaluate()
72# if is_main_process() and result is not None:
73# for k, v in result.items():
74# assert (
75# k not in results
76# ), "Different evaluators produce results with the same key {}".format(k)
77# results[k] = v
78# return results
79
80
◆ preprocess_inputs()
| fastreid.evaluation.evaluator.DatasetEvaluator.preprocess_inputs | ( | self, | |
| inputs ) |
Definition at line 28 of file evaluator.py.
28 def preprocess_inputs(self, inputs):
29 pass
30
◆ process()
| fastreid.evaluation.evaluator.DatasetEvaluator.process | ( | self, | |
| inputs, | |||
| outputs ) |
Process an input/output pair.
Args:
inputs: the inputs that's used to call the model.
outputs: the return value of `model(input)`
Reimplemented in fastreid.evaluation.reid_evaluation.ReidEvaluator.
Definition at line 31 of file evaluator.py.
31 def process(self, inputs, outputs):
32 """
33 Process an input/output pair.
34 Args:
35 inputs: the inputs that's used to call the model.
36 outputs: the return value of `model(input)`
37 """
38 pass
39
◆ reset()
| fastreid.evaluation.evaluator.DatasetEvaluator.reset | ( | self | ) |
Preparation for a new round of evaluation. Should be called before starting a round of evaluation.
Reimplemented in fastreid.evaluation.reid_evaluation.ReidEvaluator.
Definition at line 21 of file evaluator.py.
21 def reset(self):
22 """
23 Preparation for a new round of evaluation.
24 Should be called before starting a round of evaluation.
25 """
26 pass
27
The documentation for this class was generated from the following file:
- smreid/fastreid/evaluation/evaluator.py
