Inheritance diagram for fastreid.utils.events.JSONWriter:
Public Member Functions | |
| __init__ (self, json_file, window_size=20) | |
| write (self) | |
| close (self) | |
Protected Attributes | |
| _file_handle | |
| _window_size | |
| _last_write | |
Detailed Description
Write scalars to a json file.
It saves scalars as one json per line (instead of a big json) for easy parsing.
Examples parsing such a json file:
::
$ cat metrics.json | jq -s '.[0:2]'
[
{
"data_time": 0.008433341979980469,
"iteration": 20,
"loss": 1.9228371381759644,
"loss_box_reg": 0.050025828182697296,
"loss_classifier": 0.5316952466964722,
"loss_mask": 0.7236229181289673,
"loss_rpn_box": 0.0856662318110466,
"loss_rpn_cls": 0.48198649287223816,
"lr": 0.007173333333333333,
"time": 0.25401854515075684
},
{
"data_time": 0.007216215133666992,
"iteration": 40,
"loss": 1.282649278640747,
"loss_box_reg": 0.06222952902317047,
"loss_classifier": 0.30682939291000366,
"loss_mask": 0.6970193982124329,
"loss_rpn_box": 0.038663312792778015,
"loss_rpn_cls": 0.1471673548221588,
"lr": 0.007706666666666667,
"time": 0.2490077018737793
}
]
$ cat metrics.json | jq '.loss_mask'
0.7126231789588928
0.689423680305481
0.6776131987571716
...
Constructor & Destructor Documentation
◆ __init__()
| fastreid.utils.events.JSONWriter.__init__ | ( | self, | |
| json_file, | |||
| window_size = 20 ) |
Args:
json_file (str): path to the json file. New data will be appended if the file exists.
window_size (int): the window size of median smoothing for the scalars whose
`smoothing_hint` are True.
Definition at line 88 of file events.py.
88 def __init__(self, json_file, window_size=20):
89 """
90 Args:
91 json_file (str): path to the json file. New data will be appended if the file exists.
92 window_size (int): the window size of median smoothing for the scalars whose
93 `smoothing_hint` are True.
94 """
95 self._file_handle = PathManager.open(json_file, "a")
96 self._window_size = window_size
97 self._last_write = -1
98
Member Function Documentation
◆ close()
| fastreid.utils.events.JSONWriter.close | ( | self | ) |
Reimplemented from fastreid.utils.events.EventWriter.
Definition at line 120 of file events.py.
120 def close(self):
121 self._file_handle.close()
122
123
◆ write()
| fastreid.utils.events.JSONWriter.write | ( | self | ) |
Reimplemented from fastreid.utils.events.EventWriter.
Definition at line 99 of file events.py.
99 def write(self):
100 storage = get_event_storage()
101 to_save = defaultdict(dict)
102
103 for k, (v, iter) in storage.latest_with_smoothing_hint(self._window_size).items():
104 # keep scalars that have not been written
105 if iter <= self._last_write:
106 continue
107 to_save[iter][k] = v
108 all_iters = sorted(to_save.keys())
109 self._last_write = max(all_iters)
110
111 for itr, scalars_per_iter in to_save.items():
112 scalars_per_iter["iteration"] = itr
113 self._file_handle.write(json.dumps(scalars_per_iter, sort_keys=True) + "\n")
114 self._file_handle.flush()
115 try:
116 os.fsync(self._file_handle.fileno())
117 except AttributeError:
118 pass
119
Member Data Documentation
◆ _file_handle
◆ _last_write
◆ _window_size
The documentation for this class was generated from the following file:
- smreid/fastreid/utils/events.py
