Inheritance diagram for fastreid.utils.file_io.NativePathHandler:
Protected Member Functions | |
| str | _get_local_path (self, str path, **Any kwargs) |
| Union[IO[str], IO[bytes]] | _open (self, str path, str mode="r", int buffering=-1, Optional[str] encoding=None, Optional[str] errors=None, Optional[str] newline=None, bool closefd=True, Optional[Callable] opener=None, **Any kwargs) |
| bool | _copy (self, str src_path, str dst_path, bool overwrite=False, **Any kwargs) |
| bool | _exists (self, str path, **Any kwargs) |
| bool | _isfile (self, str path, **Any kwargs) |
| bool | _isdir (self, str path, **Any kwargs) |
| List[str] | _ls (self, str path, **Any kwargs) |
| None | _mkdirs (self, str path, **Any kwargs) |
| None | _rm (self, str path, **Any kwargs) |
 Protected Member Functions inherited from fastreid.utils.file_io.PathHandler | |
| None | _check_kwargs (self, Dict[str, Any] kwargs) |
| List[str] | _get_supported_prefixes (self) |
Additional Inherited Members | |
| Static Protected Attributes inherited from fastreid.utils.file_io.PathHandler | |
| bool | _strict_kwargs_check = True |
Detailed Description
Handles paths that can be accessed using Python native system calls. This handler uses `open()` and `os.*` calls on the given path.
Definition at line 184 of file file_io.py.
Member Function Documentation
◆ _copy()
|
protected |
Copies a source path to a destination path.
Args:
src_path (str): A URI supported by this PathHandler
dst_path (str): A URI supported by this PathHandler
overwrite (bool): Bool flag for forcing overwrite of existing file
Returns:
status (bool): True on success
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 254 of file file_io.py.
260 ) -> bool:
261 """
262 Copies a source path to a destination path.
263 Args:
264 src_path (str): A URI supported by this PathHandler
265 dst_path (str): A URI supported by this PathHandler
266 overwrite (bool): Bool flag for forcing overwrite of existing file
267 Returns:
268 status (bool): True on success
269 """
270 self._check_kwargs(kwargs)
271
272 if os.path.exists(dst_path) and not overwrite:
273 logger = logging.getLogger(__name__)
274 logger.error("Destination file {} already exists.".format(dst_path))
275 return False
276
277 try:
278 shutil.copyfile(src_path, dst_path)
279 return True
280 except Exception as e:
281 logger = logging.getLogger(__name__)
282 logger.error("Error in file copy - {}".format(str(e)))
283 return False
284
◆ _exists()
|
protected |
Checks if there is a resource at the given URI.
Args:
path (str): A URI supported by this PathHandler
Returns:
bool: true if the path exists
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 285 of file file_io.py.
285 def _exists(self, path: str, **kwargs: Any) -> bool:
286 self._check_kwargs(kwargs)
287 return os.path.exists(path)
288
◆ _get_local_path()
|
protected |
Get a filepath which is compatible with native Python I/O such as `open`
and `os.path`.
If URI points to a remote resource, this function may download and cache
the resource to local disk. In this case, this function is meant to be
used with read-only resources.
Args:
path (str): A URI supported by this PathHandler
Returns:
local_path (str): a file path which exists on the local file system
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 190 of file file_io.py.
190 def _get_local_path(self, path: str, **kwargs: Any) -> str:
191 self._check_kwargs(kwargs)
192 return path
193
◆ _isdir()
|
protected |
Checks if the resource at the given URI is a directory.
Args:
path (str): A URI supported by this PathHandler
Returns:
bool: true if the path is a directory
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 293 of file file_io.py.
293 def _isdir(self, path: str, **kwargs: Any) -> bool:
294 self._check_kwargs(kwargs)
295 return os.path.isdir(path)
296
◆ _isfile()
|
protected |
Checks if the resource at the given URI is a file.
Args:
path (str): A URI supported by this PathHandler
Returns:
bool: true if the path is a file
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 289 of file file_io.py.
289 def _isfile(self, path: str, **kwargs: Any) -> bool:
290 self._check_kwargs(kwargs)
291 return os.path.isfile(path)
292
◆ _ls()
|
protected |
List the contents of the directory at the provided URI.
Args:
path (str): A URI supported by this PathHandler
Returns:
List[str]: list of contents in given path
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 297 of file file_io.py.
297 def _ls(self, path: str, **kwargs: Any) -> List[str]:
298 self._check_kwargs(kwargs)
299 return os.listdir(path)
300
◆ _mkdirs()
|
protected |
Recursive directory creation function. Like mkdir(), but makes all
intermediate-level directories needed to contain the leaf directory.
Similar to the native `os.makedirs`.
Args:
path (str): A URI supported by this PathHandler
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 301 of file file_io.py.
301 def _mkdirs(self, path: str, **kwargs: Any) -> None:
302 self._check_kwargs(kwargs)
303 try:
304 os.makedirs(path, exist_ok=True)
305 except OSError as e:
306 # EEXIST it can still happen if multiple processes are creating the dir
307 if e.errno != errno.EEXIST:
308 raise
309
◆ _open()
|
protected |
Open a path.
Args:
path (str): A URI supported by this PathHandler
mode (str): Specifies the mode in which the file is opened. It defaults
to 'r'.
buffering (int): An optional integer used to set the buffering policy.
Pass 0 to switch buffering off and an integer >= 1 to indicate the
size in bytes of a fixed-size chunk buffer. When no buffering
argument is given, the default buffering policy works as follows:
* Binary files are buffered in fixed-size chunks; the size of
the buffer is chosen using a heuristic trying to determine the
underlying device’s “block size” and falling back on
io.DEFAULT_BUFFER_SIZE. On many systems, the buffer will
typically be 4096 or 8192 bytes long.
encoding (Optional[str]): the name of the encoding used to decode or
encode the file. This should only be used in text mode.
errors (Optional[str]): an optional string that specifies how encoding
and decoding errors are to be handled. This cannot be used in binary
mode.
newline (Optional[str]): controls how universal newlines mode works
(it only applies to text mode). It can be None, '', '\n', '\r',
and '\r\n'.
closefd (bool): If closefd is False and a file descriptor rather than
a filename was given, the underlying file descriptor will be kept
open when the file is closed. If a filename is given closefd must
be True (the default) otherwise an error will be raised.
opener (Optional[Callable]): A custom opener can be used by passing
a callable as opener. The underlying file descriptor for the file
object is then obtained by calling opener with (file, flags).
opener must return an open file descriptor (passing os.open as opener
results in functionality similar to passing None).
See https://docs.python.org/3/library/functions.html#open for details.
Returns:
file: a file-like object.
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 194 of file file_io.py.
205 ) -> Union[IO[str], IO[bytes]]:
206 """
207 Open a path.
208 Args:
209 path (str): A URI supported by this PathHandler
210 mode (str): Specifies the mode in which the file is opened. It defaults
211 to 'r'.
212 buffering (int): An optional integer used to set the buffering policy.
213 Pass 0 to switch buffering off and an integer >= 1 to indicate the
214 size in bytes of a fixed-size chunk buffer. When no buffering
215 argument is given, the default buffering policy works as follows:
216 * Binary files are buffered in fixed-size chunks; the size of
217 the buffer is chosen using a heuristic trying to determine the
218 underlying device’s “block size” and falling back on
219 io.DEFAULT_BUFFER_SIZE. On many systems, the buffer will
220 typically be 4096 or 8192 bytes long.
221 encoding (Optional[str]): the name of the encoding used to decode or
222 encode the file. This should only be used in text mode.
223 errors (Optional[str]): an optional string that specifies how encoding
224 and decoding errors are to be handled. This cannot be used in binary
225 mode.
226 newline (Optional[str]): controls how universal newlines mode works
227 (it only applies to text mode). It can be None, '', '\n', '\r',
228 and '\r\n'.
229 closefd (bool): If closefd is False and a file descriptor rather than
230 a filename was given, the underlying file descriptor will be kept
231 open when the file is closed. If a filename is given closefd must
232 be True (the default) otherwise an error will be raised.
233 opener (Optional[Callable]): A custom opener can be used by passing
234 a callable as opener. The underlying file descriptor for the file
235 object is then obtained by calling opener with (file, flags).
236 opener must return an open file descriptor (passing os.open as opener
237 results in functionality similar to passing None).
238 See https://docs.python.org/3/library/functions.html#open for details.
239 Returns:
240 file: a file-like object.
241 """
242 self._check_kwargs(kwargs)
243 return open( # type: ignore
244 path,
245 mode,
246 buffering=buffering,
247 encoding=encoding,
248 errors=errors,
249 newline=newline,
250 closefd=closefd,
251 opener=opener,
252 )
253
◆ _rm()
|
protected |
Remove the file (not directory) at the provided URI.
Args:
path (str): A URI supported by this PathHandler
Reimplemented from fastreid.utils.file_io.PathHandler.
Definition at line 310 of file file_io.py.
310 def _rm(self, path: str, **kwargs: Any) -> None:
311 self._check_kwargs(kwargs)
312 os.remove(path)
313
314
The documentation for this class was generated from the following file:
- smreid/fastreid/utils/file_io.py
