fastreid.utils.file_io.PathManager Class Reference

Static Public Member Functions
Union[IO[str], IO[bytes]]	open (str path, str mode="r", int buffering=-1, **Any kwargs)
bool	copy (str src_path, str dst_path, bool overwrite=False, **Any kwargs)
str	get_local_path (str path, **Any kwargs)
bool	exists (str path, **Any kwargs)
bool	isfile (str path, **Any kwargs)
bool	isdir (str path, **Any kwargs)
List[str]	ls (str path, **Any kwargs)
None	mkdirs (str path, **Any kwargs)
None	rm (str path, **Any kwargs)
None	register_handler (PathHandler handler)
None	set_strict_kwargs_checking (bool enable)

Static Protected Attributes
MutableMapping	_PATH_HANDLERS = OrderedDict()
	_NATIVE_PATH_HANDLER = NativePathHandler()

Detailed Description

A class for users to open generic paths or translate generic paths to file names.

Definition at line 315 of file file_io.py.

Member Function Documentation

copy()

bool fastreid.utils.file_io.PathManager.copy ( str src_path, str dst_path, bool overwrite = False, **Any kwargs )

static

Copies a source path to a destination path.
Args:
    src_path (str): A URI supported by this PathHandler
    dst_path (str): A URI supported by this PathHandler
    overwrite (bool): Bool flag for forcing overwrite of existing file
Returns:
    status (bool): True on success

Definition at line 361 of file file_io.py.

    ) -> bool:
        """
        Copies a source path to a destination path.
        Args:
            src_path (str): A URI supported by this PathHandler
            dst_path (str): A URI supported by this PathHandler
            overwrite (bool): Bool flag for forcing overwrite of existing file
        Returns:
            status (bool): True on success
        """
 
        # Copying across handlers is not supported.
        assert PathManager.__get_path_handler(  # type: ignore
            src_path
        ) == PathManager.__get_path_handler(dst_path)
        return PathManager.__get_path_handler(src_path)._copy(
            src_path, dst_path, overwrite, **kwargs
        )
 

exists()

bool fastreid.utils.file_io.PathManager.exists ( str path, **Any kwargs )

static

Checks if there is a resource at the given URI.
Args:
    path (str): A URI supported by this PathHandler
Returns:
    bool: true if the path exists

Definition at line 399 of file file_io.py.

    def exists(path: str, **kwargs: Any) -> bool:
        """
        Checks if there is a resource at the given URI.
        Args:
            path (str): A URI supported by this PathHandler
        Returns:
            bool: true if the path exists
        """
        return PathManager.__get_path_handler(path)._exists(  # type: ignore
            path, **kwargs
        )
 

get_local_path()

str fastreid.utils.file_io.PathManager.get_local_path ( str path, **Any kwargs )

static

Get a filepath which is compatible with native Python I/O such as `open`
and `os.path`.
If URI points to a remote resource, this function may download and cache
the resource to local disk.
Args:
    path (str): A URI supported by this PathHandler
Returns:
    local_path (str): a file path which exists on the local file system

Definition at line 383 of file file_io.py.

    def get_local_path(path: str, **kwargs: Any) -> str:
        """
        Get a filepath which is compatible with native Python I/O such as `open`
        and `os.path`.
        If URI points to a remote resource, this function may download and cache
        the resource to local disk.
        Args:
            path (str): A URI supported by this PathHandler
        Returns:
            local_path (str): a file path which exists on the local file system
        """
        return PathManager.__get_path_handler(  # type: ignore
            path
        )._get_local_path(path, **kwargs)
 

isdir()

bool fastreid.utils.file_io.PathManager.isdir ( str path, **Any kwargs )

static

Checks if the resource at the given URI is a directory.
Args:
    path (str): A URI supported by this PathHandler
Returns:
    bool: true if the path is a directory

Definition at line 425 of file file_io.py.

    def isdir(path: str, **kwargs: Any) -> bool:
        """
        Checks if the resource at the given URI is a directory.
        Args:
            path (str): A URI supported by this PathHandler
        Returns:
            bool: true if the path is a directory
        """
        return PathManager.__get_path_handler(path)._isdir(  # type: ignore
            path, **kwargs
        )
 

isfile()

bool fastreid.utils.file_io.PathManager.isfile ( str path, **Any kwargs )

static

Checks if there the resource at the given URI is a file.
Args:
    path (str): A URI supported by this PathHandler
Returns:
    bool: true if the path is a file

Definition at line 412 of file file_io.py.

    def isfile(path: str, **kwargs: Any) -> bool:
        """
        Checks if there the resource at the given URI is a file.
        Args:
            path (str): A URI supported by this PathHandler
        Returns:
            bool: true if the path is a file
        """
        return PathManager.__get_path_handler(path)._isfile(  # type: ignore
            path, **kwargs
        )
 

ls()

List[str] fastreid.utils.file_io.PathManager.ls ( str path, **Any kwargs )

static

List the contents of the directory at the provided URI.
Args:
    path (str): A URI supported by this PathHandler
Returns:
    List[str]: list of contents in given path

Definition at line 438 of file file_io.py.

    def ls(path: str, **kwargs: Any) -> List[str]:
        """
        List the contents of the directory at the provided URI.
        Args:
            path (str): A URI supported by this PathHandler
        Returns:
            List[str]: list of contents in given path
        """
        return PathManager.__get_path_handler(path)._ls(  # type: ignore
            path, **kwargs
        )
 

mkdirs()

None fastreid.utils.file_io.PathManager.mkdirs ( str path, **Any kwargs )

static

Recursive directory creation function. Like mkdir(), but makes all
intermediate-level directories needed to contain the leaf directory.
Similar to the native `os.makedirs`.
Args:
    path (str): A URI supported by this PathHandler

Definition at line 451 of file file_io.py.

    def mkdirs(path: str, **kwargs: Any) -> None:
        """
        Recursive directory creation function. Like mkdir(), but makes all
        intermediate-level directories needed to contain the leaf directory.
        Similar to the native `os.makedirs`.
        Args:
            path (str): A URI supported by this PathHandler
        """
        return PathManager.__get_path_handler(path)._mkdirs(  # type: ignore
            path, **kwargs
        )
 

open()

Union[IO[str], IO[bytes]] fastreid.utils.file_io.PathManager.open ( str path, str mode = "r", int buffering = -1, **Any kwargs )

static

Open a stream to a URI, similar to the built-in `open`.
Args:
    path (str): A URI supported by this PathHandler
    mode (str): Specifies the mode in which the file is opened. It defaults
        to 'r'.
    buffering (int): An optional integer used to set the buffering policy.
        Pass 0 to switch buffering off and an integer >= 1 to indicate the
        size in bytes of a fixed-size chunk buffer. When no buffering
        argument is given, the default buffering policy depends on the
        underlying I/O implementation.
Returns:
    file: a file-like object.

Definition at line 339 of file file_io.py.

    ) -> Union[IO[str], IO[bytes]]:
        """
        Open a stream to a URI, similar to the built-in `open`.
        Args:
            path (str): A URI supported by this PathHandler
            mode (str): Specifies the mode in which the file is opened. It defaults
                to 'r'.
            buffering (int): An optional integer used to set the buffering policy.
                Pass 0 to switch buffering off and an integer >= 1 to indicate the
                size in bytes of a fixed-size chunk buffer. When no buffering
                argument is given, the default buffering policy depends on the
                underlying I/O implementation.
        Returns:
            file: a file-like object.
        """
        return PathManager.__get_path_handler(path)._open(  # type: ignore
            path, mode, buffering=buffering, **kwargs
        )
 

register_handler()

None fastreid.utils.file_io.PathManager.register_handler ( PathHandler handler )

static

Register a path handler associated with `handler._get_supported_prefixes`
URI prefixes.
Args:
    handler (PathHandler)

Definition at line 475 of file file_io.py.

    def register_handler(handler: PathHandler) -> None:
        """
        Register a path handler associated with `handler._get_supported_prefixes`
        URI prefixes.
        Args:
            handler (PathHandler)
        """
        assert isinstance(handler, PathHandler), handler
        for prefix in handler._get_supported_prefixes():
            assert prefix not in PathManager._PATH_HANDLERS
            PathManager._PATH_HANDLERS[prefix] = handler
 
        # Sort path handlers in reverse order so longer prefixes take priority,
        # eg: http://foo/bar before http://foo
        PathManager._PATH_HANDLERS = OrderedDict(
            sorted(
                PathManager._PATH_HANDLERS.items(),
                key=lambda t: t[0],
                reverse=True,
            )
        )
 

rm()

None fastreid.utils.file_io.PathManager.rm ( str path, **Any kwargs )

static

Remove the file (not directory) at the provided URI.
Args:
    path (str): A URI supported by this PathHandler

Definition at line 464 of file file_io.py.

    def rm(path: str, **kwargs: Any) -> None:
        """
        Remove the file (not directory) at the provided URI.
        Args:
            path (str): A URI supported by this PathHandler
        """
        return PathManager.__get_path_handler(path)._rm(  # type: ignore
            path, **kwargs
        )
 

set_strict_kwargs_checking()

None fastreid.utils.file_io.PathManager.set_strict_kwargs_checking ( bool enable )

static

Toggles strict kwargs checking. If enabled, a ValueError is thrown if any
unused parameters are passed to a PathHandler function. If disabled, only
a warning is given.
With a centralized file API, there's a tradeoff of convenience and
correctness delegating arguments to the proper I/O layers. An underlying
`PathHandler` may support custom arguments which should not be statically
exposed on the `PathManager` function. For example, a custom `HTTPURLHandler`
may want to expose a `cache_timeout` argument for `open()` which specifies
how old a locally cached resource can be before it's refetched from the
remote server. This argument would not make sense for a `NativePathHandler`.
If strict kwargs checking is disabled, `cache_timeout` can be passed to
`PathManager.open` which will forward the arguments to the underlying
handler. By default, checking is enabled since it is innately unsafe:
multiple `PathHandler`s could reuse arguments with different semantic
meanings or types.
Args:
    enable (bool)

Definition at line 498 of file file_io.py.

    def set_strict_kwargs_checking(enable: bool) -> None:
        """
        Toggles strict kwargs checking. If enabled, a ValueError is thrown if any
        unused parameters are passed to a PathHandler function. If disabled, only
        a warning is given.
        With a centralized file API, there's a tradeoff of convenience and
        correctness delegating arguments to the proper I/O layers. An underlying
        `PathHandler` may support custom arguments which should not be statically
        exposed on the `PathManager` function. For example, a custom `HTTPURLHandler`
        may want to expose a `cache_timeout` argument for `open()` which specifies
        how old a locally cached resource can be before it's refetched from the
        remote server. This argument would not make sense for a `NativePathHandler`.
        If strict kwargs checking is disabled, `cache_timeout` can be passed to
        `PathManager.open` which will forward the arguments to the underlying
        handler. By default, checking is enabled since it is innately unsafe:
        multiple `PathHandler`s could reuse arguments with different semantic
        meanings or types.
        Args:
            enable (bool)
        """
        PathManager._NATIVE_PATH_HANDLER._strict_kwargs_check = enable
        for handler in PathManager._PATH_HANDLERS.values():
            handler._strict_kwargs_check = enable

Member Data Documentation

_NATIVE_PATH_HANDLER

fastreid.utils.file_io.PathManager._NATIVE_PATH_HANDLER = NativePathHandler()

staticprotected

Definition at line 321 of file file_io.py.

_PATH_HANDLERS

MutableMapping fastreid.utils.file_io.PathManager._PATH_HANDLERS = OrderedDict()

staticprotected

Definition at line 320 of file file_io.py.

---

The documentation for this class was generated from the following file:

• smreid/fastreid/utils/file_io.py