proxystore.endpoint.config

proxystore/endpoint/config.py

Endpoint configuration.

MAX_OBJECT_SIZE_DEFAULT module-attribute

MAX_OBJECT_SIZE_DEFAULT = 100000000

Default maximum endpoint object size in bytes.

DEFAULT_DATABASE_PATH module-attribute

DEFAULT_DATABASE_PATH = 'blobs.db'

Default path of the SQLite database, relative to the endpoint directory.

CONFIG_VERSION module-attribute

CONFIG_VERSION = 1

Format version of the endpoint configuration file.

EndpointStorageConfig

Bases: BaseModel

Endpoint data storage configuration.

Attributes:

• backend (Literal['memory', 'sqlite']) –

  Storage backend of the endpoint. "memory" stores objects in memory so objects are lost when the endpoint stops. "sqlite" stores objects in a SQLite database.

• database_path (str | None) –

  Path of the SQLite database. Only valid with the "sqlite" backend, and defaults to DEFAULT_DATABASE_PATH. A relative path is relative to the endpoint directory and an absolute path can be used to store the database elsewhere (e.g., on a larger file system). ~ is expanded to the user's home directory.

• max_storage_size (int) –

  Maximum total size in bytes of the objects stored in memory. Only valid with the "memory" backend. A request to store an object which does not fit fails with a StorageFullError, and stored objects are never removed to make room. If 0, there is no limit. A string with units is also accepted (e.g., "8 GiB", see readable_to_bytes()).

• object_ttl (float) –

  Seconds after an object is stored that it expires and is removed, for both backends. Storing an object again with the same key restarts its time. When the endpoint starts with a SQLite database, objects without an expiry time (e.g., stored while object_ttl was 0) expire object_ttl seconds later. If 0, objects do not expire.

Raises:

• ValueError –

  If database_path is set with the "memory" backend, if max_storage_size is set with the "sqlite" backend, or if max_storage_size or object_ttl is negative.

storage_size_limit property

storage_size_limit: int | None

Maximum storage size in bytes or None if there is no limit.

ttl property

ttl: float | None

Seconds until objects expire or None if they do not expire.

EndpointP2PConfig

Bases: BaseModel

Endpoint peer-to-peer configuration.

Attributes:

• enabled (bool) –

  Enable communication with peer endpoints. Only endpoints in the allowlist of peers (peers.toml in the endpoint directory) can communicate with this endpoint.

• relays (Literal['n0', 'none'] | list[str]) –

  Relay servers used to establish connections with peers and to relay traffic when a direct connection is not possible. "n0" uses the public relays operated by n0 (the developers of iroh), "none" disables relays, and a list of URLs uses self-hosted iroh-relay servers.

• discovery (Literal['n0', 'none']) –

  Service used to publish the addresses of this endpoint and find the addresses of peers. "n0" uses the public DNS discovery service operated by n0. "none" disables discovery so peers can only be reached at their cached addresses (see PeerAddrCache) or by connecting to this endpoint first.

• request_timeout (float) –

  Seconds a request to or from a peer can go without sending or receiving any data before it is abandoned. A request to a peer includes the time the peer spends handling it, but a request from a peer does not include the time this endpoint spends handling it. If 0, requests have no timeout.

Raises:

• ValueError –

  If relays is an empty list or contains an invalid URL, or if request_timeout is negative.

EndpointConfig

Bases: VersionedFile

Endpoint configuration.

Attributes:

• version (int) –

  Format version of the configuration.

• name (str) –

  Endpoint name. Must match the name of the endpoint directory.

• id (EndpointId) –

  Endpoint ID. This is the public key of the endpoint's secret key which is stored separately in the endpoint directory.

• host (str) –

  Address clients use to connect to the endpoint. "ip" or "fqdn" (case-insensitive) use the IP address or fully-qualified domain name of the host, determined each time the endpoint starts. Any other value is used as a static address (e.g., "127.0.0.1").

• port (int) –

  Port endpoint is running on.

• tls (bool) –

  Encrypt connections between clients and the endpoint with TLS. The endpoint generates a self-signed certificate each time it starts, and clients only trust that certificate. On by default.

• max_object_size (int) –

  Maximum size in bytes of an object that clients or peers can set on the endpoint. If 0, there is no limit. A string with units is also accepted (e.g., "100 MB" or "1 GiB", see readable_to_bytes()).

• p2p (EndpointP2PConfig) –

  Peer-to-peer configuration.

• storage (EndpointStorageConfig) –

  Storage configuration.

Raises:

• ValueError –

  If the name does not contain only alphanumeric, dash, or underscore characters, if the ID cannot be parsed, if the port is not in the range [1, 65535], if the host is empty, if the version is not supported, if the maximum object size is negative, if the maximum storage size is smaller than the maximum object size (or the object size has no limit), or if there are unknown fields.

object_size_limit property

object_size_limit: int | None

Maximum object size in bytes or None if there is no limit.

resolve_host

resolve_host(host: str) -> str

Resolve the address clients use to connect to the endpoint.

Parameters:

• host (str) –

  "ip", "fqdn", or a static address (see EndpointConfig.host).

Returns:

• str –

  The IP address or fully-qualified domain name of this host, or the static address.

Source code in proxystore/endpoint/config.py

def resolve_host(host: str) -> str:
    """Resolve the address clients use to connect to the endpoint.

    Args:
        host: `"ip"`, `"fqdn"`, or a static address (see
            [`EndpointConfig.host`][proxystore.endpoint.config.EndpointConfig]).

    Returns:
        The IP address or fully-qualified domain name of this host, or the \
        static address.
    """
    if host == 'fqdn':
        return socket.getfqdn()
    if host == 'ip':
        return socket.gethostbyname(hostname())
    return host

check_name

check_name(name: str, kind: str) -> str

Check that a name only contains alphanumeric, dash, or underscore chars.

Parameters:

• name (str) –

  Name to check.

• kind (str) –

  Kind of name for the error message (e.g., "Peer").

Returns:

• str –

  The name.

Raises:

• ValueError –

  If the name is empty or contains other characters.

Source code in proxystore/endpoint/config.py

def check_name(name: str, kind: str) -> str:
    """Check that a name only contains alphanumeric, dash, or underscore chars.

    Args:
        name: Name to check.
        kind: Kind of name for the error message (e.g., `"Peer"`).

    Returns:
        The name.

    Raises:
        ValueError: If the name is empty or contains other characters.
    """
    if _NAME_PATTERN.fullmatch(name) is None:
        raise ValueError(
            f'{kind} names must only contain alphanumeric characters, '
            f'dashes, and underscores. Got {name!r}.',
        )
    return name