proxystore.store.config

proxystore/store/config.py

Store configuration model.

ConnectorConfig

Bases: BaseModel

Connector configuration.

Example

from proxystore.connectors.redis import RedisConnector
from proxystore.store.config import ConnectorConfig

config = ConnectorConfig(
    kind='redis',
    options={'hostname': 'localhost', 'port': 1234},
)

connector = config.get_connector()
assert isinstance(connector, RedisConnector)

Attributes:

• kind (str) –

  Fully-qualified path used to import a Connector type or a shortened name for a builtin Connector type. E.g., 'file' or 'FileConnector' are valid shortcuts for 'proxystore.connectors.file.FileConnector'.

• options (Dict[str, Any]) –

  Dictionary of keyword arguments to pass to the Connector constructor.

get_connector_type

get_connector_type() -> type[Connector[Any]]

Resolve the class type for the specified connector.

This method works by first attempting to import the class by treating kind as a fully-qualified path. For example, if kind='proxystore.connectors.local.LocalConnector', the LocalConnector is imported from proxystore.connectors.local.

If the import fails, kind will be checked against a list of known (i.e., builtin) Connector types. kind will be psuedo-fuzzy matched against the class names of the known Connector types. For example, kind='local' and kind='LocalConnector' will both match to 'proxystore.connectors.local.LocalConnector'.

Returns:

• type[Connector[Any]] –

  Connector type.

Raises:

• ValueError –

  If a Connector named kind failed to import or if kind does not match a builtin connector.

Source code in proxystore/store/config.py

def get_connector_type(self) -> type[Connector[Any]]:
    """Resolve the class type for the specified connector.

    This method works by first attempting to import the class
    by treating `kind` as a fully-qualified path. For example,
    if `kind='proxystore.connectors.local.LocalConnector'`, the
    [`LocalConnector`][proxystore.connectors.local.LocalConnector] is
    imported from
    [`proxystore.connectors.local`][proxystore.connectors.local].

    If the import fails, `kind` will be checked against a list of known
    (i.e., builtin)
    [`Connector`][proxystore.connectors.protocols.Connector] types.
    `kind` will be psuedo-fuzzy matched against the class names of the
    known [`Connector`][proxystore.connectors.protocols.Connector] types.
    For example, `kind='local'` and `kind='LocalConnector'` will both
    match to `'proxystore.connectors.local.LocalConnector'`.

    Returns:
        [`Connector`][proxystore.connectors.protocols.Connector] type.

    Raises:
        ValueError: If a
            [`Connector`][proxystore.connectors.protocols.Connector]
            named `kind` failed to import or if `kind` does not match
            a builtin connector.
    """
    try:
        return import_from_path(self.kind)
    except ImportError as e:
        for path in _KNOWN_CONNECTORS:
            _, name = path.rsplit('.', 1)
            name = name.lower()
            choices = [name, name.replace('connector', '')]
            if self.kind.lower() in choices:
                return import_from_path(path)
        raise ValueError(f'Unknown connector type "{self.kind}".') from e

get_connector

get_connector() -> Connector[Any]

Get the connector specified by the configuration.

Returns:

• Connector[Any] –

  A Connector instance.

Source code in proxystore/store/config.py

def get_connector(self) -> Connector[Any]:
    """Get the connector specified by the configuration.

    Returns:
        A [`Connector`][proxystore.connectors.protocols.Connector] \
        instance.
    """
    connector_type = self.get_connector_type()
    return connector_type(**self.options)

StoreConfig

Bases: BaseModel

Store configuration.

Tip

See the Store parameters for more information about each configuration option.

Attributes:

• name (str) –

  Store name.

• connector (ConnectorConfig) –

  Connector configuration.

• serializer (Optional[SerializerT]) –

  Optional serializer.

• deserializer (Optional[DeserializerT]) –

  Optional deserializer.

• cache_size (int) –

  Cache size.

• metrics (bool) –

  Enable recording operation metrics.

• populate_target (bool) –

  Set the default value for the populate_target parameter of proxy methods.

• auto_register (bool) –

  Auto-register the store.

from_toml classmethod

from_toml(filepath: str | Path) -> Self

Create a configuration file from a TOML file.

Example

See write_toml().

Parameters:

• filepath (str | Path) –

  Path to TOML file to load.

Source code in proxystore/store/config.py

@classmethod
def from_toml(cls, filepath: str | pathlib.Path) -> Self:
    """Create a configuration file from a TOML file.

    Example:
        See
        [`write_toml()`][proxystore.store.config.StoreConfig.write_toml].

    Args:
        filepath: Path to TOML file to load.
    """
    with open(filepath, 'rb') as f:
        return load(cls, f)

write_toml

write_toml(filepath: str | Path) -> None

Write a configuration to a TOML file.

Example

from proxystore.store.config import ConnectorConfig
from proxystore.store.config import StoreConfig

config = StoreConfig(
    name='example',
    connector=ConnectorConfig(
        kind='file',
        options={'store_dir': '/tmp/proxystore-cache'},
    ),
)

config.write_toml('config.toml')

The resulting TOML file contains the full configuration, including default options, and can be loaded again using StoreConfig.from_toml('config.toml').

config.toml

name = "example"
cache_size = 16
metrics = false
populate_target = true
auto_register = false

[connector]
kind = "file"

[connector.options]
store_dir = "/tmp/proxystore-cache"

Parameters:

• filepath (str | Path) –

  Path to TOML file to write.

Source code in proxystore/store/config.py

def write_toml(self, filepath: str | pathlib.Path) -> None:
    """Write a configuration to a TOML file.

    Example:
        ```python
        from proxystore.store.config import ConnectorConfig
        from proxystore.store.config import StoreConfig

        config = StoreConfig(
            name='example',
            connector=ConnectorConfig(
                kind='file',
                options={'store_dir': '/tmp/proxystore-cache'},
            ),
        )

        config.write_toml('config.toml')
        ```
        The resulting TOML file contains the full configuration,
        including default options, and can be loaded again
        using `#!python StoreConfig.from_toml('config.toml')`.
        ```toml title="config.toml"
        name = "example"
        cache_size = 16
        metrics = false
        populate_target = true
        auto_register = false

        [connector]
        kind = "file"

        [connector.options]
        store_dir = "/tmp/proxystore-cache"
        ```

    Args:
        filepath: Path to TOML file to write.
    """
    filepath = pathlib.Path(filepath)
    filepath.parent.mkdir(parents=True, exist_ok=True)
    with open(filepath, 'wb') as f:
        dump(self, f)