proxystore.proxy¶

proxystore/proxy/__init__.py

Proxy implementation and helpers.

ProxyOr `module-attribute` ¶

ProxyOr = TypeAliasType(
    "ProxyOr", Union[Proxy[T], T], type_params=(T)
)

Type alias for a union of a type T or a Proxy[T].

This type alias is useful for typing functions that operate on or return mixed types involving proxies.

Example

from typing import TypeVar
from proxystore.proxy import Proxy, ProxyOr, extract

T = TypeVar('T')

def extract_if_proxy(value: ProxyOr[T]) -> T:
    return extract(value) if isinstance(value, Proxy) else value

Proxy ¶

Proxy(
    factory: FactoryType[T],
    *,
    cache_defaults: bool = False,
    target: T | None = None
)

Bases: as_metaclass(ProxyMetaType), Generic[T]

Lazy object proxy.

An extension of the Proxy from lazy-object-proxy with modified attribute lookup and pickling behavior.

An object proxy acts as a thin wrapper around a Python object, i.e. the proxy behaves identically to the underlying object. The proxy is initialized with a callable factory object. The factory returns the underlying object when called, i.e. resolves the proxy. This means a proxy performs lazy/just-in-time resolution, i.e., the proxy does not call the factory until the first access to the proxy.

from proxystore.proxy import Proxy

def factory() -> list[int]:
    return [1, 2, 3]

proxy = Proxy(factory)
assert isinstance(proxy, list)
assert proxy == [1, 2, 3]

Note

The factory, by default, is only ever called once during the lifetime of a proxy instance.

Note

When a proxy instance is pickled, only the factory is pickled, not the wrapped object. Thus, proxy instances can be pickled and passed around cheaply, and once the proxy is unpickled and used, the factory will be called again to resolve the object.

Tip

Common data structures (e.g., dict or set) and operations (e.g., isinstance) will resolve an unresolved proxy. This can result in unintentional performance degradation for expensive factories, such as those that require significant I/O or produce target objects that require a lot of memory. The target and cache_defaults parameters of Proxy can prevent these unintenional proxy resolves by caching the __class__ and __hash__ values of the target object in the proxy.

from proxystore.proxy import Proxy
from proxystore.proxy import is_resolved

proxy = Proxy(lambda: 'value')
assert not is_resolved(proxy)

assert isinstance(proxy, str)  # (1)!
assert is_resolved(proxy)

value = 'value'
proxy = Proxy(lambda: value, cache_defaults=True, target=value)  # (2)!
assert not is_resolved(proxy)

assert isinstance(proxy, str)  # (3)!
assert not is_resolved(proxy)

Using isinstance calls __class__ on the target object which requires the proxy to be resolved. In many cases, it may be desirable to check the type of a proxy's target object without incurring the cost of resolving the target.
If the target is available when constructing the proxy, the proxy can precompute and cache the __class__ and __hash__ values of the target.
Using isinstance no longer requires the proxy to be resolved, instead using the precomputed value.

Warning

A proxy of a singleton type (e.g., True, False, and None) will not behave exactly as a singleton type would. This is because the proxy itself is not a singleton.

>>> from proxystore.proxy import Proxy
>>> p = Proxy(lambda: True)
>>> p == True
True
>>> p is True
False

Warning

Python bindings to other languages (e.g., C, C++) may throw type errors when receiving a Proxy instance. Casting the proxy or extracting the target object may be needed.

>>> import io
>>> from proxystore.proxy import Proxy
>>> s = 'mystring'
>>> p = Proxy(lambda: s)
>>> io.StringIO(p)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: initial_value must be str or None, not Proxy
>>> io.StringIO(str(p))  # succeeds

Attributes:

__proxy_factory__ (FactoryType[T]) –

Factory function which resolves to the target object.
__proxy_target__ (T) –

The target object once resolved.
__proxy_resolved__ (bool) –

True if __proxy_target__ is set.
__proxy_wrapped__ (T) –

A property that either returns __proxy_target__ if it exists else calls __proxy_factory__, saving the result to __proxy_target__ and returning said result.
__proxy_default_class__ (DefaultClassType) –

Optional default class type value to use when a proxy is in the unresolved state. This avoids needing to resolve the proxy to perform isinstance checks. This value is always ignored while the proxy is resolved because __class__ is a writable property of the cached target and could be altered.
__proxy_default_hash__ (DefaultHashType) –

Optional default hash value to use when a proxy is in the unresolved state and hash() is called. This avoids needing to resolve the proxy for simple operations like dictionary updates. This value is always ignored while the proxy is resolved because the cached target may be modified which can alter the value of the hash.

Parameters:

factory (FactoryType[T]) –

Callable object that returns the underlying object when called. The factory should be pure meaning that every call of the factory returns the same object.
cache_defaults (bool, default: False ) –

Precompute and cache the __proxy_default_class__ and __proxy_default_hash__ attributes of the proxy instance from target. Ignored if target is not provided.
target (T | None, default: None ) –

Optionally preset the target object.

Raises:

TypeError –

If factory is not callable.

Source code in proxystore/proxy/__init__.py

def __init__(
    self,
    factory: FactoryType[T],
    *,
    cache_defaults: bool = False,
    target: T | None = None,
) -> None:
    if not callable(factory):
        raise TypeError('Factory must be callable.')
    object.__setattr__(self, '__proxy_factory__', factory)

    default_class: DefaultClassType = None
    default_hash: DefaultHashType = None

    if target is not None:
        object.__setattr__(self, '__proxy_target__', target)
        if cache_defaults:
            default_class = target.__class__
            try:
                default_hash = hash(target)
            except TypeError as e:
                default_hash = e

    object.__setattr__(self, '__proxy_default_class__', default_class)
    object.__setattr__(self, '__proxy_default_hash__', default_hash)

ProxyLocker ¶

ProxyLocker(proxy: Proxy[T])

Bases: Generic[T]

Proxy locker that prevents resolution of wrapped proxies.

The class prevents unintended access to a wrapped proxy to ensure a proxy is not resolved. The wrapped proxy can be retrieved with proxy = ProxyLocker(proxy).unlock().

Parameters:

proxy (Proxy[T]) –

Proxy to lock.

Source code in proxystore/proxy/__init__.py

def __init__(self, proxy: Proxy[T]) -> None:
    self._proxy = proxy

unlock ¶

unlock() -> Proxy[T]

Retrieve the locked proxy.

Returns:

Proxy[T] –

Proxy object.

Source code in proxystore/proxy/__init__.py

def unlock(self) -> Proxy[T]:
    """Retrieve the locked proxy.

    Returns:
        Proxy object.
    """
    return super().__getattribute__('_proxy')

get_factory ¶

get_factory(proxy: Proxy[T]) -> FactoryType[T]

Get the factory contained in a proxy.

Parameters:

proxy (Proxy[T]) –

Proxy instance to get the factory from.

Returns:

FactoryType[T] –

The factory, a callable object which, when invoked, returns an object
FactoryType[T] –

of type T.

Source code in proxystore/proxy/__init__.py

def get_factory(proxy: Proxy[T]) -> FactoryType[T]:
    """Get the factory contained in a proxy.

    Args:
        proxy: Proxy instance to get the factory from.

    Returns:
        The factory, a callable object which, when invoked, returns an object
        of type `T`.
    """
    return proxy.__proxy_factory__

extract ¶

extract(proxy: Proxy[T]) -> T

Return object wrapped by proxy.

If the proxy has not been resolved yet, this will force the proxy to be resolved prior.

Parameters:

proxy (Proxy[T]) –

Proxy instance to extract from.

Returns:

T –

Object wrapped by proxy.

Source code in proxystore/proxy/__init__.py

def extract(proxy: Proxy[T]) -> T:
    """Return object wrapped by proxy.

    If the proxy has not been resolved yet, this will force
    the proxy to be resolved prior.

    Args:
        proxy: Proxy instance to extract from.

    Returns:
        Object wrapped by proxy.
    """
    return proxy.__proxy_wrapped__

is_resolved ¶

is_resolved(proxy: Proxy[T]) -> bool

Check if a proxy is resolved.

Parameters:

proxy (Proxy[T]) –

Proxy instance to check.

Returns:

bool –

True if proxy is resolved (i.e., the factory has been called) and False otherwise.

Source code in proxystore/proxy/__init__.py

def is_resolved(proxy: Proxy[T]) -> bool:
    """Check if a proxy is resolved.

    Args:
        proxy: Proxy instance to check.

    Returns:
        `True` if `proxy` is resolved (i.e., the `factory` has been called) \
        and `False` otherwise.
    """
    return proxy.__proxy_resolved__

resolve ¶

resolve(proxy: Proxy[T]) -> None

Force a proxy to resolve itself.

Parameters:

proxy (Proxy[T]) –

Proxy instance to force resolve.

Source code in proxystore/proxy/__init__.py

def resolve(proxy: Proxy[T]) -> None:
    """Force a proxy to resolve itself.

    Args:
        proxy: Proxy instance to force resolve.
    """
    proxy.__proxy_wrapped__  # noqa: B018

proxystore.proxy¶

ProxyOr module-attribute ¶

Proxy ¶

ProxyLocker ¶

unlock ¶

get_factory ¶

extract ¶

is_resolved ¶

resolve ¶

ProxyOr `module-attribute` ¶