API reference

API reference#

Read backend#

class xarray_binfile.read.RawBinaryEntrypoint#

Bases: BackendEntrypoint

Backend entry point for reading binary files in Xarray.

Variables:

open_dataset_parameters (ClassVar[tuple | None]) – Parameters accepted by the open_dataset method.
description (ClassVar[str]) – Description of the backend.
url (ClassVar[str]) – URL to the backend documentation.

open_dataset(filename_or_obj: str | PathLike[Any], *, read_specs_getter: ReadSpecsGetterProtocol, drop_variables: str | Iterable[str] | None = None) → Dataset#

Open a dataset from a binary file.

Parameters:

filename_or_obj – Path to the binary file or a file-like object.
read_specs_getter – A callable that generates read specifications for the binary file.
drop_variables – Variables to drop from the dataset. Defaults to None.

Returns:

The opened Xarray dataset.

Raises:

ValueError – If filename_or_obj is not a valid file path.
ValueError – If there is an error reading the metadata from the file path.

class xarray_binfile.read.ReadSpecs(filepath: Path, dtype: DTypeLike, coords: Mapping[str, ArrayLike], name: str, attrs: Mapping[Any, Any] | None = None, order: Literal['C', 'F'] = 'C', coord_attrs: Mapping[str, Mapping[Any, Any]] | None = None)#

Bases: object

Immutable metadata contract for interpreting one binary file.

A raw binary file does not carry enough semantic information to be decoded safely by itself. This object provides that missing context so the backend can expose the data as a labeled xarray variable.

Notes

Dimension order is inferred from coords key order.
shape is inferred from coordinate lengths.
dtype should be explicit about byte order when portability matters (for example "<f4" for little-endian float32).
order tells which axis varies fastest on disk. NumPy’s tofile and C programs write "C" order (last axis fastest); Fortran programs, including 2DECOMP&FFT and Xcompact3d, write "F" order (first axis fastest). Both describe the same bytes, so a file written in "F" order with dims (x, y, z) can also be read in "C" order with dims (z, y, x); order just lets you keep the dimension names in their natural order.

Variables:

filepath (pathlib._local.Path) – Path to the binary file.
dtype (DTypeLike) – Data type of the binary file.
coords (Mapping[str, ArrayLike]) – Coordinates of the data in the binary file.
name (str) – Name of the dataset or variable.
attrs (Mapping[Any, Any] | None) – Additional attributes for the dataset or variable.
order (Literal['C', 'F']) – Memory layout of the file, "C" (default) or "F".
coord_attrs (collections.abc.Mapping[str, Mapping[Any, Any]] | None) – Optional attributes attached to each coordinate, keyed by coordinate name (for example {"x": {"units": "m"}}).

property dims: tuple[str, ...]#

Gets the dimension names of the data.

Returns:: Dimension names.

property shape: tuple[int, ...]#

Gets the shape of the data based on the coordinates.

Returns:: Shape of the data.

class xarray_binfile.read.ReadSpecsGetterProtocol(*args, **kwargs)#

Bases: Protocol

Structural protocol for read spec getter implementations.

Any callable matching (path: Path) -> ReadSpecs can be used as a read specs getter. No inheritance is required.

Typical responsibilities:

Parse filename conventions.
Resolve variable identity (for example variable name, step index).
Provide explicit dtype, coordinates, and optional attrs.

__call__(path: Path) → ReadSpecs#

Generate read specifications for a binary file.

Parameters:: path – Path to the binary file.
Returns:: The metadata required by the binary backend to decode path.

Write accessors#

class xarray_binfile.write.BinaryEngineDataArray(data_array: DataArray)#

Bases: object

An accessor with extra utilities for xarray.DataArray.

to_file(write_specs_getter: WriteSpecsGetterProtocol, directory: str | PathLike[str] | None = None, *, progress: Callable[[Iterable[WriteSpecs]], Iterable[WriteSpecs]] | None = None) → None#

Writes the data array to binary files.

Writes are eager and whole-file only. For each write specification, the entire sub_array is loaded into memory (triggering a Dask compute for lazy data) and the target file is written in full, in a single pass. There is no partial, appending, or resuming write mode: re-writing a file always replaces its whole content instead of trying to guess or patch existing bytes, which avoids leaving files in a partially-updated, corrupted state.

Writes are also atomic per file: the bytes are first serialized into a temporary file created next to the destination (so the final move stays on the same filesystem), and each file is moved to its final path with os.replace() only once it is complete. An interrupted write never leaves a truncated file at the destination, and the temporary file is removed automatically.

Plan the write specifications so that every individual output file fits comfortably in memory, for example by splitting the array into one file per time step. If you need streaming, incremental, or partial writes, prefer one of the other file formats supported by xarray, such as NetCDF or Zarr.

Each file is written with the in-memory dtype and native byte order, unless the write specification sets dtype, in which case the values are cast right before serialization. Bytes are laid out in the memory order declared by WriteSpecs.order: "C" (last axis fastest, the default) or "F" (first axis fastest).

A relative WriteSpecs.filename is resolved against directory, which must already exist, and may contain sub-folders (for example "3d/ux-0001.bin"), which are created on demand. It must stay inside directory: paths escaping it through .. are rejected (symbolic links inside directory are not followed for this check, so a linked sub-folder is fine). An absolute filename is used as is, which lets one call target several locations.

Parameters:

write_specs_getter – A callable that generates write specifications for the data array.
directory – The base directory for relative filenames. Defaults to the current working directory.
progress – Optional wrapper applied to the iterator of write specs before files are written, for example tqdm or functools.partial(tqdm, desc="ux"), to report progress. It receives the specs lazily, so it must not consume them ahead of iteration.

Raises:

FileNotFoundError – If a relative filename is used and directory does not exist.
ValueError – If a relative filename escapes directory.

class xarray_binfile.write.BinaryEngineDataset(data_set: Dataset)#

Bases: object

An accessor with extra utilities for xarray.Dataset.

to_file(write_specs_getter: WriteSpecsGetterProtocol, directory: str | PathLike[str] | None = None, *, progress: Callable[[Iterable[WriteSpecs]], Iterable[WriteSpecs]] | None = None) → None#

Writes the dataset to binary files.

Every data variable is delegated to BinaryEngineDataArray.to_file(), so the same eager, atomic, whole-file write semantics apply: each output file is fully materialized in memory (triggering a Dask compute for lazy variables), written in a single pass, and moved into place only once complete. See that method for guidance on sizing files and on alternatives when streaming or partial writes are needed.

Parameters:

write_specs_getter – A callable that generates write specifications for the data arrays.
directory – The directory where the binary files will be written. Defaults to the current working directory.
progress – Optional wrapper applied to the write specs of each variable, for example tqdm, to report progress.

class xarray_binfile.write.WriteSpecs(filename: str, sub_array: DataArray, dtype: DTypeLike | None = None, order: Literal['C', 'F'] = 'C')#

Bases: NamedTuple

Immutable write instruction for one output binary file.

Each item defines both the output filename and the exact DataArray slice to serialize into that file.

Variables:

filename (str) – Path of the binary file. A relative path is resolved against the (existing) output directory passed to to_file and may include sub-folders, which are created on demand (for example "3d/ux-0001.bin"); it must not escape that directory. An absolute path is written as is.
sub_array (xarray.core.dataarray.DataArray) – The portion of the DataArray to be written.
dtype (DTypeLike | None) – Optional on-disk data type. When set, the sub-array is cast to this dtype (including byte order, for example "<f4") right before serialization. When None, the in-memory dtype and native byte order are written as-is, so make sure they match what your read specs getter declares.
order (Literal['C', 'F']) – Memory layout of the file: "C" (default, last axis of sub_array varies fastest) or "F" (first axis varies fastest, as Fortran programs expect). Must match the order declared by the paired read specs.

dtype: DTypeLike | None#: Alias for field number 2

filename: str#: Alias for field number 0

order: Literal['C', 'F']#: Alias for field number 3

sub_array: DataArray#: Alias for field number 1

class xarray_binfile.write.WriteSpecsGetterProtocol(*args, **kwargs)#

Bases: Protocol

Structural protocol for write spec getter implementations.

Any callable matching (data_array: xr.DataArray) -> Iterator[WriteSpecs] can be used as a write specs getter. No inheritance is required.

Typical responsibilities:

Define filename conventions.
Split a DataArray into one or more per-file slices.
Ensure each slice is transposed to the expected on-disk dimension order.
Keep each per-file slice small enough to fit in memory, since every sub_array is fully materialized (Dask compute) before its file is written in a single pass.

__call__(data_array: DataArray) → Iterator[WriteSpecs]#

Generate write instructions for a DataArray.

Parameters:: data_array – The data array for which to generate write specifications.
Returns:: An iterator over per-file write instructions.

Conventions#

Ready-to-use file conventions built on the read and write spec protocols.

Start from the convention closest to your project’s naming scheme, or copy one and adapt it when none fits.

class xarray_binfile.conventions.Convention#

Bases: object

Base class for conventions: reader and writer plus sensible defaults.

Subclasses implement reader() and writer(). The defaults derive everything else from them, and can be overridden when a cheaper or smarter implementation exists (the shipped conventions decode names from the filename alone, for instance).

accepts(path: Path) → bool#

Tell whether the convention can decode path.

Parameters:: path – The candidate file.
Returns:: True if name_of_file() succeeds.

classmethod adapt(convention: ConventionProtocol) → Convention#

Give any reader/writer object the defaults of this class.

Parameters:: convention – A Convention, returned as is, or any object following ConventionProtocol.
Returns:: A Convention.

files(directory: str | PathLike[str]) → list[Path]#

List the files in directory that the convention accepts.

Parameters:: directory – The folder to scan (not recursively).
Returns:: The matching paths, sorted.

name_of_file(path: Path) → str#

The variable name stored in path, without reading it.

Parameters:: path – A binary file.
Returns:: The decoded variable name.
Raises:: ValueError – If the convention does not accept path.

open(directory: str | PathLike[str], *, variables: Iterable[str] | None = None, stack: bool = True, **open_mfdataset_kwargs: Any) → Dataset#

Open every accepted file in directory as one lazy dataset.

Parameters:

directory – The folder holding the files.
variables – Optional names to keep, as found on disk ("ux", not "u"); other files are not opened.
stack – Whether to rebuild the stacked arrays.
**open_mfdataset_kwargs – Forwarded to xarray.open_mfdataset(), for example chunks={"time": 1} or parallel=True.

Returns:

The lazily opened dataset, combined by coordinates.

Raises:

FileNotFoundError – If no file matches.

stack(dataset: Dataset) → Dataset#

Rebuild the arrays that the convention splits across variables.

The default keeps the dataset unchanged.

Parameters:: dataset – A dataset as returned by the binfile engine.
Returns:: The dataset, stacked when the convention declares stacks.

class xarray_binfile.conventions.ConventionProtocol(*args, **kwargs)#

Bases: Protocol

Structural protocol for a file convention.

A convention bundles a read specs getter and a write specs getter that agree on the same filename pattern and on-disk layout, so files written by one can always be read back by the other. No inheritance is required: any object exposing reader and writer attributes with the right signatures is a convention, including the classes shipped in xarray_binfile.conventions.

property reader: ReadSpecsGetterProtocol#

Read specs getter for files that follow the convention.

Returns:: A callable matching ReadSpecsGetterProtocol.

property writer: WriteSpecsGetterProtocol#

Write specs getter producing files that follow the convention.

Returns:: A callable matching WriteSpecsGetterProtocol.

class xarray_binfile.conventions.FilenamePattern(template: str, exact_width: bool = False)#

Bases: object

Format and parse filenames from a single str.format-style template.

The regular expression used by parse() is derived from template, so writing with format() and reading with parse() are always consistent. Matching is anchored to the whole filename, so a trailing .bak or a different suffix is rejected.

Supported field specifications:

{name} or {name:s}: a word made of letters, digits and underscores. It is matched lazily, so "{name}{step:03d}" splits ux001 into ux and 1. When formatting, the value may carry folder segments (geometry/epsilon), which are kept in front of the filename and turned into sub-folders by the write accessor; parsing always works on the bare filename.
{step:d}, {step:04d}: an integer. By default a zero-fill width is treated as a minimum width, matching str.format semantics, so step 12345 written with 04d can still be read back. Without a separator, names ending in a digit are then ambiguous (phi1000 reads as phi + 1000); set exact_width=True to match exactly the declared width instead, so phi1000 reads as phi1 + 000 and steps wider than the template are refused at write time.
{time:.3f}, {time:f}, {time:e}, {time:g}: a float.

Examples

>>> pattern = FilenamePattern("{name}-{step:04d}.bin")
>>> pattern.format(name="ux", step=7)
'ux-0007.bin'
>>> pattern.parse("ux-0007.bin")
{'name': 'ux', 'step': 7}
>>> pattern.parse("ux-12345.bin")
{'name': 'ux', 'step': 12345}
>>> pattern.format(name="geometry/epsi", step=0)
'geometry/epsi-0000.bin'
>>> pattern.glob()
'*-*.bin'
>>> pattern.glob(name="ux")
'ux-*.bin'
>>> FilenamePattern("{name}{step:03d}").parse("ux001")
{'name': 'ux', 'step': 1}
>>> FilenamePattern("{name}{step:03d}").parse("phi1000")
{'name': 'phi', 'step': 1000}
>>> FilenamePattern("{name}{step:03d}", exact_width=True).parse("phi1000")
{'name': 'phi1', 'step': 0}

Variables:

template (str) – The str.format template.
exact_width (bool) – Whether zero-filled integers match exactly their width.
fields (tuple[str, ...]) – Field names in the order they appear in template.
regex (re.Pattern[str]) – Compiled regular expression equivalent to template.

__post_init__() → None#

Derive fields, regex and the field types from template.

Raises:: ValueError – If the template has no fields, repeats a field, uses a conversion flag (!r), or uses an unsupported specification.

format(**fields: Any) → str#

Build a filename from field values.

The result is checked against the derived regular expression, so a filename that parse() could not read back (for example a name containing . or -) is rejected here, at write time. A name value may start with folder segments ("geometry/epsi"); they are kept in front of the checked filename ("geometry/epsi.bin").

Parameters:: **fields – One value per field declared in the template.
Returns:: The formatted filename.
Raises:: ValueError – If the formatted filename does not match the pattern, or if the folder prefix of name is malformed.

glob(**fixed: Any) → str#

Build a glob pattern that matches filenames following the template.

Every field is replaced by * unless a value for it is given, in which case the value is formatted with the field specification. The result is coarser than matches() (* also accepts text the regular expression would reject), so filter candidates with matches() or parse() after globbing.

Parameters:: **fixed – Values for the fields to pin, for example name="ux".
Returns:: A pattern for glob.glob() or pathlib.Path.glob().
Raises:: ValueError – If fixed names a field the template does not declare.

matches(filename: str) → bool#

Check whether filename follows the template.

Parameters:: filename – The file name (without directories) to test.
Returns:: True if the whole filename matches the template.

parse(filename: str) → dict[str, Any]#

Extract typed field values from filename.

Parameters:: filename – The file name (without directories) to parse.
Returns:: A mapping from field name to its value, converted to str, int or float according to the template specification.
Raises:: ValueError – If filename does not follow the template.

class xarray_binfile.conventions.FolderConventions(conventions: Mapping[str, ConventionProtocol])#

Bases: Convention

Dispatch to one convention per sub-folder.

Projects often keep arrays of different shapes in different folders, for example xy_planes/ux-0001.bin next to 3d/ux-0001.bin. Reading picks the convention whose folder matches the end of the file’s parent path, preferring the most specific (longest) folder when several match. The root of the dataset is registered as "."; it matches any file not claimed by a more specific folder. Writing offers the array to every convention and requires exactly one of them to accept it, then prefixes the resulting filenames with that folder; a mismatch raises instead of guessing. An array whose name starts with a registered folder ("geometry/epsi") is sent to that folder’s convention directly, under the remaining name ("epsi"), which resolves the ambiguity between folders sharing one layout.

Variables:: conventions (collections.abc.Mapping[str, xarray_binfile.conventions.protocol.ConventionProtocol]) – Mapping of folder (relative to the dataset root, possibly nested such as "snapshots/3d", or "." for the root) to the convention used inside it.

__post_init__() → None#: Give every member the Convention defaults.

files(directory: str | PathLike[str]) → list[Path]#

List the files of every registered folder inside directory.

Parameters:: directory – The dataset root.
Returns:: The matching paths, sorted. Folders that do not exist are skipped.

name_of_file(path: Path) → str#

The variable name of path, decoded by the convention of its folder.

Parameters:: path – Path to the binary file.
Returns:: The decoded name.
Raises:: ValueError – If no convention is registered for the file’s folder, or if that convention rejects the file.

reader(path: Path) → ReadSpecs#

Build read specs using the convention of the file’s folder.

Parameters:: path – Path to the binary file, inside one of the known folders.
Returns:: The metadata required to decode path.
Raises:: ValueError – If no convention is registered for the file’s folder.

stack(dataset: Dataset) → Dataset#

Rebuild the stacked arrays declared by the member conventions.

Parameters:: dataset – A dataset as returned by the binfile engine.
Returns:: The stacked dataset.

writer(data_array: DataArray) → Iterator[WriteSpecs]#

Yield write specs from the single convention that accepts the array.

Parameters:: data_array – The array to write.
Yields:: The accepting convention’s write specs, with filenames prefixed by its folder (always using / as separator).
Raises:: LayoutMismatchError – If no convention or more than one convention accepts the array.

class xarray_binfile.conventions.Layout(coords: ~collections.abc.Mapping[str, ArrayLike], dtype: DTypeLike = <class 'numpy.float64'>, order: ~typing.Literal['C', 'F'] = 'C', coord_attrs: ~collections.abc.Mapping[str, ~typing.Mapping[~typing.Any, ~typing.Any]] | None = None)#

Bases: object

Dimension order, coordinate values and dtype of the arrays stored on disk.

Raw binary files carry none of this information, so a layout is the shared contract between the read and write sides of a convention: readers attach these coordinates to every file they decode, and writers refuse arrays that do not match them instead of guessing.

Variables:

coords (collections.abc.Mapping[str, ArrayLike]) – Mapping of dimension name to coordinate values, in the order the dimensions are declared. Which of them varies fastest on disk is set by order.
dtype (DTypeLike) – On-disk data type. Be explicit about byte order when files move between machines (for example "<f4").
order (Literal['C', 'F']) – Memory layout of each file. "C" (default) means the last dimension in coords varies fastest, as NumPy’s tofile and C programs write. "F" means the first dimension varies fastest, as Fortran programs (2DECOMP&FFT, Xcompact3d) write. Layout({"x": x, "y": y, "z": z}, order="F") describes the same bytes as Layout({"z": z, "y": y, "x": x}); pick the one that keeps your dimension names in their natural order.
coord_attrs (collections.abc.Mapping[str, Mapping[Any, Any]] | None) – Optional attributes attached to each coordinate on read, keyed by dimension name (for example {"x": {"units": "m"}}).

property dims: tuple[str, ...]#

Dimension names in on-disk order.

Returns:: The dimension names.

dtype#: alias of float64

property shape: tuple[int, ...]#

Array shape derived from the coordinate lengths.

Returns:: The shape of one array on disk.

transpose(data_array: DataArray) → DataArray#

Reorder data_array to the on-disk dimension order.

Parameters:: data_array – An array holding exactly the layout dimensions.
Returns:: The transposed array.

validate(data_array: DataArray, *, extra_dims: Iterable[str] = (), check_coords: bool = True) → None#

Check that data_array can be written with this layout.

Parameters:

data_array – The array about to be written.
extra_dims – Dimensions the array may have on top of the layout dimensions, typically the dimension that is split across files.
check_coords – Also require the coordinate values to be equal, not only the dimension names and sizes.

Raises:

LayoutMismatchError – If the dimensions, sizes or coordinate values of data_array differ from the layout.

exception xarray_binfile.conventions.LayoutMismatchError#

Bases: ValueError

Raised when a DataArray does not fit the on-disk layout of a convention.

class xarray_binfile.conventions.PatternConventions(conventions: Sequence[ConventionProtocol])#

Bases: Convention

Dispatch to the first convention whose filename pattern matches.

Use it when files following different conventions share one folder, for example ux-0001.bin (a StepIndexedFiles) next to epsi.bin (a StaticFiles). Reading tries the conventions in order and keeps the first whose reader accepts the filename, so list the most specific pattern first. Writing offers the array to every convention and requires exactly one of them to accept it.

Variables:: conventions (collections.abc.Sequence[xarray_binfile.conventions.protocol.ConventionProtocol]) – The conventions to try, most specific first.

__post_init__() → None#: Give every member the Convention defaults.

files(directory: str | PathLike[str]) → list[Path]#

List the files in directory accepted by any of the conventions.

Parameters:: directory – The folder to scan (not recursively).
Returns:: The matching paths, sorted.

name_of_file(path: Path) → str#

The variable name of path, from the first member accepting it.

Parameters:: path – Path to the binary file.
Returns:: The decoded name.
Raises:: ValueError – If no member accepts the file.

reader(path: Path) → ReadSpecs#

Build read specs with the first convention that accepts the filename.

Parameters:: path – Path to the binary file.
Returns:: The metadata required to decode path.
Raises:: ValueError – If no convention accepts the filename.

stack(dataset: Dataset) → Dataset#

Rebuild the stacked arrays declared by the member conventions.

Parameters:: dataset – A dataset as returned by the binfile engine.
Returns:: The stacked dataset.

writer(data_array: DataArray) → Iterator[WriteSpecs]#

Yield write specs from the single convention that accepts the array.

Parameters:: data_array – The array to write.
Yields:: The accepting convention’s write specs.
Raises:: LayoutMismatchError – If no convention or more than one convention accepts the array.

class xarray_binfile.conventions.StaticFiles(layout: Layout, pattern: FilenamePattern | str = '{name}.bin', check_coords: bool = True, stacks: Sequence[VariableStack] = (), name_of: Callable[[DataArray], str] | None = None, names: Collection[str] | None = None)#

Bases: _ConventionBase

One file per variable, with no dimension split across files.

Typical for fields that do not change in time, such as a mask or a geometry (epsi.bin). Reading attaches exactly the layout coordinates; writing produces a single file per DataArray.

Examples

>>> import numpy as np
>>> convention = StaticFiles(
...     layout=Layout({"x": np.arange(4), "y": np.arange(3)}, dtype="<f4"),
...     pattern="{name}.bin",
... )
>>> specs = convention.reader(Path("epsi.bin"))
>>> specs.name, specs.dims
('epsi', ('x', 'y'))

Variables:: pattern (xarray_binfile.conventions.filename_pattern.FilenamePattern | str) – Filename pattern declaring {name}. Defaults to "{name}.bin". The other attributes are shared with every shipped convention: layout, check_coords, stacks, name_of and names.

class xarray_binfile.conventions.StepIndexedFiles(layout: Layout, pattern: FilenamePattern | str = '{name}-{step:04d}.bin', check_coords: bool = True, stacks: Sequence[VariableStack] = (), name_of: Callable[[DataArray], str] | None = None, names: Collection[str] | None = None, time_dim: str = 'time', time_dtype: DTypeLike | None = None, time_step: float | None = None)#

Bases: _SplitAlongDimension

One file per snapshot, numbered with an integer step counter.

This is the most common solver convention, for example ux-0001.bin, ux.0002 or ux0003.dat. The time coordinate is derived from the step: it is the step itself by default, or step * time_step when the interval between snapshots is known. Writing inverts that mapping and refuses coordinate values that do not land on an integer step.

Examples

>>> import numpy as np
>>> convention = StepIndexedFiles(
...     layout=Layout({"x": np.arange(4), "y": np.arange(3)}),
...     pattern="{name}-{step:04d}.bin",
...     time_step=0.5,
... )
>>> specs = convention.reader(Path("ux-0007.bin"))
>>> specs.name, specs.coords["time"]
('ux', array([3.5]))
>>> da = xr.DataArray(
...     np.zeros((4, 3, 2)),
...     coords={"x": np.arange(4), "y": np.arange(3), "time": [1.0, 1.5]},
...     name="ux",
... )
>>> [spec.filename for spec in convention.writer(da)]
['ux-0002.bin', 'ux-0003.bin']

Variables:

pattern (xarray_binfile.conventions.filename_pattern.FilenamePattern | str) – Filename pattern declaring {name} and {step}. Defaults to "{name}-{step:04d}.bin".
time_dim (str) – Name of the dimension split across files. Defaults to "time".
time_dtype (DTypeLike | None) – Optional dtype for the time coordinate on read.
time_step (float | None) – Interval between consecutive steps. When None the time coordinate is the integer step itself.

class xarray_binfile.conventions.TimeStampedFiles(layout: Layout, pattern: FilenamePattern | str = '{name}-{time:.3f}.bin', check_coords: bool = True, stacks: Sequence[VariableStack] = (), name_of: Callable[[DataArray], str] | None = None, names: Collection[str] | None = None, time_dim: str = 'time', time_dtype: DTypeLike | None = None)#

Bases: _SplitAlongDimension

One file per snapshot, named with the physical time of the snapshot.

Some solvers write ux-0.250.bin instead of a step counter. The time coordinate is the float parsed from the filename, so what is on disk is the single source of truth. Writing formats each coordinate value with the pattern precision and refuses arrays whose values collide at that precision.

Examples

>>> import numpy as np
>>> convention = TimeStampedFiles(
...     layout=Layout({"x": np.arange(4)}),
...     pattern="{name}-{time:.3f}.bin",
... )
>>> convention.reader(Path("ux-0.250.bin")).coords["time"]
array([0.25])

Variables:

pattern (xarray_binfile.conventions.filename_pattern.FilenamePattern | str) – Filename pattern declaring {name} and {time}. Defaults to "{name}-{time:.3f}.bin".
time_dim (str) – Name of the dimension split across files. Defaults to "time".
time_dtype (DTypeLike | None) – Optional dtype for the time coordinate on read.

class xarray_binfile.conventions.VariableStack(dim: str, template: str, values: Sequence[Any], names: Collection[str] | None = None, attrs: Mapping[str, Any] | None = None, min_components: int | None = None)#

Bases: object

A dimension encoded in variable names instead of in the files.

Examples

>>> import numpy as np
>>> velocity = VariableStack("i", "{name}{i}", values=("x", "y", "z"))
>>> scalars = VariableStack("n", "{name}{n:d}", values=range(1, 10))
>>> u = xr.DataArray(
...     np.zeros((3, 2)), coords={"i": ["x", "y", "z"], "x": [0, 1]}, name="u"
... )
>>> [piece.name for piece in velocity.split(u)]
['ux', 'uy', 'uz']
>>> ds = xr.Dataset({name: ("x", np.zeros(2)) for name in ("ux", "uy", "pp")})
>>> stacked = velocity.stack(ds)
>>> sorted(stacked.data_vars), stacked["i"].values.tolist()
(['pp', 'u'], ['x', 'y'])

Variables:

dim (str) – Name of the stacked dimension ("i").
template (str) – str.format template building a variable name from {name} and {dim} ("{name}{i}", "{name}{n:d}"). Format specifications apply both ways, so integer values can use zero-fill ("{name}{n:02d}").
values (collections.abc.Sequence[Any]) – The values the dimension may take, in the order the stacked coordinate should follow (("x", "y", "z"), range(1, 10)). Splitting refuses coordinate values outside this set, so every file written can be stacked back. Required because templates without a separator match almost any name (pp fits "{name}{i}" as p + p).
names (collections.abc.Collection[str] | None) – Optional base names to restrict stacking to ({"u"}), any collection (not a bare string), kept as a frozenset. When None, every group of at least min_components matching variables is stacked.
attrs (collections.abc.Mapping[str, Any] | None) – Optional attributes attached to the dim coordinate when stacking (for example {"long_name": "velocity component"}).
min_components (int | None) – How many components a group needs before it is stacked. Defaults to 1 when names lists the fields explicitly and to 2 otherwise, so that a lone vorticity is not read as vorticit + y when guessing.

__post_init__() → None#

Validate the template and freeze values.

Raises:

ValueError – If template does not declare exactly {name} and {dim}, or if values is empty.
TypeError – If names is a bare string.

split(data_array: DataArray) → Iterator[DataArray]#

Yield one named slice per value along dim.

Arrays without dim are yielded unchanged. The slice name is built from data_array.name and the coordinate value (or the position, when dim has no coordinate).

Parameters:: data_array – A named array.
Yields:: The slices, with dim dropped.
Raises:: ValueError – If the array has no name, or if a coordinate value is not listed in values.

stack(dataset: Dataset) → Dataset#

Combine the variables encoding dim into one array per base name.

Variables whose name parses with template and whose decoded value is in values are grouped by base name and concatenated along a new dimension dim, whose coordinate holds the values found, in the order of values. Everything else is left untouched. The operation is lazy for Dask-backed data.

Parameters:: dataset – The dataset holding the split variables.
Returns:: A new dataset where each group is replaced by its stacked array.
Raises:: ValueError – If a group’s base name is already a variable of the dataset, which would silently overwrite it.

xarray_binfile.conventions.split_variables(data_array: DataArray, stacks: Sequence[VariableStack]) → Iterator[DataArray]#

Split an array along every stacked dimension it carries, in order.

Parameters:

data_array – A named array.
stacks – The stacks to apply; the first one produces the outer loop.

Yields:

Named slices free of every stacked dimension.

xarray_binfile.conventions.stack_variables(dataset: Dataset, stacks: Sequence[VariableStack]) → Dataset#

Stack a dataset along every declared dimension, undoing split_variables().

Stacks are applied in reverse order, so the innermost split is rebuilt first.

Parameters:

dataset – The dataset holding the split variables.
stacks – The stacks that were used to split.

Returns:

The stacked dataset.

Tutorial helpers#

class xarray_binfile.tutorial.DatasetGenerator(read_specs_getter: ReadSpecsGetterProtocol)#

Bases: object

Utility that generates synthetic datasets from read specs metadata.

This helper is intended for tutorials and tests where you want deterministic mock data that follows the same metadata conventions expected by the backend.

Variables:

read_specs_getter (xarray_binfile.read.file_metadata.ReadSpecsGetterProtocol) – A callable that generates read specifications for binary files.
random_generator – A random number generator for creating random data.

__call__(iter_filepath: Iterator[Path]) → Dataset#

Generate and merge datasets for the provided file paths.

Each path is converted to ReadSpecs using read_specs_getter. A synthetic variable is generated for each metadata object, and all variables are merged into one Dataset.

Parameters:: iter_filepath – An iterator over file paths.
Returns:: A merged xarray Dataset.

class xarray_binfile.tutorial.FileSpecsGetter(base_coords: dict[str, ArrayLike], dtype: DTypeLike = <class 'numpy.float64'>, filename_template: str = '{name}-{digits:04}.bin', filename_regex: ~re.Pattern = re.compile('(?P<name>\\w+)-(?P<digits>\\d{4})\\.bin'))#

Bases: object

Reference implementation of both read and write spec getter protocols.

Deprecated since version Use: xarray_binfile.conventions.StepIndexedFiles instead. It derives the filename regex from the template (so the two cannot drift apart), anchors matching to the whole filename, supports an interval between snapshots through time_step, and refuses coordinate values that would collide on disk. This class is kept unchanged for backward compatibility and will be removed in a future release.

This helper is intended for tutorials, tests, and simple projects with a filename convention that encodes variable name and step index.

It is intentionally narrow: the default regex expects time-indexed files (for example ux-0001.bin). For mixed layouts that include static files (for example epsi.bin), implement a custom getter that still follows the same read/write protocol contracts.

It implements:

reader(path) -> ReadSpecs
writer(data_array) -> Iterator[WriteSpecs]

Variables:

base_coords (dict[str, ArrayLike]) – Base coordinates for the data.
dtype (DTypeLike) – Data type of the binary file. Defaults to np.float64.
filename_template (str) – Template for generating filenames.
filename_regex (re.Pattern) – Regular expression for parsing filenames.

__post_init__() → None#: Warn that this class is superseded by StepIndexedFiles.

dtype#: alias of float64

reader(path: Path) → ReadSpecs#

Build ReadSpecs from a file path.

The implementation parses path.name using filename_regex and appends a single-value time coordinate based on the extracted step number.

Parameters:: path – Path to the binary file.
Returns:: Metadata for reading the binary file.
Return type:: ReadSpecs
Raises:: ValueError – If the filename does not match the expected pattern.

writer(data_array: DataArray) → Iterator[WriteSpecs]#

Yield WriteSpecs for one DataArray.

The default behavior writes one file per time coordinate value using filename_template and transposes each slice to base_coords dimension order before serialization. Each slice is cast to dtype on write, so files always round-trip with reader().

Parameters:: data_array – The data array to generate write specifications for.
Returns:: An iterator over write specifications.

API reference

Contents

API reference#

Read backend#

Write accessors#

Conventions#

Tutorial helpers#