Skip to main content

Changelog

This page includes a running list of user-facing changes to the Arraylake Client and Service.

0.12.0 - September 25, 2024​

  • [Web, Service, Client] - Support for role-based access delegation ("customer-managed role") for AWS S3 object stores
  • [Web] - Renamed "Anonymous" credentials to "Self-Managed" credentials in the Web UI
  • [Client] - Removed deprecated bucket_nickname (now bucket_config_nickname) argument

0.11.0 - September 6, 2024​

  • [Client] - Performance improvements when fetching large or many small chunks from storage
  • [Web, CLI] - Allow org admins to set repo status to maintenance or offline mode
  • [Client] - Added an advanced configuration option (chunkstore.unsafe_use_fill_value_for_missing_chunks) to make chunkstore more fault tolerant

0.10.0 - July 12, 2024​

0.9.10 - July 05, 2024​

  • [Client] - Added the ability to login to Arraylake via python library.
  • [Client] - Added a read_only flag for getting repositories with get_repo. See client api docs.

0.9.9 - June 26, 2024​

  • [Client] - Added the ability to export Arraylake repositories to standard Zarr stores via the arraylake repo export command line utility.

0.9.8 - May 27, 2024​

  • [Client] - Added the Repo method add_kerchunk_references to support importing existing Kerchunk reference data into Arraylake repos.

0.9.7 - May 1, 2024​

  • [Client, Service] - Improve validation of new bucket configurations
  • [Client] - Fix duplicate FutureWarning in get_or_create_repo
  • [Client, Service] - Disallow deleting the main branch of a repo

0.9.6 - Apr 26, 2024​

  • [Client, Service, Web] - Arraylake now supports user managed bucket configurations
  • [Client, Service] - Arraylake now supports version tags
  • [Client] - Improve performance of add_virtual_netcdf
  • [Client] - Improve performance of repo.checkout
  • [Client] - Drop support for Python 3.9, Arraylake now supports Python 3.10, 3.11 and 3.12. See the installation page for a current list of supported dependency versions.
  • [Documentation] - Add documentation for seven storage integrations.

0.9.5 - Mar 25, 2024​

  • [Client] Improve performance of add_virtual_zarr
  • [Client] Add support for Python 3.12

0.9.4 - Mar 12, 2024​

  • [Client] Fixed incompatibility with aiobotocore>=2.12.1.
  • [Client] Retry HTTP connections when spurious network errors are encountered.
  • [Web] Improved documentation for how to authenticate to Arraylake using a service account API token.
  • [Web] Added new documentation on how to use Arraylake with Google Colab.

0.9.3 - Mar 08, 2024​

  • [Client] Improve client-side caching

0.9.2 - Mar 06, 2024​

  • [Client] Support virtual files on Google Cloud Storage
  • [Client] Performance improvements around fetching a large number of commits.
  • [Web] Add documentation for configuring Google Cloud Storage buckets.

0.9.1 - Feb 15, 2024​

  • [Client] Improve stability of HTTP connections.
  • [Client, Service] Allow listing active sessions

0.9.0 - Feb 2, 2024​

  • [Client] Support for Google Cloud Storage buckets.
  • [Client] Significant read performance improvements when fetching multiple Zarr chunks at once.
  • [Client] Server-managed sessions on by default.

0.8.1 - Jan 19, 2024​

0.8.0 - Jan 15, 2024​

  • [Client] Upgrade to Pydantic 2
  • [Client] Drop Python 3.8, Arraylake now supports Python 3.9, 3.10, and 3.11.

0.7.8 - Jan 11, 2024​

  • [Client, Service] Implement server-managed session support

0.7.7 - Dec 18, 2023​

  • [Client, Service] Major performance improvements in many aspects.
  • [Client] repo.to_xarray prefetches necessary data early, and is now significantly faster than the equivalent xarray.open_zarr call.
  • [Client] Pin ecmwflibs to 0.5 to work around upstream packaging issues.

v0.7.6 - Nov 30, 2023​

  • [Web] Search in docs.
  • [Web] Major update to Concepts section.
  • [Service, Client] Large performance improvement for repo.checkout

v0.7.5 - Nov 22, 2023​

  • [Client] Improved error handling for service exceptions.

v0.7.4 - Nov 13, 2023​

  • [Client] Fixed issue where some inline references were rejected by the Arraylake service.
  • [Client] Fixed issue where Repo.checkout('main') failed when there are only commits on other branches.

v0.7.3 - Nov 10, 2023​

  • [Client] Add support for ingesting virtual TIFF and GRIB datasets with add_virtual_tiff and add_virtual_grib.
  • [Client] Change of add_virtual_netcdf and add_virtual_zarr to match the cp src dest syntax.
  • [Client] Improve type annotations and distributed arraylake with the py.typed module.
  • [Client] Move filter_metadata out of the experimental module.
  • [Client] Mark xarray as an optional dependency.
  • [Client] Arraylake now requires the CLI dependencies (rich and typer).
  • [Service] Alert clients if they are using an outdated version of the arraylake client.
  • [Client, Service] Support inlining small chunks in the Arraylake metastore.

v0.7.2 - Oct 27, 2023​

  • [Client] The client package was changed from arraylake_client to arraylake. A consequence of this change is that Arraylake's default config directory has moved from ~/.config/arraylake_client to ~/.config/arraylake. Importing arraylake will attempt to migrate the exiting config files.
  • [Client] Client.get_repo and Client.create_repo now automatically checks out the head of the main branch. Calling Client.checkout() is no longer required after these methods.
  • [Client] Fixed issue where arraylake auth login would fail if refresh tokens were expired.
  • [Service] Fixed issue where some user names were shown as None, None.
  • [Web] Fixed issue where web app used incorrect refresh tokens.

v0.7.1 - Oct 20, 2023​

  • [Service, Client] zarr.Group.move is now a metadata-only change
  • [Service, Client] Add --diagnostics flag to CLI, log diagnostics to service when logging in
  • [Client] Add aioitertools as a required dependency
  • [Client] Improve error messages when chunkstore.uri is not configured correctly
  • [Client] Arraylake CLI is now also distributed under the al command
  • [Client] Improved CLI login experience with new log and clickable authorization URL.

v0.7.0 - Oct 13, 2023​

  • [Client] Disallow dependency on numcodecs version 0.12.0
  • [Client] Use uvloop library when available
  • [Client] Add s3.anon boolean configuration value, to support anonymous access to chunkstore
  • [Client] list_repos now returns Sequence[Repo]

v0.6.0 - Sept 22, 2023​

  • [Client] Fix memory leak when using Arraylake with Dask Distributed
  • [Service, Client] - Improve performance of repo.store.getsize()
  • [Client] Remove anyio as a dependency
  • [Client] Repo.close() is no longer required and will now emit a deprecation warning
  • [Client] Improve type annotations and documentation for Repo.to_xarray()

v0.5.0 - Sept 6, 2023​

  • [Client] Remove PyMongo as a dependency
  • [Client] Remove types-aiobotocore as a dependency
  • [Client] Remove uvloop as a required dependency
  • [Client] Improve type annotations for Repo.checkout() method
  • [Client] Added support for Zarr Version 2.16
  • [Service, Client] Integrated the experimental metadata search functionality into the Repo.tree() method
  • [Client] Added new Repo.to_xarray() method to simply reading Xarray datasets from Arraylake
  • [Client] Support initializing client M2M tokens from environment variable (ARRAYLAKE_CLIENT_TOKEN) or arraylake_client.config

v0.4.7 - Aug 25, 2023​

  • [Client] Improved retry logic when experiencing service rate limiting
  • [Client] Publish the Arraylake-client package on Conda-Forge

v0.4.6 - Aug 18, 2023​

  • [Service, Client] Introduced experimental metadata search functionality accessible via the arraylake client
  • [Service] Improved login flow web experience

v0.4.5 - July 31, 2023​

  • [Service, Client] Improved error messaging for cases when a requested commit is not found in the target ref's history
  • [Service, Client] Addressed a bug that caused tree generation to fail when the array compressor configuration was null

v0.4.4 - July 18, 2023​

  • [Client] Addressed a bug that caused failures when multiple writers attempted to commit at the same time
  • [Client] Better client side validation for M2M tokens

v0.4.3 - July 10, 2023​

  • [Client] Update user auth handling for SSO integrations. For customers using an SSO integration, please see this section of our documentation for additional configuration that should be set to support this update.
  • [Client] Additional improvements for client memory management

v0.4.2 - July 7, 2023​

  • [Client] Add Client.get_or_create_repo method
  • [Client] Support implicit groups when navigating an Arraylake store
  • [Client] Improvements for client memory management

v0.4.1 - June 15, 2023​

  • [Client] - Added support for virtual Zarr V2 datasets
  • [Service, Client] - Performance and reliability improvements for queries that include many docs / commits

v0.4.0 - June 12, 2023​

  • [Client] Improve serialization performance for Repo.get_docs
  • [Client, Service] Fix for complex data types in Zarr arrays

v0.3.8 - May 31, 2023​

  • [Client] Richer debug logging support
  • [Client] Improved version spec for broader installation support

v0.3.7 - May 22, 2023​

  • [Service, Client] Fix bug in store prefix listing

v0.3.6 - May 16, 2023​

  • [Service, Client] - Performance and stability improvements for repos with a large number of commits

v0.3.4 - May 8, 2023​

  • [Client] - Support Ipytree in repo.tree()
  • [Client] - Performance and stability improvements

v0.3.3 - May 5, 2023​

  • [Client] - Delete support
  • [Client, Service] - Efficient tree, accessible via repo.tree() and arraylake repo tree

v0.3.2 - May 2, 2023​

  • [Client] - Fix token refresh bug
  • [Client, Service] - Implement getsize, available as repo.store.getsize("some-prefix")

v0.3.1 - Apr 27, 2023​

  • [Client] Support limiting response size of .status(limit=N). .status changed from property to method.

v0.3.0 - Apr 25, 2023​

  • [Client] Fix for malformed OAuth tokens after login
  • [Client] Fix for committing large number of changes

v0.2.3 - Apr 21, 2023​

  • [Client] Adapt Author information to support service accounts

v0.2.2 - Apr 12, 2023​

  • [Client, Service] - Added support for Machine-to-machine access tokens

v0.2.1 - Apr 12, 2023​

  • [Client] Fix for intermittent httpx.RemoteProtocolError

v0.2.0 - Apr 11, 2023​

  • [Client] - Added chunk key cache to avoid repeated writes of the same chunk key.
  • [Client] - Fix for missing retries on POST requests
  • [Client] - Automatically refresh expired OAuth credentials
  • [Client] - Added support for virtual NetCDF3 datasets

v0.1.0 - Apr 7, 2023​

  • [Client] Support Zarr arrays without compression
  • [Client] Added JSON output to some CLI commands
  • [Client] Improved formatting of arraylake repo list when no repos are found
  • [Client] Fixed duplicate path in repo status

v0.0.10 - Mar 24, 2023​

  • [Client] Set username/email from authenticated user
  • [Client] Emit warning if ZARR_V3_EXPERIMENTAL_API is not set
  • [Client] Prevent writes when not on branch tip
  • [Client] Increase HTTP timeout in client
  • [Client] Raise error if chunkstore_uri is None

v0.0.9 - Mar 10, 2023​

  • [Client] Fixed Python string/repr formatting for MetastoreDatabase classes
  • [Client] Support bulk get_docs and add_docs

v0.0.8 - Feb 27, 2023​

  • [Service] Added support for org-wide access control
  • [Client] Added backoff and retries for failed requests to service
  • [Client] Fixed listing directories with trailing slashes
  • [Client] Improved styling and documentation CLI

v0.0.5 - Feb 14, 2023​

  • [Client] Refactor Arraylake client Python API

v0.0.4 - Feb 10, 2023​

  • [Documentation] - Added user guide tutorials from Jupyter notebooks

v0.0.3 - Feb 2, 2023​

  • [Client] Updated S3 endpoint configuration
  • [Service] Added health check to Arraylake service
  • [Client] Added documentation to CLI commands
  • [Documentation] Added Arraylake overview documentation