Modal

Modal is a serverless platform that allows you to develop code locally and then spawn cloud resources with which to execute the code.

Modal let's you specify the necessary computing environment for your code with modal.Image. With this, you can specify environment variables related to your Arraylake credentials, allowing you to connect to and use Arraylake in your Modal workflows executed on the cloud. See the Modal guide on Custom containers for more detail. The script below demonstrates writing and executing a Modal function that connects to Arraylake, writes data and commits it to a Repo. If you haven't already, make sure Modal is installed in your local environment with pip install modal.

import modal
from modal import Image

#create image to define environment for modal app
al_image = (
    Image.debian_slim(python_version='3.11') #specify your python version
    .pip_install('arraylake') #specify any needed pip packages
    .env({'ARRAYLAKE_TOKEN':'ema_XXXXXX'}) #pass AL token as environment variable
)

app = modal.App('earthmover_example') #create a modal App

#decorate a function that uses Arraylake with the modal app
@app.function(image=al_image) #pass image to app function
def write_data():

    import arraylake as al
    import numpy as np

    client = al.Client()
    repo = client.get_or_create_repo('earthmover/compute_integration_examples')
    toy_group = repo.root_group.create_group('toy_group')
    zeros_array = toy_group.create(
        'zeros',
        shape=100, chunks=10, dtype='f4', fill_value = 0
    )
    repo.commit('Created a group and array with zeros')

Execute the script on resources spawned by modal by running modal run my_script.py

In the Arraylake web app, we can see that our function executed and wrote data to the repo.