Configuration Client() on Heroku

Support
1

Hi all,

I want to run pyODK scripts on Heroku for automated processing (e.g. daily uploading of entities to our entity list, daily downloads etc). However, still figuring out how I can get the ODK Central client running since Heroku works with environmental variables, not with config files. With pyODK you open a client using the files: config.toml and cache.toml (see below).

Client(config_path="config.toml", cache_path="cache.toml")

Since the config.toml file has a table name [Central]:

[central]
base_url = "https://www.example.com"
username = "my_user"
password = "my_password"
default_project_id = 123

I decided to convert the .toml syntax to json:

{"Central" : {"base_url" : "https://www.example.com", "username" : "my_user", etc}}.

My idea was then to replace the settings (values) of the json with environmental variables from Heroku. However, when testing this (starting a Client() with a json structure instead of a .toml file):

client = Client(config_path={"Central": {"base_url": "https://organisation.getodk.cloud", "username": "xxxxxxxx",
"password" : "xxxxxxx", "default_project_id" : 1}}, cache_path="xxxxxxx")

I still receive errors regarding the configuration file path.

I realise that this might be an issue outside the scope of ODK, but maybe there are some thoughts or experiences how I could make this work on Heroku without using the .toml files(?) Thanks for the help!

2

I think Heroku can still write to local files during the lifecycle of the app. So as part of your app initialisation, before you invoke pyodk, grab the environment variables and put them into files. e.g.

import os

# Retrieve environment variables
base_url = os.getenv("CENTRAL_BASE_URL", "https://www.example.com")
username = os.getenv("CENTRAL_USERNAME", "my_user")
password = os.getenv("CENTRAL_PASSWORD", "my_password")
default_project_id = os.getenv("CENTRAL_DEFAULT_PROJECT_ID", "123")

# Define the file content
file_content = f"""[central]
base_url = "{base_url}"
username = "{username}"
password = "{password}"
default_project_id = {default_project_id}
"""

# Define a writable path (/app/tmp is a writable directory on Heroku)
file_path = "/app/tmp/pyodk_config.ini"

# Create the directory if it doesn't exist
os.makedirs(os.path.dirname(file_path), exist_ok=True)

# Write the configuration to the file
with open(file_path, "w") as file:
    file.write(file_content)

Then use it:

from pyodk import Client
client = Client(config_path="/app/tmp/pyodk_config.ini", cache_path="/app/tmp/pyodk_cache.ini")

There might be better support for this sort of thing once this issue is addressed.

2 Likes
3

Great feedback. I indeed read that it is still possible to create a temp file on Heroku but still hadn't figured how to put this into practice. This helps a lot. I'm going to try and let you know. Many thanks!

4

Hi Lindsay, your solution worked! Excellent. I copy paste your solution in my script and it worked straight away. Many thanks for this concrete feedback!

1 Like