Direct backup of Central using pyODK or python

Development
1

1. What is the issue? Please be detailed.
I am trying to trigger a direct backup on self hosted ODK Central v1.5.3
Using Jupyter-Lab I can connect to the server (so have correctly configured the environment) but I can't get the correct response that will download the backup.

Here is my code:

from pyodk.client import Client
client = Client().open()
r = client.post('/backup', json={"passname": "[my-pass]"})
r.content

I get a memory error (seems to fall over at: "Convert a json object to a bytes")
I tried using r.json() instead but this doesn't work either.
In both cases the server appears to be working hard, so it has probably triggered a backup, but the issue might be that the data is not being sent in a format that python is expecting?

As an alternative, I had a go at using 'straight' python code as provided in the APIary (which doesn't work as is because urllib2 isn't part of python 3). With some minor changes below, this is what I tried:

from urllib.request import urlopen, Request
values = """
  {
    "passphrase": "[my-pass]"
  }
"""
headers = {
  'Content-Type': 'application/json'
}
request = Request('https://[mysite]/v1/backup', data=values, headers=headers)

response_body = urlopen(request).read()
print (response_body)

Unfortunately this fails with:

TypeError: POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.

I am a little stuck. But it is probably a simple problem that I can't see past! As usual, a little knowledge in the wrong hands is quite dangerous.

Any help gratefully received...

2

Hi @seewhy,

I am not able to reproduce the memory error you are getting, but I have only tested your code on a test server with limited data (backup < 600 MB).

Two comments from my side:

The following code successfully allowed me to retrieve a ~8 GB backup.

backup_passphrase = "my_secret_passphrase"

from pyodk.client import Client
with Client(config_path=".pyodk_config.toml") as client:
    r = client.post('/backup', json={"passphrase": backup_passphrase})
    if r.status_code == 200:
        with open('backup_test.zip', 'wb') as f:
            f.write(r.content)

image

2 Likes
3

Hi @Thalie
Well, I knew it would be simple! I couldn't see for looking at it - no idea how I translated passphrase to passname, but I'm glad you spotted that.

Actually, I think the key to moving forward here is

I had tried without a passphrase and got nowhere with the r.content (and might then have retyped it as passname!). So that exposes my lack of familiarity with python for how to stream the data into a file, which is what it appears to have done now... And that is in the same folder as my .toml file...

To make this a very useful script, I just need to work out how get feedback from the server that the file has completed...

Thanks - hopefully this worked example might help others too :slight_smile:

1 Like