Unable to download submissions from two forms of encrypted project

[Khine]

1. What is the issue? Please be detailed.
I could not download the encrypted submissions, although the password was correct.
I used the project management encryption option for ODK central project. I have six forms in a single project and had a single password for the whole project. Four forms have been closed, and they had a small number of records (18-25). I could easily download the submissions (records) from closed forms. So, the password worked.

But when I tried to download the submissions of active forms, the other two forms having ~160 records, one each, it took longer to respond, and I got the error message which said, "something was wrong when requesting your data".

2. What steps can we take to reproduce this issue?

3. What have you tried to fix the issue?
I tried downloading submissions with similar data structures but a smaller number of records that were closed and stored in the same project. It worked well.
I used different browsers for checking - Chrome, Firefox and Microsoft Edge. The same problem happened in all three browsers.
I used VPN, but the error remained. The ODK central server I used is based in Sweden, and I am outside Sweden. So, I change the IP to Sweden using a VPN. But it did not work.
The internet speed without VPN- download = 13.39 Mbps and upload = 39.01 Mbps.

4. Upload any test forms or screenshots below.
The first screenshot shows the message after entering the password and clicking "CSV". After waiting 2-3 minutes, we got another error message, as shown in the second screenshot.
Sometimes, the first situation last longer, 5-10 minutes, and nothing happens.

[Matthew_White]

Hi @Khine! It sounds like the server is having trouble starting the download. A download or other request will usually time out if the server does not transmit anything for two minutes. Long downloads are normal, but it's unusual that a download wouldn't begin within two minutes.

Could you tell us what version of ODK Central you're using?

In some cases, media files can make downloads difficult. Are large media files or many media files collected as part of this form?

Form state (whether the form is open, closing, or closed) doesn't affect whether submissions can be downloaded, so I don't think it's likely that that's the issue.

As a temporary measure, you may be able to get your data out by using the submission filters to limit the number of submissions that you are downloading at once.

[Khine]

Hi @Matthew_White ,
Thanks a lot for the feedback. I use ODK Central v1.5.4.
I've just contacted my colleagues from Sweden, and they could manage to download the zip file.
Both forms worked well when I clicked "zip" for download too. The data file we have on ODK central contains 164 records and more than 394columns in one form and around 15 columns in another. We did not have any media files.
I could download five records by clicking the CSV file. However, CSV did not work when I tried to download the whole data file (165 records). So, is it correct to say that we need to download in a zip file when we have a large data set?
Thanks. Best wishes, Khine

[Matthew_White]

@Khine

164 records and more than 394columns

This seems like a small enough data size that it shouldn't cause the sort of issues you're seeing, especially since you're not collecting media files. When you download the .zip file, then unzip the .csv file, what is the size of the .csv file?

I'm realizing that what you describe sounds similar to this topic:

ODK Central v1.2 - Primary data table (.csv) download

1. What is the problem? Be very detailed. I cannot download data from an encrypted project when selecting the Primary data table (.csv) export. ~ 550 submissions / 322 columns [image] [image] Both All data and media files (.zip) and All data without media files (.zip) however work fine and ruODK functions that rely on the RESTful API retrieve the complete data set without any problem. 2. What app or server are you using and on what device and operating system? Include version numbers. ODK …

If possible, it'd be very helpful if you could share some of your server logs with us. (You can send them via a direct message to me or anyone else on the core team.) To retrieve those logs, first trigger the issue by attempting to download the CSV file, then waiting for the error. Next, follow these steps to retrieve the logs from that time period for the service and nginx containers. I suspect that anyone on your team who tries to download the unzipped CSV file will encounter the same issue you're seeing.

[Matthew_White]

@Khine

Both forms worked well when I clicked "zip" for download

Hi @Khine! I just wanted to check: after you unzipped the .zip file, did you view the .csv file? Did the .csv file open without issue, and did it seem to contain all the records?

[Khine]

@Matthew_White

This seems like a small enough data size that it shouldn't cause the sort of issues you're seeing, especially since you're not collecting media files. When you download the .zip file, then unzip the .csv file, what is the size of the .csv file?

Yes. You are right. It is not that big. It is only 140 KB.
For the second part, does it need to be done by the server's host?

[Khine]

The CSV file within the zip file worked perfectly. It containd all the data that we have on the server.

[Matthew_White]

Thank you for that information!

@Khine

For the second part, does it need to be done by the server's host?

It can be done by anyone who has access to the server's command line prompt. That might be whoever first set up the server or whoever continues to keep the server updated.