@MatthewMac did we manage to try this solution so we can test if it fixes issue our end?
@dr_michaelmarks @issa I added the extra lines from the diff into odk.conf but the download is still failing with a Network error. Only now after 120 seconds.
well, that's extraordinarily strange. how sensitive is the data? it would help if i had it to work with directly.
It seems specifically a problem with a specific survey I think....
I have a different project with 200+ forms running on the same server no problem.
baseline_household_survey.xlsx (16.6 KB)
baseline_household_survey.xml (29.9 KB)
Added points for future (things I mentioned at Convening)
- Turn off decryption for projects (obviously wont decrypt existing data)
- 'Better' control of the decryption (i.e not just weak passphrases)
- Question is how on the fly-decryption will work on when form number is high
- Resume download from last time point
1 Like
disabling decryption:
it's interesting to me that it's obvious it wouldn't decrypt existing data. i had planned on automatically (with good messaging of course) decrypting existing data on project encrypt disable. what's the case you envision where you'd want to disable but leave existing records encrypted? it sounded like the main reason you wanted disable was because it made trying out project encryption a lower commitment/barrier, in which case you'd only have test data right?
i think it's actually not too hard to provide the option, but i do want to be sure i understand the reasoning on your side to inform our decision making.
something other than passphrases:
my logic behind starting with passphrases was that pem files are really hard to handle safely and securely, and people are likely to do really insecure things with them (email them around, etc). certainly larger organizations with good infrastructure and practices like your own are more equipped to establish the right procedures.
passphrases at least can be stored in password managers, etc which do a good job of balancing user ease and security.
i think it makes sense to pursue something more secure, though i also wonder whether in this case it makes more sense to support something like a yubikey which is a more automatic and guaranteed-secure process than throwing a pem file around.
in either case, central supports the old aggregate encryption, so you can just continue your current practices for now.
on the fly decryption performance:
i don't really see a reason this would necessarily be bad. if it isn't good now we can improve it.
resume download:
with the REST api, building a simple tool that does this should be pretty straightforward. briefcase already does it. adding windowing to the odata api (and possibly the csv api) should also not be all that difficult. just a matter of scheduling.
2 Likes
We also need to be able to set default collect settings via central so that we can then send out QR codes that don't change collect settings back to default. I've been trying to get something working in Python but i'm not having any luck at the moment
My cheap solution is to scan the QR code, then adjust settings in Collect, then export those settings again via QR code. That second QR code and some written instructions live in our access restricted Wiki which my local coordinators can read and scan from.
That's what i'm doing at the moment but i think it would be more scalable if you could set basic settings in Central for all QR codes (i.e everyone uses forward and back buttons rather than swipes, send on finalise and check for updates every 24 hours) and then email directly from central
