Is there any supported method of syncing ODK2?

Support
1

1. What is the problem? Be very detailed.

I can not work out how to sync ODK2 forms or data. I have been trialling it casually over the past 6 months, and was using the Google App Designer with aggregate, However that will no longer install, It seems the Google App Designer Page has changed, and the Aggregate v1.4.15. installer has not been updated for over 6 months.

Following the ODK2 documentation, it suggests using using a virtual machine, but that only works with Aggregate installs with OVA files, and those versions only work for transferring XMLs, which makes them useless for ODK2.

I tried using Tomcat following the instructions in the documentation, but that only works for more recent versions of Aggregate, which, again, only work for transferring XMLs, which makes them useless for ODK2.

There seems to be a serious problem with trying to produce two different versions of ODK, when only one is being supported. The online documentation and support software such as Aggregate is being changed to reflect ODK, but the incoming links form the ODK2 documntation are remaining the same. The result is that following the instructions for ODK2 no longer work once they link to a generic ODK page, such as the instructions for installing Aggregate, which start by instructing users to download the latest version of Aggregate from Github, even though the latest versions do not seem to work with ODK2, and then instructing the user to follow directions that do not work using the older versions of Aggregate that do sork with ODK2.

This is a serious barrier to anyone adopting ODK2. I am recently technically proficient, and I have now spent 3 days trying to follow the online instructions, only to find multiple dead ends.

So, is there any way that a user who isn't highly technically proficient can actually upload ODK2 forms to a tablet and download the files? If so can you please provide me with instructions.

Please do not point me to the online documentation. I have followed those instructions to multiple dead ends. They simply do not work because of the issues outlined above.

2. What app or server are you using and on what device and operating system? Include version numbers.

I am using ODK2. I am not using any server because I can not get any server to work.

3. What you have you tried to fix the problem?

I have followed every single page of the online documentation. They do not work because the ODK2 Aggregate instructions link into the ODK pages for "Installing Aggregate" and "Planning Your Aggregate Deployment", and those instructions only work for the latest version of aggregate, which only accept xml files, which do not work with ODK2.

4. What steps can we take to reproduce the problem?

Simply follow the online ODK2 documentation, starting with the "ODK Aggregate Tables Extension" page. You will find that you can not install a working version of aggregate following the instructions.

For example, if you follow the first options ("Installing on Tomcat") you will be directed to a page that tells you to "Install ODK Aggregate. Select the latest release for your operating system." But the latest version of aggregate linked to only works with xml files.

There seems ot be an urgent need to have two different series of instructions pages for the two different version of ODK. The two have now diverged so far that having links between them is not viable.

5. Anything else we should know or have? If you have a test form or screenshots or logs, attach below.

2

Hi @Blake! As you are new to the community, when you get a chance, please introduce yourself here. I’d also encourage you to add a real picture as your avatar because it helps build community!

I am sorry you are having a frustrating experience setting up ODK-X (formerly ODK2). The tools are transitioning away from using Aggregate to using Sync Endpoint, the documentation for that is here: https://docs.opendatakit.org/odk-x/sync-endpoint/

Hopefully that setup will be more successful!

Best,
Caroline

1 Like
3

Thanks for your feedback, but it's not really useful. The level of technical knowledge required to use Sync Endpoint is beyond what any normal user would have. Just following the link to Docker, I am told that I need to complete a tutorial before I can create a swarm. The tutorial informs me that it "requires three Linux hosts which have Docker installed and can communicate over a network." And that is just the tutorial for Swarm. The use of Docker itself involves several thousand words of extremely dense techno-jargon just to get through the "Getting started" section.

The level of technical ability (and time) required here is well outside the abilities of the vast majority of users of a form creation app. As I said above, I am reasonably technically proficient to the extent that I can code in several languages including Java and VB, design Sql and Access databases and set up a LAN, but this is well beyond me, even though I am sure I could learn this given several days.

In reality, this set up requires that in order to even trial ODK2, a user must be prepared to spend several days mastering Docker/Swarm, and then spend more time installing Sync Endpoint on all their remote machines, since there is no way that the remote staff could be expected to follow these instructions. In my case this would involve weeks or work and travel.

There is also no indication whatsoever of of how to actually use Sync Endpoint for transferring forms or data. There is information on admin tasks such as assigning users to groups, but nowhere do the instruction tell me how to install my newly created forms onto the tablet, or how to transfer collected data from the tablet to a a PC/server.

So i am not even convinced that, after I spend several days learning how to use Docker and configuring a new server, I am actually going to be able to do those two simple and essential tasks.

In short, there is now an impractically high barrier to most people and organisations using ODK2.

I'm not sure why there isn't a simple set of instructions for installing Sync Endpoint and using it to transfer files as there was for the other methods. But requiring users to train themselves in the use of two third party programs and then teach themselves how to transfer files in order to use ODK is not practical. It pretty much guarantees that ODK2 will never be adopted by any organisation without a dedicated IT department.

2 Likes
4

I do concur with Blake on the difficulty of ODK2 even to a technically sound user. My hope of having ODK2 to address the limitations of having to programme from scratch seem to be really fading for now, after 6months+ of trying this. Nomatter the route you took there is always a dead end.
After so much excitement about it, my organisation is starting to see the darkness ahead and we are starting to think around going back to programming again.

5

I am sorry to hear you are having a frustrating time with ODK-X! Are there any specific issues we can help explain or address?

6

We are in the process of reorganizing the documentation, could you let us know the issues you had?

7

Thank you for the response.

Maybe to start with I would be happy to know the following:

  1. ODK2 documentation which I have refers to Aggregate 1.4.5. The only download available currently is 2.x. So I did installations and designed the forms. However I reached a snag after all the hard work noticing that it doesn't have the ODK Aggregate Tables Extension.
  2. Is there documentation around ODK Sync Endpoint and can I install it on my organisational server.
  3. I am also worried that ODK2 requires that, after developing the forms, load them in application design you need to transfer them to a tablet/mobile device from which you can then sync back the forms before other distributed devices can access it. Is there any other way with this.
  4. I wonder if the whole setup is possible if being done by an average technical person. Everything I have read so far seem to be for highly proficient technical people.
    Regards;

Cephas.

8

  1. For older versions of aggregate, you just have to go to earlier releases on Github. Here includes 1.4.15 (I think this is what you mean, as this is what the docs mention): https://github.com/opendatakit/aggregate/releases?after=v1.7.1

  2. Here are the docs for ODK-X sync endpoint; we do recommend this as are not supporting Aggregate going forward. https://docs.opendatakit.org/odk-x/sync-endpoint/

  3. I am not quite sure what the issue here is; you push your survey to a tablet and reset the server from the admin. tablet, and that is how you get the survey on all the tablets. Can you explain where you are having a specific problem with this process?

  4. We successfully designed and implemented the ODK-X tools in collecting nationally representative data, and aren't programmers. Definitely we asked a lot of specific questions on the Forum as we did so, but were able to work through issues.

1 Like
9

For 3,
It is also possible to use Suitcase, instead of an Android device, to upload forms to the server.

1 Like
10

Thanks Caroline for your response.

  1. The GitHub link you provided does not have version 1.4.15 it only goes back up to 1.50.
  2. Does ODK Sync Endpoint conflict with ODKAggregate because same server I want to try this is already running some surveys on ODK1.
  3. is there room to do an online 1 on 1 with somebody from your team who has set this up before either via skype, zoom or any other platform?

Regards;

11

Hi @cmuchuchuti,

  1. The list of releases is long! Here is the specific 1.4.15: https://github.com/opendatakit/aggregate/releases/tag/v1.4.15_Production
  2. I don't think they would have any conflicts but I don't really know about setting them up on the same server.
  3. If you're on slack, that can be a good way to do real-time Q&A if people are available at the same time. I haven't set up sync-endpoint myself yet (did aggregate) but maybe @Jeff_Beorse or @linl33 can find a time to help?
1 Like
12

You can run Sync Endpoint and Aggregate on the same server. Just make sure they not listening on the same ports. Sync Endpoint only requires 1 port to listen for HTTP/HTTPS traffic.

2 Likes
13

If you have aggregate 1.4.15 it can support ODK and ODK-X at the same time. Aggregate uses a different table naming scheme so the data doesn't co-mingle.

1 Like