for some personal needs I just created a docker image that runs Aggregate over PostgreSQL10.
It has to be improve - I'm not a docker expert - but it works fine. For example It would be cool to test and automate the database creation.
I will soon integrate it within a docker-compose beside PostgreSQL and Lizmap websig (https://github.com/3liz/lizmap-web-client) to show how to trigger the collected data transfer from aggregate to an other (spatial) database and show those data over a map.
It reminds me that we do have a WIP PR for docker support on Aggregate that maybe we can work together on? If we can get put our effort into a single Docker image, it'd be easier maintenance for everyone
Of course @yanokwa ! --)
I would be glad to help with my little docker knowledge and it would be useful and meaningful to have a kind of official aggregate docker.
I started a branch in my fork with a Docker Compose setup that spìns a container with PostgreSQL, configures the DB, and spins another container with a Tomcat8 where it deploys a freshly built Aggregate. You can check it out here: https://github.com/ggalmazor/aggregate/tree/docker/docker
We didn't have the time and energy to put a lace on it and merge it on master but if there's interest in this I guess we can continue pushing this. Our idea was to create a new Gradle task to automatize setting up the Docker Compose setup, by exploding the WAR file into the right folder.
Building a single Docker image can be a maintenance pain, but I got promising results by using the postgres:9.6-alpine as a base. Then you need to install Tomcat8 and deploy Aggregate, which results on an image round to 250MB size.
Docker Compose is much simpler to configure but it's less portable. When we did this, there were no user-friendly GUIs that helped with the local deployment of a Docker Compose setup. This is much easier with single Docker images, which the user can one-click deploy with a variety of GUI tools.
In my recent case I just nee ded aggregate to run over an existing database so I did it simply.
2 years ago for the first course we gave, we create a full server on a raspberry pi with :
PostgreSQL/PostGIS (because we collect geo data),
Aggregate
And Lizmap websig server (base on QGIS sever)
People went back home with a fully ready-to-use (geo)form server
A docker-compose would have been really easier.
By the we, if we maintain a docker-compose, we have to maintain the sub-containers and their dockerfile. So it would not be so hard to extract and run only on of the services we need.
The thing I like most about the Docker Compose setup is that I didn't have to build any Docker image. I just used stock postgres and tomcat images and deployed software and data into them. That lowers the maintenance costs to a bare minimum and upgrading those containers to more up-to-date versions should also be easier than building an image.
@ggalmazor Now that we have the Aggregate VM, I think we've got a good solution for folks who want a turnkey install of Aggregate.
The Docker compose setup built by Gradle would be true to the nature of Docker and it sounds like it'd be useful to @mathieubossaert. Let's try to get it shipped after we merging the pending Aggregate PRs.
Using the installer, running it in unattended mode using some predefined options
Unzipping the resulting WAR file
Unzipping /WEB-INF/lib/ODKAggregate-settings.jar
Replacing jdbc.username and jdbc.password lines in jdbc.properties file to point to custom environment variables
Rezipping ODKAggregate-settings.jar
Copying the now-extracted WAR to Tomcat's webapps directory
So far it has worked nicely, though I have not yet tested the major functionalities yet. My questions are mostly for sanity check and for some design questions for my further development on this. So...
I have no prior experience with Spring/Java/Tomcat so I would like to know if Aggregate or Tomcat is doing some magic when it auto-unpacks WAR files. I've done my research and what I can surmise is that it's practically just unzipping it in the webapps directory.
Does Aggregate store [persistent] state on the filesystem or is it storing everything in the database?
If it does store in the filesystem I would really like to know the directories. It is important so that I can properly mount docker volumes for them.
Can I safely assume that Aggregate will behave properly if configuration is changed? Maybe just for some reasonable ones like the database username and password.
I may have some other questions that I'm forgetting so I'll just add them to the thread.
@markgarcia I'm combing your post with the ongoing topic where we are discussing Docker. Can you review the discussion and see if @ggalmazor's approach and my proposed timeline would work for you?
Here is the very helpful and genious work from Julien Ancelin I used to create my raspberry pi server that I mentioned few messages before : https://github.com/jancelin/geo-poppy
I just have had to add aggrregate.
If I understand your Dockerfile correctly, you are using Aggregate with an external PostgreSQL database. If this is correct I think you could improve it by doing two things:
First, you could avoid having to run the install by directly using the Gradle build result. We're already doing similar stuff for the installer and packer tasks, and it should be "easy" to repurpose one of them for the Docker image. The Gradle task would take care of producing all the conf files for a specific Docker setup.
Second, you could consider not building a Docker image at all. You could use a stock tomcat:8-alpine image and mount a volume with the Aggregate build files into the container's /var/lib/tomcat8/webapp/ROOT path
This would make your setup less portable but it would make much easier changing confs and would take care of any writing to filesystem needs.
@yanokwa No problem. So you're planning to officially support dockerized deployment?
@ggalmazor Yes, as I just want to use a single shared Postgres service/instance for both Aggregate and our own website.
I did try to use a gradle build stage (basing off https://hub.docker.com/_/gradle/) but I hit some snags and I was afraid of doing something wrong (as said I don't have prior experience with the stack). The installer looks to be the canonical way of deployment and is ready to use without me building anything so I just used it. But yes I do understand that doing a gradle build and tweaking the configuration there should be the goal.
What I really just need is a way to reconfigure Aggregate without reinstalling / rebuilding the WAR file. I think using environment variables is the most straightforward way to do this. I do realize that this will have far reaching implications on how Aggregate do and use configuration hence the questions I presented. I want to approach this first without changing anything in the Aggregate source code if possible (i.e. making a fork).
A docker deployment of Aggregate would be very helpful for the Red Cross, as I think we could then easily incorporate Aggregate into our POSM stack install.
Hi everyone,
First of all very good initiative to have created odk aggregate docker image with postgresql. I am a new user of ODK and I'm a fan.
I am interested to use this image as part of my academic background on a raspberry pi.
I'm just starting with docker so I don't understand all the details to properly install the image with docker compose (https://github.com/opendatakit/aggregate/blob/master/docs/build-and-run-with-docker-compose.md)
On the SD card of the raspberry pi, i flashed the image of hypriot (http://blog.hypriot.com/downloads/), but then I'm lost, I don't know what are the other steps.
I don't understand how to use: Build the setup with ./gradle clean docker ComposeBuild -xtest -PwarMode = complete
Do you have a little tutorial to know all the steps, please?
Thank you in advance for your help.
Regards,
