Getting a functioning ODK Aggregate server that works off the internet

I thought I had a reasonable solution with the Nafundi 1.4.7 VM - but then
it turns out there's a bug with 1.4.7 where you can't set up a filter and
export metadata (and I very much need timestamps).

I've now also tried installing Aggregate using the 1.4.5 installer to
generate a WAR, using both postgresql and mysql setups. Strangely, in both
cases, I just get a madly reloading page that blinks this on and off, and
the catalina.out file grows at a rate of 10,000's of lines per minute. A
screenshot of the blinking imagery is attached.

Right now it seems that my best bet is to stick with 1.4.7, and then just
write SQL queries myself to get my data. But I figured I'd check here if
anyone had a better idea.

Best,
Dav

image

Dav,

Would a 1.4.5 VM solve the problem? If so, glad to generate one for you.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Fri, Aug 7, 2015 at 9:48 AM, davclark@berkeley.edu wrote:

I thought I had a reasonable solution with the Nafundi 1.4.7 VM - but then
it turns out there's a bug with 1.4.7 where you can't set up a filter and
export metadata (and I very much need timestamps).

I've now also tried installing Aggregate using the 1.4.5 installer to
generate a WAR, using both postgresql and mysql setups. Strangely, in both
cases, I just get a madly reloading page that blinks this on and off, and
the catalina.out file grows at a rate of 10,000's of lines per minute. A
screenshot of the blinking imagery is attached.

Right now it seems that my best bet is to stick with 1.4.7, and then just
write SQL queries myself to get my data. But I figured I'd check here if
anyone had a better idea.

Best,
Dav

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I'm not certain, but I think that would solve my problems.

I'm deploying on Monday, though, so I'm guessing that wouldn't be done in
time! I'll likely end up going the existing VM route, I can always just get
the data out of the database myself.

D

··· On Aug 7, 2015 11:59 PM, "Yaw Anokwa" wrote:

Dav,

Would a 1.4.5 VM solve the problem? If so, glad to generate one for you.

Yaw

Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.

On Fri, Aug 7, 2015 at 9:48 AM, davclark@berkeley.edu wrote:

I thought I had a reasonable solution with the Nafundi 1.4.7 VM - but
then
it turns out there's a bug with 1.4.7 where you can't set up a filter and
export metadata (and I very much need timestamps).

I've now also tried installing Aggregate using the 1.4.5 installer to
generate a WAR, using both postgresql and mysql setups. Strangely, in
both
cases, I just get a madly reloading page that blinks this on and off, and
the catalina.out file grows at a rate of 10,000's of lines per minute. A
screenshot of the blinking imagery is attached.

Right now it seems that my best bet is to stick with 1.4.7, and then just
write SQL queries myself to get my data. But I figured I'd check here if
anyone had a better idea.

Best,
Dav

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/N9x5moZWv0E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

The team at Nafundi moves pretty quickly. See your inbox for a link.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Fri, Aug 7, 2015 at 10:51 AM, Dav Clark davclark@berkeley.edu wrote:

I'm not certain, but I think that would solve my problems.

I'm deploying on Monday, though, so I'm guessing that wouldn't be done in
time! I'll likely end up going the existing VM route, I can always just get
the data out of the database myself.

D

On Aug 7, 2015 11:59 PM, "Yaw Anokwa" yanokwa@nafundi.com wrote:

Dav,

Would a 1.4.5 VM solve the problem? If so, glad to generate one for you.

Yaw

Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.

On Fri, Aug 7, 2015 at 9:48 AM, davclark@berkeley.edu wrote:

I thought I had a reasonable solution with the Nafundi 1.4.7 VM - but
then
it turns out there's a bug with 1.4.7 where you can't set up a filter
and
export metadata (and I very much need timestamps).

I've now also tried installing Aggregate using the 1.4.5 installer to
generate a WAR, using both postgresql and mysql setups. Strangely, in
both
cases, I just get a madly reloading page that blinks this on and off,
and
the catalina.out file grows at a rate of 10,000's of lines per minute. A
screenshot of the blinking imagery is attached.

Right now it seems that my best bet is to stick with 1.4.7, and then
just
write SQL queries myself to get my data. But I figured I'd check here if
anyone had a better idea.

Best,
Dav

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/N9x5moZWv0E/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Most excellent - thanks to you and the team at Nefundi! I've downloaded the
VM and will report back.

D

··· On Saturday, August 8, 2015 at 1:14:22 AM UTC+9, Yaw Anokwa wrote: > > The team at Nafundi moves pretty quickly. See your inbox for a link. >

Well, I'm sad to report that you've replicated my problem. I'm not sure
exactly what the issue is - but the 1.4.7 VM (from Nafundi) works (with the
filter / metadata bug).

All of the 1.4.5 VMs result in this strange redirect loop. The specific
environment I'm in:

  • The ODK server is a VM and has no access to the internet (for all 4
    variations on an ODK server I've tried
  • The ODK server is reachable via the local router
  • If I have internet (i.e., via a separate network interface), the web
    management console works fine from my laptop for 1.4.7, but 1.4.5 does a
    redirect loop (lots of page reloads that look like the above)
  • If I don't have internet, the 1.4.7 server takes a very long time to
    load. But if I hit the "stop" button in the browser, the page then displays
    fine.
  • Tablets are also connected only to the local router, and accessing
    Aggregate via Collect works fine for 1.4.7 (I don't think I can send a form
    without uploading the XForm first via the web console, right?)

I'm open to ideas for figuring this out! But I suspect a better use of
development time is to get to a 1.4.8 (or similar) that fixes the metadata
export bug.

D

Actually - the behavior is incredibly strange. I tried again:

  • first boot of a fresh 1.4.5 VM, I NAT'd the VM to my wifi internet
    connection, and let it finish booting. The VM reports a URL that I couldn't
    access (unsurprising).
  • Boot again, this time using DHCP to the local router. I get a random
    IP address, and I can connect to that!
  • If I set a static IP address, then I get the redirect loop

(static IP works fine with 1.4.7)

So, closer to what the issue is, but no idea how to fix it. Clearly, I am
getting routed to the VM on the static IP address.

Note - I am in a strange enterprise environment in which I cannot reliably
assign IPs on the router. The router apparently supports it, but none of us
are smart enough to do so (I understand networks, and my Japanese
colleagues can read the web interface to the router, and we found some
stuff online about logging into it via telnet, but...)

D

I have figured out how to get the desired connection... I can leave the
device configured to dhcp, but add the following line in
/etc/network/interfaces:

post-up ip addr add 192.168.11.150/24 dev eth0

(This address is outside the range managed by DHCP, and the desired IP is
set up as a secondary address. I have no idea why 1.4.5 requires DHCP when
1.4.7 doesn't.)

Now I can connect at the above address. However, I am still unable to
export metadata. Am I missing something? I can check the metadata option to
see it displayed in html, and I can create a filter (it shows up with a big
red X next to it), but I am unable to select a filter through the drop down
or in the export interface...

I'm also unable to make a title. Strangely, the "title" field is a
greyed-out select field with column names in the filter creation interface.
I'm guessing this is the core issue.

So distressing.

D

Dav,

Sounds like your strange network env is not helping. The VM tries to
grab an IP via DHCP and Aggregate uses that IP as the hostname. As far
as I know, there are no stack differences in VM 1.4.5 vs VM 1.4.7.

At this point, might be easier to do a manual install on your computer
(e.g., no VM) to reduce the levels of abstraction. The caveat is that
you have to have an globally-accessible IP for your machine.

Yaw

··· -- Need ODK services? http://nafundi.com provides form design, server setup, professional support, and software development for ODK.

On Sat, Aug 8, 2015 at 1:35 AM, davclark@berkeley.edu wrote:

I have figured out how to get the desired connection... I can leave the
device configured to dhcp, but add the following line in
/etc/network/interfaces:

post-up ip addr add 192.168.11.150/24 dev eth0

(This address is outside the range managed by DHCP, and the desired IP is
set up as a secondary address. I have no idea why 1.4.5 requires DHCP when
1.4.7 doesn't.)

Now I can connect at the above address. However, I am still unable to export
metadata. Am I missing something? I can check the metadata option to see it
displayed in html, and I can create a filter (it shows up with a big red X
next to it), but I am unable to select a filter through the drop down or in
the export interface...

I'm also unable to make a title. Strangely, the "title" field is a
greyed-out select field with column names in the filter creation interface.
I'm guessing this is the core issue.

So distressing.

D

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

You may also have an issue with cached content in your browser.

With each upgrade to ODK Aggregate, a new set of Javascript support
libraries is built and must be downloaded to your browser. While I have not
seen this behavior with 1.4.5 and 1.4.7, I have seen problems when the
browser holds a version of these libraries for a different build of ODK
Aggregate.

If you can, clear your browser cache.

Changing the IP address effectively causes your browser to maintain a
separate copy since the server URL is now different.

··· On Sat, Aug 8, 2015 at 7:35 AM, Yaw Anokwa wrote:

Dav,

Sounds like your strange network env is not helping. The VM tries to
grab an IP via DHCP and Aggregate uses that IP as the hostname. As far
as I know, there are no stack differences in VM 1.4.5 vs VM 1.4.7.

At this point, might be easier to do a manual install on your computer
(e.g., no VM) to reduce the levels of abstraction. The caveat is that
you have to have an globally-accessible IP for your machine.

Yaw

Need ODK services? http://nafundi.com provides form design, server
setup, professional support, and software development for ODK.

On Sat, Aug 8, 2015 at 1:35 AM, davclark@berkeley.edu wrote:

I have figured out how to get the desired connection... I can leave the
device configured to dhcp, but add the following line in
/etc/network/interfaces:

post-up ip addr add 192.168.11.150/24 dev eth0

(This address is outside the range managed by DHCP, and the desired IP is
set up as a secondary address. I have no idea why 1.4.5 requires DHCP
when
1.4.7 doesn't.)

Now I can connect at the above address. However, I am still unable to
export
metadata. Am I missing something? I can check the metadata option to see
it
displayed in html, and I can create a filter (it shows up with a big red
X
next to it), but I am unable to select a filter through the drop down or
in
the export interface...

I'm also unable to make a title. Strangely, the "title" field is a
greyed-out select field with column names in the filter creation
interface.
I'm guessing this is the core issue.

So distressing.

D

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Thanks Mitch!

You may also have an issue with cached content in your browser.

...

If you can, clear your browser cache.

That was the issue with the redirect loop! Perhaps we can add some
versioning to the js files?

However, all is not sorted out... I wrote:

I'm also unable to make a title. Strangely, the "title" field is a
greyed-out select field with column names in the filter creation
interface.

This issue remains. I checked all 3 major browsers on OS X. Any idea why I
wouldn't be able to set a title? It seems the title field of the form is
populated with column names for some reason, and they are greyed out /
inactive. So, I can't make a title, and so can't select the filter for
export.

D

··· On Aug 9, 2015 1:00 AM, "Mitch Sundt" wrote:

Unfortunately, the naming / versioning of the js files is decided by the
GWT library, not me.

I haven't had time to look into the problems with the filter UI.

The GWT library was updated recently; I suspect that is the cause.

··· On Sun, Aug 9, 2015 at 5:53 PM, Dav Clark wrote:

Thanks Mitch!

On Aug 9, 2015 1:00 AM, "Mitch Sundt" mitchellsundt@gmail.com wrote:

You may also have an issue with cached content in your browser.

...

If you can, clear your browser cache.

That was the issue with the redirect loop! Perhaps we can add some
versioning to the js files?

However, all is not sorted out... I wrote:

I'm also unable to make a title. Strangely, the "title" field is a
greyed-out select field with column names in the filter creation
interface.

This issue remains. I checked all 3 major browsers on OS X. Any idea why I
wouldn't be able to set a title? It seems the title field of the form is
populated with column names for some reason, and they are greyed out /
inactive. So, I can't make a title, and so can't select the filter for
export.

D

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com