Redundant copies of ODK Aggregate

Hello guys,

I have ODK Aggregate running fine on our production server that works on a paid subscription basis under a PaaS plan.
I also have a very capable and high spec'ed PC that is connected to the Internet via high speed fiber connection in my office.
I have set up this PC to backup (on a daily basis) all my ODK Aggregate data collected on my production server.
So far so good and everything is working fine. One server collects all my data, and the other backs it all up.

Now I wish to extend my PC's functionality a little further by making it more useful.
I want to setup ODK Aggregate on this PC and have it function as a backup ODK Aggregate server, such that when my production server goes down, I can continue collecting data and submitting forms with the ODK Aggregate on my high-end PC.

I fully understand that no two ODK Aggregate instances can share the same platform configuration addresses, but I was hoping I could set up a server failover solution of some sort and use a domain name to reference one when the other goes down.
I also understand that ODK Collect's platform settings would have to be changed whenever we need to redirect the data to the backup ODK Aggregate server if the production server ever goes down. I was hoping to use this domain name to configure the app.

So my questions are:

  1. Can a server failover solution work with ODK Aggregate?
  2. Can I use a domain name to reference my ODK Aggregate production server and have my PC serverr use this same domain name in the event that my production server goes down (under the failover solution)?
  3. Is there a simpler remedy out there where ODK Aggregate can be functionally distributed that I'm just not aware of?

Please let me know, and sorry for the long post.

Regards,
ramadhan

If we were concerned about availability, we would configure multiple web
servers running ODK Aggregate behind a load balancer all interacting with
the same (single) high-availability database. We would not replicate the
databases and do live fail-over across them.

IF you did try to set this up, it is likely that ODK Briefcase would have
difficulty communicating with the 'backup' server because both of the
servers would need to be configured to work with the same hostname in order
for their interactions with ODK Collect to work. As long as the 'live'
server is up, it is likely that ODK Briefcase would only talk to it.

··· On Sun, Oct 18, 2015 at 11:23 PM, wrote:

Hello guys,

I have ODK Aggregate running fine on our production server that works on a
paid subscription basis under a PaaS plan.
I also have a very capable and high spec'ed PC that is connected to the
Internet via high speed fiber connection in my office.
I have set up this PC to backup (on a daily basis) all my ODK Aggregate
data collected on my production server.
So far so good and everything is working fine. One server collects all my
data, and the other backs it all up.

Now I wish to extend my PC's functionality a little further by making it
more useful.
I want to setup ODK Aggregate on this PC and have it function as a backup
ODK Aggregate server, such that when my production server goes down, I can
continue collecting data and submitting forms with the ODK Aggregate on my
high-end PC.

I fully understand that no two ODK Aggregate instances can share the same
platform configuration addresses, but I was hoping I could set up a server
failover solution of some sort and use a domain name to reference one when
the other goes down.
I also understand that ODK Collect's platform settings would have to be
changed whenever we need to redirect the data to the backup ODK Aggregate
server if the production server ever goes down. I was hoping to use this
domain name to configure the app.

So my questions are:

  1. Can a server failover solution work with ODK Aggregate?
  2. Can I use a domain name to reference my ODK Aggregate production server
    and have my PC serverr use this same domain name in the event that my
    production server goes down (under the failover solution)?
  3. Is there a simpler remedy out there where ODK Aggregate can be
    functionally distributed that I'm just not aware of?

Please let me know, and sorry for the long post.

Regards,
ramadhan

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

A comment on the possible ODK Briefcase issue - if both servers have static
IPs (or known IPs) then you should be able to access them via Briefcase, as
you're skipping the hostname resolution step.

Regards,
Andrew

··· On Monday, 19 October 2015 19:24:49 UTC+2, Mitch Sundt wrote: > > If we were concerned about availability, we would configure multiple web > servers running ODK Aggregate behind a load balancer all interacting with > the same (single) high-availability database. We would not replicate the > databases and do live fail-over across them. > > IF you did try to set this up, it is likely that ODK Briefcase would have > difficulty communicating with the 'backup' server because both of the > servers would need to be configured to work with the same hostname in order > for their interactions with ODK Collect to work. As long as the 'live' > server is up, it is likely that ODK Briefcase would only talk to it. > > > On Sun, Oct 18, 2015 at 11:23 PM, <washbne...@gmail.com > wrote: > >> Hello guys, >> >> I have ODK Aggregate running fine on our production server that works on >> a paid subscription basis under a PaaS plan. >> I also have a very capable and high spec'ed PC that is connected to the >> Internet via high speed fiber connection in my office. >> I have set up this PC to backup (on a daily basis) all my ODK Aggregate >> data collected on my production server. >> So far so good and everything is working fine. One server collects all my >> data, and the other backs it all up. >> >> Now I wish to extend my PC's functionality a little further by making it >> more useful. >> I want to setup ODK Aggregate on this PC and have it function as a backup >> ODK Aggregate server, such that when my production server goes down, I can >> continue collecting data and submitting forms with the ODK Aggregate on my >> high-end PC. >> >> I fully understand that no two ODK Aggregate instances can share the same >> platform configuration addresses, but I was hoping I could set up a server >> failover solution of some sort and use a domain name to reference one when >> the other goes down. >> I also understand that ODK Collect's platform settings would have to be >> changed whenever we need to redirect the data to the backup ODK Aggregate >> server if the production server ever goes down. I was hoping to use this >> domain name to configure the app. >> >> So my questions are: >> >> 1. Can a server failover solution work with ODK Aggregate? >> 2. Can I use a domain name to reference my ODK Aggregate production >> server and have my PC serverr use this same domain name in the event that >> my production server goes down (under the failover solution)? >> 3. Is there a simpler remedy out there where ODK Aggregate can be >> functionally distributed that I'm just not aware of? >> >> Please let me know, and sorry for the long post. >> >> Regards, >> ramadhan >> >> -- >> -- >> Post: opend...@googlegroups.com >> Unsubscribe: opendatakit...@googlegroups.com >> Options: http://groups.google.com/group/opendatakit?hl=en >> >> --- >> You received this message because you are subscribed to the Google Groups >> "ODK Community" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to opendatakit...@googlegroups.com . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Mitch Sundt > Software Engineer > University of Washington > mitche...@gmail.com >

Hi Mitch,

Thank you so much for the quick response! I truly appreciate it!
I just needed some clarification regarding your proposed solution, I'll just number them below:

  1. Clustering servers is something that I've never done before. But I would like to know, are there any cost implications involved? Would I need to purchase additional hardware, software, and licenses to make this work?
    I did some looking up, and this is what I was able to gather (please excuse my lengthy post):

A load balancer is essentially a (physical) server that monitors the traffic and diverts the load evenly to the server cluster. As it sits right between the clients and the servers, it becomes the initial receiver of all traffic so as to determine where to relay it to.
In my case, my cluster would be comprised of my PaaS production server, and my high-end PC server - each running their own instance of ODK Aggregate.
However, my load balancer would be another server - hence my question on if I should purchase another machine, and if so, what additional software I should get (Windows Server 2008 R2 perhaps? I did some digging and found that Windows Server 2008 R2's Server Manager can accommodate multiple web server nodes in load balancing).

  1. My second question is: Where were you suggesting I host my database? See, as it stands now, both the database and ODK Aggregate run on the same production server (PaaS). But according to your recommendation, if I do successfully set up a load balancing system, in the event of the production server going offline, so would my database. Then even though my backup server kicks in, it still wont be able to access the database - because it's not redundant.
    Were you suggesting migrating the database to another location away from the production server where I know I can always access it?

  2. I had a look at the ODK support page, and there is a mention on load balancing under Multi-host and Load-balancing Considerations here:
    https://opendatakit.org/use/aggregate/tomcat-install/

However, since I've never set up load balancing before, I don't find this information particularly comprehensive enough. Could you recommend some resources that I can find that may be able to assist me?

Sorry for the long post, but your time is greatly appreciated sir.

Regards,
Ramadhan

··· On Monday, October 19, 2015 at 8:24:49 PM UTC+3, Mitch Sundt wrote: > If we were concerned about availability, we would configure multiple web servers running ODK Aggregate behind a load balancer all interacting with the same (single) high-availability database. We would not replicate the databases and do live fail-over across them. > > > > IF you did try to set this up, it is likely that ODK Briefcase would have difficulty communicating with the 'backup' server because both of the servers would need to be configured to work with the same hostname in order for their interactions with ODK Collect to work. As long as the 'live' server is up, it is likely that ODK Briefcase would only talk to it. > > > > > On Sun, Oct 18, 2015 at 11:23 PM, wrote: > Hello guys, > > > > I have ODK Aggregate running fine on our production server that works on a paid subscription basis under a PaaS plan. > > I also have a very capable and high spec'ed PC that is connected to the Internet via high speed fiber connection in my office. > > I have set up this PC to backup (on a daily basis) all my ODK Aggregate data collected on my production server. > > So far so good and everything is working fine. One server collects all my data, and the other backs it all up. > > > > Now I wish to extend my PC's functionality a little further by making it more useful. > > I want to setup ODK Aggregate on this PC and have it function as a backup ODK Aggregate server, such that when my production server goes down, I can continue collecting data and submitting forms with the ODK Aggregate on my high-end PC. > > > > I fully understand that no two ODK Aggregate instances can share the same platform configuration addresses, but I was hoping I could set up a server failover solution of some sort and use a domain name to reference one when the other goes down. > > I also understand that ODK Collect's platform settings would have to be changed whenever we need to redirect the data to the backup ODK Aggregate server if the production server ever goes down. I was hoping to use this domain name to configure the app. > > > > So my questions are: > > > > 1. Can a server failover solution work with ODK Aggregate? > > 2. Can I use a domain name to reference my ODK Aggregate production server and have my PC serverr use this same domain name in the event that my production server goes down (under the failover solution)? > > 3. Is there a simpler remedy out there where ODK Aggregate can be functionally distributed that I'm just not aware of? > > > > Please let me know, and sorry for the long post. > > > > Regards, > > ramadhan > > > > -- > > -- > > Post: opendatakit@googlegroups.com > > Unsubscribe: opendatakit+unsubscribe@googlegroups.com > > Options: http://groups.google.com/group/opendatakit?hl=en > > > > --- > > You received this message because you are subscribed to the Google Groups "ODK Community" group. > > To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit+unsubscribe@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > Mitch Sundt > Software Engineer > University of Washington > mitchellsundt@gmail.com

Hi Ramadhan,

This seems like a lot of work and it's not clear what are you
protecting against.

If your production server goes down, the devices will hold on to the
data until you can bring a new server up. And if you bring a new
server up, you can restore easily from your Briefcase backup and be up
and running in little time.

If you still wish to proceed, Mitch's approach is a good one as far as
availability. The alternative approach is to use App Engine. It does
all pretty much what you want.

Yaw

··· -- Need ODK consultants? https://nafundi.com provides form design, server setup, in-field training, and software development for ODK.

On Tue, Oct 20, 2015 at 2:11 PM, Andrew acawood777@gmail.com wrote:

A comment on the possible ODK Briefcase issue - if both servers have static
IPs (or known IPs) then you should be able to access them via Briefcase, as
you're skipping the hostname resolution step.

Regards,
Andrew

On Monday, 19 October 2015 19:24:49 UTC+2, Mitch Sundt wrote:

If we were concerned about availability, we would configure multiple web
servers running ODK Aggregate behind a load balancer all interacting with
the same (single) high-availability database. We would not replicate the
databases and do live fail-over across them.

IF you did try to set this up, it is likely that ODK Briefcase would have
difficulty communicating with the 'backup' server because both of the
servers would need to be configured to work with the same hostname in order
for their interactions with ODK Collect to work. As long as the 'live'
server is up, it is likely that ODK Briefcase would only talk to it.

On Sun, Oct 18, 2015 at 11:23 PM, washbne...@gmail.com wrote:

Hello guys,

I have ODK Aggregate running fine on our production server that works on
a paid subscription basis under a PaaS plan.
I also have a very capable and high spec'ed PC that is connected to the
Internet via high speed fiber connection in my office.
I have set up this PC to backup (on a daily basis) all my ODK Aggregate
data collected on my production server.
So far so good and everything is working fine. One server collects all my
data, and the other backs it all up.

Now I wish to extend my PC's functionality a little further by making it
more useful.
I want to setup ODK Aggregate on this PC and have it function as a backup
ODK Aggregate server, such that when my production server goes down, I can
continue collecting data and submitting forms with the ODK Aggregate on my
high-end PC.

I fully understand that no two ODK Aggregate instances can share the same
platform configuration addresses, but I was hoping I could set up a server
failover solution of some sort and use a domain name to reference one when
the other goes down.
I also understand that ODK Collect's platform settings would have to be
changed whenever we need to redirect the data to the backup ODK Aggregate
server if the production server ever goes down. I was hoping to use this
domain name to configure the app.

So my questions are:

  1. Can a server failover solution work with ODK Aggregate?
  2. Can I use a domain name to reference my ODK Aggregate production
    server and have my PC serverr use this same domain name in the event that my
    production server goes down (under the failover solution)?
  3. Is there a simpler remedy out there where ODK Aggregate can be
    functionally distributed that I'm just not aware of?

Please let me know, and sorry for the long post.

Regards,
ramadhan

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

A comment on the possible ODK Briefcase issue - if both servers have static IPs (or known IPs) then you should be able to access them via Briefcase, as you're skipping the hostname resolution step.

Regards,
Andrew

If we were concerned about availability, we would configure multiple web servers running ODK Aggregate behind a load balancer all interacting with the same (single) high-availability database. We would not replicate the databases and do live fail-over across them.

IF you did try to set this up, it is likely that ODK Briefcase would have difficulty communicating with the 'backup' server because both of the servers would need to be configured to work with the same hostname in order for their interactions with ODK Collect to work. As long as the 'live' server is up, it is likely that ODK Briefcase would only talk to it.

Hello guys,

I have ODK Aggregate running fine on our production server that works on a paid subscription basis under a PaaS plan.

I also have a very capable and high spec'ed PC that is connected to the Internet via high speed fiber connection in my office.

I have set up this PC to backup (on a daily basis) all my ODK Aggregate data collected on my production server.

So far so good and everything is working fine. One server collects all my data, and the other backs it all up.

Now I wish to extend my PC's functionality a little further by making it more useful.

I want to setup ODK Aggregate on this PC and have it function as a backup ODK Aggregate server, such that when my production server goes down, I can continue collecting data and submitting forms with the ODK Aggregate on my high-end PC.

I fully understand that no two ODK Aggregate instances can share the same platform configuration addresses, but I was hoping I could set up a server failover solution of some sort and use a domain name to reference one when the other goes down.

I also understand that ODK Collect's platform settings would have to be changed whenever we need to redirect the data to the backup ODK Aggregate server if the production server ever goes down. I was hoping to use this domain name to configure the app.

So my questions are:

  1. Can a server failover solution work with ODK Aggregate?

  2. Can I use a domain name to reference my ODK Aggregate production server and have my PC serverr use this same domain name in the event that my production server goes down (under the failover solution)?

  3. Is there a simpler remedy out there where ODK Aggregate can be functionally distributed that I'm just not aware of?

Please let me know, and sorry for the long post.

Regards,

ramadhan

--

--

Post: opend...@googlegroups.com

Unsubscribe: opendatakit...@googlegroups.com

Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups "ODK Community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to opendatakit...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

Thanks Andrew,

I assume you mean explicitly entering the unique IP addresses and ports of the servers as the namespace, right? And not using any domain name that may be a substitute for them, I'm I right?

Correct me if I'm wrong.

Regards,
Ramadhan

··· On Tuesday, October 20, 2015 at 10:11:38 AM UTC+3, Andrew wrote: > On Monday, 19 October 2015 19:24:49 UTC+2, Mitch Sundt wrote: > On Sun, Oct 18, 2015 at 11:23 PM, wrote:

Hi Ramadhan,

This seems like a lot of work and it's not clear what are you
protecting against.

If your production server goes down, the devices will hold on to the
data until you can bring a new server up. And if you bring a new
server up, you can restore easily from your Briefcase backup and be up
and running in little time.

If you still wish to proceed, Mitch's approach is a good one as far as
availability. The alternative approach is to use App Engine. It does
all pretty much what you want.

Yaw

Need ODK consultants? https://nafundi.com provides form design, server
setup, in-field training, and software development for ODK.

A comment on the possible ODK Briefcase issue - if both servers have static
IPs (or known IPs) then you should be able to access them via Briefcase, as
you're skipping the hostname resolution step.

Regards,
Andrew

If we were concerned about availability, we would configure multiple web
servers running ODK Aggregate behind a load balancer all interacting with
the same (single) high-availability database. We would not replicate the
databases and do live fail-over across them.

IF you did try to set this up, it is likely that ODK Briefcase would have
difficulty communicating with the 'backup' server because both of the
servers would need to be configured to work with the same hostname in order
for their interactions with ODK Collect to work. As long as the 'live'
server is up, it is likely that ODK Briefcase would only talk to it.

Hello guys,

I have ODK Aggregate running fine on our production server that works on
a paid subscription basis under a PaaS plan.
I also have a very capable and high spec'ed PC that is connected to the
Internet via high speed fiber connection in my office.
I have set up this PC to backup (on a daily basis) all my ODK Aggregate
data collected on my production server.
So far so good and everything is working fine. One server collects all my
data, and the other backs it all up.

Now I wish to extend my PC's functionality a little further by making it
more useful.
I want to setup ODK Aggregate on this PC and have it function as a backup
ODK Aggregate server, such that when my production server goes down, I can
continue collecting data and submitting forms with the ODK Aggregate on my
high-end PC.

I fully understand that no two ODK Aggregate instances can share the same
platform configuration addresses, but I was hoping I could set up a server
failover solution of some sort and use a domain name to reference one when
the other goes down.
I also understand that ODK Collect's platform settings would have to be
changed whenever we need to redirect the data to the backup ODK Aggregate
server if the production server ever goes down. I was hoping to use this
domain name to configure the app.

So my questions are:

  1. Can a server failover solution work with ODK Aggregate?
  2. Can I use a domain name to reference my ODK Aggregate production
    server and have my PC serverr use this same domain name in the event that my
    production server goes down (under the failover solution)?
  3. Is there a simpler remedy out there where ODK Aggregate can be
    functionally distributed that I'm just not aware of?

Please let me know, and sorry for the long post.

Regards,
ramadhan

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Yaw!

Thank you for your post and your time as well.
I truly appreciate your recommendation.
However, might you be in a position to provide further clarification please? I looked into the App Engine and (forgive me, I definitely missed it) I couldn't find any mention on ODK Aggregate distribution or database redundancy. In your expert opinion, how would the App Engine help me achieve my needs?

Oh, and to answer your question, I'm looking to protect my organization from lengthy down-times as we had experienced when our production server went down. It affected quite a lot of our activities.

Thank you once again sir!

Best,
Ramadhan

··· On Tuesday, October 20, 2015 at 11:04:26 AM UTC+3, Yaw Anokwa wrote: > On Tue, Oct 20, 2015 at 2:11 PM, Andrew wrote: > > On Monday, 19 October 2015 19:24:49 UTC+2, Mitch Sundt wrote: > >> On Sun, Oct 18, 2015 at 11:23 PM, wrote:

Hi Ramdhan,

Yes, that's what I meant. I haven't tested Briefcase with IPs and ports,
but it could be worth trying IF you were to go that route.
But I agree with Yaw and Mitch - I think you'd be creating a LOT of extra
headache for yourself without actually gaining anything, unless there are
other reasons for wanting your own setup.

Regards,
Andrew

··· On Wednesday, 21 October 2015 18:47:36 UTC+2, washbne...@gmail.com wrote: > > On Tuesday, October 20, 2015 at 10:11:38 AM UTC+3, Andrew wrote: > > A comment on the possible ODK Briefcase issue - if both servers have > static IPs (or known IPs) then you should be able to access them via > Briefcase, as you're skipping the hostname resolution step. > > > > Regards, > > Andrew > > > > On Monday, 19 October 2015 19:24:49 UTC+2, Mitch Sundt wrote: > > > > If we were concerned about availability, we would configure multiple web > servers running ODK Aggregate behind a load balancer all interacting with > the same (single) high-availability database. We would not replicate the > databases and do live fail-over across them. > > > > > > > > IF you did try to set this up, it is likely that ODK Briefcase would > have difficulty communicating with the 'backup' server because both of the > servers would need to be configured to work with the same hostname in order > for their interactions with ODK Collect to work. As long as the 'live' > server is up, it is likely that ODK Briefcase would only talk to it. > > > > > > > > > > On Sun, Oct 18, 2015 at 11:23 PM, wrote: > > Hello guys, > > > > > > > > I have ODK Aggregate running fine on our production server that works on > a paid subscription basis under a PaaS plan. > > > > I also have a very capable and high spec'ed PC that is connected to the > Internet via high speed fiber connection in my office. > > > > I have set up this PC to backup (on a daily basis) all my ODK Aggregate > data collected on my production server. > > > > So far so good and everything is working fine. One server collects all > my data, and the other backs it all up. > > > > > > > > Now I wish to extend my PC's functionality a little further by making it > more useful. > > > > I want to setup ODK Aggregate on this PC and have it function as a > backup ODK Aggregate server, such that when my production server goes down, > I can continue collecting data and submitting forms with the ODK Aggregate > on my high-end PC. > > > > > > > > I fully understand that no two ODK Aggregate instances can share the > same platform configuration addresses, but I was hoping I could set up a > server failover solution of some sort and use a domain name to reference > one when the other goes down. > > > > I also understand that ODK Collect's platform settings would have to be > changed whenever we need to redirect the data to the backup ODK Aggregate > server if the production server ever goes down. I was hoping to use this > domain name to configure the app. > > > > > > > > So my questions are: > > > > > > > > 1. Can a server failover solution work with ODK Aggregate? > > > > 2. Can I use a domain name to reference my ODK Aggregate production > server and have my PC serverr use this same domain name in the event that > my production server goes down (under the failover solution)? > > > > 3. Is there a simpler remedy out there where ODK Aggregate can be > functionally distributed that I'm just not aware of? > > > > > > > > Please let me know, and sorry for the long post. > > > > > > > > Regards, > > > > ramadhan > > > > > > > > -- > > > > -- > > > > Post: opend...@googlegroups.com > > > > Unsubscribe: opendatakit...@googlegroups.com > > > > Options: http://groups.google.com/group/opendatakit?hl=en > > > > > > > > --- > > > > You received this message because you are subscribed to the Google > Groups "ODK Community" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > an email to opendatakit...@googlegroups.com. > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > -- > > > > Mitch Sundt > > Software Engineer > > University of Washington > > mitche...@gmail.com > > Thanks Andrew, > > I assume you mean explicitly entering the unique IP addresses and ports of > the servers as the namespace, right? And not using any domain name that may > be a substitute for them, I'm I right? > > Correct me if I'm wrong. > > Regards, > Ramadhan

These days, everything is 'in the cloud'.

Unless there is a very good reason to set any of this up yourself, it
generally is not useful to do so.

Amazon, Google and most other data center hosting services offer SQL
servers that are high-availability systems (i.e., that never go down).

If you assume you will be using the internet to submit data, then it
follows that you would want your database to reside up 'in the cloud'
either on AWS, Google, or some other data center provider.

In high-data-rate configurations, you would have traffic directed to a load
balancer, then split out to N web servers, and all of those N web servers
would interact with your one high-availability database server.

The data center hosting service would typically implement that with 2 or
more actual physical boxes and provide all the mechanisms to make the
database server high-availability.

For nearly all uses of ODK Aggregate, you do not have a high-data-rate
situation. In those circumstances, the load balancer and N web servers act
only to enhance availability by handling crashes of the web server.

Note, however, that ODK Collect is intended to work without internet
connectivity. I.e., it does not require high-availability. Other than a
failure message displayed to the end user, there is little difference
between ODK Collect running away from cell or wifi access, and it running
with access but with the ODK Aggregate server being down. It is just a
question of whether the user has to wait to get back into an access area or
whether they have to wait for the ODK Aggregate server to be restored.

So, in general, you wouldn't go to the expense of setting up a load
balancer and a second fail-over web server.

Note also that the data center hosting services (such as AWS) have
automatic alarms that you can configure that can perform actions when a
server is detected to be down. The most common such action might be
'reboot'. And if the box remains down for longer, it them might have a
'page me' action.

Just setting that up on one box (that talks to a data-center-managed
high-availability database), without any load balancer or other equipment,
with these alarms, may be sufficient for reasonably high availability.

This is how we run our XLSForm server. It is on one box hosted on AWS; we
have alarms that can reboot the box and send us e-mail. I currently have
the reboot alarm disabled so I can investigate failures, which is why the
server is down longer when it does crash. We have a VM image of that
server with everything installed on it so that if the box or disk drive
physically dies, or if I need to take it out of service for a deeper
investigation of why it crashed, I can launch a completely new box, switch
the public IP address over to that new box, and have the XLSForm page
working again within about 5 minutes.

If you do want to set up something more, most data centers have some load
balancer offering. These may only be able to route requests to boxes within
their data center, but some offer the ability to route requests off-site.

I would research that if you feel the need.

··· On Wed, Oct 21, 2015 at 2:12 AM, wrote:

On Tuesday, October 20, 2015 at 11:04:26 AM UTC+3, Yaw Anokwa wrote:

Hi Ramadhan,

This seems like a lot of work and it's not clear what are you
protecting against.

If your production server goes down, the devices will hold on to the
data until you can bring a new server up. And if you bring a new
server up, you can restore easily from your Briefcase backup and be up
and running in little time.

If you still wish to proceed, Mitch's approach is a good one as far as
availability. The alternative approach is to use App Engine. It does
all pretty much what you want.

Yaw

Need ODK consultants? https://nafundi.com provides form design, server
setup, in-field training, and software development for ODK.

On Tue, Oct 20, 2015 at 2:11 PM, Andrew acawood777@gmail.com wrote:

A comment on the possible ODK Briefcase issue - if both servers have
static
IPs (or known IPs) then you should be able to access them via
Briefcase, as
you're skipping the hostname resolution step.

Regards,
Andrew

On Monday, 19 October 2015 19:24:49 UTC+2, Mitch Sundt wrote:

If we were concerned about availability, we would configure multiple
web
servers running ODK Aggregate behind a load balancer all interacting
with
the same (single) high-availability database. We would not replicate
the
databases and do live fail-over across them.

IF you did try to set this up, it is likely that ODK Briefcase would
have
difficulty communicating with the 'backup' server because both of the
servers would need to be configured to work with the same hostname in
order
for their interactions with ODK Collect to work. As long as the
'live'
server is up, it is likely that ODK Briefcase would only talk to it.

On Sun, Oct 18, 2015 at 11:23 PM, washbne...@gmail.com wrote:

Hello guys,

I have ODK Aggregate running fine on our production server that
works on
a paid subscription basis under a PaaS plan.
I also have a very capable and high spec'ed PC that is connected to
the
Internet via high speed fiber connection in my office.
I have set up this PC to backup (on a daily basis) all my ODK
Aggregate
data collected on my production server.
So far so good and everything is working fine. One server collects
all my
data, and the other backs it all up.

Now I wish to extend my PC's functionality a little further by
making it
more useful.
I want to setup ODK Aggregate on this PC and have it function as a
backup
ODK Aggregate server, such that when my production server goes down,
I can
continue collecting data and submitting forms with the ODK Aggregate
on my
high-end PC.

I fully understand that no two ODK Aggregate instances can share the
same
platform configuration addresses, but I was hoping I could set up a
server
failover solution of some sort and use a domain name to reference
one when
the other goes down.
I also understand that ODK Collect's platform settings would have to
be
changed whenever we need to redirect the data to the backup ODK
Aggregate
server if the production server ever goes down. I was hoping to use
this
domain name to configure the app.

So my questions are:

  1. Can a server failover solution work with ODK Aggregate?
  2. Can I use a domain name to reference my ODK Aggregate production
    server and have my PC serverr use this same domain name in the event
    that my
    production server goes down (under the failover solution)?
  3. Is there a simpler remedy out there where ODK Aggregate can be
    functionally distributed that I'm just not aware of?

Please let me know, and sorry for the long post.

Regards,
ramadhan

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send an
email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Yaw!

Thank you for your post and your time as well.
I truly appreciate your recommendation.
However, might you be in a position to provide further clarification
please? I looked into the App Engine and (forgive me, I definitely missed
it) I couldn't find any mention on ODK Aggregate distribution or database
redundancy. In your expert opinion, how would the App Engine help me
achieve my needs?

Oh, and to answer your question, I'm looking to protect my organization
from lengthy down-times as we had experienced when our production server
went down. It affected quite a lot of our activities.

Thank you once again sir!

Best,
Ramadhan

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

Ramadhan,

The short answer to "How would the App Engine help me achieve my
needs" is that it automatically scales and load balances and never
goes down (or rather, goes down way way less often than you can
accomplish yourself).

And while I don't want to discourage anyone from building what they
want, even with AWS components at your disposal, a reliable
distributed system is not trivial or cheap to build and maintain. If
uptime is the problem you want to solve, use App Engine.

Yaw

··· -- Need ODK consultants? https://nafundi.com provides form design, server setup, in-field training, and software development for ODK.

On Thu, Oct 22, 2015 at 1:16 AM, Mitch Sundt mitchellsundt@gmail.com wrote:

These days, everything is 'in the cloud'.

Unless there is a very good reason to set any of this up yourself, it
generally is not useful to do so.

Amazon, Google and most other data center hosting services offer SQL servers
that are high-availability systems (i.e., that never go down).

If you assume you will be using the internet to submit data, then it follows
that you would want your database to reside up 'in the cloud' either on AWS,
Google, or some other data center provider.

In high-data-rate configurations, you would have traffic directed to a load
balancer, then split out to N web servers, and all of those N web servers
would interact with your one high-availability database server.

The data center hosting service would typically implement that with 2 or
more actual physical boxes and provide all the mechanisms to make the
database server high-availability.

For nearly all uses of ODK Aggregate, you do not have a high-data-rate
situation. In those circumstances, the load balancer and N web servers act
only to enhance availability by handling crashes of the web server.

Note, however, that ODK Collect is intended to work without internet
connectivity. I.e., it does not require high-availability. Other than a
failure message displayed to the end user, there is little difference
between ODK Collect running away from cell or wifi access, and it running
with access but with the ODK Aggregate server being down. It is just a
question of whether the user has to wait to get back into an access area or
whether they have to wait for the ODK Aggregate server to be restored.

So, in general, you wouldn't go to the expense of setting up a load balancer
and a second fail-over web server.

Note also that the data center hosting services (such as AWS) have automatic
alarms that you can configure that can perform actions when a server is
detected to be down. The most common such action might be 'reboot'. And if
the box remains down for longer, it them might have a 'page me' action.

Just setting that up on one box (that talks to a data-center-managed
high-availability database), without any load balancer or other equipment,
with these alarms, may be sufficient for reasonably high availability.

This is how we run our XLSForm server. It is on one box hosted on AWS; we
have alarms that can reboot the box and send us e-mail. I currently have the
reboot alarm disabled so I can investigate failures, which is why the server
is down longer when it does crash. We have a VM image of that server with
everything installed on it so that if the box or disk drive physically dies,
or if I need to take it out of service for a deeper investigation of why it
crashed, I can launch a completely new box, switch the public IP address
over to that new box, and have the XLSForm page working again within about 5
minutes.

If you do want to set up something more, most data centers have some load
balancer offering. These may only be able to route requests to boxes within
their data center, but some offer the ability to route requests off-site.

I would research that if you feel the need.

On Wed, Oct 21, 2015 at 2:12 AM, washbnetbook43@gmail.com wrote:

On Tuesday, October 20, 2015 at 11:04:26 AM UTC+3, Yaw Anokwa wrote:

Hi Ramadhan,

This seems like a lot of work and it's not clear what are you
protecting against.

If your production server goes down, the devices will hold on to the
data until you can bring a new server up. And if you bring a new
server up, you can restore easily from your Briefcase backup and be up
and running in little time.

If you still wish to proceed, Mitch's approach is a good one as far as
availability. The alternative approach is to use App Engine. It does
all pretty much what you want.

Yaw

Need ODK consultants? https://nafundi.com provides form design, server
setup, in-field training, and software development for ODK.

On Tue, Oct 20, 2015 at 2:11 PM, Andrew acawood777@gmail.com wrote:

A comment on the possible ODK Briefcase issue - if both servers have
static
IPs (or known IPs) then you should be able to access them via
Briefcase, as
you're skipping the hostname resolution step.

Regards,
Andrew

On Monday, 19 October 2015 19:24:49 UTC+2, Mitch Sundt wrote:

If we were concerned about availability, we would configure multiple
web
servers running ODK Aggregate behind a load balancer all interacting
with
the same (single) high-availability database. We would not replicate
the
databases and do live fail-over across them.

IF you did try to set this up, it is likely that ODK Briefcase would
have
difficulty communicating with the 'backup' server because both of the
servers would need to be configured to work with the same hostname in
order
for their interactions with ODK Collect to work. As long as the
'live'
server is up, it is likely that ODK Briefcase would only talk to it.

On Sun, Oct 18, 2015 at 11:23 PM, washbne...@gmail.com wrote:

Hello guys,

I have ODK Aggregate running fine on our production server that
works on
a paid subscription basis under a PaaS plan.
I also have a very capable and high spec'ed PC that is connected to
the
Internet via high speed fiber connection in my office.
I have set up this PC to backup (on a daily basis) all my ODK
Aggregate
data collected on my production server.
So far so good and everything is working fine. One server collects
all my
data, and the other backs it all up.

Now I wish to extend my PC's functionality a little further by
making it
more useful.
I want to setup ODK Aggregate on this PC and have it function as a
backup
ODK Aggregate server, such that when my production server goes down,
I can
continue collecting data and submitting forms with the ODK Aggregate
on my
high-end PC.

I fully understand that no two ODK Aggregate instances can share the
same
platform configuration addresses, but I was hoping I could set up a
server
failover solution of some sort and use a domain name to reference
one when
the other goes down.
I also understand that ODK Collect's platform settings would have to
be
changed whenever we need to redirect the data to the backup ODK
Aggregate
server if the production server ever goes down. I was hoping to use
this
domain name to configure the app.

So my questions are:

  1. Can a server failover solution work with ODK Aggregate?
  2. Can I use a domain name to reference my ODK Aggregate production
    server and have my PC serverr use this same domain name in the event
    that my
    production server goes down (under the failover solution)?
  3. Is there a simpler remedy out there where ODK Aggregate can be
    functionally distributed that I'm just not aware of?

Please let me know, and sorry for the long post.

Regards,
ramadhan

--

Post: opend...@googlegroups.com
Unsubscribe: opendatakit...@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it,
send an
email to opendatakit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitche...@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google
Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Hi Yaw!

Thank you for your post and your time as well.
I truly appreciate your recommendation.
However, might you be in a position to provide further clarification
please? I looked into the App Engine and (forgive me, I definitely missed
it) I couldn't find any mention on ODK Aggregate distribution or database
redundancy. In your expert opinion, how would the App Engine help me achieve
my needs?

Oh, and to answer your question, I'm looking to protect my organization
from lengthy down-times as we had experienced when our production server
went down. It affected quite a lot of our activities.

Thank you once again sir!

Best,
Ramadhan

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com

--

Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en


You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.