Dear community members,
I have aggregate 1.4 version installed on appspot account. Its working fine. But when i try to Purge some submissions based on last date. It doesnt work. It does display the message of Successful commencement but data still remains intact.
Please help
Thank you
Deletion is not immediate. It is a scheduled activity that runs slowly in
the background, deleting 100 rows at a time, every few minutes.
This is partly by design -- we didn't want a user's free quota to be
immediately consumed by the deletes.
And is is partly a necessity because of the way long-running activities had
to be implemented when AppEngine was first released.
Dear community members,
I have aggregate 1.4 version installed on appspot account. Its working
fine. But when i try to Purge some submissions based on last date. It
doesnt work. It does display the message of Successful commencement but
data still remains intact.Please help
Thank you
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
Dear Mitch,
Thank you for your response. I understand that it works bit by bit. But in my case, i have issued command repeatedly and even after two three days, data is still there.
Is there any alternative way to delete old data? may be directly from appengine etc.
Thank you very much in advance.
Does your app have "Disable faster background actions (exports, publishing,
form deletion) (slows quota usage on Google AppEngine)" checked on the Site
Admin / Preferences page?
If checked, that will greatly slow down the deletion process. What happens
if you uncheck that?
If that doesn't resolve the issue, invite me as a developer, and I can take
a look at the logs. (and let me know what the server name is )
Dear Mitch,
Thank you for your response. I understand that it works bit by bit. But in
my case, i have issued command repeatedly and even after two three days,
data is still there.Is there any alternative way to delete old data? may be directly from
appengine etc.Thank you very much in advance.
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
Dear Mitch,
That option is already unchecked. I have invited you as developer. Its citypulsedata1.appspot.com
Dear Mitch,
Did you find a chance to have a look at logs? Can you kindly share whats wrong with application?
Thank you very much in advance.
There wasn't enough information being logged.
Looking at the code, I suspect that an ill-formed or partially-deleted
submission is causing the purge to fail.
Are you now trying to delete the form? I see a delete-form task attempting
to run.
It may have the same difficulty.
Dear Mitch,
Did you find a chance to have a look at logs? Can you kindly share whats
wrong with application?Thank you very much in advance.
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
Since the size got larger and it was taking lot of time to fetch instances
(Briefcase was taking hours to go through already fetched instances before
starting new fetching), I deleted the form and reloaded a fresh copy.
Although it has eased downloading now, But still the Data Store size is
same as earlier.
Thank you for your time.
There wasn't enough information being logged.
Looking at the code, I suspect that an ill-formed or partially-deleted
submission is causing the purge to fail.Are you now trying to delete the form? I see a delete-form task attempting
to run.It may have the same difficulty.
On Sat, Dec 6, 2014 at 9:24 AM, plannerqadeer@gmail.com wrote:
Dear Mitch,
Did you find a chance to have a look at logs? Can you kindly share whats
wrong with application?Thank you very much in advance.
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/_cX5PY-GXaM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
The next update to ODK Aggregate will have better logging in this area, so
we'll hopefully be able to identify the problem(s) better.
I haven't been able to break things in a way that would explain your
experiences.
Is space an issue for you?
I can manually delete the orphaned tables to clean up the system; that
will, however, consume read/delete/write quota.
Since the size got larger and it was taking lot of time to fetch instances
(Briefcase was taking hours to go through already fetched instances before
starting new fetching), I deleted the form and reloaded a fresh copy.
Although it has eased downloading now, But still the Data Store size is
same as earlier.Thank you for your time.
On Tue, Dec 9, 2014 at 12:11 AM, Mitch Sundt mitchellsundt@gmail.com wrote:
There wasn't enough information being logged.
Looking at the code, I suspect that an ill-formed or partially-deleted
submission is causing the purge to fail.Are you now trying to delete the form? I see a delete-form task
attempting to run.It may have the same difficulty.
On Sat, Dec 6, 2014 at 9:24 AM, plannerqadeer@gmail.com wrote:
Dear Mitch,
Did you find a chance to have a look at logs? Can you kindly share whats
wrong with application?Thank you very much in advance.
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/_cX5PY-GXaM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com
Dear Mitch,
I am deeply indebted to you for your continuous support. Space is not an
issue for me. But currently we have got around 20 thousand records with
daily growth of 800 records. When we try to fetch them by Briefcase it
takes around 5 hours to go through the existing records before starting
download the new ones daily.
So I deleted the old ones so that we can have quicker download.
Can you please tell me how to delete tables manually. Quota is no issue.
Thank you in advance.
The next update to ODK Aggregate will have better logging in this area, so
we'll hopefully be able to identify the problem(s) better.I haven't been able to break things in a way that would explain your
experiences.Is space an issue for you?
I can manually delete the orphaned tables to clean up the system; that
will, however, consume read/delete/write quota.On Mon, Dec 8, 2014 at 6:34 PM, Syed Muhammad Qadeer < plannerqadeer@gmail.com> wrote:
Since the size got larger and it was taking lot of time to fetch
instances (Briefcase was taking hours to go through already fetched
instances before starting new fetching), I deleted the form and reloaded a
fresh copy. Although it has eased downloading now, But still the Data Store
size is same as earlier.Thank you for your time.
On Tue, Dec 9, 2014 at 12:11 AM, Mitch Sundt mitchellsundt@gmail.com wrote:
There wasn't enough information being logged.
Looking at the code, I suspect that an ill-formed or partially-deleted
submission is causing the purge to fail.Are you now trying to delete the form? I see a delete-form task
attempting to run.It may have the same difficulty.
On Sat, Dec 6, 2014 at 9:24 AM, plannerqadeer@gmail.com wrote:
Dear Mitch,
Did you find a chance to have a look at logs? Can you kindly share
whats wrong with application?Thank you very much in advance.
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/_cX5PY-GXaM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/_cX5PY-GXaM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
You should be able to purge data with the new tables via the Purge button.
I believe the earlier problem was a data corruption issue that was
preventing those tables from being deleted.
I will look into what I can do with ODK Briefcase to make it smarter when
resuming a download; it will probably be an extra configuration setting.
I recommend leaving the extra data tables around, as they do not affect
the performance of the current tables in any way. And they are likely not
using a huge amount of space.
w.r.t. how you could do this if you did decide to....
Go to the Application Dashboard ( via ),
click on "Datastore Viewer",
choose the "opendatakit._form_data_model" kind
expand the "Options"
paste in:
SELECT DISTINCT PERSIST_AS_TABLE_NAME FROM opendatakit._form_data_model
And Run Query
This gives you the list of form data tables that ARE ACTIVELY USED by the
system.
Form data tables are distinguished by beginning with "opendatakit."
followed by an all-captial-letter name beginning with a letter.
The tables used by ODK Aggregate to manage and control the system start
with "opendatakit." followed by a name that begins with one or two leading
underscores.
You obviously do not want to delete records from any of those!
Using the above list, you can then choose form data tables not on the list,
and select their valued to delete them (again, with this manual process,
you need to be extremely careful you are selecting un-used data tables --
not active ones or tables used by the system). I believe the only way to
delete tables within the Datastore Viewer is to check the checkboxes for
the rows and select "Delete". This is very tedious and will generally not
worth the effort except, perhaps, for the ..._BLB tables, which hold image
attachments (about 1MB per row). So deleting 1000 of those rows would free
up 1GB of space. But, at $0.006 / GByte-Day, the time needed to do this
manually would not be recovered for several months or years.
Mitch
Dear Mitch,
I am deeply indebted to you for your continuous support. Space is not an
issue for me. But currently we have got around 20 thousand records with
daily growth of 800 records. When we try to fetch them by Briefcase it
takes around 5 hours to go through the existing records before starting
download the new ones daily.So I deleted the old ones so that we can have quicker download.
Can you please tell me how to delete tables manually. Quota is no issue.
Thank you in advance.
On Wed, Dec 10, 2014 at 2:25 AM, Mitch Sundt mitchellsundt@gmail.com wrote:
The next update to ODK Aggregate will have better logging in this area,
so we'll hopefully be able to identify the problem(s) better.I haven't been able to break things in a way that would explain your
experiences.Is space an issue for you?
I can manually delete the orphaned tables to clean up the system; that
will, however, consume read/delete/write quota.On Mon, Dec 8, 2014 at 6:34 PM, Syed Muhammad Qadeer < plannerqadeer@gmail.com> wrote:
Since the size got larger and it was taking lot of time to fetch
instances (Briefcase was taking hours to go through already fetched
instances before starting new fetching), I deleted the form and reloaded a
fresh copy. Although it has eased downloading now, But still the Data Store
size is same as earlier.Thank you for your time.
On Tue, Dec 9, 2014 at 12:11 AM, Mitch Sundt mitchellsundt@gmail.com wrote:
There wasn't enough information being logged.
Looking at the code, I suspect that an ill-formed or partially-deleted
submission is causing the purge to fail.Are you now trying to delete the form? I see a delete-form task
attempting to run.It may have the same difficulty.
On Sat, Dec 6, 2014 at 9:24 AM, plannerqadeer@gmail.com wrote:
Dear Mitch,
Did you find a chance to have a look at logs? Can you kindly share
whats wrong with application?Thank you very much in advance.
--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/_cX5PY-GXaM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google
Groups "ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to a topic in the
Google Groups "ODK Community" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/opendatakit/_cX5PY-GXaM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.--
Post: opendatakit@googlegroups.com
Unsubscribe: opendatakit+unsubscribe@googlegroups.com
Options: http://groups.google.com/group/opendatakit?hl=en
You received this message because you are subscribed to the Google Groups
"ODK Community" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to opendatakit+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Mitch Sundt
Software Engineer
University of Washington
mitchellsundt@gmail.com