-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
enhancementNew feature or requestNew feature or request
Description
In https://github.com/ThreeSixtyGiving/datagetter/blob/master/getter/get.py#L297 we currently
- Take the list of publishers/files from the registry
- In each multiprocess take one of the entries from the list and add the metadata information
- recreate the list of publishers/files from the registry from the returned results of (2)
If something goes wrong we should return the original entry from the registry to rebuild the list, this is done by catching all the exceptions and returning the original data. This is fine.. but we don't want to rely on that mechanism, especially in the case where one of those multiprocesses gets terminated without an exception but doesn't terminate the parent process (if that is even possible).
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request