-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Log errors and proceed when reindexing #172
base: master
Are you sure you want to change the base?
Changes from all commits
5443f3c
ac6a523
eecaf63
4f6871c
cc7a2c3
9cc85cd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -17,3 +17,7 @@ Makefile | |
.pydevproject | ||
.project | ||
.settings | ||
|
||
# virtual environment: | ||
lib64 | ||
pip-selfcheck.json |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5,6 +5,15 @@ | |
|
||
import transaction | ||
|
||
try: | ||
# this exception must be resubmitted: | ||
from ZODB.POSException import ConflictError | ||
except ImportError: | ||
# we don't really *require* this; | ||
# if we don't have it, a dummy exception will do: | ||
class ConflictError(Exception): | ||
pass | ||
|
||
|
||
def reindex_catalog(setup_tool): | ||
"""Reindex objects to fix interfaces on the catalog.""" | ||
|
@@ -16,13 +25,24 @@ def reindex_catalog(setup_tool): | |
results = catalog() | ||
logger.info(u'Found {0} objects'.format(len(results))) | ||
n = 0 | ||
errors = 0 | ||
for obj in get_valid_objects(results): | ||
catalog.catalog_object(obj, idxs=['object_provides'], update_metadata=False) | ||
try: | ||
catalog.catalog_object(obj, idxs=['object_provides'], update_metadata=False) | ||
except ConflictError: | ||
raise | ||
except Exception as e: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this is a good idea; if you don't know what other exception can be raised here, better to remove this also or we can hide some other issues. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wouldn't this operation be better atomic? The only improvement that could be made is to log the object that failed but raise the exception right away. If the upgrade runs without error, many users may think that everything went well when it didn't. |
||
errors += 1 | ||
logger.error('{0!r} reindexing {1!r}'.format(e, obj)) | ||
if test: | ||
raise | ||
n += 1 | ||
if n % 1000 == 0 and not test: | ||
transaction.commit() | ||
logger.info('{0} items processed.'.format(n)) | ||
|
||
if not test: | ||
if errors: | ||
logger.info('{0} errors occured.'.format(errors)) | ||
transaction.commit() | ||
logger.info('Done.') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if we don't require it, better to remove it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's best to add
ZODB
to setup.py