Log errors and proceed when reindexing #172
Conversation
(... which have been created by virtualenv .)
We have a ZODB which contains at least one object (after processing of 13000 objects) which can't be reindexed. Of course we do want the remaining 50000 objects to be reindexed, and we'd like to know *which one* causes the trouble. Thus, - we log the error, and - we count the errors. ZODB ConflictErrors are not handled that way (yet?) but re-raised.
- Added ImportError to bare except statement - removed unused variable e
|
PyLint complains about an unused variable, but it is wrong: That This is a little bit "special", but there is a good reason I use this syntax all the time: So, what do we do about this? |
In my Plone 4.3 setup, I got an AttributeError rather than an KeyError. With this change, the reindexing worked, and the single broken CatalogBrain was logged while the remaining 59644 have been reindexed.
|
Sorry for the noise. |
| try: | ||
| # this exception must be resubmitted: | ||
| from ZODB.POSException import ConflictError | ||
| except ImportError: |
There was a problem hiding this comment.
if we don't require it, better to remove it.
There was a problem hiding this comment.
I think it's best to add ZODB to setup.py
| catalog.catalog_object(obj, idxs=['object_provides'], update_metadata=False) | ||
| except ConflictError: | ||
| raise | ||
| except Exception as e: |
There was a problem hiding this comment.
I don't think this is a good idea; if you don't know what other exception can be raised here, better to remove this also or we can hide some other issues.
There was a problem hiding this comment.
Wouldn't this operation be better atomic? The only improvement that could be made is to log the object that failed but raise the exception right away. If the upgrade runs without error, many users may think that everything went well when it didn't.
|
@tobiasherp does the upgrade step not stop when a ConflictErrors occurs but stop for when another error occurs? It is? |
| try: | ||
| # this exception must be resubmitted: | ||
| from ZODB.POSException import ConflictError | ||
| except ImportError: |
There was a problem hiding this comment.
I think it's best to add ZODB to setup.py
| catalog.catalog_object(obj, idxs=['object_provides'], update_metadata=False) | ||
| except ConflictError: | ||
| raise | ||
| except Exception as e: |
There was a problem hiding this comment.
Wouldn't this operation be better atomic? The only improvement that could be made is to log the object that failed but raise the exception right away. If the upgrade runs without error, many users may think that everything went well when it didn't.
We have a ZODB which contains at least one object (after processing of 13000 objects) which can't be reindexed.
Of course we do want the remaining 50000 objects to be reindexed, and we'd like to know which one causes the trouble.
Thus,
ZODB ConflictErrors are not handled that way (yet?) but re-raised.