unarist 5c8ca024ef Improve error handling on LinkCrawlWorker (#5250)
* Improve error handling on LinkCrawlWorker

* Ignore TimeoutError and InvalidURIError too
* Record errors to debug log
* Enable dead job queue on LinkCrawlWorker

Since most of acceptable errors were already ignored, only our side issue should go to dead job queue.

* Ignore all http gem errors
2017-10-06 20:39:08 +02:00
..
2017-06-14 18:01:27 +02:00
2017-09-14 00:05:25 +02:00