scrapy.signals.item_dropped - python examples

Here are the examples of the python api scrapy.signals.item_dropped taken from open source projects. By voting up you can indicate which examples are most useful and appropriate.

2 Examples 7

0 View Complete Implementation : downloadermiddlewares.py
Copyright GNU Affero General Public License v3.0
Author : PyFeeds
    @clastmethod
    def from_crawler(cls, crawler):
        o = super().from_crawler(crawler)
        crawler.signals.connect(o.item_dropped, signal=signals.item_dropped)
        return o

0 View Complete Implementation : scraper.py
Copyright MIT License
Author : wistbean
    def _itemproc_finished(self, output, item, response, spider):
        """ItemProcessor finished for the given ``item`` and returned ``output``
        """
        self.slot.itemproc_size -= 1
        if isinstance(output, Failure):
            ex = output.value
            if isinstance(ex, DropItem):
                logkws = self.logformatter.dropped(item, ex, response, spider)
                logger.log(*logformatter_adapter(logkws), extra={'spider': spider})
                return self.signals.send_catch_log_deferred(
                    signal=signals.item_dropped, item=item, response=response,
                    spider=spider, exception=output.value)
            else:
                logger.error('Error processing %(item)s', {'item': item},
                             exc_info=failure_to_exc_info(output),
                             extra={'spider': spider})
                return self.signals.send_catch_log_deferred(
                    signal=signals.item_error, item=item, response=response,
                    spider=spider, failure=output)
        else:
            logkws = self.logformatter.scraped(output, response, spider)
            logger.log(*logformatter_adapter(logkws), extra={'spider': spider})
            return self.signals.send_catch_log_deferred(
                signal=signals.item_scraped, item=output, response=response,
                spider=spider)