generic.spiders.file_download#

Module Contents#

Classes#

FileDownloadSpiderConfig

FileDownloadSpider

A spider that download files under a specific path.

API#

class generic.spiders.file_download.FileDownloadSpiderConfig(/, **data: Any)#

Bases: generic.mixins.file_downloader.FileDownloaderMixinConfig

path_regexp: str = '^/'#

A regular expression for URL path part that the spider should crawls.

class generic.spiders.file_download.FileDownloadSpider(*args, **kwargs)#

Bases: generic.spiders.base.GenericSpider[generic.spiders.file_download.FileDownloadSpiderConfig], generic.mixins.file_downloader.FileDownloaderMixin

A spider that download files under a specific path.

Initialization

name = 'file-download'#
classmethod get_config_class() → Type[generic.spiders.file_download.FileDownloadSpiderConfig]#

Returns the config class for this spider.

async start()#
parse_page(res: scrapy.http.Response)#