diff options
| author | mahesh | 2016-04-01 01:41:16 +0530 |
|---|---|---|
| committer | mahesh | 2016-04-01 01:41:16 +0530 |
| commit | cf1209063bf68e01d312bc870280cf43b2a616a9 (patch) | |
| tree | ee77994fa6e79822c7ef812519e9f2f171d482b5 /scripts/crawler/scrapy.cfg | |
| parent | 9c84c0ab3a780b8055212d613776f28907535b62 (diff) | |
| download | Python-TBC-Interface-cf1209063bf68e01d312bc870280cf43b2a616a9.tar.gz Python-TBC-Interface-cf1209063bf68e01d312bc870280cf43b2a616a9.tar.bz2 Python-TBC-Interface-cf1209063bf68e01d312bc870280cf43b2a616a9.zip | |
A crawler to crawl the tbc website to find errors on tbc notebooks and broken links
Diffstat (limited to 'scripts/crawler/scrapy.cfg')
| -rw-r--r-- | scripts/crawler/scrapy.cfg | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/scripts/crawler/scrapy.cfg b/scripts/crawler/scrapy.cfg new file mode 100644 index 0000000..b99853f --- /dev/null +++ b/scripts/crawler/scrapy.cfg @@ -0,0 +1,11 @@ +# Automatically created by: scrapy startproject +# +# For more information about the [deploy] section see: +# https://scrapyd.readthedocs.org/en/latest/deploy.html + +[settings] +default = tbc_web_crawler.settings + +[deploy] +#url = http://localhost:6800/ +project = tbc_web_crawler |