source: spiderling

Name Size Rev Age Author Last Change
util 9d3fc33   3 years vit.suchomel Catch robot parser failures
crawl.py 17.0 KB 952cd37   3 years vit.suchomel Important configuration moved to util/config.py
LICENCE 31.7 KB 0d534b9   4 years vit.suchomel Initial commit
process.py 14.9 KB 403516f   3 years vit.suchomel Catch all exceptions when decoding html data
README 3.6 KB 39d1612   3 years vit.suchomel Better estimate of RAM cost of big crawls
spiderling.py 15.6 KB 952cd37   3 years vit.suchomel Important configuration moved to util/config.py
TODO 133 bytes be862ea   3 years vit.suchomel TODO update
version 5 bytes 7aa1fc0   3 years vit.suchomel v. 0.77
Note: See TracBrowser for help on using the repository browser.