robotstxt
type: Item
instance of: R package
last update: 2020-09-03T00:00:00Z
software version identifier: 0.7.13
author: Peter Meissner
author: Kun Ren
maintained by: Peter Meissner
copyright license: MIT license
copyright license: File License
imports: spiderbar
imports: utils
imports: stringr
imports: httr
imports: future
imports: future.apply
imports: magrittr
CRAN project: robotstxt
programmed in: R
depends on software: R
MaRDI profile type: Software
description: A 'robots.txt' Parser and 'Webbot'/'Spider'/'Crawler' Permissions Checker