spiderbar (0.2.2)

0 users

Parse and Test Robots Exclusion Protocol Files and Rules.

https://gitlab.com/hrbrmstr/spiderbar
http://cran.r-project.org/web/packages/spiderbar

The 'Robots Exclusion Protocol' documents a set of standards for allowing or excluding robot/spider crawling of different areas of site content. Tools are provided which wrap The 'rep-cpp' C++ library for processing these 'robots.txt' files.

Maintainer: Bob Rudis
Author(s): Bob Rudis (bob@rud.is) [aut, cre], SEOmoz, Inc [aut]

License: MIT + file LICENSE

Uses: Rcpp, testthat, covr, robotstxt

Released 4 months ago.


2 previous versions

Ratings

Overall:

  (0 votes)

Documentation:

  (0 votes)

Log in to vote.

Reviews

No one has written a review of spiderbar yet. Want to be the first? Write one now.


Related packages: Rserve, XML, httpRequest, rjson, RCurl, OAIHarvester, RgoogleMaps, sendmailR, twitteR, scrapeR, RJSONIO, imguR, googleVis, plotGoogleMaps, factualR, ROAuth, Rook, osmar, whisker, FastRWeb(20 best matches, based on common tags.)


Search for spiderbar on google, google scholar, r-help, r-devel.

Visit spiderbar on R Graphical Manual.