Rewrite `get_robotstxt_http_get`#94

jrdnbradford merged 2 commits intomainropensci/robotstxt:mainfrom

simplify-get_robotstxt_http_getropensci/robotstxt:simplify-get_robotstxt_http_getCopy head branch name to clipboard