Adds robots.txt entries from Faculty Digital Archive#340

spilth merged 1 commit intomainNYULibraries/spatial_data_repository:mainfrom

316-add-more-robots-txt-entriesNYULibraries/spatial_data_repository:316-add-more-robots-txt-entriesCopy head branch name to clipboard