

If there is a way to combine this with something like anubis this could be very interesting, then you are also not dependent on the honesty of reading the robots.txt
Have a RSL license? here you go scrape the content.
Trying to freeload? anubis time.
Qatar had American Patriot and THAAD defenses and Israël could strike anyway with nothing being shot down.