2 results found Sort:

lord-alfred
ipranges lord-alfred
79
439
cc0-1.0
32
🔨 List all IP ranges from: Google (Cloud & GoogleBot), Bing (Bingbot), Amazon (AWS), Microsoft, Oracle (Cloud), GitHub, Facebook (Meta), OpenAI (GPTBot) and other with daily updates.
bing cidr azure osint iplist oracle bingbot twitter facebook bugbounty googlebot ip-ranges microsoft amazon-aws bug-bounty pentesting cidr-ranges digitalocean google-cloud network-security
Created 2021-07-29
1,580 commits to main branch, last one 10 hours ago
jonasjacek
robots.txt jonasjacek
37
83
other
7
Simple robots.txt template. Keep unwanted robots out (disallow). White lists (allow) legitimate user-agents. Useful for all websites.
seo bingbot crawlers crawling googlebot whitelist robots-txt twitterbot user-agent web-robots baiduspider web-crawling blocking-bots search-engine robots-exclusion-standard search-engine-optimization
Created 2016-01-08
303 commits to master branch, last one 4 months ago