4 results found Sort:

lord-alfred
ipranges lord-alfred
103
633
cc0-1.0
44
🔨 List all IP ranges from: Google (Cloud & GoogleBot), Bing (Bingbot), Amazon (AWS), Microsoft, Oracle (Cloud), GitHub, Facebook (Meta), OpenAI (GPTBot) and other with daily updates.
bing cidr azure osint iplist oracle bingbot twitter facebook bugbounty googlebot ip-ranges microsoft amazon-aws bug-bounty pentesting cidr-ranges digitalocean google-cloud network-security
Created 2021-07-29
1,900 commits to main branch, last one 8 hours ago
jonasjacek
robots.txt jonasjacek
38
87
other
7
Simple robots.txt template. Keep unwanted robots out (disallow). White lists (allow) legitimate user-agents. Useful for all websites.
seo bingbot crawlers crawling googlebot whitelist robots-txt twitterbot user-agent web-robots baiduspider web-crawling blocking-bots search-engine robots-exclusion-standard search-engine-optimization
Created 2016-01-08
303 commits to master branch, last one 9 months ago
mantoufan
yzhanIP mantoufan
3
59
unknown
2
Crawl, match, parse IP or IP range, check if IP or range is in another range. Support IPv4, IPv6, IP Interval, Wildcard and CIDR. Check if IP is Cloudflare node IP, Google bot IP. 爬取，正则匹配，解析 IP 和 IP 范...
ipv4 ipv6 googlebot cloudflare
Created 2020-08-19
13 commits to master branch, last one 2 years ago
AnTheMaker
GoodBots AnTheMaker
12
45
unknown
4
Updated lists of IP addresses/whitelists of good bots and crawlers. Includes GoogleBot, BingBot, DuckDuckBot, etc.
bot crawler firewall googlebot whitelist ip-addresses
Created 2023-01-03
755 commits to main branch, last one 8 hours ago