spatie / robots-txt

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

spatie
spatie
Date Created 2018-02-26 (7 years ago)
Commits 177 (last one about a month ago)
Stargazers 235 (0 this week)
Watchers 9 (0 this week)
Forks 41
License mit
Ranking

RepositoryStats indexes 634,026 repositories, of these spatie/robots-txt is ranked #167,827 (74th percentile) for total stargazers, and #218,136 for total watchers. Github reports the primary language for this repository as PHP, for repositories using this language it is ranked #4,754/15,782.

spatie/robots-txt is also tagged with popular topics, for these it's ranked: php (#2,386/5931),  crawler (#267/594)

Other Information

There have been 25 releases, the latest one was published on 2025-01-31 (about a month ago) with the name 2.3.1.

Homepage URL: https://spatie.be/en/opensource/php

All Topics
php crawler robots-txt

Star History

Github stargazers over time

25025020020015015010010050500020192019202020202021202120222022202320232024202420252025

Watcher History

Github watchers over time, collection started in '23

111110.510.510109.59.5998.58.58820232023Jul '23Jul '2320242024Jul '24Jul '2420252025

Recent Commit History

64 commits on the default branch (main) since jan '22

707060605050404030302020101000Jul '22Jul '2220232023Jul '23Jul '2320242024Jul '24Jul '2420252025

Yearly Commits

Commits to the default branch (main) per year

3535303025252020151510105500201820182019201920202020202120212022202220242024

Issue History

Total Issues
Open Issues
Closed Issues
1616141412121010886644220020192019202020202021202120222022202320232024202420252025

Languages

The primary language is PHP but there's also others...

PHPPHPJavaScriptJavaScriptHTMLHTMLShellShell

updated: 2025-03-21 @ 02:35am, id: 122979707 / R_kgDOB1SFew