spatie / robots-txt

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

Date Created 2018-02-26 (6 years ago)

Commits 161 (last one about a month ago)

Stargazers 223 (0 this week)

Watchers 10 (0 this week)

Forks 38

License mit

Ranking

RepositoryStats indexes 589,134 repositories, of these spatie/robots-txt is ranked #166,160 (72nd percentile) for total stargazers, and #206,941 for total watchers. Github reports the primary language for this repository as PHP, for repositories using this language it is ranked #4,809/15,159.

spatie/robots-txt is also tagged with popular topics, for these it's ranked: php (#2,412/5727), crawler (#260/567)

Other Information

spatie/robots-txt has Github issues enabled, there is 1 open issue and 15 closed issues.

There have been 23 releases, the latest one was published on 2024-10-22 (about a month ago) with the name 2.2.3.

Homepage URL: https://spatie.be/en/opensource/php

All Topics

php crawler robots-txt

Star History

Github stargazers over time

Watcher History

Github watchers over time, collection started in '23

Recent Commit History

48 commits on the default branch (main) since jan '22

Yearly Commits

Commits to the default branch (main) per year

Issue History

Languages

The primary language is PHP but there's also others...

updated: 2024-12-03 @ 03:17am, id: 122979707 / R_kgDOB1SFew