spatie / robots-txt

Determine if a page may be crawled from robots.txt, robots meta tags and robot headers

Date Created 2018-02-26 (6 years ago)

Commits 159 (last one 5 days ago)

Stargazers 218 (0 this week)

Watchers 11 (0 this week)

Forks 36

License mit

Ranking

RepositoryStats indexes 565,600 repositories, of these spatie/robots-txt is ranked #164,305 (71st percentile) for total stargazers, and #188,904 for total watchers. Github reports the primary language for this repository as PHP, for repositories using this language it is ranked #4,825/14,815.

spatie/robots-txt is also tagged with popular topics, for these it's ranked: php (#2,409/5606), crawler (#254/547)

Other Information

There have been 22 releases, the latest one was published on 2024-09-25 (5 days ago) with the name 2.2.2.

Homepage URL: https://spatie.be/en/opensource/php

All Topics

php crawler robots-txt

Star History

Github stargazers over time

Watcher History

Github watchers over time, collection started in '23

Recent Commit History

46 commits on the default branch (main) since jan '22

Yearly Commits

Commits to the default branch (main) per year

Issue History

Languages

The primary language is PHP but there's also others...

updated: 2024-09-26 @ 10:20am, id: 122979707 / R_kgDOB1SFew