Compare commits

base: antanst:02015faa81b0fd4fa800672b1722e3cfe8ba22ca

Branches Tags

antanst:main

..

compare: antanst:daaa61c884ab44458d4b4fea8539e068865d33c0

Branches Tags

antanst:main

4 Commits

02015faa81 ... daaa61c884

Author	SHA1	Message	Date
antanst	daaa61c884	Add robots.txt checking ... Still needs periodic cache refresh	2024-10-23 14:26:42 +03:00
antanst	1ac250ca6e	Revert "refactor: Improve robots.txt parsing and caching" ... This reverts commit 6a96fb26cc.	2024-10-23 14:07:14 +03:00
antanst (aider)	6a96fb26cc	refactor: Improve robots.txt parsing and caching	2024-10-23 14:06:56 +03:00
antanst	3e01cb1819	refactor: Simplify robots.txt parsing logic	2024-10-23 14:06:55 +03:00

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Expand all files Collapse all files

Diff Content Not Available