Improve crawler performance and logging

- Optimize job scheduler to use NumOfWorkers for URL limits
- Clean up verbose logging in worker processing
- Update log messages for better clarity
This commit is contained in:
antanst
2025-06-29 22:27:20 +03:00
parent 9a09dd7735
commit db3448f448
4 changed files with 14 additions and 7 deletions

View File

@@ -107,6 +107,9 @@ You can access the snapshot history using the included `snapshot_history.sh` scr
Good starting points:
gemini://warmedal.se/~antenna/
gemini://tlgs.one/
gopher://i-logout.cz:70/1/bongusta/
gopher://gopher.quux.org:70/