Commit Graph

17 Commits

Author SHA1 Message Date
antanst
330b596497 Enhance crawler with seed list and SQL utilities
Add seedList module for URL initialization, comprehensive SQL utilities for database analysis, and update project configuration.
2025-06-16 12:29:33 +03:00
bfaa857fae Update and refactor core functionality
- Update common package utilities
- Refactor network code for better error handling
- Remove deprecated files and functionality
- Enhance blacklist and filtering capabilities
- Improve snapshot handling and processing
2025-05-22 12:47:01 +03:00
5940a117fd Add context-aware network operations
- Implement context-aware versions of network operations
- Add request cancellation support throughout network code
- Use structured logging with context metadata
- Support timeout management with contexts
- Improve error handling with detailed logging

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-22 12:45:58 +03:00
d1c326f868 Improve error handling with xerrors package
- Replace custom error handling with xerrors package
- Enhance error descriptions for better debugging
- Add text utilities for string processing
- Update error tests to use standard errors package
- Add String() method to GeminiError

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-22 12:45:46 +03:00
a55f820f62 Implement structured logging with slog
- Replace zerolog with Go's standard slog package
- Add ColorHandler for terminal color output
- Add context-aware logging system
- Format attributes on the same line as log messages
- Use green color for INFO level logs
- Set up context value extraction helpers

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-05-22 12:44:08 +03:00
658c5f5471 Fix linter warnings in gemini/network.go
Remove redundant nil checks before len() operations as len() for nil slices is defined as zero in Go.

🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
2025-03-10 11:34:29 +02:00
efaedcc6b2 Improvements in error handling & descriptions 2025-02-27 09:20:22 +02:00
9dc008cb0f Use go_errors library everywhere. 2025-02-26 13:31:46 +02:00
4bceb75695 Reorganize code for more granular imports 2025-02-26 10:34:46 +02:00
a9983f3531 Reorganize errors 2025-02-26 10:32:38 +02:00
5cf720103f Improve blacklist to use regex matching 2025-02-26 10:32:01 +02:00
b30b7274ec Simplify duplicate code 2025-01-16 22:37:39 +02:00
9ade26b6e8 Simplify IP pool and convert it to host pool 2025-01-16 10:04:02 +02:00
4a345a1763 Break up Gemtext link parsing code and improve tests. 2025-01-16 10:04:02 +02:00
64f98bb37c Add mode that prints multiple worker status in console 2025-01-16 10:04:02 +02:00
4e6fad873b Break up common functions and small refactor. 2025-01-04 15:31:26 +02:00
b52df073e9 Add first version of gemini-grc. 2024-12-27 12:09:55 +02:00