|
|
6cf507bdc9
|
DB scripts and migrations
|
2024-12-09 19:54:00 +02:00 |
|
|
|
825c7e3391
|
.
|
2024-11-18 16:28:45 +02:00 |
|
|
|
2bb8589eb7
|
feat: Improve error handling with custom error types and detailed messages
|
2024-11-11 15:01:20 +02:00 |
|
|
|
6346c9a829
|
refactor: Remove default configuration constants and simplify config parsing
|
2024-11-11 15:01:18 +02:00 |
|
|
|
c7b0778b77
|
refactor: Add config validation, defaults, and unit tests
|
2024-11-11 14:58:09 +02:00 |
|
|
|
bea0d22c26
|
refactor: Replace switch statement with map-based config parsing
|
2024-11-11 14:56:41 +02:00 |
|
|
|
cd60c1363b
|
Lots of features, first version that reliably crawls Geminispace.
- [x] Concurrent downloading with workers
- [x] Concurrent connection limit per host
- [x] URL Blacklist
- [x] Save image/* and text/* files
- [x] Configuration via environment variables
- [x] Storing snapshots in PostgreSQL
- [x] Proper response header & body UTF-8 and format validation
.
.
.
|
2024-10-21 20:04:09 +03:00 |
|
