Commit Graph

4 Commits

Author SHA1 Message Date
antanst
98d3ed6707 Fix infinite recrawl loop with skip-identical-content
Add last_crawled timestamp tracking to fix fetchSnapshotsFromHistory()
infinite loop when SkipIdenticalContent=true. Now tracks actual crawl
attempts separately from content changes via database DEFAULT timestamps.
2025-06-29 22:38:38 +03:00
c54c093a10 Implement context-aware database operations
- Add context support to database operations
- Implement versioned snapshots for URL history
- Update database queries to support URL timestamps
- Improve transaction handling with context
- Add utility functions for snapshot history
2025-06-29 22:38:38 +03:00
ca008b0796 Reorganize code for more granular imports 2025-02-26 10:34:46 +02:00
4e6fad873b Break up common functions and small refactor. 2025-01-04 15:31:26 +02:00