2 Commits

Author	SHA1	Message	Date
antanst	349968d019	Improve error handling and add duplicate snapshot cleanup	2025-06-29 22:38:38 +03:00
antanst	2357135d5a	Fix snapshot overwrite logic to preserve successful responses ... - Prevent overwriting snapshots that have valid response codes - Ensure URL is removed from queue when snapshot update is skipped - Add last_crawled timestamp tracking for better crawl scheduling - Remove SkipIdenticalContent flag, simplify content deduplication logic - Update database schema with last_crawled column and indexes	2025-06-29 22:38:38 +03:00