Enhance crawler with seed list and SQL utilities

Add seedList module for URL initialization, comprehensive SQL utilities for database analysis, and update project configuration.
2025-06-16 12:29:33 +03:00
parent dfb050588c
commit 37d5e7cd78
37 changed files with 742 additions and 682 deletions
--- a/misc/sql/mark_urls_processed_false.sql
+++ b/misc/sql/mark_urls_processed_false.sql
@@ -0,0 +1 @@
+update urls set being_processed=false where being_processed is true;
				`@@ -0,0 +1 @@`
				`update urls set being_processed=false where being_processed is true;`