Enhance crawler with seed list and SQL utilities

Add seedList module for URL initialization, comprehensive SQL utilities for database analysis, and update project configuration.
This commit is contained in:
antanst
2025-06-16 12:29:33 +03:00
parent 5e6dabf1e7
commit 8588414b14
37 changed files with 742 additions and 682 deletions

9
.gitignore vendored
View File

@@ -1,5 +1,6 @@
**/.#*
**/*~
**/.DS_Store
/.idea
/.goroot
/dist/**
@@ -18,3 +19,11 @@ run*.sh
/db/sql/**
**/.claude/settings.local.json
/crawl.sh
/crawler.sh
/get.sh
/snapshot_history.sh
/whitelist.txt
/CLAUDE.md