antanst
fee7d3e01c
- [x] Concurrent downloading with workers - [x] Concurrent connection limit per host - [x] URL Blacklist - [x] Save image/* and text/* files - [x] Configuration via environment variables - [x] Storing snapshots in PostgreSQL - [x] Proper response header & body UTF-8 and format validation . .
10 lines
204 B
SQL
10 lines
204 B
SQL
WITH DuplicateSnapshots AS (
|
|
SELECT id,
|
|
url,
|
|
ROW_NUMBER() OVER (PARTITION BY url ORDER BY id) AS row_num
|
|
FROM snapshots
|
|
)
|
|
SELECT *
|
|
FROM DuplicateSnapshots
|
|
WHERE row_num > 1;
|