antanst
cd60c1363b
- [x] Concurrent downloading with workers - [x] Concurrent connection limit per host - [x] URL Blacklist - [x] Save image/* and text/* files - [x] Configuration via environment variables - [x] Storing snapshots in PostgreSQL - [x] Proper response header & body UTF-8 and format validation . . .
15 lines
246 B
Bash
Executable File
15 lines
246 B
Bash
Executable File
#!/bin/sh
|
|
set -eu
|
|
|
|
MAX_RESPONSE_SIZE=10485760 \
|
|
LOG_LEVEL=info \
|
|
ROOT_PATH=./snaps \
|
|
RESPONSE_TIMEOUT=10 \
|
|
NUM_OF_WORKERS=5 \
|
|
PG_DATABASE=gemini \
|
|
PG_HOST=127.0.0.1 \
|
|
PG_PORT=5433 \
|
|
PG_USER=gemini \
|
|
PG_PASSWORD=gemini \
|
|
go run ./migrate1_host.go
|