.
This commit is contained in:
43
AGENTS.md
43
AGENTS.md
@@ -1,12 +1,16 @@
|
|||||||
# CLAUDE.md
|
# CLAUDE.md
|
||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
This file provides guidance to AI Agents such as Claude Code or ChatGPT Codex when working with code in this repository.
|
||||||
|
|
||||||
|
## General guidelines
|
||||||
|
|
||||||
|
Use idiomatic Go as possible. Prefer simple code than complex.
|
||||||
|
|
||||||
## Project Overview
|
## Project Overview
|
||||||
|
|
||||||
Gemserve is a simple Gemini protocol server written in Go that serves static files over TLS-encrypted connections. The Gemini protocol is a lightweight, privacy-focused alternative to HTTP designed for serving text-based content.
|
Gemserve is a simple Gemini protocol server written in Go that serves static files over TLS-encrypted connections. The Gemini protocol is a lightweight, privacy-focused alternative to HTTP designed for serving text-based content.
|
||||||
|
|
||||||
## Development Commands
|
### Development Commands
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Build, test, and format everything
|
# Build, test, and format everything
|
||||||
@@ -15,7 +19,7 @@ make
|
|||||||
# Run tests only
|
# Run tests only
|
||||||
make test
|
make test
|
||||||
|
|
||||||
# Build binary to ./dist/gemserve
|
# Build binaries to ./dist/ (gemserve, gemget, gembench)
|
||||||
make build
|
make build
|
||||||
|
|
||||||
# Format code with gofumpt and gci
|
# Format code with gofumpt and gci
|
||||||
@@ -37,24 +41,28 @@ make clean
|
|||||||
certs/generate.sh
|
certs/generate.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
## Architecture
|
### Architecture
|
||||||
|
|
||||||
### Core Components
|
Core Components
|
||||||
|
|
||||||
- **main.go**: Entry point with TLS server setup, signal handling, and graceful shutdown
|
- **cmd/gemserve/gemserve.go**: Entry point with TLS server setup, signal handling, and graceful shutdown
|
||||||
|
- **cmd/gemget/**: Gemini protocol client for fetching content
|
||||||
|
- **cmd/gembench/**: Benchmarking tool for Gemini servers
|
||||||
- **server/**: Request processing, file serving, and Gemini protocol response handling
|
- **server/**: Request processing, file serving, and Gemini protocol response handling
|
||||||
- **gemini/**: Gemini protocol implementation (URL parsing, status codes, path normalization)
|
- **gemini/**: Gemini protocol implementation (URL parsing, status codes, path normalization)
|
||||||
- **config/**: CLI-based configuration system
|
- **config/**: CLI-based configuration system
|
||||||
- **uid/**: Connection ID generation for logging
|
- **lib/logging/**: Structured logging package with context-aware loggers
|
||||||
|
- **lib/apperrors/**: Application error handling (fatal vs non-fatal errors)
|
||||||
|
- **uid/**: Connection ID generation for logging (uses external vendor package)
|
||||||
|
|
||||||
### Key Patterns
|
Key Patterns
|
||||||
|
|
||||||
- **Security First**: All file operations use `filepath.IsLocal()` and path cleaning to prevent directory traversal
|
- **Security First**: All file operations use `filepath.IsLocal()` and path cleaning to prevent directory traversal
|
||||||
- **Error Handling**: Uses structured errors with `xerrors` package for consistent error propagation
|
- **Error Handling**: Uses structured errors via `lib/apperrors` package distinguishing fatal from non-fatal errors
|
||||||
- **Logging**: Structured logging with configurable levels via internal logging package
|
- **Logging**: Structured logging with configurable levels via internal logging package
|
||||||
- **Testing**: Table-driven tests with parallel execution, heavy focus on security edge cases
|
- **Testing**: Table-driven tests with parallel execution, heavy focus on security edge cases
|
||||||
|
|
||||||
### Request Flow
|
Request Flow
|
||||||
|
|
||||||
1. TLS connection established on port 1965
|
1. TLS connection established on port 1965
|
||||||
2. Read up to 1KB request (Gemini spec limit)
|
2. Read up to 1KB request (Gemini spec limit)
|
||||||
@@ -63,23 +71,26 @@ certs/generate.sh
|
|||||||
5. Serve file or directory index with appropriate MIME type
|
5. Serve file or directory index with appropriate MIME type
|
||||||
6. Send response with proper Gemini status codes
|
6. Send response with proper Gemini status codes
|
||||||
|
|
||||||
## Configuration
|
Configuration
|
||||||
|
|
||||||
Server configured via CLI flags:
|
Server configured via CLI flags:
|
||||||
- `--listen`: Server address (default: localhost:1965)
|
- `--listen`: Server address (default: localhost:1965)
|
||||||
- `--root-path`: Directory to serve files from
|
- `--root-path`: Directory to serve files from
|
||||||
- `--dir-indexing`: Enable directory browsing
|
- `--dir-indexing`: Enable directory browsing (default: false)
|
||||||
- `--log-level`: Logging verbosity
|
- `--log-level`: Logging verbosity (debug, info, warn, error; default: info)
|
||||||
- `--response-timeout`: Response timeout in seconds
|
- `--response-timeout`: Response timeout in seconds (default: 30)
|
||||||
|
- `--tls-cert`: TLS certificate file path (default: certs/server.crt)
|
||||||
|
- `--tls-key`: TLS key file path (default: certs/server.key)
|
||||||
|
- `--max-response-size`: Maximum response size in bytes (default: 5242880)
|
||||||
|
|
||||||
## Testing Strategy
|
Testing Strategy
|
||||||
|
|
||||||
- **server/server_test.go**: Path security and file serving tests
|
- **server/server_test.go**: Path security and file serving tests
|
||||||
- **gemini/url_test.go**: URL parsing and normalization tests
|
- **gemini/url_test.go**: URL parsing and normalization tests
|
||||||
- Focus on security edge cases (Unicode, traversal attempts, malformed URLs)
|
- Focus on security edge cases (Unicode, traversal attempts, malformed URLs)
|
||||||
- Use parallel test execution for performance
|
- Use parallel test execution for performance
|
||||||
|
|
||||||
## Security Considerations
|
Security Considerations
|
||||||
|
|
||||||
- All connections require TLS certificates (stored in certs/)
|
- All connections require TLS certificates (stored in certs/)
|
||||||
- Path traversal protection is critical - test thoroughly when modifying file serving logic
|
- Path traversal protection is critical - test thoroughly when modifying file serving logic
|
||||||
|
|||||||
1
Makefile
1
Makefile
@@ -36,6 +36,7 @@ build: clean
|
|||||||
mkdir -p ./dist
|
mkdir -p ./dist
|
||||||
go build -mod=vendor -o ./dist/gemserve ./cmd/gemserve/gemserve.go
|
go build -mod=vendor -o ./dist/gemserve ./cmd/gemserve/gemserve.go
|
||||||
go build -mod=vendor -o ./dist/gemget ./cmd/gemget/gemget.go
|
go build -mod=vendor -o ./dist/gemget ./cmd/gemget/gemget.go
|
||||||
|
go build -mod=vendor -o ./dist/gembench ./cmd/gembench/gembench.go
|
||||||
|
|
||||||
build-docker: build
|
build-docker: build
|
||||||
docker build -t gemserve .
|
docker build -t gemserve .
|
||||||
|
|||||||
182
cmd/gembench/gembench.go
Normal file
182
cmd/gembench/gembench.go
Normal file
@@ -0,0 +1,182 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// Benchmarks a Gemini server.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/tls"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gemserve/lib/logging"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
// Parse command-line flags
|
||||||
|
insecure := flag.Bool("insecure", false, "Skip TLS certificate verification")
|
||||||
|
totalConnections := flag.Int("total-connections", 250, "Total connections to make")
|
||||||
|
parallelism := flag.Int("parallelism", 10, "How many connections to run in parallel")
|
||||||
|
flag.Parse()
|
||||||
|
|
||||||
|
// Get the URL from arguments
|
||||||
|
args := flag.Args()
|
||||||
|
if len(args) != 1 {
|
||||||
|
fmt.Fprintf(os.Stderr, "Usage: gemget [--insecure] <gemini-url>\n")
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
|
||||||
|
logging.SetupLogging(slog.LevelInfo)
|
||||||
|
logger := logging.Logger
|
||||||
|
ctx := logging.WithLogger(context.Background(), logger)
|
||||||
|
|
||||||
|
geminiURL := args[0]
|
||||||
|
host := validateUrl(geminiURL)
|
||||||
|
if host == "" {
|
||||||
|
logger.Error("Invalid URL.")
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
start := time.Now()
|
||||||
|
err := benchmark(ctx, geminiURL, host, *insecure, *totalConnections, *parallelism)
|
||||||
|
if err != nil {
|
||||||
|
logger.Error(err.Error())
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
end := time.Now()
|
||||||
|
tookMs := end.Sub(start).Milliseconds()
|
||||||
|
logger.Info("End.", "ms", tookMs)
|
||||||
|
}
|
||||||
|
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
|
||||||
|
type ctxKey int
|
||||||
|
|
||||||
|
const ctxKeyJobIndex ctxKey = 1
|
||||||
|
|
||||||
|
func benchmark(ctx context.Context, u string, h string, insecure bool, totalConnections int, parallelism int) error {
|
||||||
|
logger := logging.FromContext(ctx)
|
||||||
|
signals := make(chan os.Signal, 1)
|
||||||
|
signal.Notify(signals, syscall.SIGINT, syscall.SIGTERM)
|
||||||
|
// Root context, used to cancel
|
||||||
|
// connections and graceful shutdown.
|
||||||
|
ctx, cancel := context.WithCancel(ctx)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
// Semaphore to limit concurrency.
|
||||||
|
// Goroutines put value to channel (acquire slot)
|
||||||
|
// and consume value from channel (release slot).
|
||||||
|
semaphore := make(chan struct{}, parallelism)
|
||||||
|
|
||||||
|
loop:
|
||||||
|
for i := 0; i < totalConnections; i++ {
|
||||||
|
select {
|
||||||
|
case <-signals:
|
||||||
|
logger.Warn("Received SIGINT or SIGTERM signal, shutting down gracefully")
|
||||||
|
cancel()
|
||||||
|
break loop
|
||||||
|
case semaphore <- struct{}{}: // Acquire slot
|
||||||
|
wg.Add(1)
|
||||||
|
go func(jobIndex int) {
|
||||||
|
defer func() {
|
||||||
|
<-semaphore // Release slot
|
||||||
|
wg.Done()
|
||||||
|
}()
|
||||||
|
ctxWithValue := context.WithValue(ctx, ctxKeyJobIndex, jobIndex)
|
||||||
|
ctxWithTimeout, cancel := context.WithTimeout(ctxWithValue, 60*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
err := connect(ctxWithTimeout, u, h, insecure)
|
||||||
|
if err != nil {
|
||||||
|
logger.Warn(fmt.Sprintf("%d error: %v", jobIndex, err))
|
||||||
|
}
|
||||||
|
}(i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func validateUrl(u string) string {
|
||||||
|
// Parse the URL
|
||||||
|
parsedURL, err := url.Parse(u)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "Error parsing URL: %v\n", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ensure it's a gemini URL
|
||||||
|
if parsedURL.Scheme != "gemini" {
|
||||||
|
fmt.Fprintf(os.Stderr, "Error: URL must use gemini:// scheme\n")
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Get host and port
|
||||||
|
host := parsedURL.Host
|
||||||
|
if !strings.Contains(host, ":") {
|
||||||
|
host = host + ":1965" // Default Gemini port
|
||||||
|
}
|
||||||
|
|
||||||
|
return host
|
||||||
|
}
|
||||||
|
|
||||||
|
func connect(ctx context.Context, url string, host string, insecure bool) error {
|
||||||
|
logger := logging.FromContext(ctx)
|
||||||
|
tlsConfig := &tls.Config{
|
||||||
|
InsecureSkipVerify: insecure,
|
||||||
|
MinVersion: tls.VersionTLS12,
|
||||||
|
}
|
||||||
|
|
||||||
|
// Context checkpoint
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Connect to the server
|
||||||
|
conn, err := tls.Dial("tcp", host, tlsConfig)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Set connection deadline based on context
|
||||||
|
if deadline, ok := ctx.Deadline(); ok {
|
||||||
|
_ = conn.SetDeadline(deadline)
|
||||||
|
}
|
||||||
|
|
||||||
|
defer func() {
|
||||||
|
_ = conn.Close()
|
||||||
|
}()
|
||||||
|
|
||||||
|
// Context checkpoint
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Send the request (URL + CRLF)
|
||||||
|
request := url + "\r\n"
|
||||||
|
_, err = conn.Write([]byte(request))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Context checkpoint
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Read and dump response
|
||||||
|
_, err = io.Copy(io.Discard, conn)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
jobIndex := ctx.Value(ctxKeyJobIndex)
|
||||||
|
logger.Debug(fmt.Sprintf("%d done", jobIndex))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -1,5 +1,7 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
|
// Simply does Gemini requests and prints output.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"crypto/tls"
|
"crypto/tls"
|
||||||
"flag"
|
"flag"
|
||||||
|
|||||||
@@ -1,5 +1,7 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
|
// A Gemini server.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"crypto/tls"
|
"crypto/tls"
|
||||||
@@ -22,7 +24,7 @@ import (
|
|||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
config.CONFIG = *config.GetConfig()
|
config.CONFIG = *config.GetConfig()
|
||||||
logging.SetupLogging()
|
logging.SetupLogging(config.CONFIG.LogLevel)
|
||||||
logger := logging.Logger
|
logger := logging.Logger
|
||||||
ctx := logging.WithLogger(context.Background(), logger)
|
ctx := logging.WithLogger(context.Background(), logger)
|
||||||
err := runApp(ctx)
|
err := runApp(ctx)
|
||||||
@@ -42,9 +44,11 @@ func runApp(ctx context.Context) error {
|
|||||||
signals := make(chan os.Signal, 1)
|
signals := make(chan os.Signal, 1)
|
||||||
signal.Notify(signals, syscall.SIGINT, syscall.SIGTERM)
|
signal.Notify(signals, syscall.SIGINT, syscall.SIGTERM)
|
||||||
|
|
||||||
|
// Only this file should send to this channel.
|
||||||
|
// All external functions should return errors.
|
||||||
fatalErrors := make(chan error)
|
fatalErrors := make(chan error)
|
||||||
|
|
||||||
// Root server context, used to cancel
|
// Root context, used to cancel
|
||||||
// connections and graceful shutdown.
|
// connections and graceful shutdown.
|
||||||
serverCtx, cancel := context.WithCancel(ctx)
|
serverCtx, cancel := context.WithCancel(ctx)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|||||||
@@ -6,8 +6,6 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
|
||||||
"gemserve/config"
|
|
||||||
|
|
||||||
"github.com/lmittmann/tint"
|
"github.com/lmittmann/tint"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -36,8 +34,8 @@ func FromContext(ctx context.Context) *slog.Logger {
|
|||||||
return slog.Default()
|
return slog.Default()
|
||||||
}
|
}
|
||||||
|
|
||||||
func SetupLogging() {
|
func SetupLogging(logLevel slog.Level) {
|
||||||
programLevel.Set(config.CONFIG.LogLevel)
|
programLevel.Set(logLevel)
|
||||||
// With coloring (uses external package)
|
// With coloring (uses external package)
|
||||||
opts := &tint.Options{
|
opts := &tint.Options{
|
||||||
AddSource: true,
|
AddSource: true,
|
||||||
|
|||||||
Reference in New Issue
Block a user