This commit is contained in:
antanst
2025-10-14 16:58:38 +03:00
parent d336bdffba
commit 8e1297a230
7 changed files with 222 additions and 24 deletions

View File

@@ -1,12 +1,16 @@
# CLAUDE.md # CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. This file provides guidance to AI Agents such as Claude Code or ChatGPT Codex when working with code in this repository.
## General guidelines
Use idiomatic Go as possible. Prefer simple code than complex.
## Project Overview ## Project Overview
Gemserve is a simple Gemini protocol server written in Go that serves static files over TLS-encrypted connections. The Gemini protocol is a lightweight, privacy-focused alternative to HTTP designed for serving text-based content. Gemserve is a simple Gemini protocol server written in Go that serves static files over TLS-encrypted connections. The Gemini protocol is a lightweight, privacy-focused alternative to HTTP designed for serving text-based content.
## Development Commands ### Development Commands
```bash ```bash
# Build, test, and format everything # Build, test, and format everything
@@ -15,7 +19,7 @@ make
# Run tests only # Run tests only
make test make test
# Build binary to ./dist/gemserve # Build binaries to ./dist/ (gemserve, gemget, gembench)
make build make build
# Format code with gofumpt and gci # Format code with gofumpt and gci
@@ -37,24 +41,28 @@ make clean
certs/generate.sh certs/generate.sh
``` ```
## Architecture ### Architecture
### Core Components Core Components
- **main.go**: Entry point with TLS server setup, signal handling, and graceful shutdown - **cmd/gemserve/gemserve.go**: Entry point with TLS server setup, signal handling, and graceful shutdown
- **cmd/gemget/**: Gemini protocol client for fetching content
- **cmd/gembench/**: Benchmarking tool for Gemini servers
- **server/**: Request processing, file serving, and Gemini protocol response handling - **server/**: Request processing, file serving, and Gemini protocol response handling
- **gemini/**: Gemini protocol implementation (URL parsing, status codes, path normalization) - **gemini/**: Gemini protocol implementation (URL parsing, status codes, path normalization)
- **config/**: CLI-based configuration system - **config/**: CLI-based configuration system
- **uid/**: Connection ID generation for logging - **lib/logging/**: Structured logging package with context-aware loggers
- **lib/apperrors/**: Application error handling (fatal vs non-fatal errors)
- **uid/**: Connection ID generation for logging (uses external vendor package)
### Key Patterns Key Patterns
- **Security First**: All file operations use `filepath.IsLocal()` and path cleaning to prevent directory traversal - **Security First**: All file operations use `filepath.IsLocal()` and path cleaning to prevent directory traversal
- **Error Handling**: Uses structured errors with `xerrors` package for consistent error propagation - **Error Handling**: Uses structured errors via `lib/apperrors` package distinguishing fatal from non-fatal errors
- **Logging**: Structured logging with configurable levels via internal logging package - **Logging**: Structured logging with configurable levels via internal logging package
- **Testing**: Table-driven tests with parallel execution, heavy focus on security edge cases - **Testing**: Table-driven tests with parallel execution, heavy focus on security edge cases
### Request Flow Request Flow
1. TLS connection established on port 1965 1. TLS connection established on port 1965
2. Read up to 1KB request (Gemini spec limit) 2. Read up to 1KB request (Gemini spec limit)
@@ -63,25 +71,28 @@ certs/generate.sh
5. Serve file or directory index with appropriate MIME type 5. Serve file or directory index with appropriate MIME type
6. Send response with proper Gemini status codes 6. Send response with proper Gemini status codes
## Configuration Configuration
Server configured via CLI flags: Server configured via CLI flags:
- `--listen`: Server address (default: localhost:1965) - `--listen`: Server address (default: localhost:1965)
- `--root-path`: Directory to serve files from - `--root-path`: Directory to serve files from
- `--dir-indexing`: Enable directory browsing - `--dir-indexing`: Enable directory browsing (default: false)
- `--log-level`: Logging verbosity - `--log-level`: Logging verbosity (debug, info, warn, error; default: info)
- `--response-timeout`: Response timeout in seconds - `--response-timeout`: Response timeout in seconds (default: 30)
- `--tls-cert`: TLS certificate file path (default: certs/server.crt)
- `--tls-key`: TLS key file path (default: certs/server.key)
- `--max-response-size`: Maximum response size in bytes (default: 5242880)
## Testing Strategy Testing Strategy
- **server/server_test.go**: Path security and file serving tests - **server/server_test.go**: Path security and file serving tests
- **gemini/url_test.go**: URL parsing and normalization tests - **gemini/url_test.go**: URL parsing and normalization tests
- Focus on security edge cases (Unicode, traversal attempts, malformed URLs) - Focus on security edge cases (Unicode, traversal attempts, malformed URLs)
- Use parallel test execution for performance - Use parallel test execution for performance
## Security Considerations Security Considerations
- All connections require TLS certificates (stored in certs/) - All connections require TLS certificates (stored in certs/)
- Path traversal protection is critical - test thoroughly when modifying file serving logic - Path traversal protection is critical - test thoroughly when modifying file serving logic
- Request size limited to 1KB per Gemini specification - Request size limited to 1KB per Gemini specification
- Input validation on all URLs and paths - Input validation on all URLs and paths

View File

@@ -1 +0,0 @@
@AGENTS.md

1
CLAUDE.md Symbolic link
View File

@@ -0,0 +1 @@
./AGENTS.md

View File

@@ -36,6 +36,7 @@ build: clean
mkdir -p ./dist mkdir -p ./dist
go build -mod=vendor -o ./dist/gemserve ./cmd/gemserve/gemserve.go go build -mod=vendor -o ./dist/gemserve ./cmd/gemserve/gemserve.go
go build -mod=vendor -o ./dist/gemget ./cmd/gemget/gemget.go go build -mod=vendor -o ./dist/gemget ./cmd/gemget/gemget.go
go build -mod=vendor -o ./dist/gembench ./cmd/gembench/gembench.go
build-docker: build build-docker: build
docker build -t gemserve . docker build -t gemserve .

182
cmd/gembench/gembench.go Normal file
View File

@@ -0,0 +1,182 @@
package main
// Benchmarks a Gemini server.
import (
"context"
"crypto/tls"
"flag"
"fmt"
"io"
"log/slog"
"net/url"
"os"
"os/signal"
"strings"
"sync"
"syscall"
"time"
"gemserve/lib/logging"
)
func main() {
// Parse command-line flags
insecure := flag.Bool("insecure", false, "Skip TLS certificate verification")
totalConnections := flag.Int("total-connections", 250, "Total connections to make")
parallelism := flag.Int("parallelism", 10, "How many connections to run in parallel")
flag.Parse()
// Get the URL from arguments
args := flag.Args()
if len(args) != 1 {
fmt.Fprintf(os.Stderr, "Usage: gemget [--insecure] <gemini-url>\n")
os.Exit(1)
}
logging.SetupLogging(slog.LevelInfo)
logger := logging.Logger
ctx := logging.WithLogger(context.Background(), logger)
geminiURL := args[0]
host := validateUrl(geminiURL)
if host == "" {
logger.Error("Invalid URL.")
os.Exit(1)
}
start := time.Now()
err := benchmark(ctx, geminiURL, host, *insecure, *totalConnections, *parallelism)
if err != nil {
logger.Error(err.Error())
os.Exit(1)
}
end := time.Now()
tookMs := end.Sub(start).Milliseconds()
logger.Info("End.", "ms", tookMs)
}
var wg sync.WaitGroup
type ctxKey int
const ctxKeyJobIndex ctxKey = 1
func benchmark(ctx context.Context, u string, h string, insecure bool, totalConnections int, parallelism int) error {
logger := logging.FromContext(ctx)
signals := make(chan os.Signal, 1)
signal.Notify(signals, syscall.SIGINT, syscall.SIGTERM)
// Root context, used to cancel
// connections and graceful shutdown.
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// Semaphore to limit concurrency.
// Goroutines put value to channel (acquire slot)
// and consume value from channel (release slot).
semaphore := make(chan struct{}, parallelism)
loop:
for i := 0; i < totalConnections; i++ {
select {
case <-signals:
logger.Warn("Received SIGINT or SIGTERM signal, shutting down gracefully")
cancel()
break loop
case semaphore <- struct{}{}: // Acquire slot
wg.Add(1)
go func(jobIndex int) {
defer func() {
<-semaphore // Release slot
wg.Done()
}()
ctxWithValue := context.WithValue(ctx, ctxKeyJobIndex, jobIndex)
ctxWithTimeout, cancel := context.WithTimeout(ctxWithValue, 60*time.Second)
defer cancel()
err := connect(ctxWithTimeout, u, h, insecure)
if err != nil {
logger.Warn(fmt.Sprintf("%d error: %v", jobIndex, err))
}
}(i)
}
}
wg.Wait()
return nil
}
func validateUrl(u string) string {
// Parse the URL
parsedURL, err := url.Parse(u)
if err != nil {
fmt.Fprintf(os.Stderr, "Error parsing URL: %v\n", err)
os.Exit(1)
}
// Ensure it's a gemini URL
if parsedURL.Scheme != "gemini" {
fmt.Fprintf(os.Stderr, "Error: URL must use gemini:// scheme\n")
os.Exit(1)
}
// Get host and port
host := parsedURL.Host
if !strings.Contains(host, ":") {
host = host + ":1965" // Default Gemini port
}
return host
}
func connect(ctx context.Context, url string, host string, insecure bool) error {
logger := logging.FromContext(ctx)
tlsConfig := &tls.Config{
InsecureSkipVerify: insecure,
MinVersion: tls.VersionTLS12,
}
// Context checkpoint
if ctx.Err() != nil {
return nil
}
// Connect to the server
conn, err := tls.Dial("tcp", host, tlsConfig)
if err != nil {
return err
}
// Set connection deadline based on context
if deadline, ok := ctx.Deadline(); ok {
_ = conn.SetDeadline(deadline)
}
defer func() {
_ = conn.Close()
}()
// Context checkpoint
if ctx.Err() != nil {
return nil
}
// Send the request (URL + CRLF)
request := url + "\r\n"
_, err = conn.Write([]byte(request))
if err != nil {
return err
}
// Context checkpoint
if ctx.Err() != nil {
return nil
}
// Read and dump response
_, err = io.Copy(io.Discard, conn)
if err != nil {
return err
}
jobIndex := ctx.Value(ctxKeyJobIndex)
logger.Debug(fmt.Sprintf("%d done", jobIndex))
return nil
}

View File

@@ -1,5 +1,7 @@
package main package main
// Simply does Gemini requests and prints output.
import ( import (
"crypto/tls" "crypto/tls"
"flag" "flag"

View File

@@ -1,5 +1,7 @@
package main package main
// A Gemini server.
import ( import (
"context" "context"
"crypto/tls" "crypto/tls"
@@ -22,7 +24,7 @@ import (
func main() { func main() {
config.CONFIG = *config.GetConfig() config.CONFIG = *config.GetConfig()
logging.SetupLogging() logging.SetupLogging(config.CONFIG.LogLevel)
logger := logging.Logger logger := logging.Logger
ctx := logging.WithLogger(context.Background(), logger) ctx := logging.WithLogger(context.Background(), logger)
err := runApp(ctx) err := runApp(ctx)
@@ -42,9 +44,11 @@ func runApp(ctx context.Context) error {
signals := make(chan os.Signal, 1) signals := make(chan os.Signal, 1)
signal.Notify(signals, syscall.SIGINT, syscall.SIGTERM) signal.Notify(signals, syscall.SIGINT, syscall.SIGTERM)
// Only this file should send to this channel.
// All external functions should return errors.
fatalErrors := make(chan error) fatalErrors := make(chan error)
// Root server context, used to cancel // Root context, used to cancel
// connections and graceful shutdown. // connections and graceful shutdown.
serverCtx, cancel := context.WithCancel(ctx) serverCtx, cancel := context.WithCancel(ctx)
defer cancel() defer cancel()

View File

@@ -6,8 +6,6 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"gemserve/config"
"github.com/lmittmann/tint" "github.com/lmittmann/tint"
) )
@@ -36,8 +34,8 @@ func FromContext(ctx context.Context) *slog.Logger {
return slog.Default() return slog.Default()
} }
func SetupLogging() { func SetupLogging(logLevel slog.Level) {
programLevel.Set(config.CONFIG.LogLevel) programLevel.Set(logLevel)
// With coloring (uses external package) // With coloring (uses external package)
opts := &tint.Options{ opts := &tint.Options{
AddSource: true, AddSource: true,