TRAWL

Appearance

Standalone Containers ​

Each service has its own Dockerfile and can be built and run independently. All builds use the repository root as their build context, including web and docs.

For deployment platforms, set the build context to . and the Dockerfile path to apps/web/Dockerfile or apps/docs/Dockerfile. The shared root .dockerignore supports tools that relocate the Dockerfile and omit its adjacent Dockerfile.dockerignore.

Scraper API ​

The API image is published to GHCR for release tags. Pull it directly - no local build needed.

bash
# Pull
docker pull ghcr.io/germondai/trawl:latest

# Run (no Redis - session caching disabled, scraping still works)
docker run -d \
  --name trawl \
  -p 8191:8191 \
  --shm-size=1gb \
  ghcr.io/germondai/trawl:latest

# Run (with external Redis)
docker run -d \
  --name trawl \
  -p 8191:8191 \
  --shm-size=1gb \
  -e REDIS_URL=redis://your-redis-host:6379 \
  -e BROWSER_POOL_SIZE=1 \
  ghcr.io/germondai/trawl:latest

Older CPUs & Synology NAS ​

If your host CPU doesn't support AVX2 - older Synology NAS units, Atom/Celeron-era hardware, pre-2013 x86_64 - use the :baseline tag instead. Same image, same commands, built on Bun's baseline runtime.

Standard Bun requires Linux kernel 5.1+ (5.6+ recommended); the baseline build degrades gracefully on kernels as old as 3.10 (RHEL 7-era) - covering most Synology DSM installs out of the box.

bash
# Pull
docker pull ghcr.io/germondai/trawl:baseline

# Run
docker run -d \
  --name trawl \
  -p 8191:8191 \
  --shm-size=1gb \
  ghcr.io/germondai/trawl:baseline

Not sure which tag you need?

Try :latest first. If the container exits immediately or crash-loops, switch to :baseline for compatibility with older CPUs and kernels. See the Docker image tags.

To build from source instead:

bash
# Run from the repo root
docker build -f apps/api/Dockerfile -t trawl .
docker run -d --name trawl -p 8191:8191 --shm-size=1gb trawl

To build the baseline variant from source, use apps/api/Dockerfile.baseline instead:

bash
docker build -f apps/api/Dockerfile.baseline -t trawl:baseline .
docker run -d --name trawl -p 8191:8191 --shm-size=1gb trawl:baseline

Both API Dockerfiles omit the spoofed macOS and Windows font bundles by default. TRAWL detects their absence and limits these compact images to Linux fingerprints, keeping the advertised OS, font probes, and rendered output consistent. To retain the complete OS pool, build with:

bash
docker build \
  --build-arg CAMOUFOX_KEEP_SPOOFED_OS_FONTS=1 \
  -f apps/api/Dockerfile \
  -t trawl .

This adds approximately 891 MB to the image and is intentionally available only as a custom build. Published images remain compact and use Linux fingerprints. Other build arguments pin or validate bundled dependencies and normally should not be overridden: UBO_VERSION, UBO_AMO_FILE_ID, UBO_SHA256, GEOLITE_CITY_MIN_BYTES, and (for the baseline image) BUN_VERSION.

Build context

Both API Dockerfiles (apps/api/Dockerfile and apps/api/Dockerfile.baseline) require the repo root as the build context because they copy workspace packages (packages/types, packages/browser, packages/tiers). Always run docker build from the repo root with -f apps/api/Dockerfile (or -f apps/api/Dockerfile.baseline).

Why pull instead of build?

Building the API image downloads the Camoufox Firefox binary (~660 MB) and compiles dependencies - takes 5-10 minutes on first run. The published image on GHCR skips all of that.

Web (landing page) ​

bash
# Build - run from repo root
docker build -f apps/web/Dockerfile -t trawl-web .

# Run
docker run -d \
  --name trawl-web \
  -p 3000:80 \
  trawl-web

Static HTML served by nginx. No runtime dependencies.

Docs ​

bash
# Build - run from repo root
docker build -f apps/docs/Dockerfile -t trawl-docs .

# Run
docker run -d \
  --name trawl-docs \
  -p 3001:80 \
  trawl-docs

Access at http://localhost:3001. Static nginx, no runtime dependencies.

For subdomain deployment (e.g. docs.yourdomain.com), point the subdomain at port 3001 via your reverse proxy - no path configuration needed.

Useful commands ​

Swap :latest for :baseline in any command below if you're on older/no-AVX2 hardware.

bash
# Check what's running
docker ps

# Tail logs
docker logs -f trawl

# Stop and remove
docker stop trawl && docker rm trawl

# Update the scraper to latest
docker pull ghcr.io/germondai/trawl:latest
docker stop trawl && docker rm trawl
docker run -d --name trawl -p 8191:8191 --shm-size=1gb ghcr.io/germondai/trawl:latest