This guide starts the supported Ember Docker Compose deployment. Install Hermes separately if you also want an agent or messaging gateway.
Requirements
Ember targets native x86_64 Linux on AMD Strix Halo (gfx1151), with approximately 128 GiB unified memory, Docker Engine and Compose v2, and working /dev/kfd and /dev/dri GPU devices. Allow at least 120 GiB of free storage for the default model downloads and staging, plus space for caches.
Other GPU architectures and Docker Desktop are outside the supported deployment. Use the repository requirements for platform details.
Start the service
Obtain the source from otheru-ai/ember and select the release you intend to run. From that checkout:
scripts/preflight.sh && docker compose up -d
The Compose configuration selects a pinned image and model revision. First startup downloads and verifies the model artifacts, then loads them. Follow progress with:
docker compose logs -f ember
The standard API address is http://127.0.0.1:8080. An existing installation may use another port; OtherU's internal host uses 8000.
Verify inference
Once startup completes, run the release's smoke check:
scripts/smoke_test.sh --generate
You can also inspect the advertised model without starting a generation:
curl --fail --show-error http://127.0.0.1:8080/v1/models
A model listing verifies the HTTP service; a successful generated response verifies the inference path. If either fails, consult Operations Runbook.
Configure access and storage
Copy .env.example to .env to override the port or model and cache directories. The standard settings keep Ember on loopback. Ember has no built-in authentication; network exposure requires access controls outside the server. See Security Model.
Use the configuration shipped with your release for model changes. The model, vision projector, and drafter must be compatible; an arbitrary GGUF is not a supported substitute.
Stop and restart
docker compose down
docker compose up -d
These commands stop and start the Compose service. Downloaded models and the persistent cache remain in their configured host directories.
For agent integration, continue with Connect Hermes Agent.