Docker Images

MetalGlot’s FastAPI backend is packaged as Docker images so the local translation service can be deployed consistently across laptops, desktops, and dedicated inference machines.

Available images

CPU image

Repository: grctest/fastapi_gemma_translate.
Intended use: CPU-only deployments and simple local testing.

This is the easiest baseline image to run when you want a straightforward local server without Nvidia GPU dependencies.

CUDA image

Repository: grctest/fastapi_gemma_translate_cuda.
Intended use: Nvidia GPU deployments.

The CUDA image is the performance path for larger TranslateGemma models and heavier workloads.

GPU variants

The project README documents three CUDA-oriented build variants:

Variant legacy for Pascal / GTX 10xx cards.
Variant mainstream for Turing through Ada, including 20xx, 30xx, 40xx, and A100 hardware.
Variant future for Blackwell / 50xx cards.

That split keeps the deployment story practical across multiple Nvidia generations instead of assuming one CUDA build fits every machine.

What the containers expect

Both CPU and CUDA containers are designed around the same basic runtime pattern:

Expose the FastAPI server on port 8080.
Mount a local models directory into /code/models.
Load GGUF model files from that mounted volume.

For the CUDA image, the documented runtime also uses:

Flag --gpus all.
Setting LLAMA_N_GPU_LAYERS=-1.

Example pull commands

docker image pull grctest/fastapi_gemma_translate
docker image pull grctest/fastapi_gemma_translate_cuda:legacy
docker image pull grctest/fastapi_gemma_translate_cuda:mainstream
docker image pull grctest/fastapi_gemma_translate_cuda:future

In MetalGlot itself, users should create, start, stop, and delete containers through the application UI rather than by manually assembling docker run commands.

Choosing the right image

Environment	Recommended image
Laptop or workstation without Nvidia CUDA	`grctest/fastapi_gemma_translate`
Nvidia workstation with a 10xx Pascal card	`grctest/fastapi_gemma_translate_cuda:legacy`
Nvidia workstation with 20xx, 30xx, 40xx, or A100 hardware	`grctest/fastapi_gemma_translate_cuda:mainstream`
Newer Blackwell / 50xx hardware	`grctest/fastapi_gemma_translate_cuda:future`