GGUF Models and Downloads
MetalGlot uses downloadable GGUF models for text translation and separate vision models for image translation. Open Manage AI models from the home page toolbar to handle GGUF downloads, vision model downloads, and model loading.
The model manager always uses the model folder assigned to the currently selected container. If that container was created with a different folder, the model manager will show a different model library.
Download GGUF Text Models
Section titled “Download GGUF Text Models”The main model list is the GGUF catalog.
Model Families
Section titled “Model Families”| Family | Best for | General tradeoff |
|---|---|---|
| Translate Gemma 4B | Lower-end systems and faster iteration | Fastest, lightest, less capable than larger families |
| Translate Gemma 12B | Balanced everyday translation work | Stronger results with higher memory and compute requirements |
| Translate Gemma 27B | High-end systems | Most demanding, slowest without substantial CPU or GPU resources |
Quantization Guidance
Section titled “Quantization Guidance”Quantization affects size, speed, and quality.
- Lower quantizations are smaller and usually faster.
- Higher quantizations, such as
Q8_0, preserve more quality but require more disk space and memory. - Full precision options are the heaviest downloads and are best reserved for systems that can comfortably handle them.
If you are not sure where to start, begin with a 4B or 12B model before moving up to 27B.
Recommended GGUF Download Flow
Section titled “Recommended GGUF Download Flow”- Select the container you want to use.
- Open Manage AI models.
- Confirm the Download location shown at the top of the dialog.
- Download the GGUF model family and quantization that fit your hardware.
- Load the model into memory when you are ready to translate.
Download Vision Models For Image Translation
Section titled “Download Vision Models For Image Translation”Image translation uses separate vision handler files in addition to the base GGUF model. These appear in Available vision handler models.
When A Vision Model Is Necessary
Section titled “When A Vision Model Is Necessary”You need a vision model when you want to use Image Contents.
In the current workflow, image translation requires loading a 12B or 27B model with vision enabled. If vision is not enabled, the image workspace warns you and blocks the task until you load a compatible setup.
How To Prepare For Image Translation
Section titled “How To Prepare For Image Translation”- Download a supported 12B or 27B GGUF model.
- Download the matching vision handler model for the same family.
- Load the base model and enable the matching vision option when the app prompts you during the load flow.
- Open Image Contents and translate the selected region.
If you are choosing between the available vision quantizations, Q8_0 is the lighter option and f16 is the larger, higher-quality option.
Important: Disable Vision Before Text Tasks
Section titled “Important: Disable Vision Before Text Tasks”After finishing an image task, do not continue directly into a text workflow while vision mode is still enabled.
Instead:
- unload the vision-enabled model
- reload the text model without a vision handler
- return to the text-based translation page you want to use
MetalGlot blocks non-image workspaces while vision mode remains active.
Pause, Resume, And Cancel Model Downloads
Section titled “Pause, Resume, And Cancel Model Downloads”The Download monitor in Manage AI models lets you control active GGUF and vision-model downloads.
| Action | What it does | When to use it |
|---|---|---|
| Pause | Stops the download and keeps the partial file | Use when you want to continue later without losing progress |
| Resume | Continues a paused download from the saved partial progress when possible | Use after a pause or interrupted session |
| Cancel | Stops the download and removes the partial file | Use when you want to abandon the current partial download and restart fresh later |
The key difference is simple:
- Pause preserves progress.
- Cancel discards progress.
Download Monitoring Expectations
Section titled “Download Monitoring Expectations”During active downloads, MetalGlot shows:
- Current progress.
- Total size.
- Transfer speed.
- Estimated remaining time.
If a model download fails or is interrupted, the model manager can show the partial state so you can decide whether to resume or start over.
Advanced Note For Local Backend Users
Section titled “Advanced Note For Local Backend Users”When the container is running and a model is loaded, MetalGlot sends text translation requests to the local backend at http://127.0.0.1:8080, using POST /translate for standard supported combinations and POST /experimental_translation for partially supported ones.
Those text requests include fields such as:
- Field
model. - Field
source_lang_code. - Field
target_lang_code. - Field
text. - Field
content_type.
If you are integrating with the local backend directly, those are the main request concepts to understand for text workflows.