Inference Settings
MetalGlot lets you optionally send inference settings to its local translation backend. Most users can keep the defaults, but these controls are useful when you need longer outputs, more stable translations, or careful sampling changes.
Open Configure Inference from the home page toolbar after your translation container is running.
Translation Quality And The Optional English Bridge
Section titled “Translation Quality And The Optional English Bridge”The English bridge is separate from the Configure Inference dialog, but it affects how a translation request is routed.
To open it, click the top circular button in the center action column between the source and target panes. That button opens the Translation Quality modal.
Inside the modal, MetalGlot shows the current quality tier for the selected language pair and, when available, a switch for Use English bridge.
When The English Bridge Appears
Section titled “When The English Bridge Appears”The switch is only available when both the source and target languages are non-English. For direct English-to-target or target-to-English work, there is no bridge path to toggle because English is already one side of the translation.
When The English Bridge Is Recommended
Section titled “When The English Bridge Is Recommended”MetalGlot compares the direct route and the route through base English. If the direct pair does not have a stronger optimized path, the app can recommend the bridge instead.
In practice, that means the bridge is most useful when:
- The direct non-English pair is still zero-shot or otherwise less optimized.
- The source to English leg is stronger.
- The English to target leg is stronger.
When that happens, the modal can show Use English bridge (recommended) and the switch may default to enabled for that language pair.
What The English Bridge Does
Section titled “What The English Bridge Does”When enabled, MetalGlot translates in two steps:
- source language to English
- English to the target language
This usually takes longer because each string is translated twice, but it can improve quality for language pairs that do not have a strong direct route.
How To Think About It
Section titled “How To Think About It”- Use the direct route when the selected pair already has a strong direct path.
- Try the English bridge when the quality modal recommends it.
- Expect bridge-enabled runs to be slower than direct translation.
- If you change the source or target language, MetalGlot recalculates the recommended route for the new pair.
Inference settings in MetalGlot are opt-in. Every setting has its own switch. If a setting is turned off, MetalGlot does not send that value to the local backend.
Default Behavior
Section titled “Default Behavior”By default, only Max new tokens is enabled.
| Setting | Default state | Default value | Range |
|---|---|---|---|
| Max new tokens | Enabled | 200 | 1 to 2000 |
| Temperature | Disabled | 0.2 | 0 to 2 |
| Top P | Disabled | 0.95 | 0 to 1 |
| Top K | Disabled | 40 | 0 to 1000 |
| Min P | Disabled | 0.05 | 0 to 1 |
| Repeat penalty | Disabled | 1.0 | 0 to 2 |
| Presence penalty | Disabled | 0.0 | 0 to 2 |
| Frequency penalty | Disabled | 0.0 | 0 to 2 |
What Each Setting Does
Section titled “What Each Setting Does”Max New Tokens
Section titled “Max New Tokens”Sets the maximum number of tokens the model can generate for each translation. If longer strings are being cut off, this is the first setting to raise.
Temperature
Section titled “Temperature”Controls randomness. Higher values produce less deterministic output. For translation work, conservative values are usually easier to review and trust.
Controls nucleus sampling. Lower values keep the output more focused on the most likely next tokens.
Limits sampling to the top K candidate tokens. This can make output more constrained.
Filters out very low-probability tokens. This is another way to keep output from becoming too loose.
Repeat Penalty
Section titled “Repeat Penalty”Discourages repeated tokens. A value of 1.0 means no repeat penalty is applied.
Presence Penalty
Section titled “Presence Penalty”Applies a penalty to tokens that are already present in the prompt.
Frequency Penalty
Section titled “Frequency Penalty”Applies a penalty based on how often a token already appears in the prompt.
Recommended Starting Point
Section titled “Recommended Starting Point”For most translation workflows, start simple:
- Leave Max new tokens enabled.
- Keep the default value unless output is getting cut off.
- Leave the other settings disabled until you have a specific quality problem to solve.
Practical Tuning Tips
Section titled “Practical Tuning Tips”- If output feels unstable or too creative, lower randomness-related settings instead of changing many controls at once.
- If the model stops early, raise Max new tokens before changing sampling controls.
- If repeated phrases are becoming a problem, try a modest Repeat penalty rather than aggressive values.
- Avoid changing every setting at once. Translation quality is easier to evaluate when you make one small change at a time.
When Inference Settings Are Available
Section titled “When Inference Settings Are Available”MetalGlot only enables inference configuration when the translation container is running. If the container is stopped, start it first and then open the inference dialog.