ONYX Ai Matte - Plugin Parameters
Complete reference for all parameters in the ONYX Ai Matte OFX plugin. Use this guide for AI-powered matting, object segmentation, and mask refinement workflows.
1. Point / Bbox Mode Workflow
Use interactive points and a bounding box to define what to segment. Points can work independently or together with bbox: the bbox gives a rough region; foreground points add "include this" and background points add "exclude this." You can animate point and bbox visibility by keyframing the Enabled parameter (0/1). Useful for tracking objects across frames or enabling prompts only on specific frames.
Tools
Enable Edit
Checkbox- Default:
- On
- Description:
- Enables or disables interactive editing of points and bounding box. When off, viewport tools are inactive.
- Best for:
- Temporarily locking your prompts while adjusting other parameters.
Draw Box
Checkbox- Default:
- On
- Description:
- Activates bounding box drawing mode. Click and drag to draw a rectangle around the object. When in Bbox mode, you can also move existing points.
- Best for:
-
- Quick rough selection of entire objects
- Combining with points for complex shapes
- Rotoscoping when object boundaries are clear
- Tips:
- Bbox alone is often sufficient for simple objects. For more precision, add foreground points. See Bounding Box → Enabled to disable bbox in inference without clearing it.
Add FG Point
Checkbox- Default:
- Off
- Description:
- Add foreground points (green). Click on the object to include that area in the mask.
- Best for:
-
- Including specific features or parts
- Refining bbox selection
- Objects with clear boundaries
- Tips:
- Foreground points say "include this". Combine with background points to exclude areas. Point mode is the fastest for quick iterations. To delete points: select them and press Del on your keyboard.
Add BG Point
Checkbox- Default:
- Off
- Description:
- Add background points (red). Click on areas you want to exclude from the mask.
- Best for:
-
- Excluding similar-looking background regions
- Fixing false positives
- Clean keying when subject and background are similar
- Tips:
- Use sparingly; one or two background points often fix incorrect inclusions. Combine with FG points and bbox for best accuracy. To delete points: select them and press Del on your keyboard.
Clear All Points
Push Button- Description:
- Removes all foreground and background points. Bounding box is not affected.
Clear Box
Push Button- Description:
- Clears the bounding box. Points are not affected.
Points List
Points List
Point Picker- Capacity:
- Up to 32 points
- Per point:
- Coord (x,y), Is Foreground (bool), Enabled (0/1)
- Default:
- All points disabled (0,0), Is Foreground = true
- Description:
- Stores all interactive points. Each point has coordinates, foreground/background type, and enabled state. Points are typically added via viewport; this list is for inspection and keyframe editing.
- Best for:
- Keyframe adjustment, fixing point positions frame-by-frame.
- Tips:
- You can keyframe the Enabled parameter (0/1) per point to animate prompt visibility on/off per frame. Useful for tracking or enabling prompts only on specific frames.
Bounding Box
Bounding Box
Point Picker (2D)- Parameters:
-
- Top Left — (x, y) of top-left corner
- Bottom Right — (x, y) of bottom-right corner
- Center — (x, y) midpoint of the bounding box. Can be linked to a Tracker via expressions to move the entire bbox across frames while preserving its size. XYAbsolute type ensures compatibility with Nuke Tracker linking.
- Enabled 1/0 — Enable/disable bbox in inference
- Default:
- Top Left (0,0), Bottom Right (0,0), Center (0,0), Enabled = 1
- Description:
- Defines the bounding box around the object. When enabled, the model uses this region for segmentation. You can disable it to use only points without clearing the box.
- Best for:
- Rotoscoping, keying, and any workflow where a rough object region is known.
- Tips:
-
- The Enabled parameter (0/1) can be keyframed to turn the bbox on/off per frame.
- In Nuke: link Center to a Tracker node via expression — the entire bbox follows the tracked position across frames while preserving its size.
UI Appearance
UI Appearance
Group- Parameters:
-
- Show / Hide Guides — Show or hide bbox and points on screen
- Show Point Labels — Display point numbers below each point
- Points & Bbox Color — RGBA color for points, bbox, and labels
- Overlay Scale — 0.5 to 2.0. Scale for points and label size
- Defaults:
- Show/Hide Guides = On, Show Point Labels = On, Color = (0, 0.65, 1, 0.8), Overlay Scale = 1.0
- Description:
- Controls the visibility and appearance of interactive prompts in the viewport.
2. Text Prompt Mode Workflow
Use text to describe objects instead of drawing. Text prompts detect all matching objects in the frame. If multiple objects are detected, place ONE foreground point on the object you want to select. The point does NOT refine the mask — it only picks which detected instance to use. You can keyframe point visibility (Enabled 0/1) to switch between objects or disable prompts on certain frames.
Pipeline Mode
Dropdown- Options:
-
- Point / Bbox — Interactive points and bounding box
- Text Prompt — Describe objects in text (e.g. "person", "car")
- Trimap Input — Draw a mask in Roto (alpha channel = trimap), refined by VitMatte. Connect a Roto node to the Mask input.
- Default:
- Point / Bbox
- Description:
- Switches between three inference pipelines. Switching pipelines swaps GPU models, which may cause a brief pause.
- Best for:
-
- Point / Bbox: Rotoscoping, precise masks, fast iterations, when you need pixel-level control
- Text Prompt: Batch detection, describing objects by class (person, dog, car). Manual clicks only needed to select which detected object you want from multiple matches.
- Trimap Input: When you already have a rough Roto mask and want VitMatte to refine it — hair, fur, fine edges.
- Tips:
- Point/Bbox is fastest for single-object iteration. Text Prompt is convenient when you know the object class but slower per frame. Trimap Input gives maximum control when combined with a Roto node.
Text Prompt
Text Input- Visible when:
- Pipeline Mode = Text Prompt
- Default:
- Empty
- Description:
- Describe the object to detect in plain text. Examples: "person", "dog", "car", "red dress". The AI finds objects matching your description across the frame.
- Best for:
-
- Keying when you know the object class
- Batch processing — detection is automatic, manual clicks only needed to select from multiple detected objects
- Detecting multiple instances of the same class
- Tips:
- Use simple, common nouns. Text prompts are slower than point/bbox; for quick iterations, prefer Point/Bbox mode. Place a point on the detected object to select the instance you want.
Score Threshold
Slider- Visible when:
- Pipeline Mode = Text Prompt
- Range:
- 0.0 – 1.0
- Default:
- 0.3
- Description:
- Minimum confidence score for text detections. Higher values return fewer but more confident detections. Lower values include weaker matches.
- Best for:
- Increase if false positives appear (wrong objects detected). Decrease if the correct object is missed.
IoU Threshold
Slider- Visible when:
- Pipeline Mode = Text Prompt
- Range:
- 0.0 – 1.0
- Default:
- 0.3
- Description:
- IoU threshold for NMS (Non-Maximum Suppression) overlap suppression. Higher values keep more overlapping detection boxes; lower values remove more duplicates.
- Best for:
- When multiple overlapping detections appear for the same object, lower this value to merge/remove duplicates. Increase if nearby objects are being incorrectly suppressed.
3. Mask Refinement
Shared for both Point/Bbox and Text Prompt modes. Pre-processing, trimap, and temporal consistency.
The HQ VitMatte refiner is always enabled internally. Not user-accessible. Adjust edge quality via Trimap Erode/Dilate below.
Pre Processing
Binary Threshold
Slider- Range:
- 0–255
- Default:
- 100
- Description:
- Binary threshold for coarse mask. Lower values make more dark pixels become foreground (brightens black holes).
- Best for:
- Fixing dark holes in coarse mask before refinement.
Black Point
Slider- Range:
- 0.0–1.0
- Default:
- 0
- Description:
- Remaps the black end of the mask alpha. Raising this value lifts the darks — transparent areas become semi-transparent. Use to recover detail in thin or semi-transparent regions.
- Best for:
- Fine-tuning mask edges; soft hair or transparent objects where pure black is too aggressive.
White Point
Slider- Range:
- 0.0–1.0
- Default:
- 1
- Description:
- Remaps the white end of the mask alpha. Lowering this value clips the brights — semi-transparent areas become more transparent. Use to suppress grey fringing or halo artifacts.
- Best for:
- Cleaning up grey fringing; tightening edges that are too soft after refinement.
- Tips:
- Use Black Point and White Point together as a levels adjustment for the mask alpha. Small adjustments (0.05–0.1) are usually enough.
Fill Holes
Checkbox- Default:
- On
- Description:
- Removes small holes inside objects for cleaner masks.
- Best for:
- Masks with scattered interior holes, noisy coarse predictions.
- Tips:
- Adjust hole size with Fill Holes Area. Disable if small interior details get filled unintentionally.
Fill Holes Area
Slider- Range:
- 0–32
- Default:
- 16
- Description:
- Maximum hole size in pixels to fill. 0 = disabled. 16–32 recommended. Higher = more aggressive hole filling.
VitMatte Trimap
The trimap defines the uncertain region fed to the VitMatte refiner. Use Erode and Dilate to control its boundaries. The trimap can be visualized directly in the viewport via Show Trimap Overlay.
Show Trimap Overlay
Checkbox- Default:
- Off
- Description:
- Displays the trimap directly in the viewport. White = definite foreground, black = definite background, grey = uncertain region processed by VitMatte. Useful for diagnosing edge quality.
- Tips:
- Enable while adjusting Erode/Dilate to see the trimap update in real time. Disable when done.
Trimap Overlay Opacity
Slider- Range:
- 0.0–1.0
- Default:
- 0.5
- Description:
- Controls the transparency of the trimap overlay. Lower values show the source image underneath; higher values make the trimap more visible.
Trimap Erode Radius
Slider- Range:
- 0–200
- Default:
- 3
- Description:
- Erodes the foreground region of the trimap inward. Increasing this value pushes the uncertain zone further inside the object boundary, resulting in tighter edge processing.
- Tips:
- Balance with Trimap Dilate Radius. Erode > Dilate = narrower uncertain zone; Dilate > Erode = wider zone. Use Show Trimap Overlay to preview changes.
Trimap Dilate Radius
Slider- Range:
- 0–200
- Default:
- 10
- Description:
- Dilates the uncertain region of the trimap outward. Increasing this value expands how far the refiner looks outside the object boundary — useful for fine detail like hair or fur.
- Tips:
- Higher Dilate = more edge detail captured, but slower refinement per tile. Use Show Trimap Overlay to preview the affected region.
Temporal Consistency
Temporal Mask Consistency
Group- Parameters:
-
- Enable Temporal Consistency — Dual-layer stability (default: On)
- Temporal Weight (%) — 0–100. Weight of temporal IoU in mask selection (default: 50)
- Suppression Threshold (%) — 0–100. IoU threshold for removing jumping regions (default: 30)
- Description:
- Prevents mask switching between frames. Layer 1 chooses the mask closest to the previous frame. Layer 2 removes regions that suddenly appeared.
- Best for:
- Rotoscoping and keying when body parts flicker or switch between frames.
Temporal Smoothing Motion
Group- Parameters:
-
- Enable Temporal — Smoothing across frames (default: Off)
- Smooth Method — Dropdown (default: Motion Stabilize)
- Motion Stabilize — Compares RGB between frames; freezes alpha where image is static, suppresses VitMatte edge jitter on still areas. Smoothing = freeze strength; Edge width = alpha tolerance threshold.
- Median — Pixel-wise median filter across frames. Smoothing = temporal strength; Edge width = edge zone width in pixels.
- Smoothing — 0–100. Strength (default: 50)
- Smooth edges only — Motion-aware mode (default: Off)
- Edge width — 1–100. Edge zone width (Median) or alpha tolerance threshold (Motion Stabilize), default: 50
- Description:
- Stabilizes mask flickering across frames. Motion Stabilize freezes alpha where source is static, suppresses VitMatte temporal jitter. Median blends frames using pixel-wise median. Both modes are effective — choose based on footage.
- Tips:
-
- Disabled by default. Enable for sequences with noticeable flicker.
- Motion Stabilize: best for static/slow camera shots with jittery edges.
- Median: better for fast motion where frame blending is acceptable.
- Use "Smooth edges only" to reduce ghosting in fast-moving areas.
Final Binary Sharp
Checkbox- Default:
- Off
- Description:
- Final binarization (threshold 128) with corner smoothing. Converts mask to binary using majority voting. After enabling, you can manually adjust edge softness with Edge Feather.
- Best for:
- When you need hard binary edges instead of soft alpha.
4. Output Adjustments
Display, overlay, alpha mode, levels, expand, feather, and offset.
Output Alpha Mode
Dropdown- Options:
-
- Straight — Alpha stored separately; RGB channels are not premultiplied
- Premultiplied — RGB channels multiplied by alpha
- Default:
- Straight
- Description:
- Controls how the alpha channel is composited into the output image. Use Straight for most hosts. Switch to Premultiplied for correct display in DaVinci Resolve.
- Tips:
- In DaVinci Resolve, enable Premultiplied for correct alpha blending. In After Effects and Nuke, Straight is the standard.
Display Mode
Dropdown- Options:
-
- Overlay — Colored mask overlay on source image
- Matte — Black/white mask
- Default:
- Overlay
- Description:
- Both modes copy mask to alpha; Overlay shows a colored preview, Matte shows raw mask.
Show Mask Overlay
Checkbox- Default:
- On
- Description:
- Toggle visibility of the colored mask overlay in the viewport.
Overlay Color
Color Picker (RGBA)- Default:
- (0.5, 0, 0, 0.5)
- Description:
- Color and opacity of the mask overlay when Display Mode = Overlay.
Output Levels
Group- Parameters:
-
- Levels Black — 0.0–1.0 (default: 0.0). Values below this are clipped to fully black. Use to remove dark noise and thin transparent fringing from mask edges.
- Levels White — 0.0–1.0 (default: 1.0). Values above this are clipped to fully white. Use to remove bright noise or background contamination.
- Levels Gamma — 0.1–2.0 (default: 1.0). Midtone shift. Values below 1.0 darken and tighten the mask edge; values above 1.0 brighten and widen it. 1.0 = no change.
- Description:
- Output levels for the final mask, applied before Mask Shrink/Grow in the post-processing chain. Uses a 256-entry LUT for fast per-pixel evaluation. Useful for clipping dark/bright noise and removing moiré artifacts from mask edges.
- Best for:
-
- Clipping semi-transparent noise that survived VitMatte refinement
- Tightening or softening the transition zone at mask edges
- Removing moiré or gradient contamination in background regions
- Tips:
- Start with small adjustments (Black 0.02–0.05, White 0.95–0.98). Gamma values of 0.8–1.2 cover most use cases. Use together with Edge Feather for smooth compositing.
Mask Shrink/Grow
Slider- Range:
- −50 to +50 pixels
- Default:
- 0
- Description:
- Expand (+) or contract (−) mask boundaries. Positive = fill gaps; negative = remove thin edges.
- Best for:
-
- +5 to +15: Keying when edges are slightly too tight
- −5 to −10: Remove noise or thin halos
Edge Feather
Slider- Range:
- 0–100
- Default:
- 0
- Description:
- Feather mask edges only. Use for softer blend into background.
Offset Mask X / Y
Slider- Range:
- −100 to +100 per axis
- Default:
- 0, 0
- Description:
- Offset mask horizontally and vertically on screen. Useful for alignment or compensation.
5. Performance & Cache
Encoder settings, crop padding, and persistent disk cache.
Crop Padding %
Slider- Range:
- 0–50%
- Default:
- 20%
- Description:
- Extra padding around the bounding box when cropping for inference. Increase if object edges get clipped when the selection is tight.
- Best for:
-
- 0%: Tight crop
- 10–20%: Normal (recommended)
- 30–50%: Large padding for objects near frame edges
Disk Cache %
Slider- Range:
- 10–90%
- Default:
- 30%
- Description:
- Percentage of free disk space allocated to the persistent mask cache. Processed masks are stored to disk and survive host restarts and project reloads. When the limit is reached, the oldest cached files are automatically removed (LRU eviction).
- Best for:
- 20–50% recommended. Higher values cache more frames for large sequences. Masks are compressed with zlib (~80× size reduction — typically ~100 KB per 4K frame instead of ~8 MB).
- Tips:
- A 500 MB cache budget holds thousands of frames. On cache hit, only lightweight post-processing runs — typical hit time is under 50 ms vs. 1–2 s for full inference.
Cache Folder
Directory Path- Default:
- Empty (system default location)
- Description:
- Folder where disk mask cache files are stored. Leave empty to use the default system location. Set to a fast SSD path for best performance, or a network drive for shared cache across workstations.
Clear Disk Cache
Push Button- Description:
- Deletes all cached mask files from disk immediately. Use when cache is stale or you need to free disk space. Does not affect in-memory state.
Cache Info
Read-only- Description:
- Shows current disk cache usage: number of cached frames and total size on disk. Updates automatically. Read-only.
6. Debug & Advanced
Reset, license, and internal controls.
Reset All Parameters
Push Button- Description:
- Resets all parameters to their default values. Use when experimenting or recovering from bad settings.
License
Group- Parameters:
-
- Buy at onyxofx.com — Opens website for license purchase
- License Key — Text field for entering license key
- Activate License — One-time online activation, then works offline
- Deactivate License — Transfer license to another machine
- Status — Read-only license status
- Description:
- License management. One-time activation with internet; afterward works fully offline. License key is not saved in the project file for security.
Quick Reference: Which Mode for Which Task
| Task | Recommended Mode | Key Parameters |
|---|---|---|
| Rotoscoping (single object) | Point / Bbox | Bbox + FG points, Temporal Consistency, Edge Feather |
| Keying | Point / Bbox | Trimap Erode/Dilate, Black & White Points, Fill Holes |
| Quick masks | Point / Bbox | Bbox only, Enable Edit, Disk Cache 30% |
| Batch detection | Text Prompt | Text Prompt ("person", "car"), Score Threshold, point to select instance |
| Complex shapes | Point / Bbox | Bbox + FG + BG points, Mask Shrink/Grow, Output Levels |
| Hair / fur / fine edges | Any | Trimap Dilate Radius, Output Levels Gamma, Edge Feather |
| Existing Roto refinement | Trimap Input | Connect Roto to Mask input, Trimap Erode/Dilate |
| Tracker-driven bbox (Nuke) | Point / Bbox | Link Bounding Box → Center to Tracker expression |
| Edge jitter / flicker | Any | Temporal Smoothing → Motion Stabilize, Stabilize Strength, Alpha Tolerance |
| DaVinci Resolve compositing | Any | Output Alpha Mode = Premultiplied |
Need Help?
Get fast help from our community and support team:
- Discord: Join Discord — real-time help from developers and users
- Email: support@onyxofx.com — 48hr response for license holders
Licenses
All third-party licenses and notices included with plugin installation in the installation directory.