Last updated: 2026-07-17

ONYX Ai Matte v3 — Plugin Parameters

Complete reference for the ONYX Ai Matte v3 OFX plugin — AI-powered matting, object segmentation, tracking, and mask refinement built on SAM3.

What ONYX Actually Is

ONYX isn't a magic button, and it doesn't claim to be. It's built to get you 60-80% of the way to a finished matte, fast — especially on the hard cases: motion blur, transparency, hair, difficult greenscreen and non-greenscreen shots. Any professional roto or keying shot is always a stack of nodes; ONYX replaces most of that stack, not all of it. An experienced artist still finishes the last mile.

Field Recommendations

Hard-won advice from production use — short and practical. Read this before you start: it saves hours.

First: feed video into the Source input

Where it all starts:
Connect your footage (or a still) to the node's Source input — this is the image the AI detects the object and builds the mask on. Without a Source there is nothing for the plugin to analyse: no detection, no tracking.

Getting started: select an object and press Play

To start tracking:
Select the object you want to track — the box tool is on by default: just drag a box on screen around the object, and the mask appears. Now press the regular Play — the object is tracked frame by frame from the frame where you created it.

Interior alpha not a solid 1.0? Lower White to 0.99

Do:
If the object's fill in the mask is not fully opaque in places — the interior alpha sits a touch below 1.0 (e.g. 0.98–0.99, "almost white" instead of a clean 1.0) — open the Post Processing group and lower Output Levels: White to about 0.99. Everything that was near 1.0 snaps to a solid alpha of 1.0.
Why:
The AI outputs a soft, probabilistic alpha, and the object's interior can sit a hair below 1.0 — this is a property of the network's probabilistic output, not a plugin defect. A white point of 0.99 remaps those near-1 values to a clean 1.0; it barely affects the soft edge (an overall lift of about 1%). It is a standard levels operation applied to the finished matte in Post Processing, so it does not touch segmentation or tracking.

Turn Loop playback OFF when tracking — on every host

Do:
Before tracking with Play, switch the player's repeat mode to a single pass (Once / Loop off) — in every host.
Why:
Tracking is strictly sequential: the model's memory walks frame by frame. With Loop on, playback reaches the end and jumps back to the start — the playhead lands outside the tracked range, the track is interrupted and the plugin shows the "GO INSIDE FRAMES…" banner. The banner says it directly: "TURN OFF LOOP PLAYBACK".

Mask slips mid-track? Give SAM3 more context

Do:
Increase Crop Padding — or switch Crop Mode off entirely. Change it on the init frame (in Dynamic mode — anywhere).
Why:
SAM3 recognizes an object inside its surroundings, on a fixed 1008×1008 input. The crop exists so that objects in 4K footage don't shrink into a few pixels — but cropped too tight, the model loses the context it needs to understand what it is looking at. When tracking gets unstable, more context beats more zoom.

Mask clipped at the crop edge on fast motion? Raise Crop Padding

Do:
On fast object motion or a whip pan, raise Crop Padding — or switch Crop Mode off. Change it on the init frame (in Dynamic mode — anywhere).
Why:
The crop is a window that follows the object with this margin, but lags one frame behind — each frame's window is built from where the object was on the previous frame. If the object travels farther in one frame than the margin covers, part of it falls outside the encode window and the mask is clipped in a straight line along the crop edge. The tell that separates this from the drifting track above is that the mask is cleanly cut along a straight line, not smeared. More Crop Padding = more room for per-frame movement; Off = full frame, no clipping, at the cost of slightly less edge detail.

Bake writes exactly what you see

The rule:
With Bake on, only the frames that play on screen are written to disk. If the mask is gone from the screen and you press Play, empty frames are written over that range — a baked range of nothing means the track was lost while baking.
Overwriting:
Un-bake, change the tracker, bake again — that is a new pass: with the same Name Prefix it overwrites the previous files. Change the prefix to keep both — or re-bake only the frame ranges you fixed.
Workflow:
A proven pattern: bring the baked sequence back into the project as an image Read, then with new bakes touch only the ranges that need fixing — the final matte is assembled from passes. And decide the Tracking Mode before you start tracking: it directly shapes how the plugin behaves.

Hard shot? Prep the image first

Do:
Feed ONYX a prepped branch — e.g. a high-pass / local-contrast boost (the Nuke Survival Toolkit HighPass in high-contrast mode works well). Keep it gentle.
Why:
The AI sees only what you feed it: lifting local contrast makes a dim object stand out. But don't over-crank — push it too far and the edges turn jagged, because the refined edge follows the image you feed in.

SAM3 sees objects, not backgrounds

The rule:
Keying in the usual sense does not work here — don't click the background hoping to "key" it out. SAM3 segments objects, not backgrounds. This fully applies to chroma key (green screen): select the objects standing on it, not the green background (a point or a bbox each), refine them, and invert / comp downstream — we segment the objects; the background comes from the inversion.
One object — one point:
Don't drop several point objects onto one thing. Place one point to create the object, then grow and correct it with FG/BG refine points — that tells SAM3 "this is a single object and we are refining it". Scattered point objects read as separate objects and fight each other.

Finished tuning an object? Don't touch it — add a second node

The problem:
You selected a tricky object (say, a forest) with a point, cut the holes, tuned it — and the next click nearby wrecks the work: new prompts near a finished object can re-assign or correct it.
Do:
Leave the finished node alone and build the next selection in a second ONYX node, then merge the mattes downstream (a Merge/Plus — or feed one node's result into the other's Mask Composite input, Mode = Add).
Why it's free:
Any number of ONYX nodes share one inference server under the hood — extra nodes do not multiply GPU memory use.

The Nature of the Edge, and Where ONYX Sits in the Pipeline

A short, technical explanation of where edge jitter comes from and how ONYX is positioned. Read it to set the right expectations and build your processing chain.

Why the edge "boils" from frame to frame

The HQ Refiner (VitMatte / MEMatte, see Refiners) is a per-frame edge refiner. It processes each frame independently: it recomputes the soft alpha inside the trimap band from the actual pixels of that frame, with no memory of how the edge looked on the neighbouring ones. SAM3's segmentation does carry memory between frames — that is what tracking runs on — but the edge refinement does not.

The consequence: wherever the real pixels along the edge shift slightly from frame to frame — sensor noise, grain, compression, the micro-movement of hair — the boundary of the soft alpha shifts slightly too. Visually this reads as a fine tremor, or "boiling", of the edge (edge boiling). This is expected behavior of per-frame AI refinement, not a defect: the refiner faithfully reflects what each frame actually contains.

The built-in answer — Edge Stabilization

Edge jitter is not a bug to be "fixed in code"; it is the nature of per-frame edge refinement, and the plugin has a built-in tool for it — Edge Stabilization. It supplies the missing temporal continuity: it blends the edge band with the previous frame, compensating for camera motion, damping the frame-to-frame tremor. It is not a patch over the problem but the pipeline layer that deliberately adds temporal continuity where a per-frame refiner has none by design. It works best on static or slow-camera shots; the Stabilization Strength and Motion Sensitivity parameters balance a calm edge against a slight trail (ghost) on fast motion — details in Edge Stabilization.

ONYX is a pipeline, not a one-click button

ONYX covers a spectrum of tasks: from fast garbage / rough mattes to final-quality masks. Where a given result lands on that spectrum depends not on a single checkbox but on how the artist built the chain:

Input prep:
Out-of-range values (negatives, super-white) are clamped for you: on input the frame is converted to sRGB by default (the Apply sRGB LUT checkbox at the top of the panel — turned off for ACES / your own colour management) and every channel is hard-clamped into 0–1 (negatives → 0, brighter than 1 → 1) in either mode, so you do not need a Clamp before the node — SAM3 always receives a correct range (this covers any source outside 0–1, not only EXR). Just note that a hard clamp collapses detail in deep highlights: if the subject is hiding in super-white on HDR/log, don't just Clamp — bring the image into 0–1 with a grade or lift local contrast. The refiner resolves the edge against the image you feed it: the cleaner and more legible the input, the cleaner the edge (see Field Recommendations).
Refiner choice:
RAW / Hard Mask for fast, hard masks; the HQ Refiner (VitMatte or MEMatte) for hair, fur and soft / semi-transparent edges (Refiners).
Trimap tuning:
Erode / Dilate set the band the refiner is allowed to recompute the edge in: narrower is faster and tighter, wider gives fine hair more room (Refiners).
Edge Stabilization:
Temporal edge stabilization where frame-to-frame continuity is needed (Edge Stabilization).

The practical point: ONYX is a set of tools inside one pipeline, not a one-click solution. The same shot can yield a rough garbage mask in a minute or a final matte with hair — the difference is how the chain of prep, refinement and stabilization is built. If a result comes out rougher than expected, that is a signal to refine the chain (input, refiner, trimap, stabilization), not the tool's ceiling. The result follows predictably from that chain.

The Scene

ONYX Ai Matte works as a single unified pipeline: one scene can hold every prompt type at the same time — bounding boxes, points, text, and a mask-prompt — and they do not exclude one another. You mix and match as needed; one node, one SAM3 model path.

One scene, all prompt types

Object limit:
Up to 16 objects per scene. Adding a 17th is blocked with a "MAXIMUM 16 OBJECTS" banner.
Init frame:
The init frame is set by the first object you create — the frame you placed it on. Its number is shown in the Object Panel (the INIT FRAME line). Where you can create/delete objects depends on the Tracking Mode.
How it works:
SAM3 runs at a fixed 1008×1008 resolution. Whatever region is sent in is squeezed to that size, then segmented; tracking propagates the mask frame-by-frame through the model's memory.
Tips:
Start an object with a box or a point, add text to grab objects by name, refine with extra points — all in the same node. Switching the Tracking Mode preserves your objects.

Node Inputs

The node has one required input — the image itself — and three optional inputs. All three take a mask, but each uses it for a different job: Mask Prompt tells the AI what to track. Refiner Direct sends your mask straight to the edge refiner to get its edges polished. Mask Composite stamps a mask on top of the finished result. Use them together with AI objects — or on their own.

Source

Description:
The footage you want to matte and track. Everything the node does reads this image.

Mask Prompt

What it does:
Takes a mask you already have and uses it as a prompt — "track whatever this mask covers". Connect it, and one mask-driven object appears on the init frame. From there the AI tracks it through the shot like any other object. And the mask does not have to be a whole object: it can be a sub-object — a part of something bigger (a sleeve, a door of a car, a logo on a shirt) — and the AI will track exactly that part.
When to use:
Two cases. A part of an object — boxes and points snap to whole objects, while a mask lets you point at exactly "this piece": roto the sleeve or the car door once, and the AI tracks just it. A ready shape — you already have a roto, a luma key or a paint-out on one frame and want the AI to do the tracking. Example: a quick roto of a person + the HQ Refiner = hair-level edges on every frame.
Notes:
It appears in the Object Panel as a MASK row. It has no delete X — to remove it, disconnect the input. The mask is only read on the init frame; after that the object tracks by memory, like a box or point object.
Correcting mid-track:
The AI drifted somewhere mid-shot? Stand on the bad frame and move or reshape your input mask — a correction key is set right there. On that frame your mask becomes the result, and tracking continues from it. Frames without a key ignore the input mask, and an animated (interpolating) mask does not create keys on the in-between frames. Keys show as diamonds on the MASK row; clicking a diamond removes that frame's key. Keys survive saving and reopening the project.

Refiner Direct

What it does:
Sends an external matte straight into the HQ Refiner. Nothing is segmented or tracked here: the node takes your matte as it is and re-draws its edges against the Source image — hair, fur, semi-transparency. If there are AI objects too, the polished matte is merged (union) with the AI mask.
When to use:
Three ways. Alone — no AI objects at all: the node becomes a pure edge polisher for a matte you made elsewhere. A typical case: a mask tracked in another system, such as Mocha or a planar tracker, comes in with hard, rough edges — feed it here and every frame gets refined edges (hair, fur, soft transitions). Connect the matte, pick the Refiner Model, done. Combined — the AI tracks the object while your matte covers a part the AI misses (a prop, a logo patch); both get the same edge treatment and are merged. Removing seams between ONYX objects — masks tracked by ONYX itself can come back here too: when two tracked objects overlap, merge their mattes into a single mask and feed it in. The refiner sees one shape, so the seam where the objects meet disappears, and the outer edge is rebuilt clean.
Notes:
Connecting this input turns the refiner on automatically. The edge band is built from your matte — adjust its width with Trimap Erode / Dilate, and check it visually with Show Trimap (in the Matte Overlay display mode). The input is read every frame, so an animated matte is refined frame by frame.

Mask Composite

What it does:
Stamps an external mask onto the finished matte — after the AI, the refiner and all cleanup. The Mask Composite Mode decides how: Add = the mask grows the result, Subtract = the mask is cut out of the result, Intersect = only the overlap stays.
When to use:
Garbage mattes and hold-outs. Example: the AI tracks a person and a lamp post passes in front — feed a lamp-post mask, set Subtract, and the post is cut out of every frame.
Notes:
This input never touches the AI or the refiner. It is a plain compositing step on the final mask — applied the same way in every Display Mode, and baked into the Bake Tracking Cache files.

The Object Manager

The on-canvas panel that lists every object in your scene. Drag it by the "OBJECTS" title bar to move it anywhere on the viewport; the arrow to the left of "OBJECTS" collapses it to a single line. Under the title are the INIT FRAME and TRACKED FRAMES lines.

INIT FRAME — click to jump there. Hover the INIT FRAME line and it turns green — the colour that marks the init frame — then click to send the playhead to it. It does the same as the Go to Init Frame button in the parameters. On the init frame itself the line is inactive (you are already there), and it stays inactive on hosts that ignore playhead jumps — DaVinci Resolve, for example, where the button is hidden for the same reason. Nuke and Fusion Studio support it.

An object row

Solo (eye, on the left):
Click the eye to isolate an object. When any object is soloed, only the soloed objects appear in the result and the rest are hidden. Soloing is additive: solo several objects and they all show together. Click the eye again to un-solo.
Visibility key (on the right):
Click to show or hide the object on the current frame. This sets a visibility keyframe that holds until the next one — so an object can appear or disappear at the exact frames you choose. When an object has visibility keys, its circle fills green.
Refine marker:
Appears when the object has refine points on the current frame. Click it to clear this frame's refine touch-ups for that object. On the MASK row the diamond shows mask correction keys (see Mask Prompt) — clicking it removes the current frame's key.
Delete (X):
Remove the object. It is enabled only where the Tracking Mode lets you add or delete objects: on the init frame in Locked Init Frame (the default), inside the tracked range in Dynamic, or on any frame in No Init / Manual. Off an allowed frame the X is inactive.

Text-prompt groups

Description:
Objects found by a text prompt are grouped under a badge (the phrase you typed), with their rows indented below it. The group badge has its own delete X to remove the whole group at once.

Input Color

The topmost group on the panel (open by default). It controls the colour in which ONYX feeds the frame to SAM3.

Apply sRGB LUT

Default:
On.
Description:
SAM3 was trained on standard sRGB images, so ONYX must feed it an sRGB picture. On (default): the plugin converts a LINEAR input into sRGB for you — keep it on for a normal linear pipeline (Nuke's default working space). Off: pixels are sent as-is — use this when the input is already sRGB, or when you colour-manage the project yourself. Example: in an ACES project the plugin behaves differently on a linear EXR — turn the checkbox off and prepare the picture with an OCIO node to suit your pipeline. Values are always clamped to 0–1 in either mode; affects float input only (8/16-bit is already display-referred).
When to change it:
It is a seed parameter — it changes the colour the AI sees — so it is available on the init frame (like the crop parameters). Off the init frame the checkbox is greyed out; changing it on the init frame re-seeds the track with the new colour.

1. Object Tools

The prompt-creation tools. Exactly one tool is armed at a time (radio behaviour); the default is Add Bbox Object. Tools, text, and detail level live in the Object Tools group at the top of the UI.

Select / Edit

Description:
Select and edit existing objects — drag boxes and points, pick rows in the Object Panel. Creates nothing. Dragging a box corner or point on a tracked frame is recorded as a correction on that frame, and tracking continues with it applied.

Add Bbox Object

Description:
Draw a rectangle around an object — one rectangle = one object. The fastest way to start for most footage.

Add Point Object

Description:
A single foreground click creates a new object. The captured size is set by Point Detail Level.
Not for editing an existing object:
This tool always creates a new, separate object. To add or remove part of an existing object (an edge, a bit sticking out, a leaked chunk), use the FG / BG Refine Points, not Add Point Object. A common mistake is to keep dropping new point-objects to "grow" an object — the right tool for that is refine points.

Add FG Refine Point / Add BG Refine Point

Description:
Refine points are correction clicks that fix a specific object's mask where the AI got the edge wrong — you don't redraw the mask, you just nudge it. Pick Add FG Refine Point (foreground) and click an area that should belong to the object but is missing — the mask grows to include it. Pick Add BG Refine Point (background) and click an area that leaked into the mask but shouldn't be there — the mask shrinks to exclude it. Each click is tied to the frame you click on: the AI recomputes that object's mask on this frame with your hint and then carries the correction forward as it keeps tracking, so one good fix usually holds on the following frames too, not only the frame you clicked. Drop several FG/BG points together to sculpt a difficult edge (hair, a thin limb, a hole in the middle). How big an area a single point adds or removes is set by Point Detail Level. Refine points only correct an existing object — they never create a new one; to add an object use the Bbox / Point / Text tools.
Important — which object the point attaches to:
Place a refine point outside all boxes and it attaches to the currently selected bbox. Place it inside a box and that bbox becomes selected and the point attaches to it. So before you drop a refine point, always check which bbox is currently selected — otherwise the point may attach to a different object than you meant.
Where:
Refine points are not limited by the AI-CROP outline: click outside it and the processing window of that frame widens (using the crop padding from the UI) to fit the new point — a click far away simply grows the window up to the full frame. On the init frame, an FG point placed outside the drawn box grows the box itself. With Crop Mode = Off there is no outline at all.
Delete a point:
Select it and press Del. Move: click and drag.

Point Detail Level

Options:
Tiny detail, Object part, Large object part, Whole object (default: Whole object)
Description:
How big a piece a point grabs. For Add Point Object it sets the size of the new object (Whole object = the whole thing under the point, Tiny detail = a small part, e.g. a hand instead of the whole person, with two sizes in between). For refine points it sets how big a part to add (FG) or remove (BG); the FG/BG Refine tools default to Object part. It is per-point — each point remembers the level chosen when placed, and a selected point's level can be changed later; selecting a point also shows its own level in the dropdown. It works the same on the init frame and while tracking — every point, including a refine point, keeps its own size. It applies to points only: it does not change how a drawn Bbox itself is segmented.

Text Prompt

Add Object by Text

Description:
Type what to find (e.g. "person", "red car", "dog"). In Nuke press Enter; in DaVinci/Fusion click Submit Prompt. Each submit adds the objects it finds to the scene.
Several concepts at once:
A comma-separated line finds each phrase as a separate group: berry, bread, milk creates three groups, each with its own badge (the full phrase) and colour. Prompts are cumulative — submit several in a row, each adds what it finds.
Creation rules:
Text creates objects under the same per-mode rules as boxes and points (see Tracking Mode).

Submit Prompt

Description:
Detect objects from the prompt above and add them. Use this in Fusion/DaVinci, where pressing Enter does not commit the field.

Score threshold

Range / Default:
0.0 – 1.0 / 0.3
Description:
Minimum confidence to keep a detection. Higher = fewer, surer objects; lower = more, looser ones. Raise it if wrong objects are detected; lower it if the correct object is missed.

IoU threshold

Range / Default:
0.0 – 1.0 / 0.3
Description:
How aggressively overlapping detections of the same thing are merged (NMS). Higher = keep more near-duplicates; lower = stricter de-dupe.

Mask Prompt (a node input): connect an external mask and it creates one object on the init frame — see Node Inputs.

2. Tracking Mode

How an object is tracked across frames, and on which frames you can create/delete objects. You can switch mode at any time (except while Bake/Freeze is on — then it is greyed out); switching re-seeds the track on the current frame (which becomes the new init frame), and the objects stay.

Mode

Options:
  • Locked Init Frame (default) — strict memory tracking. Create objects on the init frame, press Play, the mask propagates by memory. Objects can be created/deleted only on the init frame. The most reliable mode, and a saved project reopens with the track restored.
  • Dynamic Init Frame — like Locked, but you can also add/delete objects anywhere inside the already-tracked range; the init frame moves to the current frame and the scene re-seeds from there (your refine touch-ups are kept). The earlier section is re-tracked. Reopens with the track restored.
  • No Init Frame — no fixed init frame: create/change objects on any frame, the scene re-seeds from there. The freest and fastest. The track is not restored on reopen (turn on Bake to keep masks on disk). Shows "MODE : NO INIT" in the panel.
  • Manual — hand-keyframe mode for rare hard frames where the auto-track breaks. Each frame is computed independently (no memory); set a box/point key on the frames you need and the geometry is interpolated between keys. The box/point is BRIGHT on a key and DIM between them. Shows "MODE : MANUAL".
Default:
Locked Init Frame
Reopening a saved project:
In Locked and Dynamic the track is restored on reopen, but it replays sequentially: go to the init frame and play forward to re-build it (the AI's memory is sequential — you cannot jump over it). A banner "GO TO INIT FRAME N TO START TRACKING" reminds you when you are off it. With Bake on, the baked masks play straight from disk — no engine, no VRAM, no re-track; turn Bake off to re-establish the live track from the init frame. No Init and Manual do not restore the track on reopen — turn on Bake first if you need to keep the result.
Tips:
Refine touch-ups are kept when moving between the auto modes (Locked / Dynamic / No Init) and cleared when moving into or out of Manual. Reset Scene returns to Locked Init Frame.

Init Frame

A line under the Mode dropdown shows the scene's init frame — the frame tracking started on. It reads a dash when there is no single init frame yet: an empty scene, or No Init / Manual mode.

Go to Init Frame

Jumps the host playhead to the init frame in one click — handy for getting back to the start of the track (e.g. after the "GO TO INIT FRAME" banner). Works in hosts that let a plugin move the playhead: Nuke, Fusion Studio, Natron. In DaVinci the button is disabled — that host does not let a plugin move the playhead.

3. Tracking Options

How the crop window frames your objects, plus seed and box behaviour. The crop parameters and Enrich First Frame are "seed settings" — in Locked mode they can be changed only on the init frame (greyed out elsewhere); in Dynamic also inside the tracked range; in No Init on any frame.

Crop Mode

Options:
  • Off — encode the whole frame (the plain SAM3 path, maximum context, but maximum compression at 4K).
  • Union (Recommended) (default) — one crop around all objects plus padding. Fast, good quality.
  • Per-Object (Slower, Higher Quality) — a separate full-resolution crop per object. Sharper on small, widely separated objects, but slower (one encode each).
Description:
Which part of the frame is sent to SAM3. A crop fills the 1008×1008 budget so the object gets more pixels (sharper edge) — but with less surrounding context. If a tight crop makes the mask break up or grab the wrong thing, raise Crop Padding or switch to Off.

Crop Padding (%)

Range / Default:
0 – 100% (display 0–50) / 20
Description:
Extra margin the AI keeps around the object inside its crop, as a percentage of object size. More padding = more room for the object to move between frames without leaving the crop.

Crop Stabilize %

Range / Default:
0 – 20 / 0 (off)
Description:
Steadies the crop window so the mask edge does not shimmer on near-static frames. It is the % the object may drift inside the window before it re-centres. 0 = the crop follows the object every frame; raise to ~2 only if a near-static edge shimmers.

Crop Aspect Limit

Range / Default:
1.0 – 1.8 / 1.5
Description:
SAM3 always squashes the crop window to a square before inference. A long, thin window gets distorted, which can make the mask wobble or "melt" on elongated objects. This caps the distortion: lower pads the window toward a square with real background (steadier, slightly less fine detail); higher keeps it tighter (more detail) but distorts more. Affects only elongated objects; near-square ones are unchanged. Above 1.8 = out of the model's training range.

Edge Padding

Range / Default:
0 – 100 px / 20
Description:
Recovers the mask along the frame edge, where SAM3 is weakest: an object cut off by the border has no data beyond it, so it comes out slightly cut in. This grows the picture by a small mirrored margin before the AI looks — the object continues past the frame with its own texture and no seam — then trims it back. A plain stretch would read as a separate object and get cut at the seam; a mirror has none. Only affects objects touching a frame edge (interior objects are unchanged). Measured in frame pixels; 0 = off. Default 20; raise toward 50–100 for a stronger edge or heavy motion blur.

Enrich First Frame

Default:
On
Description:
On the first seed the model has no memory yet, so the starting mask can come out rough or partial. With this on, the init frame is run through the model's own memory twice (a draft pass then a clean pass), so the starting mask is as clean as a tracked frame. Costs one extra pass on the seed frame only — no cost while tracking. Turn off to use the raw first-frame mask.

Auto Fit BBox to Mask

Default:
On
Description:
While tracking, the box hugs the mask as it grows/shrinks (smoothed). Off: the box keeps the size you drew and just follows the object's centre — steadier for an object that stays roughly the same size. On the init frame the box always stays exactly as you drew it (this setting is ignored there).

Re-assign Object on Drag

Default:
Off
Description:
On: dragging a tracked object's box/point onto a different object replaces it (the old one is forgotten). Off (default): the drag re-locates the same object (use it when your object moved or was briefly lost and you point to where it is now).

4. Refiners — finishing the mask edge

Choose how the mask edge is finished, and (for the HQ Refiner) control the trimap band the AI works on.

Mask Output

Options:
  • RAW Mask — the soft grayscale straight from SAM3 (shape it with the Input Levels).
  • Hard Mask — thresholded hard edges, no refiner (clean, tight).
  • HQ Refiner (default) — AI edge matting for hair / fur / soft or translucent edges. Best quality, a little slower.
Default:
HQ Refiner

Refiner Model

Options:
VitMatte (default) — often softer/cleaner edges on hair and soft borders. MEMatte — lighter, can be crisper in places.
Default:
VitMatte
Description:
HQ Refiner only — which AI edge-matting model to use. Both models are warmed up at first launch, so switching is instant; compare them on your own footage.

Trimap (HQ Refiner only)

The refiner works only on a narrow band along the edge — the trimap. It splits the area into "definitely object", "definitely background", and an "unsure" working band (where the soft alpha is recomputed from the real video pixels). The band is measured from the mask outline: inward by Erode, outward by Dilate. At the frame border itself the bands are deliberately not built (there is no off-frame data).

Show Trimap

Default:
Off
Description:
Paint the trimap on screen so you can see the real band the AI works on. Green = erode band (inside), cyan = dilate band (outside), white = contour. Turn off for render.

Trimap Opacity

Range / Default:
0.0 – 1.0 / 1.0
Description:
With Show Trimap on: how strongly the coloured bands are painted over the image. Lower it to see more of the real matte underneath. View only.

Trimap Pre-Blur

Default:
Off
Description:
Smooth the mask before building the trimap, to soften the upscale "staircase". Turn on if the edge band looks jagged.

Trimap Erode (px)

Range / Default:
0 – 255 / 5
Description:
How far inside the mask edge is treated as "definitely foreground". Larger = a thinner working band right at the edge (faster, but can clip fine detail). Try 5; raise to ~15 to speed up on big objects.

Trimap Dilate (px)

Range / Default:
0 – 255 / 20
Description:
How far outside the mask edge the AI may grow detail (hair / fur). This is the main speed knob: a wider band = more tiles to process = slower. Try 20; lower to ~8 for speed, raise to ~40 for long wispy hair.

Binarize Threshold

Range / Default:
0 – 255 / 100
Description:
Grayscale level that splits foreground from background. Drives the Binary output and the HQ Refiner input (the trimap is built from it), and the tracked bbox/crop. Higher = tighter foreground. (Lives in the Pre Processing group.)

5. Pre Processing

Shapes the mask the AI works on, in pipeline order: Input Levels (shape the raw soft mask) → Binarize Threshold → binary cleanup. These feed both the Binary output and the refiner. Group is collapsed by default.

Input Levels: Black / White / Gamma

Ranges / Defaults:
Black 0.0–1.0 / 0.0 · White 0.0–1.0 / 1.0 · Gamma 0.05–5.0 / 1.0
Description:
Shape the raw soft mask before binarize (mainly Raw mode; in Binary/Refiner they bias where the threshold cuts). The Output Levels in Post Processing remap the finished matte instead.

Remove Sprinkles (px)

Range / Default:
0 – 1,000,000 (display 0–20,000) / 100
Description:
Drop disconnected islands up to this area in full-resolution pixels. 0 = off; higher removes bigger islands. Applied per-object (not on the whole merged frame), so a small whole object is kept even next to a large one, and a fragment ≥ half the object is always kept.

Fill Holes (px)

Range / Default:
0 – 100,000 (display 0–10,000) / 0 (off)
Description:
Fill enclosed background holes up to this area in pixels (absolute, like SAM3's max hole area).

Binary Edge Smooth

Range / Default:
0 – 50 (display 0–25) / 0 (off)
Description:
Round the upscale "staircase" of the binary edge (majority-voting re-binarize). Radius in pixels; 0 = off.

Mask Shrink/Grow

Range / Default:
−50 to +50 / 0
Description:
Contract (−) or expand (+) the binary mask edge in pixels (erode/dilate), before the refiner.

6. Post Processing

Finishes the final matte after the refiner: mask compositing with an external clip, output levels, guided filter, feather, and offset. Group is collapsed by default.

Mask Composite Mode

Options / Default:
Off, Add to Mask (default), Subtract from Mask, Intersect with Mask
Description:
Combine the external "Mask Composite" input clip with the AI mask. Add = union (grows the mask); Subtract = cut the input out of the AI mask; Intersect = keep only the overlap. Example: feed a garbage matte and choose Subtract to carve it out.

Output Levels: Black / White / Gamma

Ranges / Defaults:
Black 0.0–1.0 / 0.0 · White 0.0–1.0 / 1.0 · Gamma 0.05–5.0 / 1.0
Description:
Remap the finished matte (after the refiner/cleanup). Separate from Input Levels. Start with small adjustments.

Guided Filter Radius / Eps

Ranges / Defaults:
Radius 0–48 (display 0–24) / 0 (off) · Eps 0.0–0.1 / 0.001
Description:
Edge-aware filter guided by the video. Radius 0 = off. Eps is edge sensitivity (must be > 0): smaller = sharper edge preservation; larger = softer.

Feather

Range / Default:
0.0 – 100.0 / 0.0 (off)
Description:
Gaussian-blur the mask edge for softer compositing.

Mask Offset X / Y

Range / Default:
−100 to +100 per axis / 0, 0
Description:
Offset the mask horizontally (X) and vertically (Y). Useful for compensating misalignment or creative offset.

7. Edge Stabilization

Reduces per-frame edge "boiling" with a camera-motion-compensated temporal blend in the edge band of the final mask. Off by default; best on static or slow-camera shots. Group is collapsed by default.

Stabilize Edges Over Time

Default:
Off
Description:
Blend the mask edge with the previous frame, compensated for camera motion. Best on static or slow-camera shots; on fast or close-up movement it can leave a faint trail (ghost). Play forward to build it up — it needs consecutive frames, scrubbing resets it.

Stabilization Strength

Range / Default:
0 – 100 / 60
Description:
How strongly to hold the edge steady. Higher = calmer edges but more risk of a trail on motion. For a locked-off shot push it to 90–100.

Motion Sensitivity

Range / Default:
0 – 100 / 40
Description:
How readily the edge follows movement instead of smoothing it. Low = smooth aggressively (kills shimmer on static shots, but can trail a fast edge); high = follow movement (safe on motion, a little shimmer stays). Static shot: low (~20) + high Strength. Hand-held: ~60–80.

8. Display Mode (on-screen view only)

The top group of the UI — how the node shows the result on screen. This is independent of what Bake writes to disk (that is set by Cache Format). The alpha channel always carries the mask; the modes differ in RGB only.

Mode

Options:
  • Matte Overlay (default) — footage with the mask tinted on top. On Nuke the alpha carries the mask (a real matte); on DaVinci/Fusion the overlay is kept opaque to avoid a white edge halo.
  • Matte Only — the mask as a black & white image (alpha = mask).
  • RGB*A (Premultiplied) — footage cut out by the mask (RGB × alpha) + alpha = mask; the object on black, ready to drop into a comp.
  • RGB+A (Straight) — full footage (RGB untouched) + alpha = mask, not premultiplied (cleaner edges for compositing / After Effects).
Default:
Matte Overlay

9. Bake Tracking Cache

Compute the masks once, write them to disk, then play them back from disk with no re-inference. With Bake on, opening the project does NOT start the AI engine and uses NO video memory (VRAM) — the node simply plays the ready masks from disk. The engine starts (and uses VRAM) only when you turn Bake off. The cache folder is portable — hand a finished cache to another machine with the project.

Freeze and Bake Masks to Disk (Press Play to Write)

Default:
Off
Workflow:
  1. Set the Cache Folder path below (the first time, a folder picker opens — no silent save).
  2. Enable Bake and press Play — inference runs and every frame is written to disk. Play through the whole timeline at least once for full coverage.
  3. Once cached, playback reads from disk — no inference runs.
  4. Disable Bake to re-render.
When inputs change:
Editing objects or tracking while Bake is on disables it. Changing mask / refiner / doctoring settings (e.g. trimap) re-bakes only the affected frames in place — and pressing Bake again on a folder that already holds masks for a changed scene prompts "Overwrite baked masks?". Cache settings (prefix, format) restart the cache session without disabling Bake.

Cache Format

Options:
  • PNG — Mask (8-bit, Luma + Alpha) (default) — the mask only, the same mask in Luma and Alpha. Small files, mask delivery.
  • EXR — Mask (Alpha only) — a single "A" channel, clean alpha with no RGB.
  • EXR — Premultiplied (RGB * Alpha) — RGBA where RGB is already cut by the mask, plus alpha; written with AdobeAlphaMode = Premultiplied.
  • EXR — Straight (RGB + Alpha) — clean RGB + separate alpha; AdobeAlphaMode = Straight (Nuke assumes premultiplied by EXR convention — turn the Read node's premultiplied toggle off for Straight).
Notes:
All EXR variants use PIZ compression (lossless), 16-bit half by default. EXR colour space is the host's linear working space, written unchanged (only the alpha form differs per variant). On-screen playback always recomposites the alpha by Display Mode, whatever the format here.

32-bit Float EXR (off = 16-bit half)

Default:
Off (16-bit half)
Description:
EXR is saved as 16-bit half by default — the standard delivery depth and lossless for the mask. Turn on for full 32-bit float (doubles file size, slower); useful only for technical/data passes. Only affects the EXR formats, not PNG. (Greyed out when Cache Format = PNG.)

Name Prefix

Default:
onyx_matte.####
Description:
Prefix for the cache subfolder and the frame filenames. Use #### for frame-number padding (one digit per #): default gives files onyx_matte.0001.png in subfolder onyx_matte/. Use a different prefix per node so caches in a shared folder don't mix.

Cache Folder

Description:
Folder where the cache is saved. Each node saves to <this folder>/<Name Prefix>/. Pick a different folder (or prefix) per node so two caches do not mix.

Open Cache Folder

Description:
Opens the selected Cache Folder in the system file manager.

10. Scene Cleanup & UI Appearance

Reset Scene

Description:
Remove all tracked objects and reset the init frame, so your next object starts a fresh scene on whatever frame you are on (and returns to Locked Init Frame mode). Your processing and appearance settings are kept.

Reset Parameters

Description:
Reset all processing and appearance settings (Pre/Post Processing, levels, trimap, crop, output mode, refiner, overlay colours and sizes) back to defaults. Does not delete your objects, change your license, or clear the Bake cache.

UI Appearance (group)

Parameters & defaults:
  • Pin Object Panel On Screen (Off) — keep the object panel fixed in one screen corner while you zoom or pan (by default it moves with the image). Handy when the panel scrolls out of view as you zoom in. Saved globally on this computer, so it applies to every project. Note: in Nuke the frame-edge line may draw over the panel (a host limitation).
  • Show Canvas Overlay (On) — show/hide all on-canvas editing overlays (boxes, points, handles, labels, object panel). Status banners stay visible.
  • Show Labels (On) — show/hide per-object labels and AI CROP badges. Requires Show Canvas Overlay.
  • Mask Overlay Color — colour and opacity of the mask drawn over the video in Overlay mode. Default (1, 0, 0, 0.5).
  • Overlay UI Color — colour/opacity of the on-screen guides (bbox frame, handles, cross, dashed crop outline). Default azure (0, 0.652, 1, 1.0).
  • Canvas Overlay Scale — uniform size of overlay elements. Range 0.25–3.0, default 1.0 (try 1.5 on a 4K monitor).

11. Server Tools & License

Server Tools (group)

Open Logs Folder:
Opens the ONYX logs folder (onyx3_plugin.log, onyx3_server.log). Send these to support when reporting a problem.
Detailed Logging (default Off):
Writes a detailed diagnostic log. To report a bug: turn this on, re-open the host (so the engine picks it up), reproduce the problem, then Open Logs Folder and send the files.

License (group)

Parameters:
  • Buy License — opens onyxofx.com to purchase.
  • License Key — paste your key here, then press Activate.
  • Activate License — one-time online activation (or paste an offline key starting with ONYX-); afterward the plugin runs fully offline.
  • Deactivate License — release this computer's activation to move the license to another machine.
  • Copy Hardware ID — copy this computer's Hardware ID for an offline activation key.
  • Status — read-only license status (e.g. "Licensed (online)", "Trial: 5 days remaining").
Note:
The license key is never saved into the project file (security). A 2.5.6 license activates v3 automatically — no re-entry needed.

Quick Reference: Which Tool / Setting for Which Task

TaskTool / ModeKey Parameters
Rotoscoping (single object)Add Bbox Object, Locked Init FrameBbox + FG refine points, Auto Fit BBox, Edge Feather
Keying (hair, fur, fine edges)Add Bbox Object, HQ RefinerRefiner Model, Trimap Erode/Dilate, Input/Output Levels
Quick masksAdd Bbox Object, Locked Init FrameBbox only, Crop Mode = Union
Detection by classAdd Object by TextText prompt ("person, car"), Score / IoU threshold
Multi-object selectionAdd Point Object (no bbox)One FG point per object (up to 16 objects)
Add an object mid-trackDynamic Init FrameCreate inside the tracked range; scene re-seeds
Existing Roto refinement (track it)Mask Prompt inputConnect a mask; HQ Refiner, Trimap Erode/Dilate
Edge-refine an external matte (no tracking)Refiner Direct inputConnect a matte; Refiner Model, Trimap Erode/Dilate, Show Trimap
Carve out a garbage matteMask Composite inputMask Composite Mode = Subtract
Hard shot the AI can't trackManualKeyframe the box/point; geometry interpolates between keys
Edge jitter / flicker fixAnyEdge Stabilization (static BG), Crop Stabilize %
Soft / grey mask, interior not solid 1.0AnyOutput Levels: White ~0.99 (Post Processing)
Deliver / lock the result, no VRAM on reopenBake Tracking CacheSet Cache Folder, enable Bake, play through once
DaVinci Resolve compositingAnyDisplay Mode = RGB*A (Premultiplied)
Nuke / After Effects compositingAnyDisplay Mode = RGB+A (Straight)

Processing Pipeline Order

Parameters earlier in the chain affect everything downstream.

  1. Input Color (Apply sRGB LUT) — the input frame converted to sRGB (on by default) + every channel hard-clamped into 0–1 — the picture that goes to the encoder.
  2. Encoder Crop — Crop Mode + padding region sent to SAM3 (Off = whole frame), resized to 1008×1008.
  3. SAM3 Decoder — generates the coarse mask from your prompts (box / point / text / mask).
  4. Memory Tracking — the mask propagates frame-by-frame per the Tracking Mode.
  5. Input Levels — shape the raw soft mask (Black / White / Gamma).
  6. Binarize Threshold + binary cleanup — Remove Sprinkles, Fill Holes, Binary Edge Smooth, Mask Shrink/Grow.
  7. Trimap + HQ Refiner — VitMatte / MEMatte recompute the soft alpha in the Erode/Dilate band (HQ Refiner mode).
  8. Post Processing — Mask Composite, Output Levels, Guided Filter, Feather, Offset.
  9. Edge Stabilization — temporal edge blend on the final mask (if enabled).
  10. Display Mode — how it is shown on screen (and, separately, Cache Format for what Bake writes to disk).

System requirements: RTX NVIDIA GPU (RTX 20-series or newer), NVIDIA driver R560+ (CUDA 12.6), 6 GB VRAM, 16 GB RAM, 8 GB free space. Windows 10/11 (64-bit) or Linux (glibc 2.28+). Hosts: Nuke, DaVinci Resolve, Fusion, Fusion Studio Standalone, Natron.