Everything about the ONYX Ai Matte v3 OFX plugin — installation and licensing, how the tool works in a shot, and a full reference for every parameter. AI-powered matting, object segmentation, tracking, and mask refinement built on SAM3.
Installing ONYX Ai Matte
ONYX ships as a disk image on macOS and an archive on Windows and Linux. Install it, restart your host, and the effect appears in the OFX list as ONYX Ai Matte v3. A 7-day trial starts automatically the first time you add the node.
Windows
Before you start
Close your host application — Nuke, DaVinci Resolve or Fusion Studio.
Install
Run ONYX-Ai-Matte-v3.4.2-Setup.exe.
Follow the installer and accept the license agreement.
Reopen your host. The effect appears as ONYX Ai Matte v3.
Run it elevated — as SYSTEM, or from an already elevated process. Windows asks for administrator rights on its own, and no switch suppresses that prompt.
Detect an existing installation from HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{B9B09E20-FC31-4F7C-AE45-5285D00F04E5}_is1.
Exit codes.0 — installed. 7 — nothing was changed, because a host still had the plugin loaded or the previous version could not be fully removed; retry when the machines are idle.
A running host is never closed for you. If Nuke, DaVinci Resolve, Fusion or Natron holds the plugin, the installer stops and leaves the machine exactly as it was.
Activation survives an update — a machine is activated once, not on every update.
macOS
Before you start
Quit your host application completely — Flame, Nuke, DaVinci Resolve, Fusion Studio or Natron. Closing the project is not enough.
Install
Double-click the downloaded ONYX-Ai-Matte-v3.4.2-macOS.dmg.
Double-click Install ONYX Ai Matte inside it.
Follow the installer window: welcome, license agreement, Install.
Enter your Mac administrator password when asked. It stays invisible while you type — that is normal.
Reopen your host.
No Terminal needed. A previously installed version is removed automatically and your license is kept.
Installs to
/Library/OFX/Plugins/ONYXAiMatte3.ofx.bundle
Requires
macOS 15 Sequoia or newer on Apple Silicon — M1, M2, M3, M4, M5 or newer. Intel Macs are not supported.
Deploying to many machines
Inside the disk image is a standard .pkg, so Jamf, Intune, Munki and installer itself handle it with no interface:
hdiutil attach ONYX-Ai-Matte-<version>-macOS.dmg
sudo installer -pkg "/Volumes/ONYX Ai Matte v<version>/Install ONYX Ai Matte.pkg" -target /
hdiutil detach "/Volumes/ONYX Ai Matte v<version>"
Exit codes.0 — installed. Anything else — nothing was installed; run with -verbose, or read /var/log/install.log.
Activation survives an update — a machine is activated once, not on every update.
A running host does not stop the install. macOS lets the new files land while the old ones stay alive for the running process, so the install succeeds — but that host keeps the previous version in memory until it is restarted. The package records which hosts were running, so a support case can tell whether they were restarted afterwards.
Signed and notarised by Apple, so Gatekeeper accepts it without an exception.
It worked before and now the node is gone?
Check two things, in this order — free disk space, then whether the plugin is still installed:
df -h /
ls -ld "/Library/OFX/Plugins/ONYXAiMatte3.ofx.bundle"
The bundle is ~5 GB, so on a Mac that is low on space it is a prime target: cleanup tools, or a hurried manual clear-out, remove large folders without knowing it is an installed program. "No such file or directory" means it was deleted — reinstall, your license is kept. Details: If the node disappears from your host.
Linux
Before you start
Close your host application. Copy the .tar.xz file to the Linux machine and unpack it there — not on another computer.
Install
Unpack the archive on this machine:
tar -xf ONYX-Ai-Matte-v3.4.2-linux-x86_64.tar.xz
cd into the folder it creates and run:
sudo ./install.sh
Type I AGREE when prompted.
Reopen your host.
Why it must be unpacked here
Unpacking the archive on Windows or macOS and copying the result across — or moving the unpacked folder through a share that does not carry symbolic links — leaves the bundle incomplete. The plugin is then listed in your host's plug-in preferences, but no effect appears in the OFX library.
To confirm a finished install:
ls -l /usr/OFX/Plugins/ONYXAiMatte3.ofx.bundle/Contents/Linux-x86-64/libcom_err.so.2
It must show an arrow — libcom_err.so.2 -> libcom_err.so.2.1. A plain file there means the archive was unpacked somewhere else and copied over.
Installs to
/usr/OFX/Plugins/ONYXAiMatte3.ofx.bundle
Requires
glibc 2.28 or newer (Rocky/RHEL 8+, Ubuntu 20.04+), NVIDIA GPU with a current driver.
Deploying to many machines
One switch answers both questions the script would otherwise ask — the license agreement and the removal of a previous version:
sudo ./install.sh --accept
Exit codes.0 — installed. Anything else — nothing was installed; the reason is printed on the console.
Activation survives an update — a machine is activated once, not on every update. The license lives in /var/lib/onyx and is never touched by an update.
A running host is not detected on Linux. The update replaces the bundle even while Nuke or Resolve is open, and that host keeps the previous version in memory until it is restarted. Schedule updates when the machines are idle, or restart the hosts afterwards.
Updating to a new version
Updating an existing installation
The same file you downloaded first
There is no separate update package. Download the current version from the Download page and install it the same way you installed the first time — it replaces the previous version in place.
Your license is kept
A machine is activated once. An update never asks for the key again and never spends a seat: the license is stored outside the plugin folder and is left untouched.
The previous version is removed for you
On all three systems the installer takes the old version out before writing the new one. Do not delete the plugin folder by hand first — on macOS and Linux that folder also carries files the installer expects to replace.
Close the host first
Quit your host completely before updating — closing the project is not enough. What happens if you do not differs by system:
Windows — the installer stops and changes nothing. The machine is left exactly as it was; close the host and run it again.
macOS and Linux — the update goes through, but the open host keeps the previous version in memory until it is restarted. Restart it and you are on the new version.
Confirming the version
Reopen your host and add the node — the version is printed in the plugin's Help and Diagnostics section.
Close your host before you install, and when it finishes fully quit and reopen your host; simply closing the project is not enough. This matters most in Flame, which scans OFX plugins only at startup — installing while it runs leaves it with a stale picture of what is on disk.
Activation
Licensing the plugin
Trial
Add the node and a 7-day trial starts on its own. Nothing to enter.
Online activation
Open the License section in the plugin, paste your key and activate. Needs a one-time internet connection.
Flame: where the key field is
Flame does not draw captions for text fields. The field is there and it works — only its label is not shown, so it looks like an empty box.
In Batch, the node's parameters are spread over numbered pages; the License group sits on one of the later ones. 🔑 The key goes in the empty box directly under the Buy License button — paste it there and press Activate License. Use that button as your landmark rather than a page number, which shifts with the Flame version and the width of the window.
Boxes that already show text — Licensed - Lifetime, the version number, Disabled — are read-only status lines, not inputs.
Networks, firewalls and proxies
What to allow
Activation contacts https://api.lemonsqueezy.com — HTTPS, port 443. The same address and port are listed in your license email.
Behind a proxy
The License section has Use Proxy, Proxy Server (in the form [user@]host:port) and Proxy Password. Fill them in, press Save Proxy, check the Proxy Status line reads Saved — then activate. See Proxy in the License group for the full description.
🖥️ On Flame these two fields have no captions either (see the note on activation above). They follow the Use Proxy switch, in order: the first empty box is Proxy Server, the second is Proxy Password. The box reading Disabled below them is the read-only Proxy Status line.
Custom plugin folders and shared network locations
Deploying plugins your own way
Where the bundle lives
Anywhere your host looks for OFX plugins. ONYX does not require a specific location — our installer simply uses the standard OFX path. A shared network folder for the bundle itself works fine.
What activation needs
One machine-wide folder that stores the license state. This is separate from where the bundle sits, and it must exist on every workstation.
Platform
License state folder
Windows
%PROGRAMDATA%\ONYX
macOS
/Library/Application Support/ONYX
Linux
/var/lib/onyx
Creating it manually
Our installer creates this folder for you. If you deploy plugins your own way and never run it, create the folder once per machine:
You do not create any files — the plugin writes them itself. The folder is machine-wide (shared by every OS account), so it is created once with admin rights and must be writable by the normal user your host runs as.
A shared network folder does not work for the license state. The license is bound to the hardware, so the state lives locally on each machine. Without this folder activation fails even with working internet — the key validates against the server, but there is nowhere to store the result.
If the node disappears from your host
ONYX is gone from the OFX list
known issue
Check this first:
Confirm the plugin is still on disk. Nine times out of ten it is simply no longer installed.
ls -ld "/Library/OFX/Plugins/ONYXAiMatte3.ofx.bundle" # macOS
ls -ld /usr/OFX/Plugins/ONYXAiMatte3.ofx.bundle # Linux
dir "C:\Program Files\Common Files\OFX\Plugins\ONYXAiMatte3.ofx.bundle" :: Windows
"No such file or directory" means it was deleted — just reinstall.
Why it gets deleted:
The bundle is around 5 GB because the AI models ship inside it. Disk-cleanup utilities — and people clearing space by hand — see a very large folder and remove it without realising it is an installed program. This is the single most common cause of ONYX vanishing from a host.
Prevent it:
Add the plugin folder to your cleanup tool's exclusion / ignore list: /Library/OFX/Plugins/ (macOS), /usr/OFX/Plugins/ (Linux), C:\Program Files\Common Files\OFX\Plugins\ (Windows).
Your license is safe:
The license is stored outside the plugin folder (see the table above), so reinstalling does not cost you an activation and does not require re-entering the key.
If the plugin IS still on disk:
Then the host has blacklisted it. Hosts cache the result of scanning OFX plugins, and a plugin that failed to load once — running out of disk space is enough — is skipped on every later start, even after the original problem is gone. Clear that cache and restart the host; it is rebuilt automatically.
Quit Nuke, delete the ofxplugincache folder inside that path, then start Nuke again.
Other hosts: look for a folder with ofx and cache in its name under /var/tmp (macOS, Linux) or %TEMP% (Windows).
Running out of disk space breaks ONYX silently. The plugin needs free space to compile its AI models on load, and with a full disk it cannot even write its own log — so the log will be empty rather than helpful. Keep a few GB free.
What ONYX Actually Is
ONYX isn't a magic button, and it doesn't claim to be. It's built to get you 60-80% of the way to a finished matte, fast — especially on the hard cases: motion blur, transparency, hair, difficult greenscreen and non-greenscreen shots. Any professional roto or keying shot is always a stack of nodes; ONYX replaces most of that stack, not all of it. An experienced artist still finishes the last mile.
Field Recommendations
Hard-won advice from production use — short and practical. Read this before you start: it saves hours.
First: feed video into the Source input
Where it all starts:
Connect your footage (or a still) to the node's Source input — this is the image the AI detects the object and builds the mask on. Without a Source there is nothing for the plugin to analyse: no detection, no tracking.
Getting started: select an object and press Play
To start tracking:
Select the object you want to track — the box tool is on by default: just drag a box on screen around the object, and the mask appears. Now press the regular Play — the object is tracked frame by frame from the frame where you created it.
Interior alpha not a solid 1.0? Lower White to 0.99
Do:
If the object's fill in the mask is not fully opaque in places — the interior alpha sits a touch below 1.0 (e.g. 0.98–0.99, "almost white" instead of a clean 1.0) — open the Post Processing group and lower Output Levels: White to about 0.99. Everything that was near 1.0 snaps to a solid alpha of 1.0.
Why:
The AI outputs a soft, probabilistic alpha, and the object's interior can sit a hair below 1.0 — this is a property of the network's probabilistic output, not a plugin defect. A white point of 0.99 remaps those near-1 values to a clean 1.0; it barely affects the soft edge (an overall lift of about 1%). It is a standard levels operation applied to the finished matte in Post Processing, so it does not touch segmentation or tracking.
Turn Loop playback OFF when tracking — on every host
Do:
Before tracking with Play, switch the player's repeat mode to a single pass (Once / Loop off) — in every host.
Why:
Tracking is strictly sequential: the model's memory walks frame by frame. With Loop on, playback reaches the end and jumps back to the start — the playhead lands outside the tracked range, the track is interrupted and the plugin shows the "GO INSIDE FRAMES…" banner. The banner says it directly: "TURN OFF LOOP PLAYBACK".
Cover the whole range in one pass: use Bounce (Ping-Pong) playback
Do:
To segment the entire shot from a single init frame, set the player's repeat mode to Bounce (ping-pong / back-and-forth) instead of Loop, then press Play. One pass runs forward from the init frame to the last frame, then reverses back through to the first — so wherever the init frame sits, the whole range gets tracked in one go. Works the same in any host that offers this mode (Nuke: Bounce; other players may call it Ping-Pong / Back-and-forth).
Why:
Tracking propagates in both directions from the init frame — forward covers init→end, the reverse leg covers init→start. Unlike Loop, Bounce never throws the playhead outside the range (it turns around at each end), so the track is never interrupted and no "GO INSIDE FRAMES…" banner appears. Frames already tracked simply replay from memory; only the still-untracked side is freshly segmented.
Flame: no parameters after adding the node in Batch? Pick ONYX from "Load Plugin"
When:
You added an OpenFX node in the Batch schematic and its setup shows only Resolution, Tagged Colour Space and FX Range — no ONYX controls at all, and no License group.
Do:
With the node selected, choose ONYX Ai Matte v3 from the Load Plugin drop-down in the Plugin column. The plugin's own parameters appear only after that. Then connect your source to the node's Front input.
Why:
In Batch the OpenFX node is an empty container until a plugin is loaded into it — the fields shown before that belong to the container, not to ONYX. Adding ONYX from the timeline Effects ribbon (Timeline FX) or through Batch FX skips this step entirely: there you pick the plugin itself, so its controls are present immediately.
Flame: drew a box and nothing happened? Make the node active first
When:
In the Batch schematic or in Batch FX you added an OpenFX node, picked ONYX Ai Matte v3 from Load Plugin, the picture is there — but the box does not draw at all: no box, no object, no reaction to the mouse.
Do:
Make the ONYX node the active one and the on-screen tools come alive. Batch / Batch FX: click any other node in the schematic, then click the ONYX node again — opening the Result view works too. Timeline: select the OFX — ONYX Ai Matte v3 strip under the image, or double-click into the effect editor. Effects: no extra step needed — it works straight away there.
Why:
The plugin's on-screen tools belong to the node that is currently active in the host. Right after you pick the plugin the new node is not active yet, so the tools stay with the previous node and your clicks never reach ONYX. Selecting the node hands them over. This extra step will be removed in one of the next updates.
macOS installer stops before installing anything
When:
The installer asks for your administrator password and then stops with an error. Nothing is installed, and any previous version is left exactly as it was — the installer changes nothing until it is certain it can finish.
Do:
Move the downloaded .dmg to an ordinary local folder and open it from there — not from Desktop or Documents if those are synced to iCloud Drive, Dropbox or Google Drive. Your Downloads folder is normally fine.
Why:
Cloud folders keep files in the cloud and fetch them only when something opens them. The plugin is about 5 GB, and a file that is not physically on the disk cannot be read reliably while installing with administrator rights.
Also check:
Free disk space. The installed plugin needs roughly 6 GB, and the image you downloaded takes about 5 GB more until you delete it.
Flame: "Plugin com.onyx.aimatte3 not found"? Clear Flame's OFX cache
When:
ONYX appears in the Load Plugin list, but choosing it puts PIPELINE : Plugin com.onyx.aimatte3 not found in the status line at the bottom of the screen.
Do:
First make sure the plugin actually installed — the installer must have ended with the word DONE. Clearing the cache before that achieves nothing: Flame rescans, fails to load the plugin again and records the failure again. Once the installation has succeeded, quit Flame completely, then in Terminal find Flame's OFX plugin cache and delete it:
sudo find /opt/Autodesk -name ofxPluginCache.xml
Delete the file (or files) it prints, then start Flame again — the cache is rebuilt automatically at startup. If the message persists, check the ignore list:
Anything listed inside that file is skipped by Flame at startup; remove the ONYX line, save and restart Flame.
Why:
Flame scans OFX plugins once at startup and caches the result. If the plugin failed to load on one of those scans — most often because it was installed or updated while Flame was running — the name stays in the list from the cache, but the plugin can no longer be loaded. Install ONYX with Flame closed and this does not happen.
Flame: track stuck on the first frame? Switch to "Play All Frames"
When:
Only if, tracking with Play in Flame, one frame renders and the track does not advance (it gets stuck on the first frame).
Do:
Press and hold the Play button → pick Play All Frames (instead of Play Real-Time) and the Once repeat mode. Play renders every frame in order — the track walks frame by frame to the end of the range.
Why:
In Play Real-Time mode Flame keeps real time and skips frames that did not finish rendering; an AI frame takes a few seconds, so playback runs ahead.
DaVinci Resolve: plugin interface not showing? Turn the overlay mode on
When:
Nothing is drawn on the canvas — no boxes, no points, no object panel. Most noticeable after reopening a project (with Freeze off).
Do:
Under the viewer, click the arrow next to the Qualifier icon (eyedropper) → choose Open FX Overlay.
Why:
DaVinci only draws plugin overlays in that mode. Reopening a project resets it, so the interface is not shown even though the plugin itself is working.
Linux: the Freeze checkbox blinks when you decline an overwrite — that is expected
What happens:
You press "No" in the "Overwrite baked masks?" dialog — the Freeze checkbox lights up for a moment and goes off again.
Do:
Nothing. The final state is correct: baking did not start, the checkbox is off, the files on disk are untouched.
Why:
On Linux the dialog is drawn by a separate program, and while it is open the host cannot repaint the interface. After the answer it manages to show the checkbox as on, and the revert lands right after.
Mask slips mid-track? Give SAM3 more context
Do:
Increase Crop Padding — or switch Crop Mode off entirely. Change it on the init frame (in Dynamic mode — anywhere).
Why:
SAM3 recognizes an object inside its surroundings, on a fixed 1008×1008 input. The crop exists so that objects in 4K footage don't shrink into a few pixels — but cropped too tight, the model loses the context it needs to understand what it is looking at. When tracking gets unstable, more context beats more zoom.
Mask clipped at the crop edge on fast motion? Raise Crop Padding
Do:
On fast object motion or a whip pan, raise Crop Padding — or switch Crop Mode off. Change it on the init frame (in Dynamic mode — anywhere).
Why:
The crop is a window that follows the object with this margin, but lags one frame behind — each frame's window is built from where the object was on the previous frame. If the object travels farther in one frame than the margin covers, part of it falls outside the encode window and the mask is clipped in a straight line along the crop edge. The tell that separates this from the drifting track above is that the mask is cleanly cut along a straight line, not smeared. More Crop Padding = more room for per-frame movement; Off = full frame, no clipping, at the cost of slightly less edge detail.
Bake writes exactly what you see
The rule:
With Bake on, only the frames that play on screen are written to disk. If the mask is gone from the screen and you press Play, empty frames are written over that range — a baked range of nothing means the track was lost while baking.
Overwriting:
Un-bake, change the tracker, bake again — that is a new pass: with the same Name Prefix it overwrites the previous files. Change the prefix to keep both — or re-bake only the frame ranges you fixed.
Workflow:
A proven pattern: bring the baked sequence back into the project as an image Read, then with new bakes touch only the ranges that need fixing — the final matte is assembled from passes. And decide the Tracking Mode before you start tracking: it directly shapes how the plugin behaves.
Sharing baked masks with a team
The rule:
The bake cache belongs to the node, not to the folder. A node keeps its own record of which frames it baked and with which settings; nothing of the sort is written next to the files on disk. So a second node — on another machine, or a fresh node in the same project — will not accept those frames as its own: it re-renders them and overwrites them. This is deliberate, so that two nodes writing into one folder never pass off each other's masks.
Share the sequence, not the cache:
A baked mask is an ordinary PNG or EXR image sequence — a self-contained deliverable. Your colleagues do not need ONYX for it at all: they import the sequence from the shared drive as media and use it as a matte or a layer.
Team workflow:
One person makes the selection and bakes to a folder on the shared drive; everyone else brings that sequence in as media; when something needs changing, the same person adjusts it and re-bakes, and the updated sequence is picked up automatically.
Watch out:
The Cache Folder path is stored in the node literally. If the shared drive is mounted under different paths on different machines, the plugin reports that the folder does not exist. Change the path with Bake switched off and enable it afterwards — changing it while Bake is on also clears the node's record of what it baked.
Hard shot? Prep the image first
Do:
Feed ONYX a prepped branch — e.g. a high-pass / local-contrast boost (the Nuke Survival Toolkit HighPass in high-contrast mode works well). Keep it gentle.
Why:
The AI sees only what you feed it: lifting local contrast makes a dim object stand out. But don't over-crank — push it too far and the edges turn jagged, because the refined edge follows the image you feed in.
SAM3 sees objects, not backgrounds
The rule:
Keying in the usual sense does not work here — don't click the background hoping to "key" it out. SAM3 segments objects, not backgrounds. This fully applies to chroma key (green screen): select the objects standing on it, not the green background (a point or a bbox each), refine them, and invert / comp downstream — we segment the objects; the background comes from the inversion.
One object — one point:
Don't drop several point objects onto one thing. Place one point to create the object, then grow and correct it with FG/BG refine points — that tells SAM3 "this is a single object and we are refining it". Scattered point objects read as separate objects and fight each other.
Finished tuning an object? Don't touch it — add a second node
The problem:
You selected a tricky object (say, a forest) with a point, cut the holes, tuned it — and the next click nearby wrecks the work: new prompts near a finished object can re-assign or correct it.
Do:
Leave the finished node alone and build the next selection in a second ONYX node, then merge the mattes downstream. Use a Merge set to max (Nuke) or Lighten (Flame) — not Plus/Add, which sums the two mattes and pushes anywhere they overlap above 1.0 in a float pipeline. Screen is a safe alternative — it can never exceed 1.0 either — but where two soft edges overlap it lifts them slightly (0.5 and 0.5 give 0.75, where max keeps 0.5), so it grows feathered edges a little. Alternatively feed one node's result into the other's Mask Composite input with Mode = Add — despite the name that one is a true union (a per-pixel maximum), so it can never exceed 1.0.
Why it's free:
Any number of ONYX nodes share one inference server under the hood — extra nodes do not multiply GPU memory use.
The Nature of the Edge, and Where ONYX Sits in the Pipeline
A short, technical explanation of where edge jitter comes from and how ONYX is positioned. Read it to set the right expectations and build your processing chain.
Why the edge "boils" from frame to frame
The HQ Refiner (VitMatte / MEMatte, see Refiners) is a per-frame edge refiner. It processes each frame independently: it recomputes the soft alpha inside the trimap band from the actual pixels of that frame, with no memory of how the edge looked on the neighbouring ones. SAM3's segmentation does carry memory between frames — that is what tracking runs on — but the edge refinement does not.
The consequence: wherever the real pixels along the edge shift slightly from frame to frame — sensor noise, grain, compression, the micro-movement of hair — the boundary of the soft alpha shifts slightly too. Visually this reads as a fine tremor, or "boiling", of the edge (edge boiling). This is expected behavior of per-frame AI refinement, not a defect: the refiner faithfully reflects what each frame actually contains.
The built-in answer — Edge Stabilization
Edge jitter is not a bug to be "fixed in code"; it is the nature of per-frame edge refinement, and the plugin has a built-in tool for it — Edge Stabilization. It supplies the missing temporal continuity: it blends the edge band with the previous frame, compensating for camera motion, damping the frame-to-frame tremor. It is not a patch over the problem but the pipeline layer that deliberately adds temporal continuity where a per-frame refiner has none by design. It works best on static or slow-camera shots; the Stabilization Strength and Motion Sensitivity parameters balance a calm edge against a slight trail (ghost) on fast motion — details in Edge Stabilization.
ONYX is a pipeline, not a one-click button
ONYX covers a spectrum of tasks: from fast garbage / rough mattes to final-quality masks. Where a given result lands on that spectrum depends not on a single checkbox but on how the artist built the chain:
Input prep:
Out-of-range values (negatives, super-white) are clamped for you: on input the frame is converted to sRGB by default (the Apply sRGB LUT checkbox at the top of the panel — turned off for ACES / your own colour management) and every channel is hard-clamped into 0–1 (negatives → 0, brighter than 1 → 1) in either mode, so you do not need a Clamp before the node — SAM3 always receives a correct range (this covers any source outside 0–1, not only EXR). Just note that a hard clamp collapses detail in deep highlights: if the subject is hiding in super-white on HDR/log, don't just Clamp — bring the image into 0–1 with a grade or lift local contrast. The refiner resolves the edge against the image you feed it: the cleaner and more legible the input, the cleaner the edge (see Field Recommendations).
Refiner choice:
RAW / Hard Mask for fast, hard masks; the HQ Refiner (VitMatte or MEMatte) for hair, fur and soft / semi-transparent edges (Refiners).
Trimap tuning:
Erode / Dilate set the band the refiner is allowed to recompute the edge in: narrower is faster and tighter, wider gives fine hair more room (Refiners).
Edge Stabilization:
Temporal edge stabilization where frame-to-frame continuity is needed (Edge Stabilization).
The practical point: ONYX is a set of tools inside one pipeline, not a one-click solution. The same shot can yield a rough garbage mask in a minute or a final matte with hair — the difference is how the chain of prep, refinement and stabilization is built. If a result comes out rougher than expected, that is a signal to refine the chain (input, refiner, trimap, stabilization), not the tool's ceiling. The result follows predictably from that chain.
The Scene
ONYX Ai Matte works as a single unified pipeline: one scene can hold every prompt type at the same time — bounding boxes, points, text, and a mask-prompt — and they do not exclude one another. You mix and match as needed; one node, one SAM3 model path.
One scene, all prompt types
Object limit:
Up to 16 objects per scene. Adding a 17th is blocked with a "MAXIMUM 16 OBJECTS" banner.
Init frame:
The init frame is set by the first object you create — the frame you placed it on. Its number is shown in the Object Panel (the INIT FRAME line). Where you can create/delete objects depends on the Tracking Mode.
How it works:
SAM3 runs at a fixed 1008×1008 resolution. Whatever region is sent in is squeezed to that size, then segmented; tracking propagates the mask frame-by-frame through the model's memory.
Tips:
Start an object with a box or a point, add text to grab objects by name, refine with extra points — all in the same node. Switching the Tracking Mode preserves your objects.
Node Inputs
The node has one required input — the image itself — and three optional inputs. All three take a mask, but each uses it for a different job: Mask Prompt tells the AI what to track. Refiner Direct sends your mask straight to the edge refiner to get its edges polished. Mask Composite stamps a mask on top of the finished result. Use them together with AI objects — or on their own.
Source
Description:
The footage you want to matte and track. Everything the node does reads this image.
Mask Prompt
What it does:
Takes a mask you already have and uses it as a prompt — "track whatever this mask covers". Connect it, and one mask-driven object appears on the init frame. From there the AI tracks it through the shot like any other object. And the mask does not have to be a whole object: it can be a sub-object — a part of something bigger (a sleeve, a door of a car, a logo on a shirt) — and the AI will track exactly that part.
When to use:
Two cases. A part of an object — boxes and points snap to whole objects, while a mask lets you point at exactly "this piece": roto the sleeve or the car door once, and the AI tracks just it. A ready shape — you already have a roto, a luma key or a paint-out on one frame and want the AI to do the tracking. Example: a quick roto of a person + the HQ Refiner = hair-level edges on every frame.
Notes:
It appears in the Object Panel as a MASK row. It has no delete X — to remove it, disconnect the input. The mask is only read on the init frame; after that the object tracks by memory, like a box or point object.
Correcting mid-track:
The AI drifted somewhere mid-shot? Stand on the bad frame and move or reshape your input mask — a correction key is set right there. On that frame your mask becomes the result, and tracking continues from it. Frames without a key ignore the input mask, and an animated (interpolating) mask does not create keys on the in-between frames. Keys show as diamonds on the MASK row; clicking a diamond removes that frame's key. Keys survive saving and reopening the project.
Refiner Direct
What it does:
Sends an external matte straight into the HQ Refiner. Nothing is segmented or tracked here: the node takes your matte as it is and re-draws its edges against the Source image — hair, fur, semi-transparency. If there are AI objects too, the polished matte is merged (union) with the AI mask.
This one input carries two things, told apart by channel: Refiner Direct Input: Object Mask says which channel holds the matte described here, and Refiner Direct Input: Unknown Zone says which channel holds an area for the refiner to work out for itself. Put them in different channels — the mask in red and the zone in green, say — and both are resolved in a single pass.
When to use:
Three ways. Alone — no AI objects at all: the node becomes a pure edge polisher for a matte you made elsewhere. A typical case: a mask tracked in another system, such as Mocha or a planar tracker, comes in with hard, rough edges — feed it here and every frame gets refined edges (hair, fur, soft transitions). Connect the matte, pick the Refiner Model, done. Combined — the AI tracks the object while your matte covers a part the AI misses (a prop, a logo patch); both get the same edge treatment and are merged. Removing seams between ONYX objects — masks tracked by ONYX itself can come back here too: when two tracked objects overlap, merge their mattes into a single mask and feed it in. The refiner sees one shape, so the seam where the objects meet disappears, and the outer edge is rebuilt clean.
Notes:
Connecting this input turns the refiner on automatically. The edge band is built from your matte — adjust its width with Trimap Erode / Dilate, and check it visually with Show Trimap (in the Matte Overlay display mode). The input is read every frame, so an animated matte is refined frame by frame.
Mask Composite
What it does:
Stamps an external mask onto the finished matte — after the AI, the refiner and all cleanup. The Mask Composite Mode decides how: Add = the mask grows the result, Subtract = the mask is cut out of the result, Intersect = only the overlap stays, Multiply = the two are multiplied.
Four masks are read from this one input: the usual one plus each colour channel, each with its own operation — see Mask Composite Mode R / G / B.
When to use:
Garbage mattes and hold-outs. Example: the AI tracks a person and a lamp post passes in front — feed a lamp-post mask, set Subtract, and the post is cut out of every frame.
Notes:
This input never touches the AI or the refiner. It is a plain compositing step on the final mask — applied the same way in every Display Mode, and baked into the Bake Tracking Cache files.
The Object Manager
The on-canvas panel that lists every object in your scene — shown as soon as the node exists, with OBJECTS 0 and dashes before the first object. Drag it by the "OBJECTS" title bar or by the strip above the toolbar to move it anywhere on the viewport; the arrow to the left of "OBJECTS" collapses the object list (the two toolbar rows stay). Above the title sit the two rows of the toolbar (see below); under the title are the INIT FRAME and TRACKED FRAMES lines.
The toolbar — two rows above the title. Every button mirrors a parameter of the node: click it here or in the parameter panel, the other follows. Row one: the five Object Tools (Select / Edit, Add Bbox Object, Add Point Object, Add FG / BG Refine Point) — the armed tool sits on a light plate; four growing dots = Point Detail Level, taking the shape and colour of the point kind in use (dots for a point object, green / red diamonds for refine points; grey and inactive while no point tool is armed and no point is selected); a padlock; and Reset Scene. Row two:INIT with an arrow that points to where the init frame is (◀| behind, |▶ ahead, ● you are on it — inactive) — a click jumps there, the same as Go to Init Frame; and the four Display Mode options OVERLAY · MATTE · RGB*A · RGB+A.
The padlock. Closed (the default): the armed tool never switches by itself — only turning Freeze on parks it, as before. Open: after you create an object (a box or a point) the tool returns to Select / Edit; refine points do not switch it (they are placed in series). The padlock is a setting of this computer, not of the project: it applies to every project and nothing resets it.
INIT FRAME — click to jump there. Hover the INIT FRAME line and it turns green — the colour that marks the init frame — then click to send the playhead to it. It does the same as the Go to Init Frame button in the parameters. On the init frame itself the line is inactive (you are already there), and it stays inactive on hosts that ignore playhead jumps — DaVinci Resolve and Flame, where the button is hidden for the same reason. Nuke and Fusion Studio support it.
Reset Scene — the ↻ button at the right end of the toolbar. Press and release on it to wipe the whole scene (the same as the Reset Scene parameter). Dark red when idle, bright red under the cursor and while pressed; on an empty scene it is inactive. The action can't be undone, so there's a safeguard: move the cursor off the button before releasing and the reset is cancelled. Works on every host.
An object row
Solo (eye, on the left):
Click the eye to isolate an object. When any object is soloed, only the soloed objects appear in the result and the rest are hidden. Soloing is additive: solo several objects and they all show together. Click the eye again to un-solo.
Visibility key (on the right):
Click to show or hide the object on the current frame. This sets a visibility keyframe that holds until the next one — so an object can appear or disappear at the exact frames you choose. When an object has visibility keys, its circle fills green.
Refine marker:
Appears when the object has refine points on the current frame. Click it to clear this frame's refine touch-ups for that object. On the MASK row the diamond shows mask correction keys (see Mask Prompt) — clicking it removes the current frame's key.
Delete (X):
Remove the object. It is enabled only where the Tracking Mode lets you add or delete objects: on the init frame in Locked Init Frame (the default), inside the tracked range in Dynamic, or on any frame in No Init / Manual. Off an allowed frame the X is inactive.
Text-prompt groups
Description:
Objects found by a text prompt are grouped under a badge (the phrase you typed), with their rows indented below it. The group badge has its own delete X to remove the whole group at once.
Input Color
The topmost group on the panel (open by default). It controls the colour in which ONYX feeds the frame to SAM3.
Apply sRGB LUT
Default:
On.
Description:
SAM3 was trained on standard sRGB images, so ONYX must feed it an sRGB picture. On (default): the plugin converts a LINEAR input into sRGB for you — keep it on for a normal linear pipeline (Nuke's default working space). Off: pixels are sent as-is — use this when the input is already sRGB, or when you colour-manage the project yourself. Example: in an ACES project the plugin behaves differently on a linear EXR — turn the checkbox off and prepare the picture with an OCIO node to suit your pipeline. Values are always clamped to 0–1 in either mode; affects float input only (8/16-bit is already display-referred).
When to change it:
It is a seed parameter — it changes the colour the AI sees — so it is available on the init frame (like the crop parameters). Off the init frame the checkbox is greyed out; changing it on the init frame re-seeds the track with the new colour.
ONYX Ai Matte Parameter Reference
The numbered sections below walk through the parameter groups one by one. The reading order is chosen for clarity and does not have to match the order of the groups in the panel: that layout follows how people work and changes over time.
1. Object Tools
The prompt-creation tools. Exactly one tool is armed at a time (radio behaviour); the default is Add Bbox Object. The active tool is shown grayed out (you cannot click it off), so exactly one is always armed. Tools, text, and detail level live in the Object Tools group at the top of the UI. The same five tools also sit on the toolbar at the top of the on-canvas Object Manager, where the armed one is highlighted; its padlock decides whether the tool jumps back to Select / Edit after you create an object (closed, the default — never; open — after every new box or point).
Select / Edit
Description:
Select and edit existing objects — drag boxes and points, pick rows in the Object Panel. Creates nothing. Dragging a box corner or point on a tracked frame is recorded as a correction on that frame, and tracking continues with it applied.
Add Bbox Object
Description:
Draw a rectangle around an object — one rectangle = one object. The fastest way to start for most footage.
Add Point Object
Description:
A single foreground click creates a new object. The captured size is set by Point Detail Level.
Not for editing an existing object:
This tool always creates a new, separate object. To add or remove part of an existing object (an edge, a bit sticking out, a leaked chunk), use the FG / BG Refine Points, not Add Point Object. A common mistake is to keep dropping new point-objects to "grow" an object — the right tool for that is refine points.
Add FG Refine Point / Add BG Refine Point
Description:
Refine points are correction clicks that fix a specific object's mask where the AI got the edge wrong — you don't redraw the mask, you just nudge it. Pick Add FG Refine Point (foreground) and click an area that should belong to the object but is missing — the mask grows to include it. Pick Add BG Refine Point (background) and click an area that leaked into the mask but shouldn't be there — the mask shrinks to exclude it. Each click is tied to the frame you click on: the AI recomputes that object's mask on this frame with your hint and then carries the correction forward as it keeps tracking, so one good fix usually holds on the following frames too, not only the frame you clicked. Drop several FG/BG points together to sculpt a difficult edge (hair, a thin limb, a hole in the middle). How big an area a single point adds or removes is set by Point Detail Level. Refine points only correct an existing object — they never create a new one; to add an object use the Bbox / Point / Text tools.
Important — which object the point attaches to:
Place a refine point outside all boxes and it attaches to the currently selected bbox. Place it inside a box and that bbox becomes selected and the point attaches to it. So before you drop a refine point, always check which bbox is currently selected — otherwise the point may attach to a different object than you meant.
Where:
Refine points are not limited by the AI-CROP outline: click outside it and the processing window of that frame widens (using the crop padding from the UI) to fit the new point — a click far away simply grows the window up to the full frame. On the init frame, an FG point placed outside the drawn box grows the box itself. With Crop Mode = Off there is no outline at all.
Delete a point:
Select it and press Del. Move: click and drag.
Point Detail Level
Options:
Tiny detail, Object part, Large object part, Whole object (default: Whole object) — on the Object Manager's toolbar the same setting is the row of four growing dots, coloured like the kind of point in use.
Description:
How big a piece a point grabs. For Add Point Object it sets the size of the new object (Whole object = the whole thing under the point, Tiny detail = a small part, e.g. a hand instead of the whole person, with two sizes in between). For refine points it sets how big a part to add (FG) or remove (BG); the FG/BG Refine tools default to Object part. It is per-point — each point remembers the level chosen when placed, and a selected point's level can be changed later; selecting a point also shows its own level in the dropdown. A point stays selected until you click empty canvas, select another point, or switch tools — switching tools drops the selection, so the dropdown (and the size dots on the toolbar) go back to being the default for the next point. It works the same on the init frame and while tracking — every point, including a refine point, keeps its own size. It applies to points only: it does not change how a drawn Bbox itself is segmented.
Text Prompt
Add Object by Text
Description:
Type what to find (e.g. "person", "red car", "dog"). In Nuke press Enter; in DaVinci/Fusion click Submit Prompt. Each submit adds the objects it finds to the scene.
Several concepts at once:
A comma-separated line finds each phrase as a separate group: berry, bread, milk creates three groups, each with its own badge (the full phrase) and colour. Prompts are cumulative — submit several in a row, each adds what it finds.
Creation rules:
Text creates objects under the same per-mode rules as boxes and points (see Tracking Mode).
Submit Prompt
Description:
Detect objects from the prompt above and add them. Use this in Fusion/DaVinci, where pressing Enter does not commit the field.
Score threshold
Range / Default:
0.0 – 1.0 / 0.3
Description:
Minimum confidence to keep a detection. Higher = fewer, surer objects; lower = more, looser ones. Raise it if wrong objects are detected; lower it if the correct object is missed.
IoU threshold
Range / Default:
0.0 – 1.0 / 0.3
Description:
How aggressively overlapping detections of the same thing are merged (NMS). Higher = keep more near-duplicates; lower = stricter de-dupe.
Mask Prompt (a node input): connect an external mask and it creates one object on the init frame — see Node Inputs.
2. Tracking Mode
How an object is tracked across frames, and on which frames you can create/delete objects. You can switch mode at any time (except while Bake/Freeze is on — then it is greyed out); switching re-seeds the track on the current frame (which becomes the new init frame), and the objects stay.
Mode
Options:
Locked Init Frame (default) — strict memory tracking. Create objects on the init frame, press Play, the mask propagates by memory. Objects can be created/deleted only on the init frame. The most reliable mode, and a saved project reopens with the track restored.
Dynamic Init Frame — like Locked, but you can also add/delete objects anywhere inside the already-tracked range; the init frame moves to the current frame and the scene re-seeds from there (your refine touch-ups are kept). The earlier section is re-tracked. Reopens with the track restored.
No Init Frame — no fixed init frame: create/change objects on any frame, the scene re-seeds from there. The freest and fastest. The track is not restored on reopen (turn on Bake to keep masks on disk). Shows "MODE : NO INIT" in the panel.
Manual — hand-keyframe mode for rare hard frames where the auto-track breaks. Each frame is computed independently (no memory); set a box/point key on the frames you need and the geometry is interpolated between keys. The box/point is BRIGHT on a key and DIM between them. Shows "MODE : MANUAL".
Default:
Locked Init Frame
Reopening a saved project:
In Locked and Dynamic the track is restored on reopen, but it replays sequentially: go to the init frame and play forward to re-build it (the AI's memory is sequential — you cannot jump over it). A banner "GO TO INIT FRAME N TO START TRACKING" reminds you when you are off it. With Bake on, the baked masks play straight from disk — no engine, no VRAM, no re-track; turn Bake off to re-establish the live track from the init frame. No Init and Manual do not restore the track on reopen — turn on Bake first if you need to keep the result.
Tips:
Refine touch-ups are kept when moving between the auto modes (Locked / Dynamic / No Init) and cleared when moving into or out of Manual. Reset Scene returns to Locked Init Frame.
Init Frame
A line under the Mode dropdown shows the scene's init frame — the frame tracking started on. It reads a dash when there is no single init frame yet: an empty scene, or No Init / Manual mode.
Go to Init Frame
Jumps the host playhead to the init frame in one click — handy for getting back to the start of the track (e.g. after the "GO TO INIT FRAME" banner). Works in hosts that let a plugin move the playhead: Nuke, Fusion Studio, Natron. In DaVinci and Flame the button is disabled — these hosts do not let a plugin move the playhead. On the canvas, the INIT button in the Object Manager's toolbar does the same — its arrow shows where the init frame is (◀| behind, |▶ ahead, ● you are on it).
3. Tracking Options
How the crop window frames your objects, plus seed and box behaviour. The crop parameters and Enrich First Frame are "seed settings" — in Locked mode they can be changed only on the init frame (greyed out elsewhere); in Dynamic also inside the tracked range; in No Init on any frame.
Crop Mode
Options:
Off — encode the whole frame (the plain SAM3 path, maximum context, but maximum compression at 4K).
Union (Recommended) (default) — one crop around all objects plus padding. Fast, good quality.
Per-Object (Slower, Higher Quality) — a separate full-resolution crop per object. Sharper on small, widely separated objects, but slower (one encode each).
Description:
Which part of the frame is sent to SAM3. A crop fills the 1008×1008 budget so the object gets more pixels (sharper edge) — but with less surrounding context. If a tight crop makes the mask break up or grab the wrong thing, raise Crop Padding or switch to Off.
Crop Padding (%)
Range / Default:
0 – 100% (display 0–50) / 20
Description:
Extra margin the AI keeps around the object inside its crop, as a percentage of object size. More padding = more room for the object to move between frames without leaving the crop.
Crop Stabilize %
Range / Default:
0 – 20 / 0 (off)
Description:
Steadies the crop window so the mask edge does not shimmer on near-static frames. It is the % the object may drift inside the window before it re-centres. 0 = the crop follows the object every frame; raise to ~2 only if a near-static edge shimmers.
Crop Aspect Limit
Range / Default:
1.0 – 1.8 / 1.5
Description:
SAM3 always squashes the crop window to a square before inference. A long, thin window gets distorted, which can make the mask wobble or "melt" on elongated objects. This caps the distortion: lower pads the window toward a square with real background (steadier, slightly less fine detail); higher keeps it tighter (more detail) but distorts more. Affects only elongated objects; near-square ones are unchanged. Above 1.8 = out of the model's training range.
Edge Padding
Range / Default:
0 – 100 px / 20
Description:
Recovers the mask along the frame edge, where SAM3 is weakest: an object cut off by the border has no data beyond it, so it comes out slightly cut in. This grows the picture by a small mirrored margin before the AI looks — the object continues past the frame with its own texture and no seam — then trims it back. A plain stretch would read as a separate object and get cut at the seam; a mirror has none. Only affects objects touching a frame edge (interior objects are unchanged). Measured in frame pixels; 0 = off. Default 20; raise toward 50–100 for a stronger edge or heavy motion blur.
Enrich First Frame
Default:
On
Description:
On the first seed the model has no memory yet, so the starting mask can come out rough or partial. With this on, the init frame is run through the model's own memory twice (a draft pass then a clean pass), so the starting mask is as clean as a tracked frame. Costs one extra pass on the seed frame only — no cost while tracking. Turn off to use the raw first-frame mask.
Auto Fit BBox to Mask
Default:
On
Description:
While tracking, the box hugs the mask as it grows/shrinks (smoothed). Off: the box keeps the size you drew and just follows the object's centre — steadier for an object that stays roughly the same size. On the init frame the box always stays exactly as you drew it (this setting is ignored there).
Re-assign Object on Drag
Default:
Off
Description:
On: dragging a tracked object's box/point onto a different object replaces it (the old one is forgotten). Off (default): the drag re-locates the same object (use it when your object moved or was briefly lost and you point to where it is now).
4. Refiners — finishing the mask edge
Choose how the mask edge is finished, and (for the HQ Refiner) control the trimap band the AI works on.
Mask Output
Options:
RAW Mask — the soft grayscale straight from SAM3 (shape it with the Input Levels).
Hard Mask — thresholded hard edges, no refiner (clean, tight).
HQ Refiner (default) — AI edge matting for hair / fur / soft or translucent edges. Best quality, a little slower.
Default:
HQ Refiner
Refiner Model
Options:
VitMatte (default) — often softer/cleaner edges on hair and soft borders. MEMatte — lighter, can be crisper in places.
Default:
VitMatte
Description:
HQ Refiner only — which AI edge-matting model to use. Both models are warmed up at first launch, so switching is instant; compare them on your own footage.
Refiner Direct Input: Object Mask
Options:
Off · Alpha (default) · Red · Green · Blue — which channel of the Refiner Direct image holds your object mask.
Default:
Alpha
Description:
HQ Refiner only, and only while something is wired into Refiner Direct — otherwise the list is greyed out.
The object mask is your own stand-in for what the AI would have found: a trimap is built from it and it goes through the refiner exactly as a tracked object does. Use it when you already have a matte — from roto, from another key, from anywhere — and you only want its edges re-drawn against the Source.
Off means there is no object mask on this input at all, only an Unknown Zone.
📌 At the defaults everything behaves exactly as it did before. The one case that changed: a mask delivered as three colour channels with no alpha used to be taken from red automatically — now say which channel it is.
Refiner Direct Input: Unknown Zone
Options:
Off (default) · Alpha · Red · Green · Blue — which channel of the same image holds the unknown zone: not an object, but an area where the refiner decides for itself.
Default:
Off
Description:
HQ Refiner only, and only while something is wired into Refiner Direct.
Two systems, one input. The object mask and the unknown zone travel into the same arrow in different channels — paint the mask into red and the zone into green, for instance, and the refiner resolves both in a single pass. 🔴 The two channels must be different: one channel cannot mean "definitely the object" and "unsure here" at the same time. Put both lists on the same channel and the zone switches itself off — the mask is what older projects depend on.
Why the zone exists. The refiner only reworks the narrow trimap band along the mask outline. A strand of hair that reaches past that band sits in "definitely background", and background is never reconsidered — so it is lost however high the quality settings go. Painting over that spot marks it "unsure" instead, which is the one thing that lets the refiner look there at all.
How to use it. Rough is fine — a brush stroke, a roto shape, a matte from anywhere: precision buys you nothing, covering the problem is what matters. Turn on Show Trimap and the area you painted appears in its own colour, distinct from the band the plugin builds by itself.
What the zone needs to work against. The refiner resolves an edge between something definitely foreground and something definitely background, so there has to be a mask for it to work from — either a tracked AI object or your own object mask on this same input. Every combination works: your mask + zone · AI object + zone · AI object + your mask + zone · zone on its own with the object mask list set to Off, as long as an AI object is in the scene. With nothing definite anywhere, the refiner is being asked to settle a border with only one side given, and it answers "unsure" across the whole painted area — a flat, see-through matte rather than hair.
🔑 Paint only the spot where the hair is, and leave part of the head and body unpainted. Whatever provides the mask needs to see what is definitely the person and what is definitely the background — that is how it works out where one ends and the other begins. Paint over everything and there is nothing left to tell apart: you get grey haze instead of hair.
How it differs from Trimap Erode / Dilate. Those sliders widen the unsure band everywhere at once — including where the edge is already clean, paying for it in render time — and they stop at 255 pixels. A painted zone is the same thing applied only where you put it, with no ceiling.
Cost. It scales with the painted area, because the refiner processes it in tiles. Paint the problem, not the whole frame.
⚠️ Very large frames. Carrying two masks instead of one makes each request bigger, so extremely high resolutions — roughly 55 megapixels and up — can run past what a single request holds. The plugin says so plainly in the log rather than dropping the connection.
Unknown Zone Scope
Options:
Override the AI mask (default) — the painted zone wins even where the mask was certain. Outside the AI mask — that verdict is kept and the zone only reaches what was called background.
Default:
Override the AI mask
Description:
Read whenever an Unknown Zone is supplied, wherever the zone came from. It answers one question: what happens where your paint covers the mask that was already found?
Override the AI mask — your paint wins there too, and the refiner has another go at that spot. Use it when the hair came out as one solid blob: separate strands stuck together, the gaps between them filled in. It does the same job as the Trimap Erode slider, but only where you paint instead of all the way round.
Outside the AI mask — whatever the mask was sure about stays as it is, and your paint only works outside it. Use it when the mask itself is fine and hair is simply missing past the edge. Same job as Trimap Dilate, only where you paint.
⚠️ Either way, leave part of the head and body unpainted. Something has to be definitely the person, or there is nothing to work from and the area comes back as grey haze instead of hair. If that happens, the log says so in plain words.
Trimap (HQ Refiner only)
The refiner works only on a narrow band along the edge — the trimap. It splits the area into "definitely object", "definitely background", and an "unsure" working band (where the soft alpha is recomputed from the real video pixels). The band is measured from the mask outline: inward by Erode, outward by Dilate. At the frame border itself the bands are deliberately not built (there is no off-frame data).
Show Trimap
Default:
Off
Description:
Paint the trimap on screen so you can see the real band the AI works on. Green = erode band (inside), cyan = dilate band (outside), white = contour. Turn off for render.
Trimap Opacity
Range / Default:
0.0 – 1.0 / 1.0
Description:
With Show Trimap on: how strongly the coloured bands are painted over the image. Lower it to see more of the real matte underneath. View only.
Trimap Pre-Blur
Default:
Off
Description:
Smooth the mask before building the trimap, to soften the upscale "staircase". Turn on if the edge band looks jagged.
Trimap Erode (px)
Range / Default:
0 – 255 / 5
Description:
How far inside the mask edge is treated as "definitely foreground". Larger = a thinner working band right at the edge (faster, but can clip fine detail). Try 5; raise to ~15 to speed up on big objects.
Trimap Dilate (px)
Range / Default:
0 – 255 / 20
Description:
How far outside the mask edge the AI may grow detail (hair / fur). This is the main speed knob: a wider band = more tiles to process = slower. Try 20; lower to ~8 for speed, raise to ~40 for long wispy hair.
Binarize Threshold
Range / Default:
0 – 255 / 100
Description:
Grayscale level that splits foreground from background. Drives the Binary output and the HQ Refiner input (the trimap is built from it), and the tracked bbox/crop. Higher = tighter foreground. (Lives in the Pre Processing group.)
5. Pre Processing
Shapes the mask the AI works on, in pipeline order: Input Levels (shape the raw soft mask) → Binarize Threshold → binary cleanup. These feed both the Binary output and the refiner. Group is collapsed by default.
Input Levels: Black / White / Gamma
Ranges / Defaults:
Black 0.0–1.0 / 0.0 · White 0.0–1.0 / 1.0 · Gamma 0.1–2.0 / 1.0
Description:
Shape the raw soft mask before binarize (mainly Raw mode; in Binary/Refiner they bias where the threshold cuts). The Output Levels in Post Processing remap the finished matte instead.
Remove Sprinkles (px)
Range / Default:
0 – 1,000,000 (display 0–20,000) / 100
Description:
Drop disconnected islands up to this area in full-resolution pixels. 0 = off; higher removes bigger islands. Applied per-object (not on the whole merged frame), so a small whole object is kept even next to a large one, and a fragment ≥ half the object is always kept.
Fill Holes (px)
Range / Default:
0 – 100,000 (display 0–10,000) / 0 (off)
Description:
Fill enclosed background holes up to this area in pixels (absolute, like SAM3's max hole area).
Binary Edge Smooth
Range / Default:
0 – 50 (display 0–25) / 0 (off)
Description:
Round the upscale "staircase" of the binary edge (majority-voting re-binarize). Radius in pixels; 0 = off.
Mask Shrink/Grow
Range / Default:
−50 to +50 / 0
Description:
Contract (−) or expand (+) the binary mask edge in pixels (erode/dilate), before the refiner.
6. Post Processing
Finishes the final matte after the refiner: mask compositing with an external clip, output levels, guided filter, feather, and offset. Group is collapsed by default.
Mask Composite Mode
Options / Default:
Off, Add to Mask (default), Subtract from Mask, Intersect with Mask, Multiply with Mask
Description:
Combine the external "Mask Composite" input clip with the AI mask. Add = union (grows the mask) — a per-pixel maximum, not an arithmetic sum, so the result never goes above 1.0; Subtract = cut the input out of the AI mask; Intersect = keep only the overlap; Multiply = the two are multiplied together. Example: feed a garbage matte and choose Subtract to carve it out.
This list reads the input the way it always has: the alpha channel if there is one, red otherwise. Three more masks can be read from the same clip — see Mask Composite Mode R / G / B.
Multiply with Mask vs Intersect. They differ on semi-transparent edges: Intersect keeps the darker of the two values and can leave a step, Multiply multiplies them and keeps the edge soft.
Mask Composite Mode R / G / B
Options / Default:
Off (default), Add to Mask, Subtract from Mask, Intersect with Mask, Multiply with Mask — one list per colour channel of the same input clip.
Description:
The Mask Composite input stays a single arrow, but four masks are read from it: the usual one plus the red, green and blue channels, each with its own operation. So one mask can be added while another is subtracted in the same pass — previously every mask had to be merged into one before the node, which forced a single operation on all of them.
🔴 Order is fixed: main → red → green → blue. It only matters when operations of different kinds meet: two "add" give the same result in any order, but "add" together with "intersect" does not. If something must be subtracted after everything else is combined, put it in a later channel.
📌 All three are Off by default, so nothing changes for anyone already using this input.
⚠️ A channel that is not in the incoming clip does nothing — feed a lone alpha and the red, green and blue lists have nothing to read. Turn on Detailed Logging and the log says so.
⚠️ These lists are greyed out while the input is unconnected, and while Freeze and Bake Masks to Disk is on — like every other setting that would make baked frames out of date. Switching one on does make baked frames stale when the mask in that channel changes; that is intended. While the channels are Off, baked frames are left alone entirely.
Output Levels: Black / White / Gamma
Ranges / Defaults:
Black 0.0–1.0 / 0.0 · White 0.0–1.0 / 1.0 · Gamma 0.1–2.0 / 1.0
Description:
Remap the finished matte (after the refiner/cleanup). Separate from Input Levels. Start with small adjustments.
Edge-aware filter guided by the video. Radius 0 = off. Eps is edge sensitivity (must be > 0): smaller = sharper edge preservation; larger = softer.
Feather
Range / Default:
0.0 – 100.0 / 0.0 (off)
Description:
Gaussian-blur the mask edge for softer compositing.
Mask Offset X / Y
Range / Default:
−100 to +100 per axis / 0, 0
Description:
Offset the mask horizontally (X) and vertically (Y). Useful for compensating misalignment or creative offset.
7. Edge Stabilization
Reduces per-frame edge "boiling" with a camera-motion-compensated temporal blend in the edge band of the final mask. Off by default; best on static or slow-camera shots. Group is collapsed by default.
Stabilize Edges Over Time
Default:
Off
Description:
Blend the mask edge with the previous frame, compensated for camera motion. Best on static or slow-camera shots; on fast or close-up movement it can leave a faint trail (ghost). Play forward to build it up — it needs consecutive frames, scrubbing resets it.
Stabilization Strength
Range / Default:
0 – 100 / 60
Description:
How strongly to hold the edge steady. Higher = calmer edges but more risk of a trail on motion. For a locked-off shot push it to 90–100.
Motion Sensitivity
Range / Default:
0 – 100 / 40
Description:
How readily the edge follows movement instead of smoothing it. Low = smooth aggressively (kills shimmer on static shots, but can trail a fast edge); high = follow movement (safe on motion, a little shimmer stays). Static shot: low (~20) + high Strength. Hand-held: ~60–80.
8. Display Mode (on-screen view only)
The top group of the UI — how the node shows the result on screen. This is independent of what Bake writes to disk (that is set by Cache Format). The alpha channel always carries the mask; the modes differ in RGB only. The same four modes are on the Object Manager's toolbar on the canvas (OVERLAY · MATTE · RGB*A · RGB+A).
Mode
Options:
Matte Overlay (default) — footage with the mask tinted on top. On Nuke the alpha carries the mask (a real matte); on DaVinci/Fusion the overlay is kept opaque to avoid a white edge halo.
Matte Only — the mask as a black & white image (alpha = mask).
RGB*A (Premultiplied) — footage cut out by the mask (RGB × alpha) + alpha = mask; the object on black, ready to drop into a comp.
RGB+A (Straight) — full footage (RGB untouched) + alpha = mask, not premultiplied (cleaner edges for compositing / After Effects).
Default:
Matte Overlay
9. Bake Tracking Cache
Compute the masks once, write them to disk, then play them back from disk with no re-inference. With Bake on, opening the project does NOT start the AI engine and uses NO video memory (VRAM) — the node simply plays the ready masks from disk. The engine starts (and uses VRAM) only when you turn Bake off. The cache folder is portable — hand a finished cache to another machine with the project.
Freeze and Bake Masks to Disk (Press Play to Write)
Default:
Off
Workflow:
Set the Cache Folder path below (the first time, a folder picker opens — no silent save).
Enable Bake and press Play — inference runs and every frame is written to disk. Play through the whole timeline at least once for full coverage.
Once cached, playback reads from disk — no inference runs.
Disable Bake to re-render.
When inputs change:
⚠️ Disconnecting any input while Bake is on re-bakes the frames it touched. That is correct — the inputs are part of what was baked, so the result on disk no longer matches — but it does mean losing work you had already written. Disconnect before you bake, not after.
Editing objects or tracking while Bake is on disables it. Changing mask / refiner / doctoring settings (e.g. trimap) re-bakes only the affected frames in place — and pressing Bake again on a folder that already holds masks for a changed scene prompts "Overwrite baked masks?". Cache settings (prefix, format) restart the cache session without disabling Bake.
Limit Bake to Frame Range / Bake From / Bake To
Default:
Off (bakes every frame you play)
Description:
With Limit Bake to Frame Range on, only frames inside [Bake From, Bake To] are baked and shown; frames outside the range pass the source through untouched — no inference runs and nothing is written to disk. Turning it on sets Bake From / Bake To to the scene's init frame (where the object was seeded) — extend the range by hand. Any frame number is valid (the small slider is only a drag helper, not a cap).
Why:
Bake a shot in sections that join on disk: one node bakes frames 1–100; a second node with the same Name Prefix and Cache Folder bakes 101–200; the two halves live side by side in one folder. Read the assembled sequence back with an ordinary Read node.
Cache Format
Options:
PNG — Mask (8-bit, Luma + Alpha) (default) — the mask only, the same mask in Luma and Alpha. Small files, mask delivery.
EXR — Mask (Alpha only) — a single "A" channel, clean alpha with no RGB.
EXR — Premultiplied (RGB * Alpha) — RGBA where RGB is already cut by the mask, plus alpha; written with AdobeAlphaMode = Premultiplied.
EXR — Straight (RGB + Alpha) — clean RGB + separate alpha; AdobeAlphaMode = Straight (Nuke assumes premultiplied by EXR convention — turn the Read node's premultiplied toggle off for Straight).
Notes:
All EXR variants use PIZ compression (lossless), 16-bit half by default. EXR colour space is the host's linear working space, written unchanged (only the alpha form differs per variant). On-screen playback always recomposites the alpha by Display Mode, whatever the format here.
32-bit Float EXR (off = 16-bit half)
Default:
Off (16-bit half)
Description:
EXR is saved as 16-bit half by default — the standard delivery depth and lossless for the mask. Turn on for full 32-bit float (doubles file size, slower); useful only for technical/data passes. Only affects the EXR formats, not PNG. (Greyed out when Cache Format = PNG.)
Name Prefix
Default:
onyx_matte.####
Description:
Prefix for the cache subfolder and the frame filenames. Use #### for frame-number padding (one digit per #): default gives files onyx_matte.0001.png in subfolder onyx_matte/. Use a different prefix per node so caches in a shared folder don't mix.
Cache Folder
Description:
Folder where the cache is saved. Each node saves to <this folder>/<Name Prefix>/. Pick a different folder (or prefix) per node so two caches do not mix. Leave it empty and the first Freeze opens a folder picker.
Open Cache Folder
Description:
Opens the selected Cache Folder in the system file manager.
Open Bake Folder
Description:
Opens the folder where this node saves its baked frames — the
subfolder named after the Name Prefix, not the cache root. Useful for the common finishing
workflow: bake with Freeze on, switch Freeze off, then bring the frames back in with a Read
node. The host file browser opens elsewhere by default, so this button saves hunting for the
path.
Note:
Grayed out until a Cache Folder is set — until then there is nothing to open. The
folder is created on the first press if it does not exist yet.
10. Scene Cleanup & UI Appearance
Reset Scene
Description:
Remove all tracked objects and reset the init frame, so your next object starts a fresh scene on whatever frame you are on (and returns to Locked Init Frame mode). Your processing and appearance settings are kept. The same action is available as a button right on the object panel — the purple circle at the right of the "OBJECTS" title bar.
Reset Track
Description:
Clear the tracking — the AI's computed masks and memory — so you can re-track cleanly from the init frame. Useful when a track went wrong after re-seeding or refining in several places: your objects, seed points and refines are kept, so just go to the init frame and play to re-track from scratch. Unlike Reset Scene, it does not remove objects and does not touch the Bake cache. Grayed out while Bake is on.
Reset Parameters
Description:
Reset all processing and appearance settings (Pre/Post Processing, levels, trimap, crop, output mode, refiner, overlay colours and sizes) back to defaults. Does not delete your objects, change your license, or clear the Bake cache.
UI Appearance (group)
Parameters & defaults:
Pin Object Panel On Screen (Off) — keep the object panel fixed in one screen corner while you zoom or pan (by default it moves with the image). Handy when the panel scrolls out of view as you zoom in. Saved globally on this computer, so it applies to every project. Note: in Nuke the frame-edge line may draw over the panel (a host limitation).
Show Canvas Overlay (On) — show/hide all on-canvas editing overlays (boxes, points, handles, labels, object panel). Status banners stay visible.
Show Labels (On) — show/hide per-object labels and AI CROP badges. Requires Show Canvas Overlay.
Mask Overlay Color — colour and opacity of the mask drawn over the video in Overlay mode. Default (1, 0, 0, 0.5).
Overlay UI Color — colour/opacity of the on-screen guides (bbox frame, handles, cross, dashed crop outline). Default azure (0, 0.652, 1, 1.0).
Canvas Overlay Scale — uniform size of overlay elements. Range 0.25–3.0, default 1.0 (try 1.5 on a 4K monitor).
All three above (Mask Overlay Color / Overlay UI Color / Canvas Overlay Scale) are saved globally on this computer and apply to every project — per-monitor appearance prefs, not project settings, so you set them once and they work everywhere and after a restart. The mask colour is not keyframe-animatable, and "Reset Parameters" does not reset these three. On DaVinci Resolve / Fusion the first open of an older project may revert them to the defaults once.
11. Help and Diagnostics & License
Help and Diagnostics (group)
Open Documentation:
Opens this manual in your web browser. It uses the copy installed with the plugin, so it works with no internet connection; if that copy is missing, onyxofx.com/docs opens instead.
Open Logs Folder:
Opens the ONYX logs folder (onyx3_plugin.log, onyx3_server.log). Send these to support when reporting a problem.
Detailed Logging (default Off):
Writes a detailed diagnostic log. To report a bug: turn this on, re-open the host (so the engine picks it up), reproduce the problem, then Open Logs Folder and send the files.
License (group)
Parameters:
Buy License — opens onyxofx.com to purchase.
License Key — paste your key here, then press Activate.
Deactivate License — release this computer's activation to move the license to another machine.
Copy Hardware ID — copies this computer's Hardware ID to the clipboard. Support may ask for it when looking into a license question.
Status — read-only license status: whether this machine is licensed, running the trial, or needs to be activated.
Note:
The license key is never saved into the project file (security). A 2.5.6 license activates v3 automatically — no re-entry needed.
Proxy (License group)
When you need it:
Only if this computer reaches the internet through a company proxy — common on locked-down workstations with no direct internet. Off by default, so normal machines are unaffected.
Parameters:
Use Proxy (default Off) — the master switch. On: activation is sent through the proxy below. Off: activation connects directly, as usual.
Proxy Server — the address, in the form [user@]host:port. The login is optional and goes before the @. Examples: proxy.company.com:3128 or jdoe@proxy.company.com:8080. Greyed out until Use Proxy is on.
Proxy Password — password for the login above; leave empty if your proxy needs none. Shows as ******** once saved — retype only to change it.
Proxy Status — read-only result of the last save: Disabled, Saved: host:port, or the reason an address was rejected.
Save Proxy — saves and checks the settings above.
Press Save Proxy:
In Nuke, pressing Enter in the address field also commits. On hosts where Enter in a text field does nothing — DaVinci Resolve — the button is the only way to save. Check the Proxy Status line to confirm it took.
Note:
Proxy settings are stored encrypted on this computer, machine-wide, and are never written into the project file. If Use Proxy is on but no address is set, Activate and Deactivate are blocked with a message instead of failing silently.
Quick Reference: Which Tool / Setting for Which Task
Task
Tool / Mode
Key Parameters
Rotoscoping (single object)
Add Bbox Object, Locked Init Frame
Bbox + FG refine points, Auto Fit BBox, Edge Feather
Parameters earlier in the chain affect everything downstream.
Input Color (Apply sRGB LUT) — the input frame converted to sRGB (on by default) + every channel hard-clamped into 0–1 — the picture that goes to the encoder.
Encoder Crop — Crop Mode + padding region sent to SAM3 (Off = whole frame), resized to 1008×1008.
SAM3 Decoder — generates the coarse mask from your prompts (box / point / text / mask).
Memory Tracking — the mask propagates frame-by-frame per the Tracking Mode.
Input Levels — shape the raw soft mask (Black / White / Gamma).
Edge Stabilization — temporal edge blend on the final mask (if enabled).
Display Mode — how it is shown on screen (and, separately, Cache Format for what Bake writes to disk).
System requirements: 16 GB RAM, 8 GB free space. Windows 10/11 (64-bit) or Linux (glibc 2.28+): RTX NVIDIA GPU (RTX 20-series or newer), NVIDIA driver R560+ (CUDA 12.6), 6 GB VRAM. macOS 15 Sequoia or newer: Apple Silicon — M1, M2, M3, M4, M5 or newer — inference runs on Apple's built-in neural engine; no separate GPU required. Intel Macs are not supported. Hosts: Nuke, DaVinci Resolve, Fusion, Fusion Studio Standalone, Flame, Natron. Nuke: 13 or newer on Windows and Linux; 15 or newer on macOS — earlier Nuke versions run on Apple Silicon through Rosetta as Intel applications and cannot load an Apple Silicon plugin.
Unsupported GPU? If your GPU does not meet the above (no NVIDIA CUDA GPU, or a card older than RTX 2060 / pre-Turing), ONYX shows a red "GPU NOT SUPPORTED" banner in the center of the viewer with the detected card name. It is a warning — the node stays usable — so you know why nothing is tracking. No banner on a supported RTX 2060+.