← Back to blog
AiAbout 6 min read

Replace the Person in the Video, Then Put the Voice Back

Published Nov 8, 2026
Replace the Person in the Video, Then Put the Voice Back

A repository called Genjustsu appeared on October 6 with 1,454 stars and a sentence for a title: your scene, new characters.

It is a local UI plus an agent workflow for swapping the subject of a video. Colored depth, SAM 3 masks, and Seedance Edit or Omni through Enhancor. Draft first, approve before 1080p, and the final export restores the entire original source audio rather than anything the model generated.

That last clause is the one worth reading twice.

What the pipeline actually does

The README lists nine steps and they are unusually specific, which makes them checkable rather than aspirational.

Save the source unchanged. Extract the audio and separate vocals from music with Demucs on Replicate. Pitch the isolated vocals up three semitones, holding the duration steady with Rubber Band. Normalize the working video to 24 fps and at most 960 pixels. Get colored depth from `lucataco/depth-anything-video` and keep the colours it returns. Run SAM 3 to produce raw PNG masks over that depth, then check alignment, coverage and background spill. Composite the depth subject back over the original background.

Then the gate. The full clip's mask has to be reviewed visually, and the approval is bound to hashes of the current artifacts. If anything upstream changes, the approval no longer applies.

The composite MP4 carries the pitched vocals. It goes to Seedance with character references and no standalone audio track, which is the part that took a decision: giving a video model a clean isolated voice is giving it something to imitate. Edit handles image references, Omni and multi-reference handle a video identity reference. Draft by default. Poll, download, discard the generated audio, paste the complete original soundtrack back in. When you are happy, approve the 1080p pass and restore the soundtrack again.

The three semitones

The pitch shift is the most interesting unexplained step, and the README does not explain it.

The likely reason is inference rather than stated design, so treat it as such: raising the vocal three semitones while preserving duration produces an audio track that matches the mouth movements without handing over the original voice at its original pitch. Lip sync survives because duration is held. What the model hears is close enough to be plausible and far enough not to be a clean voice sample.

Whatever the intent, the effect on the finished file is unambiguous. The original soundtrack goes back in at the end, complete. The output keeps the source audio and replaces the picture, which inverts the usual trade. Most video tools generate audio to go with generated frames; this one treats audio as the invariant.

That choice has a legal shape as well as a technical one. A voice is a person's, and the workflow is constructed so the model never receives it cleanly and the export never uses a generated replacement.

Hash-bound approval is the real contribution

The mandatory mask review is not unusual. What is unusual is that the approval is bound to artifact hashes.

Read the consequence: an approval is not a state of mind, it is a signature over a specific set of files. Change the mask, change the depth pass, change the composite, and the signature is void. The generation does not proceed on a stale approval.

Anyone who has run a production pipeline with a human review step knows the failure this prevents. The reviewer approves version four. A fix goes in, version five renders, and version five goes out because the queue moved. The hash binding closes that gap by making "approved" refer to bytes rather than to a moment.

Two more details follow the same logic. The draft costs less than the final, so approval happens twice, at 960 pixels and again at 1080p. And the approval is bound to the artifacts rather than to a job id, which means the same job cannot be re-run with new inputs under the old sign-off.

What it takes to run

Python 3.12 or later, Rubber Band CLI, and cloudflared. On macOS, `brew install rubberband cloudflared`. On Ubuntu, Debian or WSL, `sudo apt install rubberband-cli python3-venv` plus cloudflared. Then:

bash setup.sh

That builds an isolated virtualenv, installs the requirements, creates a blank `.env` and checks dependencies. You fill in your own `ENHANCOR_API_KEY` and `REPLICATE_API_TOKEN`. Then `./start.command` opens the UI at `127.0.0.1:8770`, and `doctor.py --require-keys` tells you what is still missing.

The cloudflared dependency is not decorative. Enhancor requires a webhook URL, so the launcher starts a callback-only Quick Tunnel when you have not configured your own. The UI and the media stay on localhost; the tunnel only exists to receive the callback. Native Windows users are told to use WSL2.

No local model weights and no GPU are needed for the default hosted setup. That is the main architectural decision: the heavy lifting is someone else's API, and the repository is the orchestration, the review gates and the audio surgery.

There is also an agent path. Open the folder in a coding agent, point it at `docs/SETUP-AGENT.md` and `.agents/skills/tein-video-workflow/SKILL.md`, and it will detect the OS, install prerequisites, build the environment, run the local tests, start the app and verify readiness. The README includes a ready-made prompt for this, and it ends with an instruction worth noting: do not submit paid generations.

The funnel is visible, and that is fine

The repository is MIT. It also links to a TEIN AI waitlist in the banner and to an Enhancor API page described as the cheapest in the market. The workflow exists in the shape it does partly because the pipeline needs a hosted subject-replacement model, and the author sells access to one.

For a reader, that changes what the project is. You cannot run it entirely offline, so what you get is a well-documented reference implementation of a workflow around a paid API. The documentation quality is the reason to read it whether or not you buy the API.

The part that transfers is the sequence: protect the source, isolate the audio, break the identity link before generation, put a human gate on the mask, bind the approval to files, generate twice, restore the original soundtrack. None of that is specific to Seedance or to Enhancor.

Where this ends up

Subject replacement in video has been technically possible for a while. What has been missing is a production shape that a brand or a studio could defend in a review: a source kept intact, an approval with a hash, an unchanged voice.

The repository is a small open source project with a commercial tail, and it will probably be read mostly by people who never run it. The nine steps are still worth the read, because they describe what accountable video editing looks like when the model is one component among many rather than the product.

Related articles