Media Rendering

Render & Audio Queues #

The engine provides specialized queuing utilities to coordinate resource-intensive media operations. Both queues run as in-memory data structures inside the local host process and do not orchestrate tasks across multiple remote host environments.

Audio Provider Job Queue #

Created via createAudioProviderJobQueue({ registry }), this queue coordinates voice generation and audio processing tasks.

• Priority & Concurrency: Tasks are prioritized as either interactive or batch. Higher priority tasks are placed at the front of the wait list but do not preempt or abort already running tasks. Concurrency and limits can be controlled using the capacityByGroup option.
• Timeout: Unlike render jobs, the timeout for an audio job starts ticking the moment the job is accepted into the queue.
• Idempotency & Caching: In-flight requests are deduplicated by cache key, and successful outputs are cached in an in-memory registry to avoid redundant operations.
• Readiness Polling: Prior to execution, the queue checks the destination provider's readiness status. If the provider is not ready, the job is put back into the queue for a retry after a fixed delay. Callers should await the job status using the wait(jobId) API, rather than assuming that the queue's drain() method will make all pending readiness-waiting jobs terminal.

Render Provider Job Queue #

Created via createRenderProviderJobQueue({ registry }), this queue schedules browser capture and frame processing jobs.

• Concurrency & Progress: Supports global concurrency parameters, stage monitoring events, cancel/abort triggers, and cleanups.
• Timeout Behavior: The timeout countdown starts after dequeue (meaning when the job moves from the queue into active execution). If the job times out, it is marked as failed with the timeout: true flag.

Local Audio Providers #

Audio providers are host-configured HTTP clients that interface with external speech-to-text or text-to-speech services using an injected fetch client and local artifact storage.

Screencast & Browser Capture #

Visual frame rendering leverages Chromium via Puppeteer to capture web-based templates. Puppeteer is required injection; execFile is optional because the provider defaults to Node's implementation.

• Deterministic Rendering: To prevent animation timing drift, the capture provider can invoke a host-managed rendering clock on the target page, manually stepping through frames rather than relying on real-time browser execution.
• Frame Partitioning: The partitionRenderFrameRanges(frameCount, workerCount) utility is a mathematical helper that divides a total frame count into ranges (e.g. startFrame, endFrame) to support parallel processing. This is a helper function and does not initialize node threads, OS processes, or distributed executors. The normalizeRenderJob helper normalizes/projects the required id, kind, and providerId fields while preserving other job fields; it does not normalize host stages/phases or perform capture.
• Sandbox & Seams: Passing --no-sandbox to Chromium is a launch preference and does not constitute a security boundary. Parallel deterministic capture requires leader-exported canonical setup state before boundary seam checks. Boundary checks do not prove every frame.

Finalize & Proof Verification #

After capturing frames and generating audio tracks, helper utilities are pure argument, projection, and report helpers. They do not merge, encode, inspect media bytes, run FFmpeg/ffprobe, or produce cryptographic attestations.

FFmpeg & ffprobe Arguments Compiler #

Utilities compile arguments for external media tools:

• buildFrameSequenceEncodeArgs: Compiles the options needed to stitch frame files and overlay caption tracks into a video.
• buildAudioConcatArgs: Compiles arguments to concatenate segments together.
• buildAudioOverlapMixArgs: Builds the command flags and filter complexes required to mix and overlap audio clips.
• buildAudioMuxArgs: Compiles arguments to merge video and audio files into a final container.
• parseFfprobeJson: Parses the JSON output of a probe command. Malformed or invalid JSON input will cause the parser to throw a custom E_FFPROBE_JSON error.

Synchronous Proof Verification Reports #

The buildRenderAudioLayerProof and buildRenderAvSyncProof functions compare caller-supplied metadata and stream timing details to generate validation reports. They are not cryptographic attestations or binary-level provenance verifications.