layers()

Composite multiple media items with precise positioning, overlays, and chroma key effects.

import { compose, layers } from "@synthome/sdk";

const execution = await compose(
  layers([
    { media: "https://example.com/background.mp4" },
    { media: "https://example.com/logo.png", placement: "top-right" },
  ]),
).execute();

Basic Usage

Picture-in-Picture

layers([
  { media: "https://example.com/main.mp4" },
  { media: "https://example.com/webcam.mp4", placement: "pip" },
]);

Logo Overlay

layers([
  { media: "https://example.com/video.mp4" },
  { media: "https://example.com/logo.png", placement: "top-right" },
]);

Centered Overlay

layers([
  { media: "https://example.com/background.mp4" },
  { media: "https://example.com/title.png", placement: "center" },
]);

Placement Presets

Use simple presets for common layouts:

Preset	Description
`"full"`	Fit to frame with letterboxing (black bars if aspect differs)
`"fill"`	Crop to cover entire frame, centered (no black bars)
`"center"`	Centered, original size
`"pip"`	Small overlay in bottom-right
`"picture-in-picture"`	Same as `pip`

full vs fill

The key difference is how they handle aspect ratio mismatches:

full (letterbox): Scales the video to fit inside the frame, adding black bars if needed. The entire video is visible.
fill (crop): Scales the video to cover the entire frame, cropping the edges if needed. No black bars, but some content may be cut off.

// Letterbox - fits 16:9 video into 9:16 frame with black bars
layers([{ media: "https://example.com/horizontal.mp4", placement: "full" }], {
  width: 1080,
  height: 1920,
});

// Crop to fill - 16:9 video fills 9:16 frame, sides cropped
layers([{ media: "https://example.com/horizontal.mp4", placement: "fill" }], {
  width: 1080,
  height: 1920,
});

Picture-in-Picture

layers([
  { media: "https://example.com/main.mp4" },
  { media: "https://example.com/overlay.mp4", placement: "pip" },
]);

Tailwind-Style Placement

Combine size and position using a familiar syntax:

Size Classes

Class	Description
`w-1/2`	50% width
`w-1/3`	33% width
`w-1/4`	25% width
`w-2/3`	66% width
`w-3/4`	75% width
`h-1/2`	50% height
`h-1/3`	33% height
`h-1/4`	25% height

Position Classes

Class	Description
`top-left`	Top-left corner
`top-right`	Top-right corner
`bottom-left`	Bottom-left corner
`bottom-right`	Bottom-right corner
`center`	Centered
`top`	Top center
`bottom`	Bottom center
`left`	Left center
`right`	Right center

Combined Examples

// 1/4 width overlay in bottom-right
layers([
  { media: "https://example.com/main.mp4" },
  { media: "https://example.com/pip.mp4", placement: "w-1/4 bottom-right" },
]);

// Half-width overlay centered
layers([
  { media: "https://example.com/background.mp4" },
  { media: "https://example.com/content.mp4", placement: "w-1/2 center" },
]);

// 1/3 height overlay at top
layers([
  { media: "https://example.com/main.mp4" },
  { media: "https://example.com/banner.png", placement: "h-1/3 top" },
]);

Custom Placement

For precise control, use custom placement objects:

layers([
  { media: "https://example.com/background.mp4" },
  {
    media: "https://example.com/overlay.png",
    placement: {
      width: "300", // Fixed width in pixels
      height: "200", // Fixed height in pixels
      position: {
        x: "50", // 50px from left
        y: "100", // 100px from top
      },
    },
  },
]);

Percentage-Based

placement: {
  width: "50%",
  height: "50%",
  position: {
    x: "25%",
    y: "25%",
  },
}

With Padding

placement: {
  width: "25%",
  position: { x: "100%", y: "100%" }, // Bottom-right
  padding: 20, // 20px padding from edge
}

Chroma Key (Green Screen)

Remove green backgrounds from overlays:

layers([
  { media: "https://example.com/background.mp4" },
  {
    media: "https://example.com/greenscreen-speaker.mp4",
    placement: "w-1/3 bottom-right",
    chromaKey: true,
  },
]);

Custom Chroma Key Settings

layers([
  { media: "https://example.com/background.mp4" },
  {
    media: "https://example.com/bluescreen.mp4",
    chromaKey: true,
    chromaKeyColor: "0000ff", // Blue instead of green
    similarity: 0.3, // Color match threshold
    blend: 0.1, // Edge blending
  },
]);

Dynamic Position Keyframes

Make overlays change position during playback, perfect for speaking head videos that move between positions at sentence boundaries.

Basic Usage

import {
  compose,
  layers,
  generatePositionKeyframes,
  transcribe,
  audioModel,
} from "@synthome/sdk";

// First, transcribe the speaking head video to get word timestamps
const transcript = transcribe({
  video: "https://example.com/speaking-head.mp4",
  model: audioModel("vaibhavs10/incredibly-fast-whisper", "replicate"),
});

// Generate position keyframes from the transcript
const positions = generatePositionKeyframes({
  timestamps: transcript,
  positions: ["w-2/3 bottom-left", "w-2/3 bottom", "w-2/3 bottom-right"],
  minInterval: 2, // Minimum 2 seconds between position changes
});

// Use in layer composition
const execution = await compose(
  layers([
    { media: "https://example.com/background.mp4", placement: "fill" },
    {
      media: "https://example.com/speaking-head.mp4",
      placement: "w-2/3 bottom", // Fallback/initial position
      positionKeyframes: positions, // Dynamic positions
      chromaKey: true,
      chromaKeyColor: "00FF00",
    },
  ]),
).execute();

How It Works

Transcription: Word-level timestamps are extracted from the audio
Sentence Detection: Sentence boundaries are detected (words ending with ., ?, !)
Position Cycling: At each sentence boundary, the overlay switches to the next position in the array
Interval Respect: minInterval prevents rapid position changes for short sentences

Position Keyframes Options

Option	Type	Description
`timestamps`	`TranscribeOperation \| string \| Array`	Word timestamps from transcription
`positions`	`string[]`	Placements to cycle through
`minInterval`	`number`	Minimum seconds between changes (default: 2)

Default Positions

If not specified, the default positions are:

"w-2/3 bottom-left"
"w-2/3 bottom"
"w-2/3 bottom-right"

With URL Timestamps

You can also provide timestamps as a URL to a JSON file:

const positions = generatePositionKeyframes({
  timestamps: "https://example.com/transcript.json",
  positions: ["w-1/2 left", "w-1/2 center", "w-1/2 right"],
});

The JSON file should contain an array of word objects:

[
  { "word": "Hello", "start": 0.0, "end": 0.5 },
  { "word": "world.", "start": 0.5, "end": 1.0 }
]

Duration Control

Main Layer

Mark a layer as the duration reference:

layers([
  { media: "https://example.com/background.mp4", main: true }, // Duration from this
  { media: "https://example.com/logo.png", placement: "top-right" },
]);

Explicit Duration

layers(
  [
    { media: "https://example.com/background.jpg" },
    { media: "https://example.com/overlay.png" },
  ],
  { duration: 10 }, // 10 second output
);

Output Dimensions

layers(
  [
    { media: "https://example.com/video.mp4" },
    { media: "https://example.com/overlay.png" },
  ],
  { width: 1920, height: 1080 },
);

Convert Aspect Ratio (16:9 to 9:16)

Convert horizontal videos to vertical format for social media:

// Crop to fill - no black bars, content centered
layers(
  [
    { media: "https://example.com/horizontal-video.mp4", placement: "fill" },
    {
      media: "https://example.com/overlay.png",
      placement: {
        width: "iw", // Full width of output
        position: { x: "(W-w)/2", y: "0" }, // Top center
      },
    },
  ],
  { width: 1080, height: 1920 }, // 9:16 vertical output
);

Timeline Arrays

Create sequential content within a layer:

layers([
  { media: "https://example.com/background.mp4", main: true },
  [
    // Timeline: plays sequentially on this layer
    { media: "https://example.com/title1.png", duration: 3 },
    { media: "https://example.com/title2.png", duration: 3 },
    { media: "https://example.com/title3.png", duration: 3 },
  ],
]);

With Generated Content

Generated Background

layers([
  {
    media: generateVideo({
      model: videoModel("bytedance/seedance-1-pro", "replicate"),
      prompt: "Abstract flowing colors",
    }),
  },
  { media: "https://example.com/logo.png", placement: "center" },
]);

Generated Overlay

layers([
  { media: "https://example.com/video.mp4" },
  {
    media: generateImage({
      model: imageModel("google/nano-banana", "replicate"),
      prompt: "Watermark logo",
    }),
    placement: "w-1/6 bottom-right",
  },
]);

Multiple Generated Layers

layers([
  {
    media: generateVideo({
      model: videoModel("minimax/video-01", "replicate"),
      prompt: "Ocean waves",
    }),
    main: true,
  },
  {
    media: generateVideo({
      model: videoModel("bytedance/seedance-1-pro", "replicate"),
      prompt: "Person speaking on green screen",
    }),
    placement: "w-1/3 bottom-right",
    chromaKey: true,
  },
]);

API Reference

layers(items, options?)

Parameter	Type	Description
`items`	`Array<LayerItem \| TimelineItem[]>`	Array of layers
`options`	`LayersOptions`	Optional configuration

LayerItem

Property	Type	Description
`media`	`string \| VideoOperation \| ImageOperation`	Media URL or generated content
`placement`	`PlacementPreset \| CustomPlacement`	Position and size
`positionKeyframes`	`PositionKeyframesOperation \| PositionKeyframe[] \| string`	Dynamic position changes over time
`chromaKey`	`boolean`	Enable green screen removal
`chromaKeyColor`	`string`	Hex color to remove (default: green)
`similarity`	`number`	Color match threshold
`blend`	`number`	Edge blending amount
`main`	`boolean`	Use this layer's duration

LayersOptions

Property	Type	Description
`duration`	`number`	Explicit output duration
`width`	`number`	Output width in pixels
`height`	`number`	Output height in pixels
`mainLayer`	`number`	Layer index to use for duration

CustomPlacement

Property	Type	Description
`width`	`string`	Width (pixels or percentage)
`height`	`string`	Height (pixels or percentage)
`position`	`{ x?: string, y?: string }`	Position coordinates
`padding`	`number`	Padding in pixels
`aspectRatio`	`string`	Aspect ratio (e.g., "16:9")

generatePositionKeyframes(options)

Parameter	Type	Description
`timestamps`	`TranscribeOperation \| string \| Array`	Word timestamps source
`positions`	`string[]`	Placement strings to cycle through
`minInterval`	`number`	Minimum seconds between changes (default: 2)

PositionKeyframe

Property	Type	Description
`time`	`number`	Time in seconds
`placement`	`string`	Placement string for this time

Layers

On this page