Smelter TypeScript SDK
Smelter is a video/audio composition framework using React components to define scenes. You write React JSX describing the layout; Smelter renders it as actual video frames.
Core Concept
// Define a scene as React JSX
function MyScene() {
return (
<View style={{ width: 1920, height: 1080 }}>
<InputStream inputId="input_1" />
</View>
);
}
// Wire it to actual media
await smelter.registerOutput("main", <MyScene />, { type: "rtmp_client", url: "..." });Runtime Packages — Choose One
| Package | Use when | Details |
|---|---|---|
@swmansion/smelter-node | Server-side Node.js app | Auto-spawns Smelter binary; live + offline modes |
@swmansion/smelter-web-client | Browser app, external server | Connects to deployed Smelter server; live + offline modes |
@swmansion/smelter-web-wasm | Browser app, no server | Runs Smelter as WASM in Web Worker; Chrome only |
→ For detailed setup and API for each runtime: references/runtimes/nodejs.md, references/runtimes/web-client.md, references/runtimes/web-wasm.md
Components
Import from @swmansion/smelter:
import { View, Text, InputStream, Tiles, Rescaler, Image, Mp4, Shader, Show, SlideShow, WebView } from "@swmansion/smelter";Layout Components
| Component | Summary | When to use |
|---|---|---|
| View | Core container, like <div> | Structure any layout; supports absolute and static positioning, overflow, background color |
| Tiles | A layout component that arranges all children side by side in equally sized, non-overlapping tiles, automatically calculating optimal rows/columns. | Multi-stream grids (e.g., video conferencing layout, side by side layouts). Use as default for multiple inputs if user does not specify a layout explicitly. |
| Rescaler | Scales single child to fit, preserving aspect ratio | Fit any stream/content into a fixed area |
→ references/components/View.md, references/components/Tiles.md, references/components/Rescaler.md
Media Components
| Component | Summary | When to use |
|---|---|---|
| InputStream | Displays a registered input stream | Show any registered input (camera, RTP, RTMP, WHIP, etc.) |
| Mp4 | Plays MP4 file directly (no registration needed) | Simple one-off MP4 playback without registration overhead |
| Image | Renders an image (URL or registered asset) | Static images, logos, overlays |
| Shader | Renders output of a WGSL GPU shader | Custom visual effects, chroma key, color grading |
| WebView | Renders a live website via Chromium | Embedding web-based graphics or interactive content |
→ references/components/InputStream.md, references/components/Mp4.md, references/components/Image.md, references/components/Shader.md, references/components/WebView.md
Utility Components
| Component | Summary | When to use |
|---|---|---|
| Text | Renders styled text | Whenever stream needs to display text, specifically for lower thirds, captions, labels, titles, etc. |
| Show | Conditionally shows children based on timestamp | Scheduling elements in offline processing |
| SlideShow | Sequences <Slide> children one after another | Intro/outro sequences, sequential content |
→ references/components/Text.md, references/components/Show.md, references/components/SlideShow.md
Props (Styling)
| Props Type | Used by | Summary |
|---|---|---|
| ViewStyleProps | <View> | Width/height, direction (row/column), absolute positioning, overflow, background, padding |
| TextStyleProps | <Text> | fontSize (required), font family/weight/style, color, alignment, wrapping |
| TilesStyleProps | <Tiles> | Width/height, tile aspect ratio, margin, padding, alignment |
| RescalerStyleProps | <Rescaler> | Rescaling mode (fit/fill), alignment, absolute positioning |
| Transition | <View>, <Tiles>, <Rescaler> | Animated scene updates with duration and easing |
| EasingFunction | Transition | "linear", "bounce", or custom cubic_bezier |
→ references/props/ViewStyleProps.md, references/props/TextStyleProps.md, references/props/TilesStyleProps.md, references/props/RescalerStyleProps.md, references/props/Transition.md, references/props/EasingFunction.md
Hooks
| Hook | Summary | When to use |
|---|---|---|
| useInputStreams() | Returns state of all registered inputs | Conditionally render based on stream status (ready/playing/finished) - TODO: Change |
| useAudioInput(id, opts) | Controls audio for an input without rendering it visually | Background audio mixing without visual component |
| useAfterTimestamp(ms) | Returns true once a timestamp passes | Time-based scene changes in offline processing |
| useBlockingTask(fn) | Runs async fn, blocks offline rendering until resolved | Load remote data before offline rendering proceeds |
→ references/hooks/useInputStreams.md, references/hooks/useAudioInput.md, references/hooks/useAfterTimestamp.md, references/hooks/useBlockingTask.md
Inputs
Registered via smelter.registerInput(id, options). Displayed via <InputStream inputId="id" />.
| Input type | Runtime | Use when |
|---|---|---|
mp4 | All | Play a local/remote MP4 file |
rtp_stream | Node.js, Web Client | Receive RTP stream over UDP/TCP |
hls | Node.js | Consume HLS playlist |
whip_server | Node.js, Web Client | Accept WebRTC stream via WHIP protocol |
whep_client | Node.js, Web Client | Pull stream from WHEP server |
rtmp_server | Node.js, Web Client | Accept RTMP stream (OBS, FFmpeg) — experimental |
camera | WASM only | Browser camera via getUserMedia() |
screen_capture | WASM only | Browser screen capture via getDisplayMedia() |
stream | WASM only | Any MediaStream object |
whep_client (WASM) | WASM only | Pull from WHEP server in browser |
→ Full details: references/inputs.md
Outputs
Registered via smelter.registerOutput(id, <ReactRoot />, options).
| Output type | Runtime | Use when |
|---|---|---|
mp4 | Node.js, Web Client | Save to MP4 file |
rtp_stream | Node.js, Web Client | Stream over RTP |
hls | Node.js, Web Client | Stream over HLS. User will need to handle serving the files on their own. |
whip_client | Node.js, Web Client | Push via WebRTC WHIP |
whep_server | Node.js, Web Client | Serve via WebRTC WHEP to multiple viewers |
rtmp_client | Node.js, Web Client | Push to RTMP server (YouTube, Twitch) |
canvas | WASM only | Render to HTMLCanvasElement |
stream | WASM only | Return a MediaStream |
whip_client (WASM) | WASM only | Push via WebRTC WHIP from browser |
→ Full details: references/outputs.md
Resources
Pre-registered assets used by components.
| Resource | Registered via | Used by component |
|---|---|---|
| Image | smelter.registerImage(id, opts) | <Image imageId="id" /> |
| Shader | smelter.registerShader(id, opts) | <Shader shaderId="id" /> |
| WebRenderer | smelter.registerWebRenderer(id, opts) | <WebView instanceId="id" /> |
| Font | smelter.registerFont(source) | <Text> components |
→ Full details: references/resources.md
Patterns & Best Practices
Real-world patterns for building Smelter apps:
| Pattern | Summary |
|---|---|
| Shader Wrapping | Recursively nest <Shader> components to chain multiple effects |
| Input State Rendering | Use useInputStreams() for spinner/offline/playing conditional rendering |
| Animations via Timers | setInterval + React state for swap transitions, marquee, fades |
| Shader Color Params | Convert hex colors to per-channel f32 fields (_r, _g, _b) |
| Multi-Output Shared Store | Same store for WHEP live + MP4 recording = identical output |
| Scrolling Text | Animate <View> position inside overflow: 'hidden' container |
→ Full details and code examples: references/patterns.md