diff --git a/AGENTS.md b/AGENTS.md index 5aa7e70..a348046 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -218,9 +218,9 @@ Narration for this scene. [#marker] A marker names a moment you can animate on. - **Show it when it is said.** A picture appears as the voice mentions it. Use word cues. -- **Something changes every 2 to 4 seconds.** A new element, an arrow, a - highlight or a camera move. `check` points out long stretches where nothing - moves. +- **Something changes every 2 to 4 seconds.** A new element, an arrow or a + highlight. The camera adds its own moves. `check` points out long stretches + where nothing new appears. - **Short screen text.** Labels, numbers and key phrases of a few words. The whole narration never goes on screen. When the narration should be readable, turn on captions (`"captions": true`). Then do not also show the sentence @@ -235,6 +235,54 @@ Narration for this scene. [#marker] A marker names a moment you can animate on. picture of the real screen is better than three boxes with general words. - **A clear last frame** that shows the main point. +### The camera + +explainroo films each scene with a camera that follows the narration. It +opens on the picture. When a scene starts small, the camera starts close and +widens as new things need room. Something new that already fits the shot +needs no move. Now and then the camera moves in on what the voice talks +about, for example a new card and the one it relates to, when the viewer has +time to look. Then it follows new things at that distance. Near the end of +the scene it pulls back, so the last frame shows the whole idea. During each +scene it also pushes in very slowly. + +While a shot holds, every card and label is either fully in the shot or out +of it. Arrows and lines may run off the edge. While the camera moves from +one shot to the next, things may pass the edge of the frame for a moment. +Shots stay clear of the captions and of the parts the app covers on Shorts, +TikTok and Reels. + +Lay out each scene as one picture on the full frame, the way it should look +at the end, and let the camera find the close-ups: + +- Put the next element near the one it follows, so one shot shows both. +- A scene that starts with one thing and grows gets the most camera moves. + The camera starts close and widens with the picture. +- Keep a box with its label, or an icon with its caption, together as one + unit. Things far apart get their own shots. +- Lines, arrows and things added inside an element already on screen, like + a line in a chart, get no close-up of their own. Use `s.focus` when one + should. +- When a scene goes on with the same picture, draw the carried elements with + `at: -1` in the same places. The camera then goes on from where it + stopped, and the two scenes fade into each other. + +The camera measures every element, including what `s.draw` draws, and +follows the `at` and `out` times. Anything drawn straight on `s.ctx` counts +as one block that is there for the whole scene, so the camera keeps it whole +and cannot film it on its own. Use `s.draw` for a drawing that appears on a +cue. Highlighted code lines and terminal commands get close-ups of their own +when they are small in the shot. `s.focus` points the camera at any part of +the picture. + +`s.camera([...])` in a scene takes over that scene's camera, and +`s.camera([])` keeps the scene still. `"camera": "still"` in `video.json` +turns the camera off for the whole video. `check` checks the layout as you +wrote it, without the camera. Stills, sheets and renders show the camera. + +The final render adds motion blur to frames where something moves, like a +film camera. Drafts skip it. `"motionBlur": false` turns it off. + ## Voices Kokoro runs on this computer and needs no key. Pick a voice with @@ -741,6 +789,8 @@ videos// | `captions` | `"auto"` | text of the narration at the bottom; auto turns it on for vertical and square videos | | `cover` | `"auto"` | full-size JPEG, without captions, at the end of the first scene, or a still spec: `12.5`, `"intro"`, `"intro@2.4"`, `"intro@end"` | | `transition` | `"auto"` | the look's default, or `fade`, `slide`, `wipe`, `zoom`, `brush`, `cut` | +| `camera` | `"auto"` | `"auto"` films each scene with a camera that follows the narration; `"still"` keeps every frame as written | +| `motionBlur` | `true` | motion blur on moving frames in the final render | | `lead`, `hold`, `end` | 0.35, 0.7, 1.4 | seconds before the voice, after it, and extra at the very end | | `sentenceGap`, `paragraphGap` | 0.3, 0.55 | pauses in generated narration; recordings keep their own pauses | | `loudness` | -14 | target loudness in LUFS | @@ -1172,8 +1222,14 @@ or `{ out, at, color }`. Options: `w`, `size`, `rows`, `title`, `prompt`, ### Camera, groups and your own drawing -- `s.camera([{ at, x, y, zoom, dur, rotate }])` moves the view. Call it first - in the scene function. +- `s.camera([{ at, x, y, zoom, dur, rotate }])` moves the view by hand + instead of the automatic camera (see The camera). Call it first in the + scene function. `s.camera([])` keeps the scene still. +- `s.focus(target, { at, out })` points the automatic camera at an element + or a region on a cue, for example one row of a table. `target` is an `id` + drawn earlier, `{ x, y, w, h }` (center and size) or `[x, y, w, h]`. It + draws nothing. Highlighted code lines and terminal commands get a + close-up on their own when they are small in the shot. - `s.group({ x, y, scale, rotate, at, out, enter }, () => { ... })` moves a set of elements together. Inside the group, positions count from `x`, `y`. - `s.draw({ x, y, at, out }, (ctx, life) => { ... })` gives you the canvas 2D diff --git a/README.md b/README.md index bc44e24..a0fa376 100644 --- a/README.md +++ b/README.md @@ -63,6 +63,11 @@ explainroo does the rest: icons from [Lucide](https://lucide.dev). A scene can also show charts, math formulas, code that changes step by step, your own screenshots and video clips such as screen recordings. +- **Camera.** A camera films each scene. It moves in on what the voice talks + about when there is time, and pulls back at the end so the last frame shows + the whole idea. Cards and labels are never cut in half while a shot holds. + Things arrive with a little spring, and the final video has motion blur, + like a film camera. - **Sound.** explainroo makes its own background music for each video and adds small sound effects. The music gets quieter while the voice speaks. - **File.** ffmpeg puts it all together into an MP4, with subtitle files, diff --git a/engine/director.js b/engine/director.js new file mode 100644 index 0000000..9ffb16b --- /dev/null +++ b/engine/director.js @@ -0,0 +1,458 @@ +// The automatic camera. Before anything is drawn, explainroo measures each +// element of a scene once, and this file plans the camera from that. The +// camera opens on the picture and widens when something new would not fit. +// Something new that already fits needs no move. Now and then the camera +// moves in on a new thing and then follows new things at that distance. It +// moves in again only five seconds after the last change of zoom. Near the +// end of the scene it shows the whole picture. During each scene it also +// pushes in very slowly. Scenes that call s.camera() keep their own camera. +import { clamp, lerp } from './util.js'; + +// Things that appear within BEAT_GAP seconds of each other are one beat +// and get one shot. +const BEAT_GAP = 0.8; +const FILL = 0.72; // share of the view a close-up's subject fills +const PICTURE_FILL = 0.82; // share of the view the whole picture fills +const MAX_ZOOM = 2.2; +const LEAD = 0.3; // the camera starts moving a moment before the element appears +const MIN_HOLD = 2.2; // shortest time between the starts of two moves +const SETTLE = 0.08; // smaller changes than this are not worth a move +const DRIFT = 0.035; // how much closer the slow push gets over a whole scene +// Moving in on something: the new shot is at least PUSH times closer, the +// viewer gets PUSH_HOLD seconds to look once the camera has arrived, and the +// zoom last changed, in or out, PUSH_GAP or more seconds ago. +const PUSH = 1.3; +const PUSH_HOLD = 1.5; +const PUSH_GAP = 5; +// A highlighted code line or terminal command gets a close-up when it is at +// least this much closer. +const FOCUS_GAIN = 1.25; +// Kinds that are a background when they cover most of the frame. +const BACKDROPS = ['shape', 'custom', 'image', 'clip', 'panel']; + +const smoother = (p) => p * p * p * (p * (p * 6 - 15) + 10); + +function union(boxes) { + let x0 = Infinity; + let y0 = Infinity; + let x1 = -Infinity; + let y1 = -Infinity; + for (const b of boxes) { + x0 = Math.min(x0, b.x - b.w / 2); + y0 = Math.min(y0, b.y - b.h / 2); + x1 = Math.max(x1, b.x + b.w / 2); + y1 = Math.max(y1, b.y + b.h / 2); + } + return { x: (x0 + x1) / 2, y: (y0 + y1) / 2, w: x1 - x0, h: y1 - y0 }; +} + +// view: the part of the frame the subject should fill (the safe area), +// frame: { W, H }. Returns { x, y, z }: the scene point shown at the view's +// center and the zoom. +export function frameBox(box, view, frame, maxZoom = MAX_ZOOM, fill = FILL) { + const z = clamp(Math.min((view.w * fill) / Math.max(1, box.w), (view.h * fill) / Math.max(1, box.h)), 1, maxZoom); + return clampShot({ x: box.x, y: box.y, z }, view, frame); +} + +// Keeps the visible part of the scene inside the frame, so backgrounds a +// scene paints itself never show their edges. +function clampShot(shot, view, frame) { + const cx = view.x + view.w / 2; + const cy = view.y + view.h / 2; + const z = shot.z; + return { + x: clamp(shot.x, cx / z, frame.W - (frame.W - cx) / z), + y: clamp(shot.y, cy / z, frame.H - (frame.H - cy) / z), + z, + }; +} + +// The part of the scene the whole frame shows. +export function frameRect(shot, view, frame) { + const cx = view.x + view.w / 2; + const cy = view.y + view.h / 2; + return { x0: shot.x - cx / shot.z, y0: shot.y - cy / shot.z, x1: shot.x + (frame.W - cx) / shot.z, y1: shot.y + (frame.H - cy) / shot.z }; +} + +export function viewRect(shot, view) { + const cx = view.x + view.w / 2; + const cy = view.y + view.h / 2; + return { x0: shot.x - (cx - view.x) / shot.z, y0: shot.y - (cy - view.y) / shot.z, x1: shot.x + (view.x + view.w - cx) / shot.z, y1: shot.y + (view.y + view.h - cy) / shot.z }; +} + +// Whether a box is partly in view. Edges within `margin` of the view's edge +// count as cut too, so nothing touches the edge of the frame. A box that +// pokes in by less than `slack` counts as out. +// `r` is the whole frame; `use` (default r) the part captions and app +// overlays leave free, which a visible box has to sit in. +function cut(box, r, margin, slack = 0, use = r) { + const x0 = box.x - box.w / 2; + const x1 = box.x + box.w / 2; + const y0 = box.y - box.h / 2; + const y1 = box.y + box.h / 2; + const outside = x1 <= r.x0 + slack || x0 >= r.x1 - slack || y1 <= r.y0 + slack || y0 >= r.y1 - slack; + const inside = x0 >= use.x0 + margin && x1 <= use.x1 - margin && y0 >= use.y0 + margin && y1 <= use.y1 - margin; + return !inside && !outside; +} + +// The free part of the frame (frame pixels) as scene coordinates for a shot. +function usableRect(shot, view, usable) { + const cx = view.x + view.w / 2; + const cy = view.y + view.h / 2; + return { x0: shot.x + (usable.x0 - cx) / shot.z, y0: shot.y + (usable.y0 - cy) / shot.z, x1: shot.x + (usable.x1 - cx) / shot.z, y1: shot.y + (usable.y1 - cy) / shot.z }; +} + +// Whether a shot shows `box` (when given) whole and cuts none of `others`, +// at its own zoom and at the tightest framing the slow drift reaches. +function clean(shot, box, others, view, frame, usable) { + const z = shot.z; + const margin = (Math.min(frame.W, frame.H) * 0.07) / z; + const tight = clampShot({ ...shot, z: z * (1 + DRIFT) }, view, frame); + if (box) { + const v = viewRect(tight, view); + if (box.x - box.w / 2 < v.x0 - 0.5 || box.x + box.w / 2 > v.x1 + 0.5 || box.y - box.h / 2 < v.y0 - 0.5 || box.y + box.h / 2 > v.y1 + 0.5) return false; + } + const r = frameRect(tight, view, frame); + const wide = frameRect(shot, view, frame); + return !others.some((e) => cut(e.box, r, margin, 1 / z, usableRect(tight, view, usable)) || cut(e.box, wide, margin, 1 / z, usableRect(shot, view, usable))); +} + +// Frames `focus` as close as it can while every other visible element is +// either fully in the shot or fully out of it. Half a label at the edge +// looks like a mistake. It tries a few zoom levels. For each it tries the +// center on the subject and the centers that put a neighbour right inside +// or right outside the frame's edge, and keeps the closest, best centered +// shot. At zoom 1 the shot is the frame as it was written, which always +// works. +export function compose(focus, visible, view, frame, usable = { x0: 0, y0: 0, x1: frame.W, y1: frame.H }, fill = FILL, maxZoom = MAX_ZOOM) { + const box = union(focus.map((e) => e.box)); + // Arrows and lines may run off the edge of a close-up, and so may an + // element the shot looks into, like a code window around its highlighted + // lines. A cut label or card may not. + const looking = focus.every((e) => e.kind === 'focus'); + const others = visible.filter((e) => !focus.includes(e) && !['arrow', 'line', 'focus'].includes(e.kind) && !(looking && contains(e.box, box))); + const cx = view.x + view.w / 2; + const cy = view.y + view.h / 2; + const best0 = frameBox(box, view, frame, Math.max(1, maxZoom), fill); + let best = null; + for (let z = best0.z; z > 1.001; z /= 1.06) { + const margin = (Math.min(frame.W, frame.H) * 0.07) / z; + // Centers that keep the subject inside the view. + const xr = [box.x + box.w / 2 + margin - (view.x + view.w - cx) / z, box.x - box.w / 2 - margin + (cx - view.x) / z]; + const yr = [box.y + box.h / 2 + margin - (view.y + view.h - cy) / z, box.y - box.h / 2 - margin + (cy - view.y) / z]; + if (xr[0] > xr[1] || yr[0] > yr[1]) continue; + const pad = 3 / z; + const xs = [box.x]; + const ys = [box.y]; + for (const e of others) { + const l = e.box.x - e.box.w / 2; + const r = e.box.x + e.box.w / 2; + const t = e.box.y - e.box.h / 2; + const b = e.box.y + e.box.h / 2; + xs.push(r + pad + cx / z, l - pad - (frame.W - cx) / z, l - margin + cx / z, r + margin - (frame.W - cx) / z); + ys.push(b + pad + cy / z, t - pad - (frame.H - cy) / z, t - margin + cy / z, b + margin - (frame.H - cy) / z); + } + for (const x0 of xs) { + if (x0 < xr[0] - 1e-6 || x0 > xr[1] + 1e-6) continue; + for (const y0 of ys) { + if (y0 < yr[0] - 1e-6 || y0 > yr[1] + 1e-6) continue; + const shot = clampShot({ x: x0, y: y0, z }, view, frame); + if (!clean(shot, box, others, view, frame, usable)) continue; + const off = Math.hypot((shot.x - box.x) * z / view.w, (shot.y - box.y) * z / view.h); + const score = Math.log(z) - 0.8 * off; + if (!best || score > best.score) best = { shot, score }; + } + } + } + if (best) return best.shot; + // Otherwise the frame as it was written. Its slow push stays off when it + // would move something into the captions or the app's buttons. + const full = frameBox(box, view, frame, 1); + const tight = clampShot({ ...full, z: 1 + DRIFT }, view, frame); + const within = (e, shot) => { + const u = usableRect(shot, view, usable); + return e.box.x - e.box.w / 2 >= u.x0 && e.box.x + e.box.w / 2 <= u.x1 && e.box.y - e.box.h / 2 >= u.y0 && e.box.y + e.box.h / 2 <= u.y1; + }; + const held = [...focus, ...others].filter((e) => !['arrow', 'line', 'focus'].includes(e.kind)); + return held.some((e) => within(e, full) && !within(e, tight)) ? { ...full, drift: 0 } : full; +} + +function contains(outer, inner) { + return outer.x - outer.w / 2 <= inner.x - inner.w / 2 + 0.5 && outer.x + outer.w / 2 >= inner.x + inner.w / 2 - 0.5 && outer.y - outer.h / 2 <= inner.y - inner.h / 2 + 0.5 && outer.y + outer.h / 2 >= inner.y + inner.h / 2 - 0.5; +} + +function same(a, b) { + return Math.abs(a.x - b.x) < 0.01 && Math.abs(a.y - b.y) < 0.01 && Math.abs(a.z - b.z) < 1e-4 && (a.drift ?? 1) === (b.drift ?? 1); +} + +// How long a move takes at pace 1: longer for a bigger change of zoom or +// place, and a little longer for the pull-back at the end of a scene. +function moveLength(from, to, kind, frame) { + const distance = Math.hypot(to.x - from.x, to.y - from.y) / (frame.W / Math.min(from.z, to.z)); + return clamp(0.9 + 0.5 * Math.abs(Math.log(to.z / from.z)) + 0.6 * distance, 0.95, 1.7) * (kind === 'reveal' ? 1.2 : 1); +} + +function differs(a, b, frame) { + return (a.drift ?? 1) !== (b.drift ?? 1) || Math.abs(Math.log(a.z / b.z)) > SETTLE || Math.hypot(a.x - b.x, a.y - b.y) > (SETTLE * frame.W) / Math.max(a.z, b.z); +} + +// elements: [{ at, out, box }] in scene coordinates. Returns null when there +// is nothing to film. +// `from`: where the camera is when the scene starts, for a scene that +// continues the one before it. +export function planCamera(elements, { dur, view, frame, pace = 1, voiceEnd = null, from = null, still = 0, usable = null }) { + usable ??= { x0: 0, y0: 0, x1: frame.W, y1: frame.H }; + const area = frame.W * frame.H; + const items = elements + // A plain shape, drawing or picture that covers most of the frame is a + // background. Charts, code, text and what a scene draws by hand ("ink") + // count however large they are. + .filter((e) => e.box && (!BACKDROPS.includes(e.kind) || e.box.w * e.box.h < area * 0.6) && (e.box.w > 2 || e.box.h > 2)) + .map((e) => ({ ...e, at: Math.max(0, e.at), out: e.out ?? Infinity })) + .filter((e) => e.at < dur - 0.3) + .sort((a, b) => a.at - b.at); + if (!items.length) return null; + + const beats = []; + for (const e of items) { + const last = beats[beats.length - 1]; + if (last && e.at - last.start <= BEAT_GAP) last.els.push(e); + else beats.push({ start: e.at, els: [e] }); + } + + const hold = MIN_HOLD / pace; + const visibleDuring = (t0, t1) => items.filter((e) => e.at <= t1 && e.out > t0 + 0.25); + const picture = (list) => list.filter((e) => e.kind !== 'focus'); + const solid = (list) => list.filter((e) => !['arrow', 'line', 'focus'].includes(e.kind)); + // Everything on screen from t0 to t1, framed as one picture. + const overview = (t0, t1) => { + const v = picture(visibleDuring(t0, t1)); + return v.length ? compose(v, v, view, frame, usable, PICTURE_FILL) : null; + }; + // Whether a shot already shows these elements whole, in the free part of + // the frame, also at the end of its slow push. A beat of only arrows and + // lines fits when they are at least partly in view. + const fits = (shot, list) => { + const body = list.filter((e) => e.kind !== 'arrow' && e.kind !== 'line'); + const tight = shot.drift === 0 ? shot : clampShot({ ...shot, z: shot.z * (1 + DRIFT) }, view, frame); + const pad = (Math.min(frame.W, frame.H) * 0.02) / shot.z; + return [shot, tight].every((s) => { + const u = usableRect(s, view, usable); + if (!body.length) return list.every((e) => e.box.x + e.box.w / 2 > u.x0 && e.box.x - e.box.w / 2 < u.x1 && e.box.y + e.box.h / 2 > u.y0 && e.box.y - e.box.h / 2 < u.y1); + return body.every((e) => e.box.x - e.box.w / 2 >= u.x0 + pad && e.box.x + e.box.w / 2 <= u.x1 - pad && e.box.y - e.box.h / 2 >= u.y0 + pad && e.box.y + e.box.h / 2 <= u.y1 - pad); + }); + }; + // The one or two elements in `before` closest to `els`. + const nearest = (els, before) => { + if (!before.length) return []; + const c = union(els.map((e) => e.box)); + const d = (e) => Math.hypot(e.box.x - c.x, e.box.y - c.y); + const sorted = [...before].sort((a, b) => d(a) - d(b)); + return sorted.slice(0, sorted.length > 1 && d(sorted[1]) < d(sorted[0]) * 1.4 ? 2 : 1); + }; + + // Near the end the camera shows the whole picture. + const lastBeat = beats[beats.length - 1].start; + const revealAt = Math.max(lastBeat + 1.8, (voiceEnd ?? lastBeat) - 0.9); + // When the next new thing or focus point after beat i would not fit the + // shot. + const heldUntil = (shot, i) => { + for (let j = i + 1; j < beats.length; j++) { + const next = beats[j].els.filter((e) => e.out > beats[j].start + 0.4); + if (next.length && !fits(shot, next)) return Math.max(first, beats[j].start - LEAD); + } + return dur; + }; + + // `still`: the camera holds while the scene dissolves in from the one it + // continues, so both pictures stay on top of each other. + const first = Math.max(0, still); + const shots = []; + let cur = from; + let last = from ? first - hold : -Infinity; + let zoomed = -Infinity; + // Takes a shot from `begin` on. The first one opens the scene. + // A required move keeps something new whole, however small it is. + const take = (shot, begin, kind, required = false) => { + if (!shot) return false; + if (cur && (required ? same(cur, shot) : !differs(cur, shot, frame))) return false; + if (cur && Math.abs(Math.log(shot.z / cur.z)) > 0.1) zoomed = begin; + shots.push({ begin: cur ? begin : 0, shot, kind, required }); + // The opening is not a move: the first move may follow it sooner. + last = cur ? begin : begin - hold * 0.4; + cur = shot; + return true; + }; + for (let i = 0; i < beats.length; i++) { + const beat = beats[i]; + const begin = Math.max(first, beat.start - LEAD); + const next = i + 1 < beats.length ? Math.max(first, beats[i + 1].start - LEAD) : dur; + const free = !cur || begin >= last + hold - 1e-9; + // Whether the camera may move closer now. + const closer = !cur || (next - begin >= (PUSH_HOLD + 1) / pace && begin - zoomed >= PUSH_GAP / pace); + const points = beat.els.filter((e) => e.kind === 'focus'); + const fresh = picture(beat.els).filter((e) => e.out > beat.start + 0.4); + + // Highlighted code lines, terminal commands and s.focus get a close-up + // when the shot does not already show them big and there is time. + if (points.length && free) { + const explicit = points.some((p) => p.explicit); + const ends = Math.min(...points.map((p) => p.out)); + const close = compose(points, visibleDuring(begin, Math.min(ends, begin + hold * 2 + LEAD)), view, frame, usable, FILL, explicit || closer ? MAX_ZOOM : cur.z); + // The close-up holds until it ends or something new would not fit it. + const until = Math.min(ends, heldUntil(close, i)); + const gain = cur ? close.z / cur.z : Infinity; + if (until - begin >= hold && ((cur && !fits(cur, points)) || gain >= (explicit ? 1.05 : FOCUS_GAIN))) { + if (take(close, begin, 'focus', Boolean(cur && !fits(cur, points)))) continue; + } + } + if (!fresh.length) continue; + const window = begin + hold + LEAD; + if (!cur) { + take(overview(begin, window), begin, 'picture'); + continue; + } + if (!free && !fits(cur, fresh)) { + // Too soon after the last move for something that does not fit: the + // last shot takes in the whole picture instead. + if (shots.length) { + const prev = shots[shots.length - 1]; + const wide = overview(prev.begin, window); + if (wide) Object.assign(prev, { shot: wide, kind: 'picture' }); + cur = prev.shot; + } + continue; + } + const before = items.filter((e) => e.at < beat.start - 0.05 && e.out > beat.start + 0.3 && e.kind !== 'focus' && e.kind !== 'ink'); + if (fits(cur, fresh)) { + // It is already in the shot. The camera moves in only on a new card, + // label or picture. A line, an arrow, or something drawn inside an + // element that is already there, like a line in a chart, gets no + // close-up. It tries the new thing with its neighbours first, then on + // its own. + const own = solid(fresh).filter((e) => !before.some((f) => f.box.w * f.box.h >= e.box.w * e.box.h * 2 && contains(f.box, e.box))); + // It is on screen already, so the push may wait a moment for its turn. + const at = Math.max(begin, last + hold); + if (!own.length || at > beat.start + 1 / pace || at - zoomed < PUSH_GAP / pace) continue; + // What comes soon after belongs to the same shot when it can. + const soon = beats.slice(i + 1).filter((b) => b.start - beat.start <= PUSH_HOLD / pace).flatMap((b) => solid(picture(b.els))); + const near = nearest(own, before); + const seen = visibleDuring(at, at + PUSH_HOLD / pace + hold); + for (const list of [[...near, ...own, ...soon], [...near, ...own], [...own, ...soon], own]) { + const shot = compose(list, seen, view, frame, usable); + const until = Math.min(heldUntil(shot, i), fits(shot, picture(visibleDuring(revealAt, revealAt + 0.1))) ? dur : revealAt); + if (shot.z >= cur.z * PUSH && until - at - moveLength(cur, shot, 'push', frame) / pace >= PUSH_HOLD / pace) { + take(shot, at, 'push'); + break; + } + } + continue; + } + // It does not fit: the new thing with what it relates to, at most as + // close as the camera may go now. + const visible = visibleDuring(begin, Math.max(next, window)); + const limit = closer ? MAX_ZOOM : cur.z; + let subject = compose([...nearest(fresh, before), ...fresh], visible, view, frame, usable, FILL, limit); + if (subject.z < Math.min(1.15, limit)) subject = compose(fresh, visible, view, frame, usable, FILL, limit); + take(fits(subject, fresh) ? subject : overview(begin, window), begin, 'follow', true); + } + if (!shots.length && !from) return null; + // Near the end the camera shows the whole picture when the shot leaves + // part of it out. + const reveal = Math.max(revealAt, last + hold); + if (cur && reveal < dur - 1.1) { + const all = picture(visibleDuring(reveal, reveal + 0.1)); + if (all.length && !fits(cur, all)) take(compose(all, visibleDuring(reveal, dur), view, frame, usable, PICTURE_FILL), reveal, 'reveal'); + } + + const start = from ?? shots[0].shot; + const moves = []; + let current = start; + for (const { begin, shot, kind, required } of from ? shots : shots.slice(1)) { + if (required ? same(current, shot) : !differs(current, shot, frame)) continue; + // Every move ends before the scene does. + const room = dur - 0.05 - begin; + if (room < 0.45 / pace) continue; + const length = Math.min(room, moveLength(current, shot, kind, frame) / pace); + moves.push({ start: begin, dur: length, to: shot, kind }); + current = shot; + } + // Each move starts from wherever the camera is at that moment, so moves + // that overlap stay smooth. + const plan = { start, moves, dur, view, frame, still, elements: items }; + for (const m of moves) { + m.from = cameraAt(plan, m.start, false); + // Between shots far apart the camera pulls back on the way, so less is + // cut off while it travels. + const a = frameRect(m.from, view, frame); + const b = frameRect(m.to, view, frame); + const shared = Math.max(0, Math.min(a.x1, b.x1) - Math.max(a.x0, b.x0)) * Math.max(0, Math.min(a.y1, b.y1) - Math.max(a.y0, b.y0)); + if (shared < 0.25 * Math.min((a.x1 - a.x0) * (a.y1 - a.y0), (b.x1 - b.x0) * (b.y1 - b.y0))) { + const both = { x: (Math.min(a.x0, b.x0) + Math.max(a.x1, b.x1)) / 2, y: (Math.min(a.y0, b.y0) + Math.max(a.y1, b.y1)) / 2, w: Math.max(a.x1, b.x1) - Math.min(a.x0, b.x0), h: Math.max(a.y1, b.y1) - Math.min(a.y0, b.y0) }; + const via = frameBox(both, view, frame, Math.min(m.from.z, m.to.z)); + if (via.z < Math.min(m.from.z, m.to.z) * 0.97) m.via = via; + } + } + return plan; +} + +// `drift` on a shot (0 or 1, default 1) says whether the slow push may run +// there; moves blend it. +export function cameraAt(plan, t, drift = true) { + let cam = plan.start; + for (const m of plan.moves) { + if (t < m.start || !m.from) break; + const p = smoother(clamp((t - m.start) / m.dur)); + const d = lerp(m.from.drift ?? 1, m.to.drift ?? 1, p); + if (m.via) { + // A curve through the wider shot in the middle. + const q = (a, b, c) => (1 - p) * (1 - p) * a + 2 * (1 - p) * p * b + p * p * c; + const mid = (k) => 2 * m.via[k] - (m.from[k] + m.to[k]) / 2; + cam = { + x: q(m.from.x, mid('x'), m.to.x), + y: q(m.from.y, mid('y'), m.to.y), + z: Math.max(1, Math.exp(q(Math.log(m.from.z), 2 * Math.log(m.via.z) - (Math.log(m.from.z) + Math.log(m.to.z)) / 2, Math.log(m.to.z)))), + }; + cam = { ...clampShot(cam, plan.view, plan.frame), drift: d }; + } else { + cam = { + x: lerp(m.from.x, m.to.x, p), + y: lerp(m.from.y, m.to.y, p), + z: Math.exp(lerp(Math.log(m.from.z), Math.log(m.to.z), p)), + drift: d, + }; + } + } + if (!drift) return cam; + // The slow push starts once a continued scene has dissolved in. + const still = plan.still || 0; + const push = 1 + DRIFT * (cam.drift ?? 1) * clamp((t - still) / Math.max(1, plan.dur - still)); + return { ...clampShot({ x: cam.x, y: cam.y, z: cam.z * push }, plan.view, plan.frame), drift: cam.drift ?? 1 }; +} + +// True while the camera is in a move (not counting the slow drift). +export function cameraMoving(plan, t) { + return plan.moves.some((m) => t > m.start && t < m.start + m.dur); +} + +// Whether scene B picks up where scene A left off: what A's last frame +// shows is mostly still there when B starts, in the same places. `last` is +// the part of the scene A's camera shows at the end. +export function continues(a, b, last) { + const inView = (e) => e.box.x + e.box.w / 2 > last.x0 && e.box.x - e.box.w / 2 < last.x1 && e.box.y + e.box.h / 2 > last.y0 && e.box.y - e.box.h / 2 < last.y1; + const end = a.filter((e) => e.out === Infinity && inView(e)); + const carried = b.filter((e) => e.at <= 0.05); + if (end.length < 2 || carried.length < 2) return false; + const same = (x, y) => x.kind === y.kind && Math.abs(x.box.x - y.box.x) < 2 && Math.abs(x.box.y - y.box.y) < 2 && Math.abs(x.box.w - y.box.w) < 2 && Math.abs(x.box.h - y.box.h) < 2; + const kept = end.filter((e) => carried.some((f) => same(e, f))); + const known = carried.filter((e) => a.some((f) => same(e, f))); + return kept.length >= end.length * 0.7 && known.length >= carried.length * 0.6; +} + +// Where the background is, for the camera at `cam`: it follows at a third of +// the camera's strength, as if it sat further back. +export function backgroundShot(plan, cam, depth = 0.33) { + const cx = plan.view.x + plan.view.w / 2; + const cy = plan.view.y + plan.view.h / 2; + return clampShot({ x: cx + (cam.x - cx) * depth, y: cy + (cam.y - cy) * depth, z: Math.pow(cam.z, depth) }, plan.view, plan.frame); +} diff --git a/engine/frame.html b/engine/frame.html index 7856d15..34d3f44 100644 --- a/engine/frame.html +++ b/engine/frame.html @@ -24,16 +24,50 @@ window.explainrooReady = (async () => { const { engine, audio, state } = await boot({ canvas, scale, transparent }); const fps = state.config.fps; + const sub = document.createElement('canvas'); + sub.width = canvas.width; + sub.height = canvas.height; + const subCtx = sub.getContext('2d', { alpha: false }); + // Draws the frame `samples` times over half a frame's time around T and + // averages them, like a film camera's shutter. The samples stay inside + // the frame's scene, so a cut never blends with the scene before it. Each + // sample loads its own clip frames, so the result does not depend on what + // was drawn before. + const blurFrame = async (T, samples) => { + const ctx = canvas.getContext('2d'); + const sc = state.timeline.scenes[engine.sceneAt(T)]; + const lo = sc.start; + const hi = Math.min(sc.start + sc.dur, state.timeline.duration) - 1e-4; + for (let k = 0; k < samples; k++) { + const t = Math.min(hi, Math.max(lo, T + ((k + 0.5) / samples - 0.5) * (0.5 / fps))); + if (engine.clips.size) await engine.prepareFrame(t, subCtx); + engine.renderFrame(t, subCtx); + ctx.save(); + ctx.setTransform(1, 0, 0, 1, 0, 0); + ctx.globalAlpha = k === 0 ? 1 : 1 / (k + 1); + ctx.drawImage(sub, 0, 0); + ctx.restore(); + } + }; window.explainroo = { info() { return { duration: state.timeline.duration, frames: state.timeline.frames, fps, width: canvas.width, height: canvas.height, scenes: state.timeline.scenes.map((s) => ({ id: s.id, start: s.start, dur: s.dur })) }; }, - async frames(list, quality) { + // The automatic camera's plan for each scene it films. + cameras() { + return Object.fromEntries([...engine.cameraPlans].map(([i, p]) => [state.timeline.scenes[i].id, { start: p.start, moves: p.moves.map(({ start, dur, from, to }) => ({ start, dur, from, to })), elements: p.elements, view: p.view }])); + }, + // samples > 1 adds motion blur: a frame in which something moves is + // drawn that many times across half a frame's time and averaged, like a + // film camera's shutter. Still frames are drawn once. + async frames(list, quality, samples = 1) { const out = []; for (const f of list) { try { if (engine.clips.size) await engine.prepareFrame(f / fps); + engine.moving = false; engine.renderFrame(f / fps); + if (samples > 1 && engine.moving && !transparent) await blurFrame(f / fps, samples); } catch (e) { throw new Error(describe(e)); } diff --git a/engine/runtime.js b/engine/runtime.js index ba1f3fa..bbd969a 100644 --- a/engine/runtime.js +++ b/engine/runtime.js @@ -1,7 +1,7 @@ // Boots the engine in a page: loads the project state, fonts, icons, images // and scenes, then exposes frame rendering, checks and the soundtrack. import rough from '../vendor/rough.js'; -import { Stage, SceneError } from './stage.js'; +import { Stage, SceneError, boundsRecorder } from './stage.js'; import { getTheme, makeBackground, fontString, colorOf } from './themes.js'; import { Pen } from './pen.js'; import { drawTransition } from './transitions.js'; @@ -11,6 +11,7 @@ import { layoutAreas } from './layout.js'; import { loadClips, ClipFrames } from './clips.js'; import { UI } from './ui.js'; import { playbackTextIssue } from './readability.js'; +import { planCamera, cameraAt, cameraMoving, continues, frameRect, backgroundShot } from './director.js'; Stage.UI = UI; @@ -160,11 +161,14 @@ export class Engine { return this.clipFrames.nearest(clip, frame); } - async prepareFrame(T) { + // Loads the clip frames that time T shows. It finds them by drawing the + // frame on `ctx`. Motion blur passes a spare canvas here, so the picture it + // is adding up on the main canvas stays as it is. + async prepareFrame(T, ctx = this.ctx) { if (!this.clips.size) return; const needed = new Map(); this.clipProbe = needed; - try { this.renderFrame(T); } + try { this.renderFrame(T, ctx); } finally { this.clipProbe = null; } await Promise.all([...needed.values()].map(({ clip, frame }) => this.clipFrames.load(clip, frame))); this.clipFrames.trim(new Set(needed.keys())); @@ -224,6 +228,9 @@ export class Engine { } transitionFor(sc) { + // A scene that goes on with the picture before it fades into it. A wipe + // or a slide would break that picture apart. + if (sc.transition === 'auto' && this.continued?.has(sc.index)) return 'fade'; const name = sc.transition === 'auto' ? this.theme.transition : sc.transition; if (this.transparent && ['slide', 'wipe', 'brush'].includes(name)) return 'fade'; return name || 'fade'; @@ -238,14 +245,59 @@ export class Engine { ctx.setTransform(this.scale, 0, 0, this.scale, 0, 0); ctx.globalAlpha = 1; ctx.globalCompositeOperation = 'source-over'; - if (this.background) ctx.drawImage(this.background, 0, 0, this.W, this.H); + const plan = !this.planning && !this.cameraOff ? this.cameraPlans?.get(index) : null; + // A scene drawn past its end, while the next one comes in, keeps the + // camera where it stopped, so a continued picture stays on top of itself. + const cam = plan ? cameraAt(plan, Math.min(t, plan.dur)) : null; + if (this.background) { + if (plan) { + // The background moves a third as much as the camera, so it looks + // further away. + const bg = backgroundShot(plan, cam); + const v = plan.view; + ctx.save(); + ctx.translate(v.x + v.w / 2, v.y + v.h / 2); + ctx.scale(bg.z, bg.z); + ctx.translate(-bg.x, -bg.y); + ctx.drawImage(this.background, 0, 0, this.W, this.H); + ctx.restore(); + } else ctx.drawImage(this.background, 0, 0, this.W, this.H); + } if (this.config.boil > 0) this.pen.seedOffset = Math.floor((sc.start + t) * this.config.boil) % 997; const saved = Math.random; Math.random = mulberry32(hashStr(`random:${sc.id}`)); - const s = new Stage(this, ctx, sc, t); ctx.save(); + // The automatic camera films the scene unless the scene moves its own. + if (plan) { + const v = plan.view; + ctx.translate(v.x + v.w / 2, v.y + v.h / 2); + ctx.scale(cam.z, cam.z); + ctx.translate(-cam.x, -cam.y); + if (cameraMoving(plan, t)) this.moving = true; + } + const s = new Stage(this, ctx, sc, t); + // While the camera is planned, anything a scene draws straight on s.ctx + // is measured as one block that the camera must not cut. + let raw = null; + let api = s; + if (this.planning && this.layoutLog) { + raw = boundsRecorder(ctx, this.scale, { W: this.W, H: this.H }); + // The UI kit draws on the same recorder; its panels and headlines + // still report their own times. + let ui = null; + api = new Proxy(s, { + get: (target, prop) => { + if (prop === 'ctx') return raw.proxy; + if (prop === 'ui') return (ui ??= new Stage.UI(api)); + const v = target[prop]; + return typeof v === 'function' ? v.bind(target) : v; + }, + }); + } try { - this.scenesModule[sc.id](s); + this.scenesModule[sc.id](api); + const ink = raw?.box(); + if (ink) this.layoutLog.push({ kind: 'ink', at: 0, out: Infinity, box: ink }); } catch (e) { const err = e instanceof Error ? e : new Error(String(e)); err.scene = sc.id; @@ -258,6 +310,51 @@ export class Engine { return s; } + // Measures every scene once and plans its camera. A scene that continues + // the picture of the scene before it starts where that camera stopped, + // and the two dissolve into each other. Scene errors are left to check + // and render, which report them with their time. + planCameras() { + this.cameraPlans = new Map(); + this.continued = new Set(); + if (this.config.camera === 'still') return; + const scratch = document.createElement('canvas'); + scratch.width = 16; + scratch.height = 16; + const ctx = scratch.getContext('2d'); + const view = this.safeArea; + const frame = { W: this.W, H: this.H }; + // The part of the frame that captions and app overlays leave free. + const usable = { x0: 0, y0: 0, x1: this.W, y1: this.H }; + if (this.appArea) Object.assign(usable, { x0: this.appArea.left, y0: this.appArea.top, x1: this.appArea.right, y1: this.appArea.bottom }); + if (this.captionBand) usable.y1 = Math.min(usable.y1, this.captionBand.top); + const layouts = this.timeline.scenes.map((sc) => { + this.layoutLog = []; + this.planning = true; + this.manualCamera = false; + try { + this.drawScene(ctx, sc.index, Math.max(0, sc.dur - 0.01)); + return this.manualCamera ? null : this.layoutLog.map((e) => ({ ...e, out: e.out ?? Infinity })); + } catch { + return null; + } finally { + this.layoutLog = null; + this.planning = false; + } + }); + this.timeline.scenes.forEach((sc, i) => { + if (!layouts[i]) return; + const prev = i > 0 ? this.cameraPlans.get(i - 1) : null; + const linked = prev && layouts[i - 1] && sc.transition !== 'none' && continues(layouts[i - 1], layouts[i], frameRect(cameraAt(prev, prev.dur), view, frame)); + const voiceEnd = sc.voice ? sc.lead + sc.voice.dur : null; + const still = linked ? Math.min(this.timeline.transitionSeconds, sc.dur * 0.5) : 0; + const plan = planCamera(layouts[i], { dur: sc.dur, view, frame, pace: this.timeline.pace || 1, voiceEnd, from: linked ? cameraAt(prev, prev.dur) : null, still, usable }); + if (!plan) return; + this.cameraPlans.set(i, plan); + if (linked) this.continued.add(i); + }); + } + renderFrame(T, ctx = this.ctx, { captions = true } = {}) { const i = this.sceneAt(T); const sc = this.timeline.scenes[i]; @@ -271,6 +368,7 @@ export class Engine { ctx.setTransform(1, 0, 0, 1, 0, 0); ctx.clearRect(0, 0, ctx.canvas.width, ctx.canvas.height); } + this.moving = true; this.drawScene(a.getContext('2d', { alpha: this.transparent }), i - 1, T - prev.start); this.drawScene(b.getContext('2d', { alpha: this.transparent }), i, t); drawTransition(trName, ctx, a, b, t / trDur, this.W, this.H, hashStr(sc.id), this.transparent); @@ -398,6 +496,18 @@ export class Engine { // Samples every scene and reports layout problems. check(step = 0.25, viewWidth = null) { + // check looks at the layout as written. The automatic camera only ever + // moves in, which makes text larger, and it may leave the edges out of + // a close-up on purpose. + this.cameraOff = true; + try { + return this._check(step, viewWidth); + } finally { + this.cameraOff = false; + } + } + + _check(step, viewWidth) { const issues = []; const add = (level, scene, t, message) => issues.push({ level, scene, t: t === null ? null : Math.round(t * 100) / 100, message }); if (this.config.look) { @@ -557,6 +667,7 @@ export async function boot({ canvas, scale = 1, transparent = false, version = ' engine.iconTags = iconTags; if (player) engine.staticMath = await (await fetch('./math.json')).json(); else await engine.loadMath(); + engine.planCameras(); return { engine, audio, state }; } diff --git a/engine/stage.js b/engine/stage.js index 0c21c7d..e2dafc4 100644 --- a/engine/stage.js +++ b/engine/stage.js @@ -4,12 +4,12 @@ // the theme decides how they enter, and entering elements add sound effects. import { clamp, lerp, ease, easeFn, rand01, noise1, hashStr, mulberry32, suggest, normWord, withAlpha, contrast, mixColor, opaque } from './util.js'; import { colorOf, tintOf, fontString } from './themes.js'; -import { scalePath, quadD, polyD, pathLength, pointAt } from './pathdata.js'; +import { scalePath, quadD, polyD, pathLength, pointAt, parsePath, absolutize } from './pathdata.js'; import { layoutText, lineStartX } from './text.js'; import { clipFrame, clipURL } from './clips.js'; import { highlightLine, codeTokens, matchTokens, resolveCodeSteps, codeMorphState } from './code.js'; -const DEFAULT_DUR = { draw: 0.9, write: 0.8, pop: 0.45, rise: 0.55, fade: 0.5, zoom: 0.6, drop: 0.55, grow: 1.0, none: 0.001, 'slide-left': 0.55, 'slide-right': 0.55, 'slide-up': 0.55, 'slide-down': 0.55 }; +const DEFAULT_DUR = { draw: 0.9, write: 0.8, pop: 0.6, rise: 0.75, fade: 0.6, zoom: 0.75, drop: 0.75, grow: 1.0, none: 0.001, 'slide-left': 0.75, 'slide-right': 0.75, 'slide-up': 0.75, 'slide-down': 0.75 }; const ENTERS = ['draw', 'write', 'type', 'words', 'sync', 'pop', 'rise', 'fade', 'zoom', 'drop', 'grow', 'none', 'slide-left', 'slide-right', 'slide-up', 'slide-down']; const EXITS = ['fade', 'pop', 'rise', 'drop', 'slide-left', 'slide-right', 'none']; const BOLD = { display: undefined, body: 600, hand: undefined, mono: 700 }; @@ -220,6 +220,19 @@ export class Stage { const base = o.dur ?? extra.dur ?? DEFAULT_DUR[enter] ?? 0.6; const dur = extra.real ? base : base / this.pace; this._record(kind, at, dur, o, extra); + if (this.engine.layoutLog && this._pendingBox) { + // The element's own scale and rotation, and parts drawn beside it + // (an icon's label), count for the camera. + let box = this._pendingBox; + const sc = o.scale ?? 1; + const rad = ((o.rotate ?? 0) * Math.PI) / 180; + if (sc !== 1 || rad) box = { ...box, w: (Math.abs(box.w * Math.cos(rad)) + Math.abs(box.h * Math.sin(rad))) * sc, h: (Math.abs(box.w * Math.sin(rad)) + Math.abs(box.h * Math.cos(rad))) * sc }; + // _pendingExtra is already measured with the element's transform. + if (this._pendingExtra) box = unionBox(box, this._pendingExtra); + this._attach(kind, at, out, box); + this._pendingBox = null; + this._pendingExtra = null; + } if (this._suppress > 0) return null; if (this.t < at) return null; let q = 0; @@ -227,7 +240,9 @@ export class Stage { q = clamp((this.t - out) / ((o.outDur ?? 0.45) / this.pace)); if (q >= 1 || o.exit === 'none') return null; } - return { p: clamp((this.t - at) / dur), q, at, dur, enter, exit: o.exit ?? 'fade', local: this.t - at }; + const p = clamp((this.t - at) / dur); + if ((p < 1 && enter !== 'none') || (q > 0 && q < 1)) this.engine.moving = true; + return { p, q, at, dur, enter, exit: o.exit ?? 'fade', local: this.t - at }; } _record(kind, at, dur, o, extra = {}) { @@ -239,37 +254,53 @@ export class Stage { rec.push({ kind, at, dur, sfx: sfx || null, gain: o.sfxGain ?? extra.gain ?? 1, pitch: extra.pitch, sustain: extra.sustain, key: `${kind}|${Math.round(at * 1000)}|${o.id ?? extra.label ?? ''}|${extra.pitch ?? ''}` }); } + // Entrances move on a spring: they go a little past their place and settle + // back. They fade in quickly. + // `lift` is 1 while an element is arriving and 0 once it has landed; + // shadows use it to rise and settle. _enterTransform(life) { const p = life.p; const E = ease.out(p); + const S = ease.spring(p); + const A = ease.out(clamp(p * 1.8)); let a = 1; let s = 1; let dx = 0; let dy = 0; + let lift = 0; switch (life.enter) { - case 'fade': a = E; break; - case 'pop': s = lerp(0.55, 1, ease.outBack(p)); a = clamp(p * 3); break; - case 'rise': dy = (1 - E) * 36; a = E; break; - case 'drop': dy = -(1 - ease.outBack(p)) * 48; a = clamp(p * 2.5); break; - case 'zoom': s = lerp(1.35, 1, E); a = E; break; - case 'slide-left': dx = (1 - E) * 150; a = E; break; - case 'slide-right': dx = -(1 - E) * 150; a = E; break; - case 'slide-up': dy = (1 - E) * 150; a = E; break; - case 'slide-down': dy = -(1 - E) * 150; a = E; break; + case 'fade': a = E; s = lerp(0.975, 1, S); break; + case 'pop': s = lerp(0.6, 1, S); a = clamp(p * 3); lift = 1 - E; break; + case 'rise': dy = (1 - S) * 44; s = lerp(0.965, 1, S); a = A; lift = 1 - E; break; + case 'drop': dy = -(1 - S) * 60; a = clamp(p * 2.5); lift = 1 - E; break; + case 'zoom': s = lerp(1.3, 1, S); a = A; break; + case 'slide-left': dx = (1 - S) * 170; a = A; break; + case 'slide-right': dx = -(1 - S) * 170; a = A; break; + case 'slide-up': dy = (1 - S) * 170; a = A; break; + case 'slide-down': dy = -(1 - S) * 170; a = A; break; default: break; } if (life.q > 0) { const Q = ease.in(life.q); switch (life.exit) { case 'pop': s *= 1 - 0.45 * Q; a *= 1 - Q; break; - case 'rise': dy -= Q * 40; a *= 1 - Q; break; + case 'rise': dy -= Q * 40; s *= 1 - 0.03 * Q; a *= 1 - Q; break; case 'drop': dy += Q * 60; a *= 1 - Q; break; case 'slide-left': dx -= Q * 170; a *= 1 - Q; break; case 'slide-right': dx += Q * 170; a *= 1 - Q; break; - default: a *= 1 - Q; + default: s *= 1 - 0.03 * Q; a *= 1 - Q; } } - return { a, s, dx, dy }; + return { a, s, dx, dy, lift }; + } + + // The look's shadow (or `fallback`), raised while the element drawn last + // is still arriving. + _shadow(tr = this._lastTr, fallback = null) { + const sh = this.theme.shadow || fallback; + if (!sh || !tr || !tr.lift) return sh; + const k = tr.lift; + return { ...sh, blur: sh.blur * (1 + 1.4 * k), y: (sh.y || 0) * (1 + 2 * k) }; } _push(x, y, life, o) { @@ -288,6 +319,7 @@ export class Stage { const sc = (o.scale ?? 1) * tr.s; if (sc !== 1) ctx.scale(sc, sc); ctx.globalAlpha *= tr.a * (o.opacity ?? 1); + this._lastTr = tr; return tr; } @@ -311,9 +343,48 @@ export class Stage { const ys = pts.map((p) => p.y); const box = { x: (Math.min(...xs) + Math.max(...xs)) / 2, y: (Math.min(...ys) + Math.max(...ys)) / 2, w: Math.max(...xs) - Math.min(...xs), h: Math.max(...ys) - Math.min(...ys), shape }; if (id !== undefined && id !== null) this._ids.set(String(id), box); + // While the camera is planned, the next _life call gives this box its + // timing. + if (this.engine.layoutLog) this._pendingBox = box; return box; } + // Records an element's place and timing for the automatic camera. x, y is + // the center in local coordinates. + _logLayout(kind, at, out, x, y, w, h) { + if (!this.engine.layoutLog) return; + this._attach(kind, at, out, this._designBox(x, y, w, h)); + } + + // A box inside an element drawn at (ax, ay), in scene coordinates, with + // the element's own scale and rotation, as _push draws it. + _elementBox(o, ax, ay, x, y, w, h) { + const sc = o.scale ?? 1; + const rad = ((o.rotate ?? 0) * Math.PI) / 180; + const m = this._matrix().translate(ax, ay).rotate((rad * 180) / Math.PI).scale(sc, sc); + const pts = [[x - w / 2, y - h / 2], [x + w / 2, y - h / 2], [x + w / 2, y + h / 2], [x - w / 2, y + h / 2]].map(([px, py]) => m.transformPoint(new DOMPoint(px - ax, py - ay))); + const xs = pts.map((p) => p.x); + const ys = pts.map((p) => p.y); + return { x: (Math.min(...xs) + Math.max(...xs)) / 2, y: (Math.min(...ys) + Math.max(...ys)) / 2, w: Math.max(...xs) - Math.min(...xs), h: Math.max(...ys) - Math.min(...ys) }; + } + + _designBox(x, y, w, h) { + const m = this._matrix(); + const pts = [[x - w / 2, y - h / 2], [x + w / 2, y - h / 2], [x + w / 2, y + h / 2], [x - w / 2, y + h / 2]].map(([px, py]) => m.transformPoint(new DOMPoint(px, py))); + const xs = pts.map((p) => p.x); + const ys = pts.map((p) => p.y); + return { x: (Math.min(...xs) + Math.max(...xs)) / 2, y: (Math.min(...ys) + Math.max(...ys)) / 2, w: Math.max(...xs) - Math.min(...xs), h: Math.max(...ys) - Math.min(...ys) }; + } + + // An element inside groups is only on screen while every group is. + _attach(kind, at, out, box, extra = null) { + for (const g of this._spans || []) { + at = Math.max(at, g.at); + out = Math.min(out, g.out); + } + if (out > at) this.engine.layoutLog.push({ kind, at, out, box, ...extra }); + } + _fromDesign(box) { const inv = this._matrix().inverse(); const c = inv.transformPoint(new DOMPoint(box.x, box.y)); @@ -408,13 +479,18 @@ export class Stage { // keys: [{ at, x, y, zoom, dur }]. Call before drawing anything else. camera(keys, o = {}) { + // A scene that moves its own camera gets no automatic camera. + if (this.engine.planning) this.engine.manualCamera = true; const list = [{ at: 0, x: this.cx, y: this.cy, zoom: 1, dur: 0 }, ...keys.map((k) => ({ x: this.cx, y: this.cy, zoom: 1, dur: 1.1, ...k, at: this.time(k.at ?? 0) }))].sort((a, b) => a.at - b.at); let cur = { ...list[0] }; + if (o.shake) this.engine.moving = true; for (let i = 1; i < list.length; i++) { const k = list[i]; if (this.t < k.at) break; const prev = cur; - const p = easeFn(k.ease ?? 'inOut')(clamp((this.t - k.at) / Math.max(0.001, k.dur / this.pace))); + const raw = (this.t - k.at) / Math.max(0.001, k.dur / this.pace); + if (raw > 0 && raw < 1) this.engine.moving = true; + const p = easeFn(k.ease ?? 'inOut')(clamp(raw)); cur = { x: lerp(prev.x, k.x, p), y: lerp(prev.y, k.y, p), zoom: lerp(prev.zoom, k.zoom, p), rotate: lerp(prev.rotate || 0, k.rotate || 0, p) }; } const ctx = this.ctx; @@ -435,23 +511,84 @@ export class Stage { group(o, fn) { if (typeof o === 'function') [o, fn] = [{}, o]; const life = this._life(o, 'group', { enter: o.enter ?? 'none', sfx: null, dur: o.dur }); - if (!life) { - this._suppress++; - try { fn(this); } finally { this._suppress--; } - return; + const planning = Boolean(this.engine.layoutLog); + if (planning) (this._spans ??= []).push({ at: this.time(o.at ?? 0), out: o.out === undefined ? Infinity : this.time(o.out) }); + try { + if (!life || planning) { + if (!life) this._suppress++; + // The camera measures what a group holds where it rests, whether it + // is hidden, arriving or leaving at that moment. + if (planning) { + this.ctx.save(); + this.ctx.translate(o.x ?? 0, o.y ?? 0); + if (o.rotate) this.ctx.rotate((o.rotate * Math.PI) / 180); + if ((o.scale ?? 1) !== 1) this.ctx.scale(o.scale, o.scale); + } + try { fn(this); } finally { + if (!life) this._suppress--; + if (planning) this.ctx.restore(); + } + return; + } + this._push(o.x ?? 0, o.y ?? 0, life, o); + try { fn(this); } finally { this.ctx.restore(); } + } finally { + if (planning) this._spans.pop(); } - this._push(o.x ?? 0, o.y ?? 0, life, o); - try { fn(this); } finally { this.ctx.restore(); } + } + + // While the camera is planned, runs a custom drawing once on a context + // that records how far its paths, shapes, text and images reach. It runs at + // its own place, scale and rotation, without its entrance or exit. + _measureDrawing(o, fn, life, random) { + const ctx = this.ctx; + const at = this.time(o.at ?? 0); + const out = o.out === undefined ? Infinity : this.time(o.out); + ctx.save(); + ctx.translate(o.x ?? 0, o.y ?? 0); + if (o.rotate) ctx.rotate((o.rotate * Math.PI) / 180); + if (o.scale) ctx.scale(o.scale, o.scale); + const rec = boundsRecorder(ctx, this.engine.scale); + const saved = Math.random; + Math.random = random; + try { + fn(rec.proxy, life ? { ...life, p: 1, q: 0 } : { p: 1, q: 0, at, dur: 0.001, enter: 'none', exit: 'fade', local: Math.max(0, this.t - at) }); + } catch { + // drawn for real later, where its errors are reported + } finally { + Math.random = saved; + ctx.restore(); + } + const box = rec.box(); + if (box) this._attach('custom', at, out, box); + } + + // Points the camera at an element or a region from `at` until `out`. It + // draws nothing. target: an id drawn earlier, { x, y, w, h } (center and + // size) or [x, y, w, h]. + focus(target, o = {}) { + const b = typeof target === 'string' ? this.get(target) : Array.isArray(target) ? { x: target[0], y: target[1], w: target[2], h: target[3] } : target; + if (!b || !Number.isFinite(b.x) || !Number.isFinite(b.y) || !(b.w > 0) || !(b.h > 0)) throw new SceneError('focus needs an element id, { x, y, w, h } or [x, y, w, h]'); + if (this.engine.layoutLog) this._attach('focus', this.time(o.at ?? 0), o.out === undefined ? Infinity : this.time(o.out), this._designBox(b.x, b.y, b.w, b.h), { explicit: true }); } // Custom drawing with the same timing rules: fn(ctx, life) where life.p is // the 0..1 entrance progress and ctx is already moved to x, y. + // Math.random inside fn has its own sequence for each drawing, so its + // numbers do not depend on which other drawings are on screen. draw(o, fn) { if (typeof o === 'function') [o, fn] = [{}, o]; const life = this._life(o, 'custom', { enter: o.enter ?? 'none', sfx: o.sfx ?? null }); + const random = mulberry32(this._seed('random', o)); + if (this.engine.layoutLog) return this._measureDrawing(o, fn, life, random); if (!life) return; this._push(o.x ?? 0, o.y ?? 0, life, o); - try { fn(this.ctx, life); } finally { this.ctx.restore(); } + const saved = Math.random; + Math.random = random; + try { fn(this.ctx, life); } finally { + Math.random = saved; + this.ctx.restore(); + } } // ---------- text ---------- @@ -491,7 +628,19 @@ export class Stage { const weight = o.weight ?? (o.bold ? BOLD[role] ?? 700 : undefined); const font = fontString(th, role, size, weight); const align = o.align ?? 'center'; - const lay = layoutText(ctx, { str: String(str), font, size, maxWidth: o.maxWidth ?? o.width ?? this._w(0.8), lineHeight: o.lineHeight ?? 1.24 }); + // Large text gets slightly tighter letter spacing in looks that set a + // tracking value. + const tracking = size >= 56 ? (th.fonts[role].tracking ?? 0) * size : 0; + // save() and restore() put the spacing back. Reading ctx.letterSpacing + // back is not reliable in Chrome. + ctx.save(); + ctx.letterSpacing = `${tracking}px`; + let lay; + try { + lay = layoutText(ctx, { str: String(str), font, size, maxWidth: o.maxWidth ?? o.width ?? this._w(0.8), lineHeight: o.lineHeight ?? 1.24 }); + } finally { + ctx.restore(); + } const x = o.x ?? this.cx; const y = o.y ?? this.cy; const left = align === 'left' ? x : align === 'right' ? x - lay.width : x - lay.width / 2; @@ -523,11 +672,12 @@ export class Stage { const accent = colorOf(th, o.mark ?? 'accent'); if (o.bg) { const bgc = colorOf(th, o.bg === true ? 'surface' : o.bg); - this.pen.draw(ctx, { kind: 'rect', w: g.w, h: g.h, r: o.radius ?? Math.min(th.radius, g.h / 2) }, { fill: bgc, stroke: o.border ? colorOf(th, o.border) : 'none', width: 2, shadow: th.shadow, seed: this._seed('textbg', o, plain), rough: false }, { stroke: 1, fill: 1 }); + this.pen.draw(ctx, { kind: 'rect', w: g.w, h: g.h, r: o.radius ?? Math.min(th.radius, g.h / 2) }, { fill: bgc, stroke: o.border ? colorOf(th, o.border) : 'none', width: 2, shadow: this._shadow(), seed: this._seed('textbg', o, plain), rough: false }, { stroke: 1, fill: 1 }); } ctx.font = font; ctx.textBaseline = 'alphabetic'; ctx.textAlign = 'left'; + ctx.letterSpacing = `${tracking}px`; const ox = -lay.width / 2; const oy = -lay.height / 2; const glow = th.stroke.glow && o.glow !== false && kind === 'title' ? th.stroke.glow : 0; @@ -680,7 +830,7 @@ export class Stage { const stroke = colorOf(th, o.stroke ?? o.color, 'ink'); const fill = o.fill !== undefined ? colorOf(th, o.fill) : o.color ? tintOf(th, o.color) : th.surface; const spec = shape === 'ellipse' || shape === 'circle' ? { kind: 'ellipse', w, h } : { kind: 'rect', w, h, r: o.radius ?? th.radius }; - this.pen.draw(ctx, spec, { stroke: o.border === false ? 'none' : stroke, width: o.width ?? th.stroke.width, fill, fillStyle: o.fillStyle, dashed: o.dashed, seed: this._seed('box', o, plain), shadow: th.shadow, glow: o.glow ?? (th.stroke.glow && o.color ? th.stroke.glow : 0) }, prog); + this.pen.draw(ctx, spec, { stroke: o.border === false ? 'none' : stroke, width: o.width ?? th.stroke.width, fill, fillStyle: o.fillStyle, dashed: o.dashed, seed: this._seed('box', o, plain), shadow: this._shadow(), glow: o.glow ?? (th.stroke.glow && o.color ? th.stroke.glow : 0) }, prog); const textColor = o.textColor ? colorOf(th, o.textColor) : this._autoTextColor(fill, th.ink); const top = -contentH / 2; const prevA = ctx.globalAlpha; @@ -715,6 +865,13 @@ export class Stage { const th = this.theme; const pts = points.map((p) => this._point(p)).map((p) => [p.x, p.y]); const life = this._life(o, 'line', { enter: o.enter ?? 'draw' }); + if (this.engine.layoutLog) { + const xs = pts.map((p) => p[0]); + const ys = pts.map((p) => p[1]); + // A line thick enough to read as a bar is a shape the camera keeps whole. + const thick = (o.width ?? th.stroke.width) >= 10; + this._logLayout(thick ? 'shape' : 'line', this.time(o.at ?? 0), o.out === undefined ? Infinity : this.time(o.out), (Math.min(...xs) + Math.max(...xs)) / 2, (Math.min(...ys) + Math.max(...ys)) / 2, Math.max(...xs) - Math.min(...xs), Math.max(...ys) - Math.min(...ys)); + } if (!life) return; this.ctx.save(); const tr = this._enterTransform(life.enter === 'draw' ? { ...life, enter: 'none' } : life); @@ -727,6 +884,19 @@ export class Stage { path(d, o = {}) { const th = this.theme; const life = this._life(o, 'line', { enter: o.enter ?? 'draw' }); + if (this.engine.layoutLog) { + const pts = []; + for (const { c, v } of absolutize(parsePath(d))) { + if (c === 'A') pts.push([v[5], v[6]], [v[5] - v[0], v[6] - v[1]], [v[5] + v[0], v[6] + v[1]]); + else for (let i = 0; i + 1 < v.length; i += 2) pts.push([v[i], v[i + 1]]); + } + if (pts.length) { + const xs = pts.map((q) => q[0] + (o.x ?? 0)); + const ys = pts.map((q) => q[1] + (o.y ?? 0)); + // A closed or filled path is a shape; an open one is a line. + this._logLayout(/z/i.test(d) || o.fill ? 'shape' : 'line', this.time(o.at ?? 0), o.out === undefined ? Infinity : this.time(o.out), (Math.min(...xs) + Math.max(...xs)) / 2, (Math.min(...ys) + Math.max(...ys)) / 2, Math.max(...xs) - Math.min(...xs), Math.max(...ys) - Math.min(...ys)); + } + } if (!life) return; this._push(o.x ?? 0, o.y ?? 0, life.enter === 'draw' ? { ...life, enter: 'none' } : life, o); const prog = this._drawProgress(life); @@ -749,6 +919,11 @@ export class Stage { const c = { x: (a.x + b.x) / 2 - dy * bend * 0.5, y: (a.y + b.y) / 2 + dx * bend * 0.5 }; const d = bend ? quadD([a.x, a.y], [c.x, c.y], [b.x, b.y]) : polyD([[a.x, a.y], [b.x, b.y]]); const life = this._life(o, 'arrow', { enter: o.enter ?? 'draw', dur: o.dur ?? 0.7 }); + if (this.engine.layoutLog) { + const xs = [a.x, b.x, c.x]; + const ys = [a.y, b.y, c.y]; + this._logLayout('arrow', this.time(o.at ?? 0), o.out === undefined ? Infinity : this.time(o.out), (Math.min(...xs) + Math.max(...xs)) / 2, (Math.min(...ys) + Math.max(...ys)) / 2, Math.max(...xs) - Math.min(...xs), Math.max(...ys) - Math.min(...ys)); + } const result = { from: a, to: b, control: c }; if (!life) return result; const ctx = this.ctx; @@ -862,6 +1037,15 @@ export class Stage { const y = o.y ?? this.cy; const bgSize = o.bg ? size * (o.bgScale ?? 1.7) : size; this._register(o.id, x, y, bgSize, bgSize, o.bg === 'square' ? 'rect' : o.bg ? 'ellipse' : 'rect'); + if (this.engine.layoutLog && o.label) { + // The label belongs to the icon for the camera, not for arrows. + const ls = o.labelSize ?? 36; + this.ctx.save(); + this.ctx.font = fontString(th, o.labelFont ?? 'body', ls, BOLD[o.labelFont ?? 'body']); + const lw = this.ctx.measureText(String(o.label)).width; + this.ctx.restore(); + this._pendingExtra = this._elementBox(o, x, y, x, y + bgSize / 2 + ls * 0.45 + ls / 2, lw, ls * 1.1); + } this._iconPaths(name, size); const life = this._life(o, 'icon', { label: name }); const g = geom(x, y, bgSize, bgSize); @@ -872,7 +1056,7 @@ export class Stage { if (o.bg) { const bgColor = o.bg === true || o.bg === 'circle' || o.bg === 'square' ? tintOf(th, o.color ?? th.accent) : colorOf(th, o.bg); const spec = o.bg === 'square' ? { kind: 'rect', w: bgSize, h: bgSize, r: th.radius } : { kind: 'ellipse', w: bgSize, h: bgSize }; - this.pen.draw(this.ctx, spec, { stroke: th.stroke.rough ? color : 'none', width: th.stroke.width * 0.8, fill: bgColor, seed: this._seed('iconbg', o, name), shadow: th.shadow }, { stroke: prog.stroke, fill: prog.fill }); + this.pen.draw(this.ctx, spec, { stroke: th.stroke.rough ? color : 'none', width: th.stroke.width * 0.8, fill: bgColor, seed: this._seed('iconbg', o, name), shadow: this._shadow() }, { stroke: prog.stroke, fill: prog.fill }); } this._iconAt(name, 0, 0, size, color, prog.stroke, this._seed('icon', o, name), o.weight); if (o.label) { @@ -942,9 +1126,9 @@ export class Stage { const ctx = this.ctx; const r = o.radius ?? (frame === 'phone' ? 46 : th.radius); if (frame === 'phone') { - this.pen.draw(ctx, { kind: 'rect', w: fw, h: fh, r: r + 10 }, { fill: '#111418', stroke: 'none', width: 0, shadow: th.shadow || { color: 'rgba(0,0,0,0.25)', blur: 30, y: 12 }, rough: false }, { stroke: 1, fill: 1 }); + this.pen.draw(ctx, { kind: 'rect', w: fw, h: fh, r: r + 10 }, { fill: '#111418', stroke: 'none', width: 0, shadow: this._shadow(undefined, { color: 'rgba(0,0,0,0.25)', blur: 30, y: 12 }), rough: false }, { stroke: 1, fill: 1 }); } else if (bar) { - this.pen.draw(ctx, { kind: 'rect', w: fw, h: fh, r }, { fill: th.dark ? '#1b2233' : '#ffffff', stroke: th.dark ? '#2b3550' : 'rgba(0,0,0,0.08)', width: 2, shadow: th.shadow || { color: 'rgba(0,0,0,0.18)', blur: 26, y: 10 }, rough: false }, { stroke: 1, fill: 1 }); + this.pen.draw(ctx, { kind: 'rect', w: fw, h: fh, r }, { fill: th.dark ? '#1b2233' : '#ffffff', stroke: th.dark ? '#2b3550' : 'rgba(0,0,0,0.08)', width: 2, shadow: this._shadow(undefined, { color: 'rgba(0,0,0,0.18)', blur: 26, y: 10 }), rough: false }, { stroke: 1, fill: 1 }); const dots = ['#ff5f57', '#febc2e', '#28c840']; dots.forEach((c, i) => { ctx.fillStyle = c; @@ -965,9 +1149,9 @@ export class Stage { ctx.restore(); } } else if (frame === 'card') { - this.pen.draw(ctx, { kind: 'rect', w: fw, h: fh, r: r + cardPad / 2 }, { fill: th.dark ? th.surface : '#ffffff', stroke: th.stroke.rough ? th.ink : 'none', width: th.stroke.width * 0.8, shadow: th.shadow || { color: 'rgba(0,0,0,0.16)', blur: 24, y: 8 }, seed: this._seed('card', o, src), fillStyle: 'solid' }, { stroke: 1, fill: 1 }); + this.pen.draw(ctx, { kind: 'rect', w: fw, h: fh, r: r + cardPad / 2 }, { fill: th.dark ? th.surface : '#ffffff', stroke: th.stroke.rough ? th.ink : 'none', width: th.stroke.width * 0.8, shadow: this._shadow(undefined, { color: 'rgba(0,0,0,0.16)', blur: 24, y: 8 }), seed: this._seed('card', o, src), fillStyle: 'solid' }, { stroke: 1, fill: 1 }); } else if (o.shadow !== false && th.shadow) { - this.pen.draw(ctx, { kind: 'rect', w, h, r }, { fill: th.surface, stroke: 'none', width: 0, shadow: th.shadow, rough: false }, { stroke: 1, fill: 1 }); + this.pen.draw(ctx, { kind: 'rect', w, h, r }, { fill: th.surface, stroke: 'none', width: 0, shadow: this._shadow(), rough: false }, { stroke: 1, fill: 1 }); } const iy = frame === 'phone' ? 0 : bar / 2; ctx.save(); @@ -1355,6 +1539,8 @@ export class Stage { const x = o.x ?? this.cx; const y = o.y ?? this.cy; this._register(o.id, x, y, w, h); + // The camera keeps the tallest step in view, not only the current one. + if (this.engine.layoutLog && steps && !o.h) this._register(undefined, x, y, w, Math.max(...steps.map((step) => step.lines.length)) * lh + pad * 1.6 + barH); const at = this.time(o.at ?? 0); const reveal = o.reveal ?? 'type'; const cps = (o.cps ?? 40) * this.pace; @@ -1366,9 +1552,29 @@ export class Stage { if (steps) steps.slice(1).forEach((step, i) => this._record('code', step.at, (o.morphDur ?? 0.8) / this.pace, { sfx: o.sfx === false || o.sfx === null ? false : undefined, sfxGain: 0.2 }, { sfx: 'whoosh', label: `morph${i + 2}` })); const life = this._life({ ...o, sfx: null }, 'code', { dur: 0.5 }); + if (this.engine.layoutLog) { + // Highlighted lines tell the camera where to look. The window is as + // tall as the step it shows, so each step has its own first line. + const focusLines = (list, from, until, rows) => { + const nums = (Array.isArray(list) ? list : list ? [list] : []).filter((n) => n >= 1 && n <= rows.length); + if (!nums.length) return; + const lineTop = y - (o.h ?? rows.length * lh + pad * 1.6 + barH) / 2 + barH + pad * 0.8; + // Whole lines across, with a little of the lines around them. + const y0 = lineTop + (Math.min(...nums) - 1) * lh - lh * 0.72; + const y1 = lineTop + Math.max(...nums) * lh + lh * 0.28; + const longest = Math.max(...rows.map((r) => r.length)); + const x0 = x - w / 2 + pad; + const x1 = Math.min(x + w / 2 - pad, x0 + numW + longest * charW); + const box = this._elementBox(o, x, y, (x0 + x1) / 2, (y0 + y1) / 2, x1 - x0, y1 - y0); + this._attach('focus', from, until, box); + }; + const out = o.out === undefined ? Infinity : this.time(o.out); + if (steps) steps.forEach((step, k) => focusLines(step.highlight, k ? step.at : this.time(o.highlightAt ?? at + 0.4 / this.pace), steps[k + 1]?.at ?? out, step.lines)); + else focusLines(o.highlight, this.time(o.highlightAt ?? at + 0.4 / this.pace), out, lines); + } if (!life) return geom(x, y, w, h); this._push(x, y, life, o); - this.pen.draw(ctx, { kind: 'rect', w, h, r: Math.min(th.radius, 18) }, { fill: C.bg, stroke: th.stroke.rough ? C.border : 'none', width: th.stroke.width * 0.8, seed: this._seed('codebg', o), shadow: th.shadow || { color: 'rgba(0,0,0,0.2)', blur: 24, y: 10 }, rough: th.stroke.rough, fillStyle: 'solid' }, { stroke: 1, fill: 1 }); + this.pen.draw(ctx, { kind: 'rect', w, h, r: Math.min(th.radius, 18) }, { fill: C.bg, stroke: th.stroke.rough ? C.border : 'none', width: th.stroke.width * 0.8, seed: this._seed('codebg', o), shadow: this._shadow(undefined, { color: 'rgba(0,0,0,0.2)', blur: 24, y: 10 }), rough: th.stroke.rough, fillStyle: 'solid' }, { stroke: 1, fill: 1 }); const left = -w / 2 + pad; const top = -h / 2 + barH + pad * 0.8; if (barH) { @@ -1522,9 +1728,28 @@ export class Stage { return r; }); const life = this._life({ ...o, sfx: null }, 'terminal', { dur: 0.5, enter: o.enter ?? (th.enter.code === 'rise' ? 'rise' : 'fade') }); + if (this.engine.layoutLog) { + // Each command with its output is a focus point while it is the + // newest thing in the terminal. + const out = o.out === undefined ? Infinity : this.time(o.out); + const lineTop = y - h / 2 + barH + pad * 0.8; + let row = 0; + const blocks = []; + for (const it of timed) { + const n = String(it.cmd ?? it.out ?? '').split('\n').length; + if (it.cmd !== undefined) blocks.push({ at: it.start, from: row, to: row + n }); + else if (blocks.length) blocks[blocks.length - 1].to = row + n; + row += n; + } + if (blocks.length > 1) blocks.forEach((b, i) => { + const y0 = lineTop + b.from * lh - lh * 0.45; + const y1 = lineTop + b.to * lh - lh * 0.1; + this._attach('focus', b.at, blocks[i + 1]?.at ?? out, this._elementBox(o, x, y, x, (y0 + y1) / 2, w - pad * 2, y1 - y0)); + }); + } if (!life) return geom(x, y, w, h); this._push(x, y, life, o); - this.pen.draw(ctx, { kind: 'rect', w, h, r: Math.min(th.radius, 16) }, { fill: T.bg, stroke: th.stroke.rough ? th.ink : 'none', width: th.stroke.width * 0.8, seed: this._seed('termbg', o), shadow: th.shadow || { color: 'rgba(0,0,0,0.22)', blur: 26, y: 10 }, rough: th.stroke.rough, fillStyle: 'solid' }, { stroke: 1, fill: 1 }); + this.pen.draw(ctx, { kind: 'rect', w, h, r: Math.min(th.radius, 16) }, { fill: T.bg, stroke: th.stroke.rough ? th.ink : 'none', width: th.stroke.width * 0.8, seed: this._seed('termbg', o), shadow: this._shadow(undefined, { color: 'rgba(0,0,0,0.22)', blur: 26, y: 10 }), rough: th.stroke.rough, fillStyle: 'solid' }, { stroke: 1, fill: 1 }); ctx.save(); const clip = new Path2D(); clip.roundRect(-w / 2, -h / 2, w, barH, [Math.min(th.radius, 16), Math.min(th.radius, 16), 0, 0]); @@ -1670,3 +1895,260 @@ function wedgeD(r, hole, a0, a1) { } export { mixColor }; + +// A drawing context that remembers the box everything drawn on it covers, +// in design pixels. Each fill, stroke, text or image counts on its own, and +// one that covers most of the frame is a background and does not count. +// Path2D objects cannot be looked into and are skipped. +export function boundsRecorder(ctx, scale, frame = null) { + let x0 = Infinity; + let y0 = Infinity; + let x1 = -Infinity; + let y1 = -Infinity; + let path = null; + const point = (x, y) => { + const m = ctx.getTransform(); + return [(m.a * x + m.c * y + m.e) / scale, (m.b * x + m.d * y + m.f) / scale]; + }; + const grow = (b, x, y) => { + const [px, py] = point(x, y); + if (!Number.isFinite(px) || !Number.isFinite(py)) return; + b.x0 = Math.min(b.x0, px); + b.y0 = Math.min(b.y0, py); + b.x1 = Math.max(b.x1, px); + b.y1 = Math.max(b.y1, py); + }; + const empty = () => ({ x0: Infinity, y0: Infinity, x1: -Infinity, y1: -Infinity }); + let paused = 0; + const commit = (b) => { + if (paused || !b || !(b.x1 >= b.x0) || !(b.y1 >= b.y0)) return; + if (frame && (b.x1 - b.x0) * (b.y1 - b.y0) > frame.W * frame.H * 0.6) return; + x0 = Math.min(x0, b.x0); + y0 = Math.min(y0, b.y0); + x1 = Math.max(x1, b.x1); + y1 = Math.max(y1, b.y1); + }; + // How far a stroke reaches past its outline: half the line width in the + // drawing's own units, through the full transform. + const widen = (b) => { + const m = ctx.getTransform(); + const r = ctx.lineWidth / 2 / scale; + const kx = r * Math.hypot(m.a, m.c); + const ky = r * Math.hypot(m.b, m.d); + return { x0: b.x0 - kx, y0: b.y0 - ky, x1: b.x1 + kx, y1: b.y1 + ky }; + }; + const onPath = (fn) => (...a) => { + path ??= empty(); + fn(path, ...a); + }; + // For strokes: the segments of each subpath of the current path, so + // sharp corners and square ends can be measured. A segment keeps its + // ends and the control points that give its direction there. + let subs = []; + let sub = null; + const dir = (p, q) => { + const dx = q[0] - p[0]; + const dy = q[1] - p[1]; + const L = Math.hypot(dx, dy); + return L > 1e-9 ? [dx / L, dy / L] : null; + }; + const begin = (p) => { + sub = { first: p, last: p, segs: [], closed: false }; + subs.push(sub); + }; + // A segment from the current point to `end`; `a` and `b` are its first + // and last control points. Without a current point a path starts at `a`. + const segment = (a, b, end) => { + if (!sub) begin(a); + const p0 = sub.last; + if (dir(p0, a) || dir(p0, b) || dir(p0, end)) sub.segs.push({ p0, a, b, p1: end }); + sub.last = end; + }; + const close = () => { + if (!sub) return; + segment(sub.first, sub.first, sub.first); + sub.closed = true; + begin(sub.first); + }; + const line = (x, y) => { + const p = point(x, y); + segment(p, p, p); + }; + // An arc as one segment, with its direction at both ends. Canvas draws a + // straight line to the arc's start first. + const arcSegment = (cx, cy, rx, ry, rot, a0, a1, ccw) => { + const full = ccw ? a0 - a1 >= Math.PI * 2 : a1 - a0 >= Math.PI * 2; + const end = full ? a0 : a1; + const sign = ccw ? -1 : 1; + const turn = (x, y) => [x * Math.cos(rot) - y * Math.sin(rot), x * Math.sin(rot) + y * Math.cos(rot)]; + const at = (t) => { const [x, y] = turn(rx * Math.cos(t), ry * Math.sin(t)); return [cx + x, cy + y]; }; + const tangent = (t) => turn(-rx * Math.sin(t) * sign, ry * Math.cos(t) * sign); + const s0 = at(a0); + const s1 = at(end); + const t0 = tangent(a0); + const t1 = tangent(end); + line(s0[0], s0[1]); + segment(point(s0[0] + t0[0], s0[1] + t0[1]), point(s1[0] - t1[0], s1[1] - t1[1]), point(s1[0], s1[1])); + }; + // Turns a point from `point` back into the current transform's units. + const toLocal = () => { + const inv = ctx.getTransform().inverse(); + if (![inv.a, inv.b, inv.c, inv.d, inv.e, inv.f].every(Number.isFinite)) return null; + return (p) => { + const q = inv.transformPoint(new DOMPoint(p[0] * scale, p[1] * scale)); + return [q.x, q.y]; + }; + }; + // Works out the corners and ends of a stroke in the current transform's + // units, where the line width applies, and adds them to the box. + const strokeEnds = (b) => { + const local = toLocal(); + if (!local) return b; + const hw = ctx.lineWidth / 2; + const miter = ctx.lineJoin === 'miter' ? ctx.miterLimit : 0; + // A mitered corner sticks out along the line that splits the corner in + // half. Canvas cuts it off when the corner is sharper than the miter + // limit allows. + const join = (u, v, p) => { + const sinHalf = Math.sqrt(Math.max(0, (1 + u[0] * v[0] + u[1] * v[1]) / 2)); + const ox = u[0] - v[0]; + const oy = u[1] - v[1]; + const L = Math.hypot(ox, oy); + if (sinHalf < 1e-6 || 1 / sinHalf > miter || L < 1e-9) return; + grow(b, p[0] + (ox / L) * (hw / sinHalf), p[1] + (oy / L) * (hw / sinHalf)); + }; + const cap = (p, d) => { + grow(b, p[0] + (d[0] - d[1]) * hw, p[1] + (d[1] + d[0]) * hw); + grow(b, p[0] + (d[0] + d[1]) * hw, p[1] + (d[1] - d[0]) * hw); + }; + for (const sp of subs) { + const segs = sp.segs.map((g) => { + const [p0, a, c, p1] = [g.p0, g.a, g.b, g.p1].map(local); + return { p0, p1, d0: dir(p0, a) ?? dir(p0, c) ?? dir(p0, p1), d1: dir(c, p1) ?? dir(a, p1) ?? dir(p0, p1) }; + }).filter((g) => g.d0 && g.d1); + const n = segs.length; + if (!n) continue; + if (miter) { + for (let i = 1; i < n; i++) join(segs[i - 1].d1, segs[i].d0, segs[i].p0); + if (sp.closed) join(segs[n - 1].d1, segs[0].d0, segs[0].p0); + } + if (ctx.lineCap === 'square' && !sp.closed) { + cap(segs[0].p0, [-segs[0].d0[0], -segs[0].d0[1]]); + cap(segs[n - 1].p1, segs[n - 1].d1); + } + } + return b; + }; + const corners = (b, x, y, w, h) => { + grow(b, x, y); + grow(b, x + w, y); + grow(b, x, y + h); + grow(b, x + w, y + h); + }; + const shape = (x, y, w, h) => { + const b = empty(); + corners(b, x, y, w, h); + commit(b); + }; + const text = (str, x, y, stroked = false) => { + const m = ctx.measureText(String(str)); + const l = m.actualBoundingBoxLeft || 0; + const a = m.actualBoundingBoxAscent || 0; + const b = empty(); + corners(b, x - l, y - a, l + (m.actualBoundingBoxRight || m.width), a + (m.actualBoundingBoxDescent || 0)); + commit(stroked ? widen(b) : b); + }; + const record = { + beginPath: () => { + path = null; + subs = []; + sub = null; + }, + moveTo: onPath((b, x, y) => { grow(b, x, y); begin(point(x, y)); }), + lineTo: onPath((b, x, y) => { grow(b, x, y); line(x, y); }), + quadraticCurveTo: onPath((b, cx, cy, x, y) => { grow(b, cx, cy); grow(b, x, y); const c = point(cx, cy); segment(c, c, point(x, y)); }), + bezierCurveTo: onPath((b, a, c, d, e, x, y) => { grow(b, a, c); grow(b, d, e); grow(b, x, y); segment(point(a, c), point(d, e), point(x, y)); }), + // A line toward the corner (x1, y1) that turns along a circle onto the + // line toward (x2, y2), as canvas draws it. + arcTo: onPath((b, x1, y1, x2, y2, r) => { + grow(b, x1, y1); + grow(b, x2, y2); + const local = toLocal(); + if (!sub || !local) return line(x1, y1); + const p0 = local(sub.last); + const u = dir([x1, y1], p0); + const v = dir([x1, y1], [x2, y2]); + const sin = u && v ? Math.abs(u[0] * v[1] - u[1] * v[0]) : 0; + if (!(r > 0) || sin < 1e-9) return line(x1, y1); + const d = r / Math.tan(Math.acos(clamp(u[0] * v[0] + u[1] * v[1], -1, 1)) / 2); + const t1 = [x1 + u[0] * d, y1 + u[1] * d]; + const t2 = [x1 + v[0] * d, y1 + v[1] * d]; + grow(b, t1[0], t1[1]); + grow(b, t2[0], t2[1]); + line(t1[0], t1[1]); + segment(point(x1, y1), point(x1, y1), point(t2[0], t2[1])); + }), + arc: onPath((b, x, y, r, a0, a1, ccw = false) => { corners(b, x - r, y - r, r * 2, r * 2); arcSegment(x, y, r, r, 0, a0, a1, ccw); }), + ellipse: onPath((b, x, y, rx, ry, rot, a0, a1, ccw = false) => { const r = Math.max(rx, ry); corners(b, x - r, y - r, r * 2, r * 2); arcSegment(x, y, rx, ry, rot, a0, a1, ccw); }), + // A rectangle is a closed path of its own, and the next one starts at + // its first corner. + rect: onPath((b, x, y, w, h) => { + corners(b, x, y, w, h); + begin(point(x, y)); + line(x + w, y); + line(x + w, y + h); + line(x, y + h); + close(); + }), + roundRect: onPath((b, x, y, w, h) => { corners(b, x, y, w, h); begin(point(x, y)); }), + closePath: close, + fill: (p) => { if (!(p instanceof Path2D)) commit(path); }, + stroke: (p) => { if (!(p instanceof Path2D) && path) commit(strokeEnds(widen(path))); }, + fillRect: shape, + // Mitered corners reach half the line width out on both sides; round + // and bevelled ones no further than the widened outline. + strokeRect: (x, y, w, h) => { + const b = empty(); + const hw = ctx.lineWidth / 2; + if (ctx.lineJoin === 'miter') { + corners(b, Math.min(x, x + w) - hw, Math.min(y, y + h) - hw, Math.abs(w) + hw * 2, Math.abs(h) + hw * 2); + commit(b); + } else { + corners(b, x, y, w, h); + commit(widen(b)); + } + }, + fillText: (str, x, y) => text(str, x, y), + strokeText: (str, x, y) => text(str, x, y, true), + drawImage: (img, ...a) => { + if (a.length === 2) shape(a[0], a[1], img.width, img.height); + else if (a.length === 4) shape(a[0], a[1], a[2], a[3]); + else if (a.length === 8) shape(a[4], a[5], a[6], a[7]); + }, + }; + // Parts that report their own place and time (UI panels, headlines) and + // decoration (floating cards) are left out. + const pause = (on) => { paused += on ? 1 : -1; }; + const proxy = new Proxy(ctx, { + get(target, prop) { + if (prop === '__inkPause') return pause; + const v = target[prop]; + if (typeof v !== 'function') return v; + const note = record[prop]; + return note ? (...args) => { note(...args); return v.apply(target, args); } : v.bind(target); + }, + set(target, prop, value) { + target[prop] = value; + return true; + }, + }); + return { proxy, box: () => (x1 >= x0 && y1 >= y0 && (x1 > x0 || y1 > y0) ? { x: (x0 + x1) / 2, y: (y0 + y1) / 2, w: x1 - x0, h: y1 - y0 } : null) }; +} + +function unionBox(a, b) { + const x0 = Math.min(a.x - a.w / 2, b.x - b.w / 2); + const y0 = Math.min(a.y - a.h / 2, b.y - b.h / 2); + const x1 = Math.max(a.x + a.w / 2, b.x + b.w / 2); + const y1 = Math.max(a.y + a.h / 2, b.y + b.h / 2); + return { x: (x0 + x1) / 2, y: (y0 + y1) / 2, w: x1 - x0, h: y1 - y0 }; +} diff --git a/engine/themes.js b/engine/themes.js index a490a79..7ebc549 100644 --- a/engine/themes.js +++ b/engine/themes.js @@ -49,7 +49,7 @@ const CLEAN = { blue: '#3b6cf6', purple: '#7c5cff', pink: '#ec4899', gray: '#98a2b3', ink: '#111827', }, fonts: { - display: { family: 'Plus Jakarta Sans', weight: 800 }, + display: { family: 'Plus Jakarta Sans', weight: 800, tracking: -0.025 }, body: { family: 'Inter', weight: 400 }, hand: { family: 'Kalam', weight: 400 }, mono: { family: 'JetBrains Mono', weight: 400 }, @@ -145,7 +145,7 @@ const MIDNIGHT = { blue: '#60a5fa', purple: '#a78bfa', pink: '#f472b6', gray: '#64748b', ink: '#e8eef8', }, fonts: { - display: { family: 'Space Grotesk', weight: 700 }, + display: { family: 'Space Grotesk', weight: 700, tracking: -0.02 }, body: { family: 'Inter', weight: 400 }, hand: { family: 'Kalam', weight: 400 }, mono: { family: 'JetBrains Mono', weight: 400 }, @@ -177,7 +177,7 @@ const EDITORIAL = { blue: '#2f5b88', purple: '#6a4e8c', pink: '#b04870', gray: '#8a847b', ink: '#1c1a17', }, fonts: { - display: { family: 'Instrument Serif', weight: 400 }, + display: { family: 'Instrument Serif', weight: 400, tracking: -0.01 }, body: { family: 'Inter', weight: 400 }, hand: { family: 'Instrument Serif', weight: 400 }, mono: { family: 'JetBrains Mono', weight: 400 }, diff --git a/engine/ui.js b/engine/ui.js index 586fd48..f14b483 100644 --- a/engine/ui.js +++ b/engine/ui.js @@ -133,6 +133,8 @@ export class UI { // An entrance, so `check` knows something new appears and a sound can play. _enter(at, dur, sfx) { this.s._record('ui', at, dur, { sfx: sfx ?? null }); + // Arriving parts get motion blur in the final render. + if (this.s.t >= at && this.s.t < at + dur / this.s.pace) this.s.engine.moving = true; } // ---------- surfaces ---------- @@ -148,7 +150,19 @@ export class UI { // Faint app cards drifting around the edges, out of focus. Each card is // blurred once and reused, because a blur filter on every frame is slow. + // The floating cards are decoration. The camera does not count them as + // part of the picture. floaters(o = {}) { + const pause = this.c.__inkPause; + pause?.(true); + try { + return this._floaters(o); + } finally { + pause?.(false); + } + } + + _floaters(o = {}) { const s = this.s; const c = this.c; const tint = o.tint ?? 'card'; @@ -249,20 +263,29 @@ export class UI { const a = this.at(o.at ?? 0); const dur = o.dur ?? 0.6; this._enter(a, dur, o.sfx); + s._logLayout('panel', a, o.out === undefined ? Infinity : this.at(o.out), x + w / 2, y + h / 2, w, h); if (s.t < a) return 0; const p = clamp((s.t - a) / (dur / s.pace)); const q = o.out !== undefined ? this._p(o.out, o.outDur ?? 0.35, 'in') : 0; if (q >= 1) return p; const c = this.c; const sc = lerp(o.from ?? 0.94, 1, outBack(p)) * (1 - 0.03 * q); + // A panel tells the camera its own place and time, so what it holds is + // not measured again as a loose drawing. + const pause = c.__inkPause; + pause?.(true); c.save(); c.globalAlpha *= clamp(p * 2.5) * (1 - q); c.translate(x + w / 2, y + h / 2 + (1 - outCubic(p)) * (o.rise ?? 30)); c.scale(sc, sc); c.translate(-(x + w / 2), -(y + h / 2)); - this.card(x, y, w, h, o); - if (fn) fn(x, y, w, h); - c.restore(); + try { + this.card(x, y, w, h, o); + if (fn) fn(x, y, w, h); + } finally { + c.restore(); + pause?.(false); + } return p; } @@ -448,7 +471,18 @@ export class UI { // A headline that blurs in word by word, in the headline font. Wrap words // in *stars* to set them in the accent color, in italic unless // italic: false (for a brand font without an italic). + // A headline reports its own place and time to the camera. headline(str, x, y, o = {}) { + const pause = this.c.__inkPause; + pause?.(true); + try { + return this._headline(str, x, y, o); + } finally { + pause?.(false); + } + } + + _headline(str, x, y, o = {}) { const s = this.s; const c = this.c; const size = o.size ?? 96; @@ -472,6 +506,7 @@ export class UI { const total = widths.reduce((a, b) => a + b, 0) + space * (tokens.length - 1); let cx = o.align === 'left' ? x : o.align === 'right' ? x - total : x - total / 2; const a = this.at(o.at ?? 0); + s._logLayout('headline', a, o.out === undefined ? Infinity : this.at(o.out), cx + total / 2, y - size * 0.35, total, size * 1.1); const stagger = (o.stagger ?? 0.07) / s.pace; const dur = (o.dur ?? 0.7) / s.pace; this._enter(a, dur + stagger * tokens.length, o.sfx); @@ -936,6 +971,7 @@ export class UI { if (s.t < a) continue; if (k.x !== undefined) { const p = inOutCubic(clamp((s.t - a) / ((k.dur ?? 0.7) / s.pace))); + if (p > 0 && p < 1) s.engine.moving = true; // A slight arc looks like a hand moved it. const dx = k.x - x; const dy = k.y - y; diff --git a/engine/util.js b/engine/util.js index 2f9525a..41f5ac4 100644 --- a/engine/util.js +++ b/engine/util.js @@ -16,6 +16,8 @@ export const ease = { outElastic: (p) => (p === 0 || p === 1 ? p : Math.pow(2, -10 * p) * Math.sin((p * 10 - 0.75) * ((2 * Math.PI) / 3)) + 1), outQuart: (p) => 1 - Math.pow(1 - p, 4), inOutSine: (p) => -(Math.cos(Math.PI * p) - 1) / 2, + // A lightly damped spring: quick, about 4% past the end, then settled. + spring: (p) => (p >= 1 ? 1 : 1 - Math.exp(-7.2 * p) * Math.cos(7.2 * p)), }; export function easeFn(name) { diff --git a/src/export.js b/src/export.js index ad56fa1..0b43138 100644 --- a/src/export.js +++ b/src/export.js @@ -64,7 +64,7 @@ export function copyPlayer(project, timeline, used, out) { if (languageFonts(project.config.language).length) copy(path.join(cacheRoot(), 'fonts', 'NotoSansCJK-OFL.txt'), 'fonts/licenses/NotoSansCJK-OFL.txt'); fs.writeFileSync(path.join(out, 'fonts/fonts.json'), JSON.stringify(fonts)); // Audio synthesis and upload code run during export, never during playback. - const modules = ['runtime', 'stage', 'themes', 'pen', 'transitions', 'captions', 'util', 'layout', 'clips', 'ui', 'readability', 'pathdata', 'text', 'code']; + const modules = ['runtime', 'stage', 'themes', 'pen', 'transitions', 'captions', 'util', 'layout', 'clips', 'ui', 'readability', 'pathdata', 'text', 'code', 'director']; for (const name of modules) copy(path.join(ROOT, 'engine', `${name}.js`), `engine/${name}.js`); copy(path.join(ROOT, 'engine/icons/lucide.json'), 'engine/icons/lucide.json'); copy(path.join(ROOT, 'engine/icons/LICENSE-lucide.txt'), 'engine/icons/LICENSE-lucide.txt'); diff --git a/src/project.js b/src/project.js index 4595ec3..3ef0c57 100644 --- a/src/project.js +++ b/src/project.js @@ -45,6 +45,8 @@ export const DEFAULTS = { captions: 'auto', cover: 'auto', transition: 'auto', + camera: 'auto', + motionBlur: true, lead: 0.35, hold: 0.7, end: 1.4, @@ -193,6 +195,8 @@ export function normalizeConfig(raw, dir = '.') { if (typeof cfg.voice === 'string' && cfg.speed * cfg.pace > 1.8) fail(`speed ${cfg.speed} times pace ${cfg.pace} makes the voice too fast; keep speed x pace at 1.8 or below`); if (![24, 25, 30, 50, 60].includes(cfg.fps)) fail('fps must be 24, 25, 30, 50 or 60'); if (!TRANSITIONS.includes(cfg.transition)) fail(`transition must be one of ${TRANSITIONS.join(', ')}`); + if (!['auto', 'still'].includes(cfg.camera)) fail('camera must be "auto" (the camera follows the narration) or "still"'); + if (typeof cfg.motionBlur !== 'boolean') fail('motionBlur must be true or false'); if (![true, false, 'minimal'].includes(cfg.sfx)) fail('sfx must be true, false or "minimal"'); if (![true, false, 'auto'].includes(cfg.captions)) fail('captions must be true, false or "auto"'); const cover = cfg.cover; diff --git a/src/render.js b/src/render.js index 5ba06da..b00d4a6 100644 --- a/src/render.js +++ b/src/render.js @@ -43,6 +43,8 @@ export async function render(project, opts = {}) { const coverTime = partial ? null : resolveTimes(timeline, [project.config.cover === 'auto' ? `${timeline.scenes[0].id}@end` : project.config.cover])[0]; const workers = Math.max(1, Math.min(opts.workers ?? Math.min(8, Math.max(1, Math.floor(os.cpus().length / 2))), to - from)); const quality = draft ? 0.82 : 0.93; + // Final renders get motion blur. Drafts, GIFs and transparent videos skip it. + const samples = draft || format === 'gif' || transparent || project.config.motionBlur === false ? 1 : 6; const segDir = path.join(project.paths.build, 'segments'); fs.rmSync(segDir, { recursive: true, force: true }); fs.mkdirSync(segDir, { recursive: true }); @@ -78,7 +80,7 @@ export async function render(project, opts = {}) { for (let f = a; f < b; f += batch) { const frames = []; for (let i = f; i < Math.min(b, f + batch); i++) frames.push(i); - const jpegs = await page.evaluate(([l, q]) => window.explainroo.frames(l, q), [frames, quality]); + const jpegs = await page.evaluate(([l, q, n]) => window.explainroo.frames(l, q, n), [frames, quality, samples]); for (const j of jpegs) await enc.write(Buffer.from(j, 'base64')); done += frames.length; if (Date.now() - lastLog > 3000) { diff --git a/test/camera-browser.test.js b/test/camera-browser.test.js new file mode 100644 index 0000000..e35f063 --- /dev/null +++ b/test/camera-browser.test.js @@ -0,0 +1,148 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { findChrome } from '../src/browser.js'; +import { loadProject } from '../src/project.js'; +import { buildTimeline } from '../src/timeline.js'; +import { withEngine } from '../src/pipeline.js'; + +let chrome; +try { chrome = findChrome(); } catch {} + +test('the camera measures what scenes draw where and as big as it is drawn', { skip: !chrome }, async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'explainroo-camera-')); + try { + fs.writeFileSync(path.join(dir, 'video.json'), JSON.stringify({ theme: 'clean', captions: false, watermark: false, music: false, sfx: false })); + fs.writeFileSync(path.join(dir, 'script.md'), '## demo {min=8}\n'); + fs.writeFileSync(path.join(dir, 'scenes.js'), 'export default { demo(s) {} };'); + const project = loadProject(dir); + await withEngine(project, buildTimeline(project, {}), {}, async (page) => { + const r = await page.evaluate(async () => { + const { Engine, fetchState } = await import('/engine/runtime.js'); + const state = await fetchState(); + const plan = (demo) => { + const engine = new Engine({ canvas: document.createElement('canvas'), state, scenes: { demo }, icons: { dot: ['M0 0L24 24'] }, images: new Map() }); + engine.planCameras(); + return engine.cameraPlans.get(0)?.elements ?? []; + }; + const out = {}; + // Random positions: planned where they are drawn, one call each. + let calls = 0; + const drawn = []; + const scene = (s) => { + for (let i = 0; i < 3; i++) s.draw({ at: 0 }, (ctx) => { calls++; const x = 200 + Math.random() * 1400; drawn.push(x); ctx.fillRect(x, 400, 100, 100); }); + }; + out.random = { planned: plan(scene).filter((e) => e.kind === 'custom').map((e) => e.box.x - 50), calls }; + const engine = new Engine({ canvas: document.createElement('canvas'), state, scenes: { demo: scene }, icons: {}, images: new Map() }); + drawn.length = 0; + engine.renderFrame(4); + out.random.drawn = [...drawn]; + // A drawing on its way out is measured at its full size. + out.leaving = plan((s) => s.draw({ at: 0, out: 7.9, exit: 'pop' }, (ctx) => ctx.fillRect(700, 400, 400, 200))).find((e) => e.kind === 'custom')?.box; + // Stroke widths count, through the transform. + out.stroked = plan((s) => s.draw({ x: 960, y: 540, at: 0 }, (ctx) => { ctx.lineWidth = 300; ctx.strokeRect(-50, -150, 100, 300); })).find((e) => e.kind === 'custom')?.box; + // A sharp miter corner reaches past half the line width. + out.miter = plan((s) => s.draw({ at: 0 }, (ctx) => { + ctx.lineWidth = 50; + ctx.beginPath(); ctx.moveTo(900, 650); ctx.lineTo(960, 400); ctx.lineTo(1020, 650); ctx.closePath(); ctx.stroke(); + })).find((e) => e.kind === 'custom')?.box; + // Sharp corners under a squeezed transform, after a rectangle, and + // on curves without a moveTo reach as far as Chrome draws them. + const ink = (paint) => { + const c = document.createElement('canvas'); + c.width = 1920; c.height = 1080; + const g = c.getContext('2d'); + g.strokeStyle = '#f00'; + paint(g); + const d = g.getImageData(0, 0, 1920, 1080).data; + const b = { x0: Infinity, y0: Infinity, x1: -Infinity, y1: -Infinity }; + for (let i = 3; i < d.length; i += 4) if (d[i] > 128) { + const x = ((i - 3) / 4) % 1920; + const y = Math.floor((i - 3) / 4 / 1920); + b.x0 = Math.min(b.x0, x); b.y0 = Math.min(b.y0, y); b.x1 = Math.max(b.x1, x + 1); b.y1 = Math.max(b.y1, y + 1); + } + return b; + }; + const strokes = { + squeezed: (ctx) => { ctx.scale(0.5, 1); ctx.lineWidth = 50; ctx.beginPath(); ctx.moveTo(1880, 650); ctx.lineTo(1920, 400); ctx.lineTo(1960, 650); ctx.closePath(); ctx.stroke(); }, + stretched: (ctx) => { ctx.scale(1, 0.5); ctx.lineWidth = 40; ctx.lineCap = 'square'; ctx.beginPath(); ctx.moveTo(800, 1000); ctx.lineTo(1100, 1200); ctx.stroke(); }, + afterRect: (ctx) => { ctx.lineWidth = 50; ctx.beginPath(); ctx.rect(930, 650, 60, 50); ctx.lineTo(960, 400); ctx.lineTo(990, 650); ctx.stroke(); }, + curve: (ctx) => { ctx.lineWidth = 40; ctx.lineCap = 'square'; ctx.beginPath(); ctx.quadraticCurveTo(700, 300, 900, 600); ctx.stroke(); }, + backwards: (ctx) => { ctx.lineWidth = 100; ctx.strokeRect(900, 500, -100, -100); ctx.strokeRect(1300, 500, -100, 100); }, + arcTo: (ctx) => { ctx.lineWidth = 40; ctx.beginPath(); ctx.moveTo(800, 700); ctx.arcTo(960, 300, 1000, 700, 10); ctx.lineTo(1300, 300); ctx.stroke(); }, + }; + out.strokes = Object.fromEntries(Object.entries(strokes).map(([k, paint]) => { + const box = plan((s) => s.draw({ at: 0 }, paint)).find((e) => e.kind === 'custom')?.box; + return [k, { box: { x0: box.x - box.w / 2, y0: box.y - box.h / 2, x1: box.x + box.w / 2, y1: box.y + box.h / 2 }, ink: ink(paint) }]; + })); + // A group on its way out is measured where it rests. + out.groupLeaving = plan((s) => s.group({ x: 960, y: 540, at: 0, out: 7.55, exit: 'pop' }, () => { + s.draw({ at: 0 }, (ctx) => ctx.fillRect(-550, -100, 1100, 200)); + })).find((e) => e.kind === 'custom')?.box; + // A drawing's random numbers do not depend on which drawings are on screen. + const xs = []; + const two = (s) => { + s.draw({ at: 0, out: 2 }, (ctx) => { const x = 200 + Math.random() * 1400; xs.push(['early', s.t, x]); ctx.fillRect(x, 400, 100, 100); }); + s.draw({ at: 0 }, (ctx) => { const x = 200 + Math.random() * 1400; xs.push(['always', s.t, x]); ctx.fillRect(x, 600, 100, 100); }); + }; + out.hidden = { planned: plan(two).filter((e) => e.kind === 'custom').map((e) => e.box.x - 50) }; + const twice = new Engine({ canvas: document.createElement('canvas'), state, scenes: { demo: two }, icons: {}, images: new Map() }); + xs.length = 0; + twice.renderFrame(1); + out.hidden.drawn = xs.map((v) => v[2]); + // An icon's label grows with the icon. + out.label = plan((s) => s.icon('dot', { x: 960, y: 400, size: 120, label: 'A label much wider than its icon', labelSize: 48, scale: 2, at: 0, enter: 'none' })).find((e) => e.kind === 'icon')?.box; + // A highlight in each code step is where that step's line is. + out.focus = plan((s) => s.code([ + { code: Array.from({ length: 12 }, (_, i) => `const line${i} = ${i};`).join('\n'), at: 0, highlight: [1] }, + { code: 'done();', at: 4, highlight: [1] }, + ], { reveal: 'none', size: 32 })).filter((e) => e.kind === 'focus').map((e) => e.box.y); + return out; + }); + assert.equal(r.random.calls, 3); + r.random.planned.forEach((x, i) => assert.ok(Math.abs(x - r.random.drawn[i]) < 1e-6, JSON.stringify(r.random))); + assert.deepEqual([r.leaving.w, r.leaving.h].map(Math.round), [400, 200]); + assert.deepEqual([r.stroked.x, r.stroked.y, r.stroked.w, r.stroked.h].map(Math.round), [960, 540, 400, 600]); + assert.ok(r.label.w > 1300, `label width ${r.label.w}`); + // The tip at (960, 400) turns by about 27 degrees: its miter reaches about 107 pixels higher. + assert.ok(r.miter.y - r.miter.h / 2 < 300, JSON.stringify(r.miter)); + assert.deepEqual(r.hidden.planned.map(Math.round), r.hidden.drawn.map(Math.round)); + for (const [k, { box, ink }] of Object.entries(r.strokes)) { + assert.ok(box.x0 <= ink.x0 + 0.5 && box.y0 <= ink.y0 + 0.5 && box.x1 >= ink.x1 - 0.5 && box.y1 >= ink.y1 - 0.5, `${k}: ${JSON.stringify({ box, ink })}`); + } + assert.equal(Math.round(r.groupLeaving.w), 1100); + // The 12-line window starts much higher than the one-line window. + assert.ok(r.focus[0] < r.focus[1] - 200, JSON.stringify(r.focus)); + }); + } finally { fs.rmSync(dir, { recursive: true, force: true }); } +}); + +test('large type tracking stays with its own text', { skip: !chrome }, async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'explainroo-tracking-')); + try { + fs.writeFileSync(path.join(dir, 'video.json'), JSON.stringify({ theme: 'clean', captions: false, watermark: false, music: false, sfx: false })); + fs.writeFileSync(path.join(dir, 'script.md'), '## demo {min=8}\n'); + fs.writeFileSync(path.join(dir, 'scenes.js'), 'export default { demo(s) {} };'); + const project = loadProject(dir); + await withEngine(project, buildTimeline(project, {}), {}, async (page) => { + const [alone, after] = await page.evaluate(async () => { + const { Engine, fetchState } = await import('/engine/runtime.js'); + const state = await fetchState(); + const box = (withTitle) => { + let b; + const engine = new Engine({ canvas: document.createElement('canvas'), state, scenes: { demo(s) { + // A hidden title is still laid out; its tracking must not reach the box. + if (withTitle) s.title('A big hidden title', { at: 6 }); + b = s.box('Some words that wrap in a narrow box', { x: 960, y: 700, w: 420, size: 40, at: 0, enter: 'none' }); + } }, icons: {}, images: new Map() }); + engine.renderFrame(1); + return b.h; + }; + return [box(false), box(true)]; + }); + assert.equal(after, alone); + }); + } finally { fs.rmSync(dir, { recursive: true, force: true }); } +}); diff --git a/test/clips-browser.test.js b/test/clips-browser.test.js index 1b93b8a..fb60ce1 100644 --- a/test/clips-browser.test.js +++ b/test/clips-browser.test.js @@ -93,3 +93,33 @@ test('stills use exact clip frames in any order and reuse extracted frames', { s }); } finally { fs.rmSync(dir, { recursive: true, force: true }); } }); + +test('motion blur keeps a clip frame that does not change during the shutter', { skip: !chrome }, async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'explainroo-clip-blur-')); + try { + fs.mkdirSync(path.join(dir, 'assets')); + await ffmpeg(['-loglevel', 'error', '-f', 'lavfi', '-i', 'color=red:s=160x90:r=30:d=1', '-f', 'lavfi', '-i', 'color=lime:s=160x90:r=30:d=1', '-filter_complex', '[0:v][1:v]concat=n=2:v=1:a=0[v]', '-map', '[v]', '-c:v', 'libx264', '-threads', '1', path.join(dir, 'assets/clip.mp4')]); + fs.writeFileSync(path.join(dir, 'video.json'), JSON.stringify({ theme: 'clean', camera: 'still', captions: false, watermark: false, music: false, sfx: false })); + fs.writeFileSync(path.join(dir, 'script.md'), '## demo\n'); + // The card keeps the frame moving, so it is drawn with motion blur. + fs.writeFileSync(path.join(dir, 'scenes.js'), `export default { demo(s) { + s.clip('assets/clip.mp4', { w: 700, at: 0, enter: 'none', radius: 0, shadow: false }); + s.box('Moving', { x: 300, y: 200, w: 200, h: 100, at: 0.9, dur: 2, enter: 'rise' }); + } };`); + const project = loadProject(dir); + const timeline = { pace: 1, duration: 3, frames: 90, fps: 30, width: 1920, height: 1080, transitionSeconds: 0, + scenes: [{ id: 'demo', index: 0, start: 0, dur: 3, words: [], marks: {}, voice: null, lead: 0, transition: 'none' }] }; + await withEngine(project, timeline, { scale: 0.25 }, async (page) => { + const [sharp, blurred] = await page.evaluate(async () => { + const c = document.querySelector('#c'); + const pixel = () => [...c.getContext('2d').getImageData(c.width / 2, c.height / 2, 1, 1).data].slice(0, 3); + await window.explainroo.frames([45], 0.9, 1); + const a = pixel(); + await window.explainroo.frames([45], 0.9, 6); + return [a, pixel()]; + }); + assert.ok(sharp[1] > 240 && sharp[0] < 15, JSON.stringify(sharp)); + assert.deepEqual(blurred, sharp); + }); + } finally { fs.rmSync(dir, { recursive: true, force: true }); } +}); diff --git a/test/director.test.js b/test/director.test.js new file mode 100644 index 0000000..928ff6b --- /dev/null +++ b/test/director.test.js @@ -0,0 +1,245 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { planCamera, cameraAt, cameraMoving, compose, continues, frameRect, viewRect } from '../engine/director.js'; + +const frame = { W: 1920, H: 1080 }; +const view = { x: 134, y: 76, w: 1652, h: 928 }; +const el = (kind, x, y, w, h, at, out = Infinity) => ({ kind, at, out, box: { x: x + w / 2, y: y + h / 2, w, h } }); + +// Every card or label is fully inside the frame or fully outside it. +function cuts(shot, elements, margin = 1) { + const r = frameRect(shot, view, frame); + return elements.filter((e) => { + if (e.kind === 'arrow' || e.kind === 'line') return false; + const b = { x0: e.box.x - e.box.w / 2, x1: e.box.x + e.box.w / 2, y0: e.box.y - e.box.h / 2, y1: e.box.y + e.box.h / 2 }; + const outside = b.x1 <= r.x0 + margin || b.x0 >= r.x1 - margin || b.y1 <= r.y0 + margin || b.y0 >= r.y1 - margin; + const inside = b.x0 >= r.x0 - margin && b.x1 <= r.x1 + margin && b.y0 >= r.y0 - margin && b.y1 <= r.y1 + margin; + return !inside && !outside; + }); +} + +const diagram = [ + el('text', 760, 90, 400, 90, 0.1), + el('shape', 200, 450, 260, 140, 1.5), + el('arrow', 470, 500, 300, 40, 2.6), + el('shape', 780, 450, 260, 140, 2.8), + el('arrow', 1050, 500, 300, 40, 4.4), + el('shape', 1360, 450, 260, 140, 4.6), + el('text', 1360, 640, 260, 40, 6.2), +]; + +test('no elements means no camera', () => { + assert.equal(planCamera([], { dur: 5, view, frame }), null); + assert.equal(planCamera([el('shape', 0, 0, 1920, 1080, 0)], { dur: 5, view, frame }), null); +}); + +// Zoom direction changes (in after out, or out after in) less than `gap` +// seconds apart. +function reversals(plan, gap = 5) { + let prev = null; + let n = 0; + for (const m of plan.moves) { + const dz = Math.log(m.to.z / m.from.z); + if (Math.abs(dz) < 0.1) continue; + if (prev && Math.sign(dz) !== prev.sign && m.start - prev.start < gap) n++; + prev = { sign: Math.sign(dz), start: m.start }; + } + return n; +} + +test('a scene that starts small starts close and widens as the picture grows', () => { + const row = [0, 1, 2, 3].map((i) => el('shape', 260 + i * 380, 470, 220, 140, 0.5 + i * 3)); + const plan = planCamera(row, { dur: 14, view, frame }); + assert.ok(plan.start.z > 1.3, 'the first shot is close'); + for (const m of plan.moves) assert.ok(m.to.z <= m.from.z + 1e-9, `no zooming in again at ${m.start}`); + const end = frameRect(cameraAt(plan, 13.9), view, frame); + for (const e of row) assert.ok(e.box.x - e.box.w / 2 >= end.x0 && e.box.x + e.box.w / 2 <= end.x1, 'the end shows everything'); +}); + +test('something new that already fits the shot needs no move', () => { + // A large chart, then a line and a label drawn inside it. + const chart = [ + el('shape', 260, 160, 1400, 760, 0.2), + el('line', 560, 540, 800, 0, 3), + el('text', 900, 600, 160, 40, 6.5), + ]; + const plan = planCamera(chart, { dur: 11, view, frame }); + assert.equal(plan.moves.length, 0); +}); + +test('the camera pushes in on something small when there is time, not again right after, and pulls back at the end', () => { + const scene = [ + el('shape', 200, 200, 600, 300, 0.2), + el('shape', 1100, 200, 600, 300, 0.2), + el('text', 1250, 700, 300, 50, 3), + el('text', 300, 700, 300, 50, 6.5), + ]; + const plan = planCamera(scene, { dur: 13, view, frame, voiceEnd: 11 }); + const pushes = plan.moves.filter((m) => m.to.z > m.from.z * 1.1); + assert.equal(pushes.length, 1, 'one push in'); + assert.ok(Math.abs(pushes[0].start - 2.7) < 1e-6, 'on the first note'); + const end = frameRect(cameraAt(plan, 12.9), view, frame); + for (const e of scene) assert.ok(e.box.x - e.box.w / 2 >= end.x0 - 0.5 && e.box.x + e.box.w / 2 <= end.x1 + 0.5, 'the end shows everything'); +}); + +test('the camera does not keep zooming in and out', () => { + // New things every two seconds, far apart from each other. + const spots = [[200, 200], [1500, 800], [300, 820], [1450, 180], [860, 500], [200, 500], [1500, 500]]; + const scene = spots.map(([x, y], i) => el('shape', x - 90, y - 50, 180, 100, 0.2 + i * 2)); + const plan = planCamera(scene, { dur: 16, view, frame, voiceEnd: 14 }); + for (let i = 1; i < plan.moves.length; i++) assert.ok(plan.moves[i].start - plan.moves[i - 1].start >= 2.2 - 1e-9, 'moves are spaced out'); + assert.ok(reversals(plan) <= 1, `reversals: ${reversals(plan)}`); + assert.ok(plan.moves.length <= 5, `moves: ${plan.moves.length}`); +}); + +test('no shot cuts a card or label in half while it holds, drift included', () => { + const plan = planCamera(diagram, { dur: 12, view, frame, voiceEnd: 9 }); + const holds = [0.05, ...plan.moves.map((m) => m.start + m.dur + 0.01), 11.9]; + for (const t of holds) { + const shot = cameraAt(plan, t); + const visible = diagram.filter((e) => e.at <= t && e.out > t); + assert.deepEqual(cuts(shot, visible).map((e) => e.kind), [], `t=${t}`); + } +}); + +test('the camera is continuous and only reports real moves as moving', () => { + const plan = planCamera(diagram, { dur: 12, view, frame, voiceEnd: 9 }); + let prev = cameraAt(plan, 0); + for (let t = 0.01; t < 12; t += 0.01) { + const c = cameraAt(plan, t); + assert.ok(Math.abs(c.x - prev.x) < 25 && Math.abs(c.y - prev.y) < 25 && Math.abs(Math.log(c.z / prev.z)) < 0.03, `jump at ${t.toFixed(2)}`); + prev = c; + } + const m = plan.moves[0]; + assert.equal(cameraMoving(plan, m.start + m.dur / 2), true); + assert.equal(cameraMoving(plan, m.start - 0.01), false); +}); + +test('a close-up leaves a neighbour out entirely or takes it in, and falls back to the full frame', () => { + const row = [0, 1, 2, 3, 4].map((i) => el('shape', 805 + i * 205, 335, 190, 130, 3 + i * 0.2)); + const music = el('icon', 1050, 700, 120, 120, 6.7); + const shot = compose([music], [...row, music], view, frame); + assert.ok(shot.z > 1.2); + assert.deepEqual(cuts(shot, row), []); + // A neighbour hugging the subject on every side cannot be avoided. + const subject = el('shape', 900, 500, 120, 80, 1); + const ring = [el('text', 700, 380, 600, 60, 0), el('text', 700, 640, 600, 60, 0), el('text', 560, 380, 60, 320, 0), el('text', 1300, 380, 60, 320, 0)]; + const tight = compose([subject], [subject, ...ring], view, frame); + assert.deepEqual(cuts(tight, ring), []); + const v = viewRect(tight, view); + assert.ok(subject.box.x - subject.box.w / 2 >= v.x0 && subject.box.x + subject.box.w / 2 <= v.x1); +}); + +test('a scene continues the one before it when its first frame is that scene\'s last picture', () => { + const a = [el('shape', 200, 450, 260, 140, 0), el('shape', 780, 450, 260, 140, 1), el('text', 780, 640, 260, 40, 2)]; + const b = [el('shape', 200, 450, 260, 140, -1), el('shape', 780, 450, 260, 140, -1), el('text', 780, 640, 260, 40, -1), el('shape', 1360, 450, 260, 140, 2)]; + const everything = { x0: 0, y0: 0, x1: 1920, y1: 1080 }; + assert.equal(continues(a, b, everything), true); + // Not when A ends on something B leaves out. + assert.equal(continues(a, b.filter((e) => e.kind !== 'text'), everything), false); + // Not when A's last close-up shows little of what B carries. + const closeUp = { x0: 760, y0: 620, x1: 1060, y1: 700 }; + assert.equal(continues(a, b, closeUp), false); + // Not when B draws its own picture. + assert.equal(continues(a, [el('shape', 100, 100, 200, 100, -1), el('shape', 400, 100, 200, 100, -1)], everything), false); +}); + +test('a continued scene starts where the last camera stopped and holds through the dissolve', () => { + const from = { x: 700, y: 520, z: 1.4 }; + const plan = planCamera(diagram, { dur: 12, view, frame, from, still: 0.6 }); + assert.deepEqual(plan.start, from); + assert.ok(plan.moves.every((m) => m.start >= 0.6)); + const c = cameraAt(plan, 0); + assert.ok(Math.abs(c.x - from.x) < 1e-9 && Math.abs(c.z - from.z) < 1e-9); +}); + +test('pace makes the camera quicker', () => { + const slow = planCamera(diagram, { dur: 12, view, frame, pace: 1 }); + const fast = planCamera(diagram, { dur: 12, view, frame, pace: 1.5 }); + assert.ok(slow.moves.length && fast.moves.length); + assert.ok(fast.moves[0].dur < slow.moves[0].dur); +}); + +test('the opening shot cuts nothing that is on screen before the first move', () => { + // Two opening cards leave at 1s; the close-up on the card at 2s would cut them. + const opening = [el('shape', 160, 150, 1600, 100, 0, 1), el('shape', 700, 440, 400, 200, 0, 1), el('shape', 1300, 465, 200, 150, 2)]; + const plan = planCamera(opening, { dur: 8, view, frame }); + for (const t of [0.05, 0.5, 0.9]) { + const visible = opening.filter((e) => e.at <= t && e.out > t); + assert.deepEqual(cuts(cameraAt(plan, t), visible).map((e) => e.box.x), [], `t=${t}`); + } +}); + +test('every move ends before its scene does', () => { + const late = [el('shape', 200, 450, 260, 140, 0), el('shape', 1500, 700, 200, 120, 7.5)]; + const plan = planCamera(late, { dur: 8, view, frame }); + for (const m of plan.moves) assert.ok(m.start + m.dur <= 8 + 1e-9, `move ${m.start}+${m.dur}`); +}); + +test('a full-frame shot does not drift what fits the free part of the frame into the app overlays', () => { + const tall = { W: 1080, H: 1920 }; + const safe = { x: 119, y: 240, w: 745, h: 1027 }; + const usable = { x0: 119, y0: 240, x1: 864, y1: 1267 }; + const cards = [el('shape', 119, 703, 200, 100, 0), el('shape', 664, 703, 200, 100, 0.2)]; + const plan = planCamera(cards, { dur: 8, view: safe, frame: tall, usable }); + for (const t of [0.5, 4, 7.9]) { + const cam = cameraAt(plan, t); + const cx = safe.x + safe.w / 2; + for (const e of cards) { + const x0 = cx + (e.box.x - e.box.w / 2 - cam.x) * cam.z; + const x1 = cx + (e.box.x + e.box.w / 2 - cam.x) * cam.z; + assert.ok(x0 >= usable.x0 - 0.5 && x1 <= usable.x1 + 0.5, `t=${t}: ${x0.toFixed(1)}..${x1.toFixed(1)}`); + } + } +}); + +// Boxes given by their center, as the engine measures them. +const centered = (kind, x, y, w, h, at, out = Infinity, extra = {}) => ({ kind, box: { x, y, w, h }, at, out, ...extra }); +const whole = (shot, e, v = view) => { + const r = frameRect(shot, v, frame); + return e.box.x - e.box.w / 2 >= r.x0 - 0.5 && e.box.x + e.box.w / 2 <= r.x1 + 0.5 && e.box.y - e.box.h / 2 >= r.y0 - 0.5 && e.box.y + e.box.h / 2 <= r.y1 + 0.5; +}; + +test('a move that keeps something new whole is made however small it is', () => { + const wide = { x: 76, y: 76, w: 1768, h: 928 }; + const items = [centered('shape', 1525, 650, 500, 100, 0), centered('shape', 1000, 450, 550, 300, 3), centered('shape', 900, 300, 400, 100, 4.5), centered('shape', 1200, 900, 500, 200, 6)]; + const plan = planCamera(items, { dur: 14, view: wide, frame, voiceEnd: 12 }); + for (let t = 6.2; t < 13.9; t += 0.1) if (!cameraMoving(plan, t)) assert.ok(whole(cameraAt(plan, t), items[3], wide), `t=${t.toFixed(1)}`); +}); + +test('a focus region is framed whole, and an arrow that fits does not cancel it', () => { + const pan = [centered('code', 960, 540, 1600, 700, 0), centered('focus', 600, 540, 200, 80, 0, 3.9, { explicit: true }), centered('focus', 1050, 540, 200, 80, 4, Infinity, { explicit: true })]; + const p1 = planCamera(pan, { dur: 14, view, frame, voiceEnd: 12 }); + assert.ok(whole(cameraAt(p1, 6), pan[2])); + const arrow = [centered('shape', 960, 540, 1400, 700, 0), centered('focus', 960, 540, 200, 80, 3, Infinity, { explicit: true }), centered('arrow', 960, 700, 500, 20, 3.9)]; + const p2 = planCamera(arrow, { dur: 14, view, frame, voiceEnd: 12 }); + assert.ok(cameraAt(p2, 6).z > 1.5, 'the close-up happens'); +}); + +test('a push in is held for a while before the camera pulls back', () => { + const items = [centered('shape', 500, 350, 600, 300, 0), centered('shape', 1400, 350, 600, 300, 0), centered('text', 1400, 725, 300, 50, 8.5)]; + const plan = planCamera(items, { dur: 12, view, frame, voiceEnd: 10 }); + for (let i = 0; i < plan.moves.length; i++) { + const m = plan.moves[i]; + if (m.to.z <= m.from.z * 1.1) continue; + const next = plan.moves[i + 1]?.start ?? 12; + assert.ok(next - (m.start + m.dur) >= 1.5 - 1e-9, `held ${(next - m.start - m.dur).toFixed(2)}s`); + } +}); + +test('a large chart is the picture, and a label inside it gets no close-up', () => { + const items = [centered('chart', 960, 540, 1600, 800, 0), centered('line', 960, 540, 1400, 0, 3), centered('text', 960, 700, 160, 40, 6.5)]; + const plan = planCamera(items, { dur: 12, view, frame, voiceEnd: 10 }); + for (let t = 0.1; t < 11.9; t += 0.5) assert.ok(whole(cameraAt(plan, t), items[0]), `t=${t.toFixed(1)}`); +}); + +test('a push in makes way for a focus point that comes soon after', () => { + const items = [centered('shape', 500, 350, 600, 300, 0), centered('shape', 1400, 350, 600, 300, 0), centered('text', 500, 725, 300, 50, 3), centered('focus', 1400, 350, 200, 80, 5.5, Infinity, { explicit: true })]; + const plan = planCamera(items, { dur: 12, view, frame, voiceEnd: 10 }); + for (let i = 0; i < plan.moves.length; i++) { + const m = plan.moves[i]; + const next = plan.moves[i + 1]?.start ?? 12; + if (m.to.z > m.from.z * 1.1) assert.ok(next - (m.start + m.dur) >= 1.5 - 1e-9, `held ${(next - m.start - m.dur).toFixed(2)}s`); + } + assert.ok(whole(cameraAt(plan, 8), items[3]) && cameraAt(plan, 8).z > 1.5, 'the focus gets its close-up'); +}); diff --git a/test/math-browser.test.js b/test/math-browser.test.js index 3d24e08..2361273 100644 --- a/test/math-browser.test.js +++ b/test/math-browser.test.js @@ -108,7 +108,7 @@ test('math boxes, transformations, text checks and late-formula errors work in t assert.match(result.error.message, /\\frac\{1\}.*Missing argument/); for (const r of result.entrances) { assert.equal(r.changing, 3, r.theme); - assert.equal(r.dur, ['clean', 'midnight'].includes(r.theme) ? 0.55 : 0.9, r.theme); + assert.equal(r.dur, ['clean', 'midnight'].includes(r.theme) ? 0.75 : 0.9, r.theme); } }); });