Commit Graph
13 Commits
Author SHA1 Message Date
ZacharyZhang-NY bf1ebfb6fe Hold hardware live mode until BGRA presentation 2026-05-12 23:42:52 -04:00
ZacharyZhang-NY 316bf6f9a4 Split Servo live rendering modules 2026-05-12 23:38:13 -04:00
ZacharyZhang-NY 0db1caad69 Route web surface scrolls to Servo hit point 2026-05-12 23:29:40 -04:00
ZacharyZhang-NY b70fa71a9f T13: push display scale factor into Servo's hidpi so Retina pages lay out at logical CSS dimensions 2026-05-11 00:46:32 -04:00
ZacharyZhang-NY 6e2bd20a9b T10.6: drop RGBA payload from the wire when the hardware path publishes a surface 2026-05-10 23:52:12 -04:00
ZacharyZhang-NY a447d52262 T10.5: render IOSurface via gpui::surface(CVPixelBuffer) 2026-05-10 23:37:00 -04:00
ZacharyZhang-NY c4a6ea3c57 T10.4: import sidecar IOSurface mach port into a cached MTLTexture 2026-05-10 23:27:56 -04:00
ZacharyZhang-NY 07b9c9da01 T10.3: publish IOSurfaceHandle once per surface, current_surface_id per frame 2026-05-10 23:17:05 -04:00
ZacharyZhang-NY f7027e6ea5 Profile frame stages via sidecar histogram → ely::servo::perf 2026-05-10 22:30:57 -04:00
ZacharyZhang-NY 18fd20b577 Plumb the rendering context kind from ely_app env var to the sidecar
T10.1: with the vendored `HardwareOffscreenContext` (048c5df) and the
host-level kind dispatch (a7d3e89) in place, the sidecar binary
still ignored the rendering context kind — every spawn was wired to
the software path regardless of how the host process was built. This
commit threads the choice end-to-end:

  * `ely_servo_sidecar` learns a `--rendering-context [software|
    hardware]` flag on its `live` subcommand. `LiveArgs` carries
    the parsed `RenderingContextKind` (defaulting to `Software` so
    existing invocations stay bit-identical) and `live.rs::run_live`
    routes it through to `SoftwareServoHost::new_with_config_dir_and_kind`.
    Unknown values produce a typed
    `SidecarArgsError::InvalidRenderingContext`; a missing value
    after the flag produces the existing `MissingArgumentValue`.

  * `ely_app` reads `ELY_SERVO_RENDERING_CONTEXT` (with values
    `software` / `hardware`, case-insensitive) and, if set, appends
    `--rendering-context VALUE` to the sidecar command line.
    Unset or unrecognised values fall through to the sidecar's own
    software default — a stale env var or a typo never breaks the
    browser startup. The sidecar arg parser is the source of truth
    for legality of explicit values; the env helper only gates
    which values reach it.

  * Five new unit tests in `args::tests` pin the new parse paths:
    default-is-software, explicit-software, explicit-hardware,
    bogus-value-rejected, missing-value-rejected. Run via
    `cargo test -p ely_servo_host --features servo-engine --bin
    ely_servo_sidecar` and now hit alongside the five existing
    snapshot tests for 10 passes.

End-to-end perf expectation: with the sidecar binary built using
`--features servo-engine,hardware-render` and the env var set to
`hardware`, every spawned sidecar webview rasterises through the
real GPU adapter (via the vendored
`HardwareOffscreenContext`/surfman/CGL chain on macOS). The host
still reads back RGBA into a `Vec<u8>` for the existing pipe
protocol; the IOSurface zero-copy bridge that deletes that
read-back is T10.2–T10.5 in docs/t10-iosurface-plan.md and lands
in subsequent commits.

cargo test --bin ely_app: 120 passed, 0 failed, 2 ignored.
cargo test -p ely_servo_host --features servo-engine --bin ely_servo_sidecar: 10 passed.
cargo test -p ely_servo_host --features servo-engine --test sidecar: 9 passed.
cargo test -p ely_servo_host --features servo-engine,hardware-render --test hardware_rendering_context: 1 passed.
2026-05-10 21:27:12 -04:00
ZacharyZhang-NY 414ba3d158 Land a red TDD guard for live-frame texture re-upload (T10)
Every `WebSurfaceFrame::from_live_frame` today calls
`Arc::new(RenderImage::new([image::Frame::new(image_buffer)]))`
unconditionally — even when the underlying RGBA bytes are
byte-for-byte identical to the previous frame. At 60 fps on a 1080p
canvas that is roughly 960 MB/s of host-side cloning + a fresh
GPUI texture upload, and the roundtable agreed it is the next
material bottleneck after the file-system pipe (a80d039).

The contract this test pins is the cheapest invariant we can hold
against today's `SoftwareRenderingContext`: two `ServoLiveFrame`
inputs whose `rgba_bytes` are bit-identical must produce the same
underlying `Arc<RenderImage>` instance. Today they do not; the
ignored run confirms two distinct pointer values for back-to-back
identical inputs.

The fix has two recognised shapes. The interim shape lives entirely
in `WebSurfaceFrame::from_parts`: remember the previous frame's
bytes (hash or pointer-eq) and reuse the existing `Arc<RenderImage>`
when they match. The endgame shape removes the host-side image step
entirely — `OffscreenRenderingContext` + IOSurface — at which point
the assertion becomes meaningless and is replaced by a frame-time
budget. Whichever lands first, the fix commit MUST delete the
`#[ignore]` attribute outright; toggling its reason is a broken
contract.

To call `WebSurfaceFrame::from_live_frame` from a unit test without
spawning a real sidecar process, `ServoLiveFrame` gains a
`#[cfg(test)] pub(crate) fn for_test(...)` constructor that wraps
the existing private `from_parts` with realistic defaults. No
production path uses it.

cargo test --bin ely_app: 112 passed, 0 failed, 2 ignored.
cargo test --bin ely_app -- --ignored: 2 failed (expected RED:
T7 click pipeline + T10 texture re-upload).
2026-05-10 19:32:43 -04:00
ZacharyZhang-NY a80d0393e9 Drop the file system from the live frame pixel pipe
Every live frame was round-tripping through the local file system:
the sidecar called `fs::write(rgba_out, frame.rgba_bytes())` in
`poll_frame`, the JSON response carried `rgba_path`, and the main
process turned around and called `fs::read(rgba_path)` to lift the
bytes back into a `Vec<u8>`. At 1080p that is 8 MB of syscall +
memcpy + page cache traffic per frame; at 60 fps it dwarfs every
other cost in the pipeline and shows up as scroll/zoom jank the user
can feel before any other bottleneck.

Replace it with a same-pipe binary protocol. The sidecar writes the
JSON `LiveResponse` line as before, then writes the raw RGBA frame
bytes on the same stdout immediately after the trailing `\n`. The
client `read_line`s the JSON, parses `rgba_byte_count` from the
header, and `read_exact`s exactly that many bytes from the same
`BufReader<ChildStdout>` (the buffered reader drains its own buffer
before pulling from the child). No tmpfs directory, no
`fs::remove_dir_all` on drop, no `rgba_path` field, no per-frame
filename plumbing.

Boundary defence on the client side: the header's `rgba_byte_count`
is cross-checked against `width * height * 4` before any allocation
or `read_exact`. A buggy or compromised sidecar can no longer ask
the GPUI process to allocate an arbitrarily large buffer or park on
`read_exact` for a payload that will never arrive.

The sidecar process boundary stays exactly where it was; only the
pixel transport between the two processes changes. The `one-shot`
sidecar binary path (used by `tests/sidecar.rs` and PRD smoke tests)
still writes to its CLI-supplied `--rgba-out` path — those tests
were untouched and continue to pass 9/9.

cargo test --bin ely_app: 112 passed.
cargo test -p ely_servo_host --features servo-engine --test sidecar: 9 passed.

Follow-ups deferred to the profile step (T9):
  * the host still copies its rendered buffer to a transient
    `Vec<u8>` via `frame.rgba_bytes().to_vec()` before write_all;
    expose `&[u8]` straight to `write_all` once the profile shows
    that allocation in the top five.
  * `poll_frame` calls `snapshot` twice per iteration; harmless
    under the software renderer but worth folding into a single
    snapshot once we have numbers.
  * raw memcpy bandwidth is still ~480 MB/s at 60 fps 1080p; the
    GPU-side fix (T10: OffscreenRenderingContext + IOSurface
    zero-copy) is the next material change.
2026-05-10 19:04:34 -04:00
ZacharyZhang-NY 4c6d9c757a Add mouse hover tracking across the Servo input pipeline
Without continuous MouseMove events, Servo never updates hover state:
no cursor changes over links, no :hover CSS effects, no mouseenter
JavaScript events. The web page appears completely non-interactive.

Track mouse position from GPUI on_mouse_move through the full IPC
pipeline to Servo. Hover position is included with each ensure
request so Servo updates hover state at frame rate (~60fps).
2026-05-09 16:21:43 -04:00