Commit Graph
363 Commits
Author SHA1 Message Date
ZacharyZhang-NY 90c029eddb Move Servo IPC off UI thread
Root cause of the post-tab lag: the GPUI 16 ms timer was calling
`WebSurfaceRuntime::ensure_tab` and `tick` on the UI thread, and each
call did a synchronous `serde_json` write plus `read_line` against the
Servo sidecar over stdin/stdout. With even one visible tab, every
frame stalled on cross-process IPC.

Introduce `web_surface_worker.rs` — a per-profile worker thread that
owns the `ServoLiveClient`, drains a coalescing request queue
(latest Ensure/Poll per tab wins, no unbounded growth), and ships
results back through a `std::sync::mpsc` channel. `WebSurfaceRuntime`
now submits work non-blockingly and drains responses in `tick`; the
UI thread never blocks on the sidecar.

Adjacent in-flight cleanup riding along: hardware IOSurface
rendering-context completion (sidecar `live_protocol`,
`hardware_rendering_context`, GPUI BGRA surface shader), CSS viewport
size + device pixel ratio plumbing into `ServoLiveFrame`, and the
Send opt-ins for `CVPixelBuffer`-bearing types so frames can cross
the thread boundary.
2026-05-15 16:41:40 -04:00
ZacharyZhang-NY f4c650c4d8 Fix Servo IOSurface orientation and resize identity 2026-05-13 10:21:50 -04:00
ZacharyZhang-NY 8238526cc1 Wait for complete live-site smoke frames 2026-05-13 04:40:14 -04:00
ZacharyZhang-NY 2302a53b39 Cover web surface live scrolling 2026-05-13 04:31:58 -04:00
ZacharyZhang-NY d4046ca4e7 Default Servo live rendering to hardware on macOS 2026-05-13 04:14:18 -04:00
ZacharyZhang-NY a743435ea1 Transfer Servo IOSurface ports over Mach 2026-05-13 04:05:07 -04:00
ZacharyZhang-NY 72f7b9761e Report Servo live IOSurface import failures 2026-05-13 03:30:55 -04:00
ZacharyZhang-NY b91f61c201 Fix live perf bench scroll page URL 2026-05-13 02:59:15 -04:00
ZacharyZhang-NY ac84a80d9e Close Servo live sessions for removed tabs 2026-05-13 02:41:54 -04:00
ZacharyZhang-NY 74731d4faa Keep loading until live content arrives 2026-05-13 02:18:48 -04:00
ZacharyZhang-NY 8fb1277ced Default Servo live rendering to software 2026-05-13 02:13:51 -04:00
ZacharyZhang-NY e7b62e2daf Scope Servo live runtimes by profile 2026-05-13 02:07:01 -04:00
ZacharyZhang-NY 05ce4afba4 Poll only visible web surfaces 2026-05-13 01:54:51 -04:00
ZacharyZhang-NY 6ce5df83a8 Skip hardware live readback 2026-05-13 01:41:58 -04:00
ZacharyZhang-NY c19e6923c3 Clean gpui harness clippy debt 2026-05-13 01:26:52 -04:00
ZacharyZhang-NY 5bdb2871ad Invert wheel deltas for Servo scroll 2026-05-13 01:20:10 -04:00
ZacharyZhang-NY 1325f043b4 Commit Servo loaded URLs to tabs 2026-05-13 01:09:39 -04:00
ZacharyZhang-NY 4cb7ff089e Wire topbar tab history navigation 2026-05-13 00:49:52 -04:00
ZacharyZhang-NY 8cd5e969dc Guard hardware surface selectors 2026-05-13 00:43:13 -04:00
ZacharyZhang-NY fec5a9b475 Move idle archiving into tab lifecycle 2026-05-13 00:38:04 -04:00
ZacharyZhang-NY 8308eae6d2 Split sidecar args tests 2026-05-13 00:34:03 -04:00
ZacharyZhang-NY 777df012e3 Validate Servo hardware frame surfaces 2026-05-13 00:29:26 -04:00
ZacharyZhang-NY 15d773b28e Prefer Cargo hardware sidecar only for workspace builds 2026-05-13 00:25:48 -04:00
ZacharyZhang-NY 9313c76743 Default Servo live rendering to hardware on macOS 2026-05-13 00:24:39 -04:00
ZacharyZhang-NY 05a7a0d67f Present Servo BGRA hardware surfaces 2026-05-13 00:22:00 -04:00
ZacharyZhang-NY bf1ebfb6fe Hold hardware live mode until BGRA presentation 2026-05-12 23:42:52 -04:00
ZacharyZhang-NY 316bf6f9a4 Split Servo live rendering modules 2026-05-12 23:38:13 -04:00
ZacharyZhang-NY 0db1caad69 Route web surface scrolls to Servo hit point 2026-05-12 23:29:40 -04:00
ZacharyZhang-NY 6f279bf3e6 T17: pixel-content smoke tests — red/blue data URLs render their actual colour through Servo 2026-05-11 01:03:52 -04:00
ZacharyZhang-NY 42b4e8c87b T15: paint barrier — pump event loop until framebuffer is consistent
`webview.paint()` dispatches a render command to Servo's paint thread
asynchronously, so the subsequent `read_to_image()` raced the paint
thread and reliably returned cleared-white pixels on data: URLs (T10.8).
Clear `has_pending_frame` before dispatching paint, then spin the Servo
event loop until `notify_new_frame_ready` re-arms it or 32 ms elapse
(override via `ELY_PAINT_BARRIER_MS`). Bench: software path now ships
real RGBA bytes for 60 frames instead of all-white.
2026-05-11 00:57:35 -04:00
ZacharyZhang-NY 5269853ec8 T14: disable T10.5 surface() path; GPUI 0.2.2 Surface asserts NV12 YUV 2026-05-11 00:53:18 -04:00
ZacharyZhang-NY b70fa71a9f T13: push display scale factor into Servo's hidpi so Retina pages lay out at logical CSS dimensions 2026-05-11 00:46:32 -04:00
ZacharyZhang-NY 60a3b9b050 T12: disarm visible-content gate on software path after first real frame 2026-05-11 00:35:11 -04:00
ZacharyZhang-NY 5d8734aefc T11: swap R↔B between Servo RGBA and GPUI BGRA so colours stop inverting 2026-05-11 00:27:09 -04:00
ZacharyZhang-NY d570b84c9c T10.7: bypass visible-content gate on hardware path (196ms → 6ms per frame) 2026-05-11 00:05:04 -04:00
ZacharyZhang-NY 6e2bd20a9b T10.6: drop RGBA payload from the wire when the hardware path publishes a surface 2026-05-10 23:52:12 -04:00
ZacharyZhang-NY a447d52262 T10.5: render IOSurface via gpui::surface(CVPixelBuffer) 2026-05-10 23:37:00 -04:00
ZacharyZhang-NY c4a6ea3c57 T10.4: import sidecar IOSurface mach port into a cached MTLTexture 2026-05-10 23:27:56 -04:00
ZacharyZhang-NY 07b9c9da01 T10.3: publish IOSurfaceHandle once per surface, current_surface_id per frame 2026-05-10 23:17:05 -04:00
ZacharyZhang-NY bb0bd0032f T10.2: extract IOSurface mach port from the hardware surface on macOS 2026-05-10 23:01:34 -04:00
ZacharyZhang-NY 1e38ace997 Add manual live_perf_bench driver for software vs hardware comparison 2026-05-10 22:56:06 -04:00
ZacharyZhang-NY 71ddadb482 Wire HardwareOffscreenContext::connection() so Servo's painter constructs 2026-05-10 22:56:03 -04:00
ZacharyZhang-NY 320da3ddb5 Make frame_perf total a real end-to-end timing and right-size buckets 2026-05-10 22:38:12 -04:00
ZacharyZhang-NY f7027e6ea5 Profile frame stages via sidecar histogram → ely::servo::perf 2026-05-10 22:30:57 -04:00
ZacharyZhang-NY 18fd20b577 Plumb the rendering context kind from ely_app env var to the sidecar
T10.1: with the vendored `HardwareOffscreenContext` (048c5df) and the
host-level kind dispatch (a7d3e89) in place, the sidecar binary
still ignored the rendering context kind — every spawn was wired to
the software path regardless of how the host process was built. This
commit threads the choice end-to-end:

  * `ely_servo_sidecar` learns a `--rendering-context [software|
    hardware]` flag on its `live` subcommand. `LiveArgs` carries
    the parsed `RenderingContextKind` (defaulting to `Software` so
    existing invocations stay bit-identical) and `live.rs::run_live`
    routes it through to `SoftwareServoHost::new_with_config_dir_and_kind`.
    Unknown values produce a typed
    `SidecarArgsError::InvalidRenderingContext`; a missing value
    after the flag produces the existing `MissingArgumentValue`.

  * `ely_app` reads `ELY_SERVO_RENDERING_CONTEXT` (with values
    `software` / `hardware`, case-insensitive) and, if set, appends
    `--rendering-context VALUE` to the sidecar command line.
    Unset or unrecognised values fall through to the sidecar's own
    software default — a stale env var or a typo never breaks the
    browser startup. The sidecar arg parser is the source of truth
    for legality of explicit values; the env helper only gates
    which values reach it.

  * Five new unit tests in `args::tests` pin the new parse paths:
    default-is-software, explicit-software, explicit-hardware,
    bogus-value-rejected, missing-value-rejected. Run via
    `cargo test -p ely_servo_host --features servo-engine --bin
    ely_servo_sidecar` and now hit alongside the five existing
    snapshot tests for 10 passes.

End-to-end perf expectation: with the sidecar binary built using
`--features servo-engine,hardware-render` and the env var set to
`hardware`, every spawned sidecar webview rasterises through the
real GPU adapter (via the vendored
`HardwareOffscreenContext`/surfman/CGL chain on macOS). The host
still reads back RGBA into a `Vec<u8>` for the existing pipe
protocol; the IOSurface zero-copy bridge that deletes that
read-back is T10.2–T10.5 in docs/t10-iosurface-plan.md and lands
in subsequent commits.

cargo test --bin ely_app: 120 passed, 0 failed, 2 ignored.
cargo test -p ely_servo_host --features servo-engine --bin ely_servo_sidecar: 10 passed.
cargo test -p ely_servo_host --features servo-engine --test sidecar: 9 passed.
cargo test -p ely_servo_host --features servo-engine,hardware-render --test hardware_rendering_context: 1 passed.
2026-05-10 21:27:12 -04:00
ZacharyZhang-NY a7d3e896bb Wire the vendored hardware context into SoftwareServoHost
`HardwareOffscreenContext` was vendored in 048c5df but the host's
per-webview `new_rendering_context` still hard-wired
`servo::SoftwareRenderingContext`. This commit threads a
`RenderingContextKind` enum through the host so existing call sites
keep their software path, and new callers can opt into the hardware
path through `SoftwareServoHost::new_with_config_dir_and_kind(...)`.

Three changes, kept tightly scoped:

  * `runtime.rs` gains a public `RenderingContextKind { Software,
    Hardware }` enum and a new constructor that takes it. The
    existing `new` and `new_with_config_dir` keep their signatures
    and default to `Software`, so the sidecar binary and the
    integration tests pick up zero behavioural change. The
    private `new_rendering_context` moves from a free function to a
    `&self` method so it can read `self.rendering_context_kind` and
    dispatch — `Software` constructs `SoftwareRenderingContext` as
    before, `Hardware` constructs the vendored
    `HardwareOffscreenContext`. When the `hardware-render` feature
    isn't compiled in, the `Hardware` arm returns
    `ServoHostError::HardwareRenderUnavailable` instead of silently
    falling back; the new constructor also rejects the request
    up-front before touching the global Servo runtime flag.

  * `error.rs` gains `HardwareRenderUnavailable` so the wrong-feature
    path is a typed error, not a panic.

  * `lib.rs` exports `RenderingContextKind` alongside
    `SoftwareServoHost` so downstream code (next commit will be the
    sidecar's `--rendering-context` CLI flag and the live.rs
    plumbing) can name the variant directly.

This is purely an extension point — no existing call path changes,
no existing test asserts on the new enum. The next commit will add
the sidecar CLI flag and wire `live.rs::run_live` to pass the kind
through to the host so users can pick the path at startup. The
follow-up commits then extract the IOSurface from the hardware
surfman surface and bridge it across the IPC channel to GPUI's
Metal renderer, deleting the host-side `Vec<u8>` from the per-frame
hot path entirely (full plan in docs/t10-iosurface-plan.md).

cargo test -p ely_servo_host --features servo-engine --lib: 2 passed.
cargo test -p ely_servo_host --features servo-engine --test sidecar: 9 passed.
cargo test -p ely_servo_host --features servo-engine,hardware-render
  --test hardware_rendering_context: 1 passed.
cargo test --bin ely_app: 120 passed, 0 failed, 2 ignored.
2026-05-10 20:55:35 -04:00
ZacharyZhang-NY 048c5dfecd Vendor a headless hardware RenderingContext for Servo
The first concrete step toward the T10 IOSurface zero-copy path
(plan in docs/t10-iosurface-plan.md). Before this commit the sidecar
process could only use `SoftwareRenderingContext` — CPU rasterising
plus an 8 MB RGBA readback per 1080p frame is most of where scroll
latency comes from after the file pipe (a80d039), Vec clone
(e02c0fd), texture dedup (7f3b8b4), and hash swap (3f184ee) have
all landed.

The blocker is purely architectural: `servo-paint-api 0.1` exposes
`OffscreenRenderingContext` only as a child of
`WindowRenderingContext`, which requires a `DisplayHandle +
WindowHandle`. The sidecar has no window. The underlying
`SurfmanRenderingContext` glue *can* drive a hardware adapter
against a `SurfaceType::Generic` offscreen surface, but its
constructor is private. Until the upstream PR lands, this commit
vendors the minimal slice of that glue into `ely_servo_host`:

  * `HardwareOffscreenContext::new(size)` uses
    `Connection::new() → create_adapter()` (real GPU, not the
    software adapter) and a `SurfaceType::Generic` offscreen
    surface. On macOS the surfman CGL backend backs that surface
    with an `IOSurface` — exactly the thing the IOSurface bridge
    in subsequent commits will reach for.
  * Implements `servo::RenderingContext` so it slots into
    `ServoBuilder::rendering_context` wherever the existing
    `SoftwareRenderingContext` does, with no other Servo-side
    knowledge.
  * Scope deliberately narrow: only the methods Servo's headless
    readback actually calls. `create_texture` /
    `destroy_texture` / `connection` / `refresh_driver` fall
    through to the trait's `None` defaults. `read_to_image` inlines
    the upstream `Framebuffer::read_framebuffer_to_image` helper so
    we don't reach for a private helper that may change shape.
  * No `RawWindowHandle` and no `RefreshDriver` — both belong to
    paths the headless sidecar doesn't take.

The whole thing is feature-gated on `hardware-render`. Default
builds compile zero new lines; the additional surfman / gleam /
glow / euclid / image / log deps are all `optional = true`. Sidecar
binary still uses `SoftwareServoHost` until a follow-up commit
threads the new context in behind a CLI flag.

A smoke test at `tests/hardware_rendering_context.rs` constructs
the context. On a host with a real GPU it returns `Ok`; on a no-GPU
CI host it logs the surfman cause and reports `ok` rather than
failing the suite — the test is guarding the wiring, not the
hardware availability. On this Mac it constructs cleanly.

The `expect_used` / `unwrap_used` workspace lints are honoured —
fallible reads return `None` instead of panicking, no `.expect()` /
`.unwrap()` survives in the vendored body. The two `unsafe` blocks
(loading GL function pointers via surfman's `get_proc_address`) are
the same blocks upstream uses, with `#[expect(unsafe_code)]` to
override the workspace `unsafe_code = "deny"` lint locally.

cargo test --bin ely_app: 120 passed.
cargo test -p ely_servo_host --features servo-engine,hardware-render
  --test hardware_rendering_context: 1 passed (constructs cleanly).
Pre-existing `manages_real_servo_webview_lifecycle` failure on
servo.org is unrelated (reproduces on b8795bf without this change,
already documented in T7's commit history).
2026-05-10 20:50:59 -04:00
ZacharyZhang-NY 3f184ee941 Swap SipHash13 → AHash for the live-frame dedup key + record T10 plan
Two T10-flavoured changes in one commit, each independently ship-able
on its own:

1. `web_surface_frame::rgba_hash` switches from std's
   `DefaultHasher` (SipHash13, ~1.5 GB/s) to `ahash::AHasher`
   (~10 GB/s). At 1080p (8 MB per frame) the dedup key drops from
   ~5 ms to ~0.8 ms per cache-miss frame, returning roughly 25 % of
   the 16 ms scroll budget that was being spent hashing the
   newly-arrived RGBA payload. ahash was already in the dependency
   graph transitively via hashbrown, so this only adds a direct
   `ahash = "0.8"` line and one Cargo.lock entry.

2. `docs/t10-iosurface-plan.md` records the full architectural
   roadmap for the actual zero-copy path that supersedes the
   software-pipe pipeline: `OffscreenRenderingContext` against a
   hardware surfman adapter, IOSurface-backed surface on macOS,
   mach-port handoff to the GPUI process, MTLTexture import as an
   external sampler. The document explains why each currently
   shipped commit (`840255f`, `a80d039`, `e02c0fd`, `7f3b8b4`, plus
   this hash swap) is a stepping stone that eventually deletes
   itself once the IOSurface path lands, and names the upstream
   API gap in `servo-paint-api` that blocks step 2.

cargo test --bin ely_app: 120 passed, 0 failed, 2 ignored.
Hash collision probability remains ~1 in 2^64; AHash uses the same
keyspace as the previous SipHash13.
2026-05-10 20:25:01 -04:00
ZacharyZhang-NY 9711167560 T13 diagnostic: T7 red is a TestAppContext hit_test quirk, not a prod bug
Three new diagnostics narrow T7 to test-mode infrastructure:

1. `diagnose_t7_hitbox_reachability_heatmap` (ignored, --nocapture):
   builds the real ElyShell, navigates to https://example.com/, then
   sweeps a 6×6 grid across the full 1920×1080 window dispatching
   mouse_move at each cell and recording hover_point. Result: **0 of
   36 grid cells trigger hover_point** — input_overlay's listener
   misses every position in the window, not just the viewport center.

2. Same test then dispatches a single MouseDown at the viewport
   center and reads back `focus_handle.is_focused`. Result: **true**.
   The root div's `track_focus` MouseDown bubble handler fires on
   the very same event. GPUI dispatch IS working at the root hitbox
   (`.size_full()`).

3. `baseline_overlay_after_gpui_component_init_receives_click` and
   `baseline_overlay_with_input_state_construction_receives_click`:
   replicate `gpui_component::init` and `InputState::new` +
   `subscribe_in` in isolation. Both still pass — neither breaks
   hit_test for an occlude div.

Together these isolate the failure to *something `ElyShell::new`
configures that interacts badly with `TestAppContext`'s simulated
executor*, after every individual ingredient passes. The user has
already confirmed (in the previous round) that "click works" after
the layout fix in 840255f lands in the real binary; the T7 red
guard is reproducing a test-mode quirk, not a production regression.

The red guard stays in the suite as documentation. Its `#[ignore]`
reason now records the diagnostic outcome so the next reader doesn't
re-walk the same bisect. Two paths forward (recorded in the
attribute):
  (a) reproduce the bug outside TestAppContext and file upstream
  (b) route web canvas input through the root div + a viewport-bounds
      gate, which sidesteps hit_test for input_overlay entirely

(b) is structurally more complex than it looks because backdrops
(workspace disclosure, hidden-sidebar overlay) sit in z-order in
front of input_overlay and don't `stop_propagation` — implementing
a clean root-level fallback requires teasing apart the cases. Not
done in this commit.

cargo test --bin ely_app: 120 passed, 0 failed, 2 ignored.
cargo test --bin ely_app -- --ignored: 1 fail (T7 red) + 1 pass
(diagnostic, which just prints the heatmap and reports findings).
2026-05-10 20:13:19 -04:00
ZacharyZhang-NY e02c0fd502 Drop the per-frame to_vec() clone in the sidecar pixel writer
`LiveOutcome::frame` was carrying the rendered bytes as a fresh
`Vec<u8>` cloned out of `RenderedFrame::rgba_bytes()`. At 60 fps
on a 1080p canvas that was an extra 8 MB allocation + memcpy per
frame on top of the clone `host.last_rendered_frame()` already
paid for. `LiveOutcome` now carries the owned `RenderedFrame`
directly, and `write_outcome` writes its `rgba_bytes()` slice
straight to stdout — same single-clone cost as the host already
incurred, no second allocation.

Small Karpathy-style follow-up from the T8 review ("the host
still copies its rendered buffer to a transient Vec<u8> before
write_all; expose &[u8] straight to write_all once the profile
shows that allocation in the top five"). Profile data is still
deferred (T9 was marked vibe-benchmarking until T10 lands a real
spec), but removing the cheap clone is structurally cleaner and
makes the dispatch path one allocation lighter regardless.

cargo test --bin ely_app: 118 passed, 0 failed, 1 ignored.
cargo test -p ely_servo_host --features servo-engine --test sidecar: 9 passed.
2026-05-10 19:54:14 -04:00