hcie-rust-v3.05/hcie-iced-app/implementation_plan_4k_opus.md

Iced GUI 4K Canvas GPU Acceleration Plan

Problem Analysis

The Iced GUI draws at ~10 FPS on 4K (3840×2160) vs ~50 FPS on 800×600. Your logs show:

Phase	Cost per 4K frame
render_composite_region (engine)	0.1ms ✅
composite_raw dirty-region copy	0.0ms ✅
ImageHandle::from_rgba (33MB Bytes::copy_from_slice)	3.8–5.2ms 🟡
view() widget tree	10–12ms 🟡
Iced wgpu layout + tessellation + present	~70–90ms 🔴
Total inter-frame	80–100ms

The engine itself is blazing fast (sub-1ms). The bottleneck is three-fold:

1. Full 33MB CPU→GPU upload every frame — ImageHandle::from_rgba(Bytes::copy_from_slice(&doc.composite_raw)) copies all 33MB even when only a 41×51 region changed
2. Iced's Canvas widget re-tessellates — the draw_image path goes through Iced's image pipeline which re-uploads the entire texture
3. No partial GPU texture update — unlike egui's tex.set_partial(), Iced's ImageHandle has no sub-region update API

How egui solves this (reference)

egui uses tex.set_partial([x0, y0], region_image, options) in render.rs — uploading only the dirty region (e.g., 41×51 = 8KB instead of 33MB). This is ~4000x less data per stroke event.

Proposed Changes

Phase 1: iced::widget::shader — Direct wgpu Texture Pipeline (Primary Fix)

Replace the Canvas widget's draw_image(ImageHandle) approach with a custom iced::widget::shader::Program that manages its own wgpu::Texture and uses queue.write_texture() for partial sub-region updates.

Important

This is the critical path — it eliminates the 33MB CPU→GPU copy bottleneck and the Iced tessellation overhead for the canvas area. Expected improvement: ~70ms → ~1ms per frame for dirty-region updates.

[NEW] shader_canvas.rs

Custom iced::widget::shader::Program implementation:

• CompositeShaderPrimitive — carries dirty region [x0, y0, x1, y1] + pixel data for that region only
• CompositeShaderPipeline — holds:
  • wgpu::Texture (RGBA8, 3840×2160, created once, resized only on canvas resize)
  • wgpu::BindGroup for the texture + sampler
  • wgpu::RenderPipeline with a simple textured-quad vertex/fragment shader
  • Uniform buffer for zoom/pan transform matrix
• prepare() — calls queue.write_texture() with only the dirty sub-region (e.g., 41×51×4 = 8KB)
• render() — single draw call: 6 vertices (textured quad), no tessellation

Frame cost estimate:
  queue.write_texture(41×51 region) ≈ 0.01ms
  Draw 1 quad                       ≈ 0.01ms
  Total                             ≈ 0.02ms (vs current 75ms)

[NEW] canvas.wgsl

Simple WGSL shader:

• Vertex shader: transforms quad vertices by zoom/pan uniforms
• Fragment shader: samples the composite texture with nearest-neighbor filtering

[MODIFY] mod.rs

• Replace CanvasProgram (which uses canvas::Program → draw_image(ImageHandle)) with the new ShaderCanvasProgram (which uses shader::Program → queue.write_texture)
• The view() function returns iced::widget::shader(program) instead of iced::widget::canvas(program)
• Keep the overlay (selection rect, vector preview, crosshair) as a small transparent Canvas widget layered on top via iced::widget::Stack
• Keep all mouse event handling unchanged

[MODIFY] app.rs

• IcedDocument changes:
  • Remove composite_handle_cache: RefCell<Option<(u64, ImageHandle)>> — no longer needed
  • Add dirty_region: Option<[u32; 4]> — stores the last dirty rect from render_composite_region()
• refresh_composite_if_needed() changes:
  • Instead of bumping render_generation and letting view() rebuild ImageHandle, store the dirty region + the partial pixel data directly
  • The shader primitive reads this data in prepare() and uploads only the dirty sub-region
• Remove Bytes::copy_from_slice call — the shader reads from composite_raw directly

---

Phase 2: Checkerboard as Shader (Eliminate ~20K rects)

[MODIFY] shader_canvas.rs

Move the checkerboard from CPU-tessellated rectangles to the fragment shader:

// In fragment shader: procedural checkerboard
let checker = floor(uv / checker_size);
let is_light = (checker.x + checker.y) % 2.0 == 0.0;
let bg = select(dark_color, light_color, is_light);
let composite = textureSample(composite_tex, sampler, uv);
let final_color = mix(bg, composite, composite.a);

This replaces ~20K CPU-generated rectangles with zero geometry cost.

---

Phase 3: Multi-Layer Performance (Already Handled)

Note

The engine already handles multi-layer compositing efficiently via below_cache (composites layers below the active layer once at stroke start). The render_composite_region() returns a flat composite — the GPU only sees one texture regardless of layer count. No additional GPU work needed for multi-layer.

The primary bottleneck for multi-layer 4K documents is the engine-side below_cache computation at stroke start (~5-10ms for 10 layers at 4K), which is a one-time cost per stroke. During the stroke, only the active layer's dirty region is re-composited (~0.1ms).

---

Phase 4: Additional Optimizations

[MODIFY] Cargo.toml

Add wgpu feature to Iced:

iced = { version = "0.13", features = ["tokio", "image", "svg", "canvas", "wgpu"] }

[MODIFY] main.rs

Configure Iced wgpu settings for performance:

• Set present_mode to Mailbox (non-blocking vsync) instead of Fifo (blocking vsync)
• This eliminates the vsync wait that contributes ~16ms to the inter-frame time

[MODIFY] Cargo.toml

Add wgpu and bytemuck dependencies for the shader pipeline:

wgpu = "23"  # Match Iced 0.13's wgpu version
bytemuck = { version = "1", features = ["derive"] }

---

Architecture Diagram

graph TD
    A["Engine: stroke_to()"] -->|"dirty_bounds [x0,y0,x1,y1]"| B["render_composite_region()"]
    B -->|"0.1ms"| C["composite_raw partial copy"]
    C -->|"dirty region pixels only"| D["ShaderCanvasProgram::prepare()"]
    D -->|"queue.write_texture(sub-region)"| E["wgpu::Texture<br/>(persistent, 3840×2160)"]
    E -->|"1 draw call"| F["Fragment Shader:<br/>checkerboard + composite"]
    F --> G["GPU Present"]
    
    style A fill:#2d5a2d
    style E fill:#2d2d5a
    style F fill:#2d2d5a
    style G fill:#5a2d2d

Expected Performance

Metric	Current (Canvas+ImageHandle)	After (Shader+write_texture)
CPU→GPU upload	33MB full copy (~4ms)	8KB dirty region (~0.01ms)
Tessellation	~20K checker rects (~5ms)	0 (procedural shader)
Iced layout overhead	~70ms (widget tree + image pipeline)	~2ms (shader widget)
Total frame	~80-100ms (~10 FPS)	~5-10ms (~100+ FPS)

User Review Required

Important

Iced wgpu feature flag: The iced::widget::shader module requires the "wgpu" feature in the Iced dependency. This is already the rendering backend Iced uses internally, but the feature flag exposes the shader widget API. This should not change binary size significantly.

Warning

Phase 1 is large: The shader canvas is ~400 lines of new code (shader primitive + wgsl + pipeline setup). The existing CanvasProgram (725 lines) will be simplified but the mouse event handling stays identical.

Open Questions

1. Overlay rendering: Should selection rects, vector previews, and crosshair cursor be rendered:

   • (a) In the same WGSL shader (pass selection rect uniforms → draw in fragment shader) — simpler, all in one draw call
   • (b) As a separate transparent Canvas widget layered on top via Stack — keeps current code, two render passes

   I recommend (b) for now because overlays are cheap (few lines/rects) and it keeps the shader simple. We can merge later.

2. Canvas resize: When the document size changes (e.g., "New Image" → 3840×2160), the wgpu texture must be recreated. Should we:

   • (a) Pre-allocate the max texture size (e.g., 8192×8192 = 256MB VRAM)
   • (b) Recreate the texture on resize (one-time ~1ms cost)

   I recommend (b) — 256MB VRAM reservation is wasteful.

Verification Plan

Automated Tests

cargo build -p hcie-iced-gui 2>&1 | head -50
cargo test -p hcie-engine-api --test visual_regression

Manual Verification

1. Launch hcie-iced with a 4K canvas (3840×2160)
2. Draw brush strokes — verify smooth visual feedback (target: 30+ FPS)
3. Zoom in/out — verify checkerboard + composite update correctly
4. Pan — verify smooth scrolling
5. Multi-layer test: 5+ layers at 4K, draw on top layer — verify no stuttering
6. Compare perf logs: [perf] inter-frame should show <20ms consistently