phantom/hcie-rust-v3.05
1
0
Fork 0
Code Issues Pull Requests Actions 12 Packages Projects Releases Wiki Activity
Files
main
hcie-rust-v3.05/test_analysis.md
T
Your Name 8447b2d796
mandatory-regression-gate / deterministic-tests (push) Has been cancelled
Details
mandatory-regression-gate / protected-performance-path (push) Has been cancelled
Details
Refactor code structure for improved readability and maintainability
2026-07-23 15:34:05 +03:00

22 KiB
Raw Permalink Blame History

HCIE-Rust v3.05 — Test Analysis

Generated: 2026-07-23 Project root: /mnt/extra/00_PROJECTS/hcie-rust-v3.05

1. Status Assessment

1.1 Active Tests (fully functional, run by default)

All #[test] functions without #[ignore] that do not depend on missing fixture files.
Approximately ~400 tests across 24 crates.

1.2 Inactive, Skipped, or Conditionally Disabled Tests

Test Crate File:Line Reason
meadow_brush_visual_check hcie-engine-api tests/meadow_check.rs:18 #[ignore = "manual visual check"] — requires human inspection of output PNG
leaves_brush_visual_check hcie-engine-api tests/leaves_check.rs:11 #[ignore = "manual visual check"] — requires human inspection of output PNG
stamp_floor_brush_visual_check hcie-engine-api tests/stamp_floor_check.rs:11 #[ignore = "manual visual check"] — requires human inspection of output PNG
diagnostic_watercolor_sample_latency hcie-brush-engine src/lib.rs:4447 #[ignore = "diagnostic benchmark"] — performance measurement, not pass/fail
diagnostic_4k_dirty_staging_latency hcie-iced-gui src/canvas/texture_update.rs:253 #[ignore = "diagnostic benchmark"] — performance measurement, not pass/fail
test_parse_asl_emboss hcie-fx src/parser.rs:1671 #[ignore = "requires /tmp/kra_test/emboss_full.asl fixture"] — missing fixture
test_psd_composite_against_ref hcie-io lib.rs:43 Runtime skip if Example3-mini.psd not found (IS found → active)
test_base_generated_2_vs_merged hcie-io lib.rs:301 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_base_generated_2_no_effects hcie-io lib.rs:378 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_base_generated_2_composite hcie-io lib.rs:420 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_hc_emboss_composite hcie-io lib.rs:428 Runtime skip if hc_emboss.psd not found (MISSING → inactive)
test_base_generated_2_effect_isolation hcie-io lib.rs:438 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_check_soft_orange_pixels hcie-io lib.rs:491 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_find_orange_pixels hcie-io lib.rs:516 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_layer_bounds hcie-io lib.rs:555 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_check_layer_offsets hcie-io lib.rs:584 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_save_layer_pixels hcie-io lib.rs:621 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_pixel_layer_contributions hcie-io lib.rs:635 Runtime skip if base_test_generated_2.psd not found (MISSING → inactive)
test_per_layer_effects_vs_photoshop_export hcie-io lib.rs:696 Runtime skip if base PSD not found (MISSING → inactive)
test_sultan_effects_mae hcie-io lib.rs:861 Runtime skip if sultan.psd not found (MISSING → inactive)
test_emboss_optimize hcie-io lib.rs:1057 Runtime skip if emboss.psd not found (FOUND → active if it uses emboss.psd)
test_effects_survive_ffi_boundary hcie-io lib.rs:1246 Runtime skip if base PSD not found (MISSING → inactive)
test_ffi_bincode_roundtrip hcie-io lib.rs:1276 Runtime skip if base PSD not found (MISSING → inactive)

Total conditionally inactive: ~17 tests (mostly hcie-io PSD composite tests needing _images/_psd_stil_test/ fixtures that do not exist in this environment).

1.3 Notable: Expensive / Long-Running Active Tests

Test Crate Est. Time Notes
benchmark_4k_stroke_on_multilayer_document hcie-engine-api ~60s 10 layers on 3840×2160, 100 stroke segments with per-segment timing
test_load_test_2_psd hcie-io ~30-60s Loads and composites all PSDs from test_2
test_emboss_optimize hcie-io ~30-60s Detailed emboss analysis with line scans, heatmaps
test_psd_composite_against_ref hcie-io ~10-30s Full PSD composite against reference PNG
test_layer_pixels_direct hcie-io ~10-30s Per-layer pixel comparison

2. Command Generation

2.1 Per-Crate Test Commands

Layer 1: DATA

# hcie-protocol — 16 tests (color packing, constants, lerp, distance, thumbnail, selection, blend modes, tools, version)
cargo test -p hcie-protocol

# hcie-color — 11 tests (gamma encode/decode, sRGB/linear, RGB/HSL)
cargo test -p hcie-color

Layer 2: ENGINE CORE

# hcie-blend — 9 tests (opacity identity, multiply darkens, screen lightens, all modes)
cargo test -p hcie-blend

# hcie-brush-engine — 13 tests (6 stamp + 7 internal)
#   (includes 1 ignored diagnostic benchmark)
cargo test -p hcie-brush-engine
cargo test -p hcie-brush-engine -- --ignored  # includes diagnostic_watercolor_sample_latency

# hcie-draw — 8 tests (rect, circle, ellipse, line, flood fill, mask)
cargo test -p hcie-draw

# hcie-composite — 5 tests (pass through, visibility toggle, basic composite)
cargo test -p hcie-composite

# hcie-filter — 13 tests (blur, gaussian, motion, invert, grayscale, sharpen, emboss, etc.)
cargo test -p hcie-filter

# hcie-selection — 15 tests (rect/ellipse mask, invert, grow, shrink, feather, magic wand, lasso)
cargo test -p hcie-selection

# hcie-vector — 4 tests (crescent, bubble, rotate point, SVG rotation)
cargo test -p hcie-vector

# hcie-history — 9 tests (new empty, push/undo/redo, max steps, jump to, entry description)
cargo test -p hcie-history

# hcie-fx — 1 ignored test (ASL emboss parser — requires fixture)
cargo test -p hcie-fx
cargo test -p hcie-fx -- --ignored  # includes test_parse_asl_emboss

# hcie-io — ~24 tests (PSD composite, layer analysis, effects)
#   Many will runtime-skip if _images/ fixtures are missing
#   Can be very slow (up to several minutes)
cargo test -p hcie-io

# hcie-psd — 8 tests (signature validation, channel RLE, file header fields)
cargo test -p hcie-psd

# hcie-vision — 2 tests (smart patch identity and gradient)
cargo test -p hcie-vision

# hcie-build-info — 1 test (build ID format)
cargo test -p hcie-build-info

Layer 4: ENGINE API

# Core engine API — 8 visual regression + 2 history + 5 visual checks + 1 bitmap + 1 selection + 1 performance + 1 internal
cargo test -p hcie-engine-api
cargo test -p hcie-engine-api -- --ignored              # includes meadow, leaves, stamp_floor checks
cargo test -p hcie-engine-api -- performance_stroke_4k   # 4K benchmark only

Layer 6: GUI — egui

# hcie-gui-egui — ~90 tests (canvas_engine, gui_audit, widget_ui, brush_import)
cargo test -p hcie-gui-egui

Layer 6: GUI — iced

# hcie-iced-gui — ~120+ tests (selection, feature scorecard, raster, app state, dock,
#   panels, widgets, color picker, AI chat, viewer, CLI, theme, settings, SVG editor, etc.)
cargo test -p hcie-iced-gui

Brush Catalog Crates

cargo test -p hcie-dry-media-brushes
cargo test -p hcie-paint-brushes
cargo test -p hcie-digital-brushes
cargo test -p hcie-watercolor-brushes
cargo test -p hcie-ink-brushes

2.2 Specific Test Filter Commands

# Run only the proximity dedup tests (recent color fix)
cargo test -p hcie-iced-gui -- proximate

# Run only the visual regression golden tests
cargo test -p hcie-engine-api -- visual_regression

# Run only the feature scorecard tests
cargo test -p hcie-iced-gui -- feature_scorecard

# Run only crop/selection tests
cargo test -p hcie-iced-gui -- selection_test

# Run only the GUI audit tests
cargo test -p hcie-gui-egui -- gui_audit

# Run only the dock sizing/layout tests
cargo test -p hcie-iced-gui -- sizing

# Run only the history tests
cargo test -p hcie-history

# Run only the PSD-related tests
cargo test -p hcie-psd
cargo test -p hcie-io -- psd  # matches "psd" in test names in hcie-io

2.3 Running Ignored Tests

# Run all ignored tests across all crates (visual checks and benchmarks)
cargo test -- --ignored                                    # runs ALL crates
cargo test -p hcie-engine-api -- --ignored
cargo test -p hcie-brush-engine -- --ignored
cargo test -p hcie-iced-gui -- --ignored
cargo test -p hcie-fx -- --ignored

3. Batch Script (.bat)

File: run_tests_categorized.bat

@echo off
setlocal enabledelayedexpansion

set ROOT_DIR=%~dp0
cd /d "%ROOT_DIR%" || exit /b 1

echo ============================================================
echo HCIE-Rust v3.05 — Categorized Test Runner (Windows .bat)
echo Root: %ROOT_DIR%
echo ============================================================

set PASS=0
set FAIL=0
set SKIP=0

:: Helper: run a category and accumulate results
call :RUN_CATEGORY "Layer 1: DATA" ^
    hcie-protocol ^
    hcie-color

call :RUN_CATEGORY "Layer 2: ENGINE CORE — Blend/Brush" ^
    hcie-blend ^
    hcie-brush-engine

call :RUN_CATEGORY "Layer 2: ENGINE CORE — Draw/Composite/Filter" ^
    hcie-draw ^
    hcie-composite ^
    hcie-filter

call :RUN_CATEGORY "Layer 2: ENGINE CORE — Selection/Vector/History" ^
    hcie-selection ^
    hcie-vector ^
    hcie-history

call :RUN_CATEGORY "Layer 2: ENGINE CORE — IO/PSD/Vision/Build" ^
    hcie-io ^
    hcie-psd ^
    hcie-vision ^
    hcie-build-info

call :RUN_CATEGORY "Layer 4: ENGINE API" ^
    hcie-engine-api

call :RUN_CATEGORY "Layer 6: GUI — egui" ^
    hcie-gui-egui

call :RUN_CATEGORY "Layer 6: GUI — iced" ^
    hcie-iced-gui

call :RUN_CATEGORY "Brush Catalogs" ^
    hcie-dry-media-brushes ^
    hcie-paint-brushes ^
    hcie-digital-brushes ^
    hcie-watercolor-brushes ^
    hcie-ink-brushes

echo ============================================================
echo SUMMARY: %PASS% passed, %FAIL% failed, %SKIP% skipped
echo ============================================================
exit /b %FAIL%

:: ============================================================
:: Subroutine: run a named category across multiple crates
:: ============================================================
:RUN_CATEGORY
set CATEGORY=%~1
shift
echo.
echo ============================================================
echo Category: %CATEGORY%
echo ============================================================

:RUN_CATEGORY_LOOP
if "%~1"=="" goto :EOF
set CRATE=%~1
shift
echo --- Running: %CRATE% ---
cargo test -p %CRATE%
if %ERRORLEVEL%==0 (
    set /a PASS+=1
) else (
    set /a FAIL+=1
)
goto RUN_CATEGORY_LOOP

4. Bulk Execution Script (bash)

File: run_all_tests.sh

#!/usr/bin/env bash
# HCIE-Rust v3.05 — Bulk test runner
# Usage: bash run_all_tests.sh [--no-io] [--no-4k] [--ignored]
set -euo pipefail

ROOT_DIR="$(cd "$(dirname "$0")" && pwd)"
cd "$ROOT_DIR"

PASS=0
FAIL=0
SKIP=0
START_TIME=$(date +%s)

# Parse flags
SKIP_IO=false
SKIP_4K=false
INCLUDE_IGNORED=false
EXTRA_ARGS=()

for arg in "$@"; do
    case "$arg" in
        --no-io)    SKIP_IO=true ;;
        --no-4k)    SKIP_4K=true ;;
        --ignored)  INCLUDE_IGNORED=true ;;
        *)          EXTRA_ARGS+=("$arg") ;;
    esac
done

SEPARATOR() {
    printf '%*s\n' 80 '' | tr ' ' '='
}

run_crate() {
    local crate="$1"
    local label="$2"
    local extra="$3"
    shift 3

    SEPARATOR
    echo "[$label] Running: $crate $extra"
    SEPARATOR

    if [ -n "$extra" ]; then
        # shellcheck disable=SC2086
        if cargo test -p "$crate" $extra "${EXTRA_ARGS[@]+${EXTRA_ARGS[@]}}" 2>&1; then
            PASS=$((PASS + 1))
        else
            FAIL=$((FAIL + 1))
        fi
    else
        if cargo test -p "$crate" "${EXTRA_ARGS[@]+${EXTRA_ARGS[@]}}" 2>&1; then
            PASS=$((PASS + 1))
        else
            FAIL=$((FAIL + 1))
        fi
    fi
}

display_summary() {
    local elapsed=$(( $(date +%s) - START_TIME ))
    echo ""
    SEPARATOR
    echo " BULK TEST EXECUTION COMPLETE"
    echo " Crates passed:  $PASS"
    echo " Crates with failures: $FAIL"
    echo " Elapsed time:   ${elapsed}s"
    SEPARATOR
    if [ "$FAIL" -eq 0 ]; then
        echo " ✓ ALL CRATES PASSED"
    else
        echo " ✗ $FAIL crate(s) have failing tests"
    fi
    exit "$FAIL"
}

# ── Layer 1: DATA ───────────────────────────────────────────────
run_crate "hcie-protocol" "Layer 1" ""
run_crate "hcie-color"    "Layer 1" ""

# ── Layer 2: ENGINE CORE ────────────────────────────────────────
run_crate "hcie-blend"       "Layer 2" ""
run_crate "hcie-brush-engine" "Layer 2" ""
run_crate "hcie-draw"         "Layer 2" ""
run_crate "hcie-composite"    "Layer 2" ""
run_crate "hcie-filter"       "Layer 2" ""
run_crate "hcie-selection"    "Layer 2" ""
run_crate "hcie-vector"       "Layer 2" ""
run_crate "hcie-history"      "Layer 2" ""

if [ "$SKIP_IO" = false ]; then
    run_crate "hcie-io" "Layer 2" ""
else
    echo "[SKIP] hcie-io (--no-io flag)"
    SKIP=$((SKIP + 1))
fi
run_crate "hcie-psd"         "Layer 2" ""
run_crate "hcie-vision"      "Layer 2" ""
run_crate "hcie-build-info"  "Layer 2" ""

# ── Layer 4: ENGINE API ─────────────────────────────────────────
if [ "$SKIP_4K" = false ]; then
    run_crate "hcie-engine-api" "Layer 4" ""
else
    run_crate "hcie-engine-api" "Layer 4" "--skip benchmark_4k_stroke_on_multilayer_document"
fi

# ── Layer 6: GUI — egui ────────────────────────────────────────
run_crate "hcie-gui-egui" "Layer 6" ""

# ── Layer 6: GUI — iced ─────────────────────────────────────────
run_crate "hcie-iced-gui" "Layer 6" ""

# ── Brush Catalogs ──────────────────────────────────────────────
run_crate "hcie-dry-media-brushes"  "Brushes" ""
run_crate "hcie-paint-brushes"      "Brushes" ""
run_crate "hcie-digital-brushes"    "Brushes" ""
run_crate "hcie-watercolor-brushes" "Brushes" ""
run_crate "hcie-ink-brushes"        "Brushes" ""

# ── Ignored tests (visual checks, benchmarks) ───────────────────
if [ "$INCLUDE_IGNORED" = true ]; then
    echo ""
    SEPARATOR
    echo " Running IGNORED tests (visual checks, benchmarks)"
    SEPARATOR
    cargo test -p hcie-engine-api -- --ignored || true
    cargo test -p hcie-brush-engine -- --ignored || true
    cargo test -p hcie-iced-gui -- --ignored || true
    cargo test -p hcie-fx -- --ignored || true
fi

display_summary

5. Gap Analysis

5.1 Crates with Zero Tests

These 21 crates have no #[test] functions, no tests/ directory, and no #[cfg(test)] modules:

Crate Layer Risk Notes
hcie-document Layer 3 HIGH Core document state management — layer CRUD, dirty tracking, zoom, blend mode changes. Zero tests despite being a critical engine crate.
hcie-tile Layer 2 HIGH Sparse tile-based layer storage — tile read/write, cache invalidation, dirty tracking. No tests for the incremental tile update mechanism.
hcie-text Layer 2 HIGH Vector text rendering via fontdue — text layout, wrapping, font loading, glyph rasterization. No tests.
hcie-native Layer 2 MEDIUM HCIE native file format I/O. Simple wrapper crate but no serialization/deserialization tests.
hcie-kra Layer 2 MEDIUM Krita (KRA) file format import/export. No roundtrip tests.
hcie-psd-saver Layer 2 MEDIUM PSD file saver. No tests.
hcie-ai Layer 5 MEDIUM Ollama/LMStudio/OpenAI chat client, templates, AI actions. No tests for the client, message building, or response parsing.
egui-panel-adapter Layer 6 MEDIUM Dynamic schema-to-widget UI binder. No tests.
egui-panel-filters Layer 6 MEDIUM Filter parameter panels. No tests for parameter binding or UI state.
egui-panel-ai-chat Layer 6 MEDIUM AI chat panel (egui). No tests.
egui-panel-script Layer 6 MEDIUM Scripting panel (egui). No tests.
egui-panel-ai-script Layer 6 MEDIUM AI-assisted script generation panel (egui). No tests.
iced-panel-adapter Layer 6 MEDIUM Schema-to-widget binder (iced). No tests.
iced-panel-ai-chat Layer 6 MEDIUM AI chat panel (iced). No tests.
iced-panel-script Layer 6 MEDIUM Scripting panel (iced). No tests.
panel-tuner Tools LOW Development tool for panel layout tuning.
screenshot-diff Tools LOW Screenshot comparison tool.
hcie-egui-app Layer 6 LOW Workspace root — no test logic expected.
hcie-iced-app Layer 6 LOW Workspace root — no test logic expected.
egui-winit Patches LOW Upstream patch — tests disabled (autotests = false).

5.2 Modules with Incomplete Test Coverage

Module Existing Tests Missing Coverage
hcie-io PSD composite 24 tests, but ~17 skip at runtime Deterministic tests that work without fixture files (generate test PSDs programmatically)
hcie-engine-api visual regression 8 golden hash tests No test for watercolor brush, pencil, gradient fill, crop, transform, layer effects
hcie-brush-engine 7 inline + 6 integration tests No tests for airbrush/spray dynamics, tilt/rotation mapping, dual-brush mode
hcie-filter 13 smoke tests No tests verifying correct pixel output values for any filter; only "does not panic"
hcie-psd 8 tests (header + signature) No tests for image resource sections, layer info (masks, effects, patterns), channel data decompression (all unimplemented! stubs exist)
hcie-composite 5 tests No tests for layer masks, clipping masks, adjustment layers, layer groups with non-PassThrough mode, blend mode combinations
hcie-iced-gui dock system ~25 tests No tests for minimizing/maximizing panels, keyboard navigation of dock, drag-and-drop reordering of tabs
hcie-iced-gui panels ~15 tests Brushes panel has catalog count checks but no interaction tests. Filters panel has no tests. Layer details/panel has no tests.

5.3 Recommended Test Additions (Priority Order)

Priority 1 (High Risk — Engine Core Integrity)

  1. hcie-document — Unit tests for:

    • Layer CRUD (add/remove/reorder)
    • Active layer switching
    • Dirty flag set/clear
    • Zoom clamp boundaries
    • Blend mode changes on layers
    • Opacity/visibility toggle propagation

  2. hcie-tile — Unit tests for:

    • Tile read/write at pixel level
    • Dirty rectangle tracking
    • Tile cache invalidation
    • Incremental tile update (the protected optimization)
    • Multi-resolution tile access

  3. hcie-text — Unit tests for:

    • Font loading (valid/invalid paths)
    • Text layout (single line, multi-line, wrapping)
    • Glyph rasterization bounds
    • Unicode / special character handling
    • Text cursor positioning

Priority 2 (Medium Risk — Feature Correctness)

  1. hcie-io — Replace fixture-dependent tests with:

    • Programmatic PSD creation using hcie-psd-saver (roundtrip test)
    • In-memory composite with known pixel values
    • PNG/JPG/WebP import/export with checksum verification

  2. hcie-filter — Add value verification tests:

    • Invert: known input → known output
    • Grayscale: R=G=B after filter
    • Brightness/Contrast: exact value mapping
    • Box blur: average of neighborhood

  3. hcie-ai — Unit tests for:

    • Message formatting (system/user/assistant)
    • API response parsing (JSON extraction)
    • Template rendering
    • Error handling (network failure, auth error)

Priority 3 (Lower Risk — GUI Completeness)

  1. egui-panel-filters — Test filter parameter schema binding and default values
  2. egui-panel-ai-chat — Test message history management, send/receive state machine
  3. iced-panel-adapter — Test dynamic widget generation from schema
  4. All panel crates — Smoke tests ensuring each panel renders without panic

5.4 Quick Wins (Easy to Add, High Impact)

Test Crate Effort Impact
Roundtrip: create layer → modify → undo → redo → equal original hcie-document 1 hour Catches state corruption bugs
Tile dirty region compute: modify tile → read dirty rect hcie-tile 2 hours Validates the optimization in AGENTS.md
Text measure: known string + font → expected width hcie-text 1 hour Prevents layout regressions
Invert filter: 5 known RGBA values → expected output hcie-filter 1 hour Upgrades smoke test to real verification
No-panic on empty input for every filter hcie-filter 30 min Edge case coverage
Fixture-free PSD composite: create 2 layers with known colors in memory, composite, verify hcie-io 3 hours Eliminates ~17 flaky/skipped tests

5.5 Test Infrastructure Improvements

Issue Recommendation
hcie-io tests silently pass when fixtures missing Change to panic! with informative message, or add #[ignore] with fixture requirement documented
No test coverage tracking Add cargo tarpaulin or cargo llvm-cov to CI and report line/region coverage
No CI integration visible Set up GitHub Actions running: Layer 1 → Layer 2 → Layer 4 → Layer 6 sequentially with --no-io --no-4k for quick CI and full suite nightly
hcie-psd has 12 unimplemented!() stubs Add #[should_panic] tests that verify stubs panic as expected, or complete the implementations
Reference in New Issue View Git Blame Copy Permalink