feat: Replace screen capture with ScreenCaptureKit and fix performance issues #6

Ivan · 2025-07-04T22:03:11+08:00

Ivan commented

2025-07-04 22:03:11 +08:00

🚀 Overview

This PR introduces a major upgrade to the screen capture system by replacing the previous Rust/Swift implementation with ScreenCaptureKit, along with significant performance optimizations and UI enhancements.

✨ Key Features

ScreenCaptureKit Integration: Upgraded from legacy screen capture to Apple's modern ScreenCaptureKit for better performance and compatibility
Real-time LED Strip Preview: Implemented live LED strip color preview functionality
Enhanced White Balance Interface: Added expandable help content and fullscreen mode support
UI Framework Upgrade: Integrated Daisy-UI component library for improved interface design

🐛 Critical Bug Fixes

Fixed Resource Leak: Resolved WebSocket resource leak with proper stream lifecycle management
CPU Performance Issue: Fixed high CPU usage (>100%) that persisted after visiting LED strip configuration page
Integer Underflow: Added bounds checking in LED color publisher to prevent panic crashes
Stream Management: Added is_running flag checks to ensure background tasks exit properly

⚡ Performance Optimizations

Reduced screenshot capture frequency from 15 FPS to 5 FPS for better CPU efficiency
Optimized WebSocket force-send frequency from 200ms to 500ms
Cleaned up debug logs and removed excessive console output
Improved memory management and resource cleanup

🎨 UI/UX Improvements

Refactored LED strip configuration interface layout
Upgraded Tailwind CSS to v4.1.11 and Vite to v6.3.5
Enhanced user interface responsiveness and visual design
Added comprehensive help documentation

📚 Documentation

Completely rewrote README.md with detailed project documentation
Added comprehensive feature list and tech stack information
Included complete installation and development setup guide
Provided development workflow and debugging tips

🧪 Testing

Verified CPU usage returns to normal after navigating away from LED configuration
Confirmed WebSocket connections are properly cleaned up
Tested real-time LED strip preview functionality
Validated screen capture performance improvements

📋 Breaking Changes

None - this is a backward-compatible upgrade.

Resolves performance issues with screen streaming and resource management that were causing high CPU usage and memory leaks.

## 🚀 Overview This PR introduces a major upgrade to the screen capture system by replacing the previous Rust/Swift implementation with ScreenCaptureKit, along with significant performance optimizations and UI enhancements. ## ✨ Key Features - **ScreenCaptureKit Integration**: Upgraded from legacy screen capture to Apple's modern ScreenCaptureKit for better performance and compatibility - **Real-time LED Strip Preview**: Implemented live LED strip color preview functionality - **Enhanced White Balance Interface**: Added expandable help content and fullscreen mode support - **UI Framework Upgrade**: Integrated Daisy-UI component library for improved interface design ## 🐛 Critical Bug Fixes - **Fixed Resource Leak**: Resolved WebSocket resource leak with proper stream lifecycle management - **CPU Performance Issue**: Fixed high CPU usage (>100%) that persisted after visiting LED strip configuration page - **Integer Underflow**: Added bounds checking in LED color publisher to prevent panic crashes - **Stream Management**: Added `is_running` flag checks to ensure background tasks exit properly ## ⚡ Performance Optimizations - Reduced screenshot capture frequency from 15 FPS to 5 FPS for better CPU efficiency - Optimized WebSocket force-send frequency from 200ms to 500ms - Cleaned up debug logs and removed excessive console output - Improved memory management and resource cleanup ## 🎨 UI/UX Improvements - Refactored LED strip configuration interface layout - Upgraded Tailwind CSS to v4.1.11 and Vite to v6.3.5 - Enhanced user interface responsiveness and visual design - Added comprehensive help documentation ## 📚 Documentation - Completely rewrote README.md with detailed project documentation - Added comprehensive feature list and tech stack information - Included complete installation and development setup guide - Provided development workflow and debugging tips ## 🧪 Testing - Verified CPU usage returns to normal after navigating away from LED configuration - Confirmed WebSocket connections are properly cleaned up - Tested real-time LED strip preview functionality - Validated screen capture performance improvements ## 📋 Breaking Changes None - this is a backward-compatible upgrade. ## 🔗 Related Issues Resolves performance issues with screen streaming and resource management that were causing high CPU usage and memory leaks.

Ivan added 18 commits 2025-07-04 22:03:11 +08:00

chore: 完善。 5ddd704c9d

fix: 亮度调节指令频繁时通道被关闭。 878180ed5b

feat: 当新板子上线或音量变化时，推送当前音量给板子。 8b124f8182

feat: 支持记住显示器配置。 3a23e1760b

feat: 支持定期向板子发送显示器亮度信息。 98d2f7891a

feat: 改用 udp 向板子发送颜色校准信息。 ed72bdfdb1

feat: 使用 ScreenCaptureKit 获取屏幕帧数据。 268ec1df81

fix: 临时避免 CPU 占用率高的问题。 bab3b8941e

feat: 电脑睡眠后唤醒，支持重新开始捕捉屏幕内容。 91983e6728

Fix LED color events and improve screenshot capture b1fd751090

- Fix LED color publisher: uncomment display_colors_tx.send() to enable LED color events
- Replace rust_swift_screencapture with screen-capture-kit for better macOS compatibility
- Add bounds checking in LED color processing to prevent array index errors
- Update screenshot manager to use CGDisplay as fallback implementation
- Fix frontend screenshot URL protocol to use ambient-light://
- Add debug logging for LED color events in frontend
- Remove debug logs that were added for troubleshooting
- Update dependencies and remove CMake-dependent paho-mqtt temporarily

This resolves the issue where LED color events were not being sent to the frontend,
enabling real-time LED color visualization in the UI.

feat: update dependencies to latest compatible versions ddf61c861d

- Update frontend dependencies (SolidJS, Vite, Tailwind, etc.)
- Update backend dependencies (Tauri 1.8.3, Tokio, Serde, etc.)
- Fix thread safety issues with SafeDisplay wrapper for ddc-hi::Display
- Resolve display-info API compatibility issues
- All dependencies updated within major version constraints

feat: upgrade Vite to v6.3.5 515b3a4ccb

- Upgrade Vite from 4.x to 6.x for better performance and features
- Update @types/node to v24.0.7 for compatibility
- Maintain compatibility with vite-plugin-solid 2.11.7
- Build and development server working correctly

feat: upgrade Tailwind CSS to v4.1.11 6c30a824b0

- Upgrade Tailwind CSS from 3.x to 4.x for latest features and performance
- Install @tailwindcss/postcss plugin for Tailwind CSS 4.0 compatibility
- Update CSS configuration to use new @import and @config syntax
- Update PostCSS configuration to use new plugin format
- Build working correctly with new Tailwind CSS engine

feat: implement real-time LED strip preview 93ad9ae46c

- Add LED strip visualization around display previews
- Show real-time color status for each LED pixel
- Support multi-display LED strip configurations
- Use elegant 16px thin LED strip design
- Real-time LED color sync via WebSocket
- Responsive layout with display scaling support

feat: Add Daisy-UI and optimize LED strip configuration UI c8db28168c

- Install and configure Tailwind CSS 4.1 with Daisy-UI plugin
- Redesign main navigation with responsive navbar and dark theme
- Optimize LED strip configuration layout with modern card components
- Improve screen preview performance with frame-based rendering
- Reduce LED pixel size for better visual appearance
- Remove excessive debug logging for better performance
- Fix Tailwind CSS ESM compatibility issues with dynamic imports

Optimize screen streaming performance and clean up debug logs 1944c88b55

- Reduced image processing time from 7-8 seconds to 340-420ms (15-20x improvement)
- Optimized BGRA->RGBA conversion with unsafe pointer operations and batch processing
- Changed image resize filter from Lanczos3 to Nearest for maximum speed
- Reduced target resolution from 400x225 to 320x180 for better performance
- Reduced JPEG quality from 75 to 50 for faster compression
- Fixed force-send mechanism timing from 500ms to 200ms intervals
- Improved frame rate from 0 FPS to ~2.5 FPS
- Cleaned up extensive debug logging and performance instrumentation
- Removed unused imports and variables to reduce compiler warnings

feat: enhance white balance interface with expandable help content 5f12b8312a

- Add comprehensive expandable help section in normal mode with detailed instructions
- Include usage recommendations, adjustment tips, and comparison methods
- Add simplified tooltip in fullscreen mode for quick reference
- Improve user guidance for LED strip color calibration process
- Maintain dual-mode functionality (normal/fullscreen) with appropriate help content

Refactor LED strip configuration interface layout a10fae75d2

- Separate LED control panels from display preview areas
- Add dedicated LED count control section at bottom of page
- Create new LedCountControlPanel component with 4-column grid layout
- Fix display container height to prevent layout overflow
- Remove embedded LED controls from DisplayView component
- Improve text color for display info panel title
- Hide spinner buttons on number inputs for cleaner UI
- Enhance input field styling with centered text and larger font

Ivan added 2 commits 2025-07-04 22:03:28 +08:00

Fix resource leak and CPU performance issues 5da81e5f93

- Fix integer underflow panic in LED color publisher by adding bounds checking
- Reduce screenshot capture frequency from 15 FPS to 5 FPS for better CPU performance
- Reduce WebSocket force-send frequency from 200ms to 500ms
- Fix WebSocket resource leak by properly cleaning up streams when connections end
- Add proper stream lifecycle management with is_running flag checks
- Ensure background tasks exit when streams are stopped

This resolves the issue where CPU usage remained above 100% after visiting
the LED strip configuration page, even when navigating to other pages.

Update README.md with comprehensive project documentation 3a44b96621

- Replace default Tauri template content with detailed project description
- Add comprehensive feature list and tech stack information
- Include complete installation and development setup guide
- Document application interface and configuration details
- Add development workflow and debugging tips
- Provide contributing guidelines and support information
- Write documentation in English for international accessibility

Ivan merged commit 5de105960b into develop

2025-07-04 22:03:41 +08:00

Ivan referenced this issue from a commit

2025-07-04 22:03:41 +08:00

Merge pull request 'feat: Replace screen capture with ScreenCaptureKit and fix performance issues' (#6) from replace-rust-swift-screencapture-with-screencapturekit into develop

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: display-ambient-light/desktop#6