Compare commits

..

20 Commits

Author SHA1 Message Date
Terrence
f4ed1a4a0d remove pm config code 2026-02-09 19:05:47 +08:00
Terrence
f8a4b8b70d chore: Update component versions and enhance UI setup across multiple boards
- Bumped uart-eth-modem version from ~0.3.2 to ~0.3.3 in idf_component.yml.
- Added SetupUI method to various display classes to ensure proper UI initialization before usage.
- Improved error handling in display classes to prevent issues when UI is not set up.
- Ensured UI customization is performed in SetupUI rather than constructors for better reliability.
2026-02-09 10:21:31 +08:00
Xiaoxia
9215a04a7e Delay init success sound playback and remove gif playback delay (#1748)
* refactor: Remove hardcoded loop delay for GIF playback in LcdDisplay class

* chore: Update esp-ml307 and uart-eth-modem component versions in idf_component.yml

- Bump esp-ml307 version from ~3.6.3 to ~3.6.4
- Update uart-eth-modem version from ~0.3.1 to ~0.3.2

* feat: Add PrintPmLocks method to SystemInfo class

- Introduced PrintPmLocks method to display power management locks using esp_pm_dump_locks.
- Updated system_info.h to declare the new method.

* refactor: Streamline audio codec initialization and enablement

- Removed redundant channel enable checks from AudioCodec::Start.
- Added channel enablement in CreateDuplexChannels for various audio codecs.
- Implemented EnableInput and EnableOutput methods in NoAudioCodec for better control over input/output states.

* refactor: Delay audio success sound playback until after activation completion

- Moved the success sound playback to a scheduled task to ensure it occurs after the activation process is complete.
- This change improves the responsiveness of the application during activation events.

* refactor: Update camera integration from EspVideo to Esp32Camera

- Replaced EspVideo with Esp32Camera for improved camera configuration and initialization.
- Streamlined camera setup by utilizing a new configuration structure for better clarity and maintainability.
- Updated README.md to remove outdated camera sensor configuration instructions.

* refactor: Update audio demuxing process in AudioService

- Replaced the existing demuxer instance with a local unique pointer in the PlaySound method for better memory management.
- Moved the OnDemuxerFinished callback setup into the PlaySound method to ensure it is correctly associated with the new demuxer instance.
- Removed the member variable demuxer_ from AudioService to streamline the class structure.
2026-02-08 22:09:45 +08:00
Wang is proud
7b7d22c495 feat: modify CircularStrip constructor parameter types and add SetMultiColors method (#1750) 2026-02-08 11:17:12 +08:00
zczc365
b4eada876a feat&fix: 小智云聊增加蓝牙功能 (#1732) 2026-02-07 11:02:03 +08:00
小林同志
49cd6625f4 增加流式ogg解封装支持 (#1705)
* 增加流式ogg解封装支持

* 增加TF卡引脚连接说明

* 修复圆角图标屏幕导致的显示问题

* 优化聊天消息显示

* 修改解封装实现

---------

Co-authored-by: smalllin0 <aslinqf@163.com>
2026-02-05 00:12:40 +08:00
Y1hsiaochunnn
6f71868bad fix: Categorizing waveshare products (#1734)
content: Update the README.md

ci: Adapt to the lower directory

ci: the lower directory support

ci: board type errors fixed.

ci: board_type errors fixed to board_leaf.

fix: A naming error problem

fix: jd9365 member error

fix: Update product naming

feat: Add manufacturer parameters to the firmware name for 'release'

fix: Verification logic

fix: Verification logic
2026-02-04 21:03:23 +08:00
baidxi
173eaa7463 在Display基类中添加SetupUI虚函数 (#1742)
Signed-off-by: jeck.chen <jeck.chen@dbappsecurity.com.cn>
2026-02-04 21:03:08 +08:00
Xiaoxia
2b025c4ea6 Enhance audio processing and wake word detection (#1739)
* Enhance audio processing and wake word detection

- Set task priority in Application::Run to improve responsiveness.
- Log detected wake words with their state in HandleWakeWordDetectedEvent.
- Streamline audio feeding in AudioService to handle both wake word and audio processor events.
- Implement input buffering in AfeAudioProcessor, AfeWakeWord, CustomWakeWord, and EspWakeWord to manage audio data more efficiently.
- Clear input buffers on stop to prevent residual data issues.

* Refactor audio processing to enhance thread safety and state management

- Implement early return checks in Feed methods of AfeAudioProcessor, AfeWakeWord, CustomWakeWord, and EspWakeWord to prevent processing when not running.
- Introduce std::atomic for running state in CustomWakeWord and EspWakeWord to ensure thread-safe access.
- Consolidate input buffer management with mutex locks to avoid race conditions during Stop and Feed operations.

* Refactor listening mode handling and wake word detection configuration

- Replace direct mode setting logic with a new GetDefaultListeningMode method for improved clarity and maintainability.
- Update HandleToggleChatEvent, HandleWakeWordDetectedEvent, and ContinueWakeWordInvoke to utilize the new method for determining listening mode.
- Introduce Kconfig option WAKE_WORD_DETECTION_IN_LISTENING to enable or disable wake word detection during listening mode, enhancing configurability.
2026-02-04 14:28:21 +08:00
Xiaoxia
37110a9d05 Fix: esp32camera pixel byte order and uart-uhci compiling error (#1728)
* Fix: uart-uhci compiling errors

* Enhance Esp32Camera functionality by adding optional byte swapping for RGB565 format. Introduce SetSwapBytes method to enable/disable byte order swapping, and update Capture method to utilize an encode buffer for improved memory management and performance during image processing.
2026-02-02 15:33:32 +08:00
小鹏
796312db4c Enhance Otto Robot camera support by adding configuration for OV3660. (#1726) 2026-02-02 10:22:53 +08:00
tkpdx01
9e1724e892 feat: add M5Stack Cardputer Adv board support (#1718)
Add support for M5Stack Cardputer Adv, a card-sized computer based on
ESP32-S3FN8 (Stamp-S3A) with the following features:

Hardware:
- MCU: ESP32-S3FN8 @ 240MHz, 8MB Flash (no PSRAM)
- Display: ST7789V2 1.14" 240x135
- Audio: ES8311 codec + NS4150B amplifier
- Keyboard: 56-key via TCA8418
- IMU: BMI270

Key configurations:
- SPI3_HOST with 3-wire SPI mode for display
- 256Hz PWM frequency for backlight (matching M5GFX)
- ES8311 with use_mclk=false (no MCLK pin)
- Display offset X=40, Y=52 for correct alignment

新增 M5Stack Cardputer Adv 开发板支持

支持基于 ESP32-S3FN8 (Stamp-S3A) 的卡片式电脑 M5Stack Cardputer Adv:

硬件规格:
- MCU: ESP32-S3FN8 @ 240MHz, 8MB Flash (无 PSRAM)
- 显示屏: ST7789V2 1.14" 240x135
- 音频: ES8311 编解码器 + NS4150B 功放
- 键盘: 56键 (TCA8418)
- IMU: BMI270

关键配置:
- 显示使用 SPI3_HOST 和 3-wire SPI 模式
- 背光 PWM 频率 256Hz (与 M5GFX 一致)
- ES8311 设置 use_mclk=false (无 MCLK 引脚)
- 显示偏移 X=40, Y=52 以正确对齐
2026-02-02 10:01:36 +08:00
Xiaoxia
0b3b98eca7 Update esp-ml307 component version to 3.6.2 to support UART DMA (#1724)
* Update esp-ml307 dependency version to ~3.6.0 in idf_component.yml

* Update .gitignore to include 'dist/' directory, add ml307 and dual_network_board source files to CMakeLists.txt, and update esp-ml307 dependency version to ~3.6.2 in idf_component.yml. Refactor CompactWifiBoard and CompactWifiBoardLCD classes to inherit from WifiBoard instead of DualNetworkBoard, simplifying network handling logic.
2026-02-02 09:53:06 +08:00
Xiaoxia
abd62648cb Implement early return in AfeWakeWord::Feed to prevent processing when detection is not running. This enhances the robustness of the wake word detection logic. (#1723) 2026-02-01 17:42:34 +08:00
Xiaoxia
0883a36537 Refactor audio channel handling and wake word detection in Application class (#1722)
- Introduced ContinueOpenAudioChannel and ContinueWakeWordInvoke methods to streamline audio channel management and wake word processing.
- Updated HandleToggleChatEvent and HandleWakeWordDetectedEvent to utilize scheduling for state changes, improving UI responsiveness.
- Simplified logic for setting listening modes based on audio channel state, enhancing code clarity and maintainability.
2026-02-01 14:55:47 +08:00
Xiaoxia
b6c61fe390 Update project version to 2.2.2, Noto fonts and emoji support. (#1720) 2026-02-01 01:04:24 +08:00
Xiaoxia
f7284a57df Enhance memory management in asset download and OTA processes by repl… (#1716)
* Enhance memory management in asset download and OTA processes by replacing static buffer allocations with dynamic memory allocation using heap capabilities. Update SPIRAM configuration values for improved memory usage. Add logging for error handling in buffer allocation failures. Introduce a new parameter in CloseAudioChannel to control goodbye message sending in MQTT and WebSocket protocols.

* Update component versions in idf_component.yml and refactor GIF decoder functions for improved performance. Bump versions for audio effects, audio codec, LED strip, and other dependencies. Change GIF read and seek functions to inline for optimization.

* Update language files to include new phrases for flight mode and connection status across multiple locales. Added translations for "FLIGHT_MODE_ON", "FLIGHT_MODE_OFF", "CONNECTION_SUCCESSFUL", and "MODEM_INIT_ERROR" in various languages, enhancing user experience and localization support.

* fix wechat display
2026-01-31 22:58:08 +08:00
小鹏
96f34ec70f Refactor emoji initialization for Electron and Otto boards to use Assets system (#1704)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters

* Update Otto controller parameters for oscillation settings

- Changed default oscillation period from 500ms to 300ms.
- Increased default steps from 5.0 to 8.0.
- Updated default amplitude from 20 degrees to 0 degrees.
- Enhanced documentation with new examples for oscillation modes and sequences.

* Fix default amplitude initialization in Otto controller to use a single zero instead of two digits.

* chore: update txp666/otto-emoji-gif-component version to 1.0.3 in idf_component.yml

* Refactor Otto controller
- Consolidated movement actions into a unified tool for the Otto robot, allowing for a single action command with various parameters.
- Removed individual movement tools (walk, turn, jump, etc.) and replaced them with a more flexible action system.

* Enhance Otto robot functionality by adding WebSocket control server and IP address retrieval feature. Updated config to support WebSocket, and revised README to include new control options and usage examples.

* Add camera support for Otto Robot board

- Introduced configuration option to enable the Otto Robot camera in Kconfig.
- Updated config.h to define camera-related GPIO pins and settings.
- Modified config.json to include camera configuration.
- Enhanced otto_robot.cc to initialize I2C and camera components when the camera is enabled.
- Adjusted power_manager.h to manage battery updates during camera operations.
- Removed unused SetupChatLabel method from OttoEmojiDisplay class.

* Refactor Otto Robot configuration and initialization

- Removed the camera configuration option from Kconfig and related code.
- Introduced a new HardwareConfig struct to encapsulate hardware pin definitions and settings.
- Updated config.h to define camera and non-camera configurations using the new struct.
- Refactored otto_controller.cc and otto_robot.cc to utilize the HardwareConfig struct for initialization.
- Enhanced camera detection and initialization logic based on hardware version.
- Improved audio codec initialization based on configuration settings.

* Refactor emoji initialization for Electron and Otto boards to use Assets system

- Removed direct emoji initialization from `InitializeElectronEmojis` and `InitializeOttoEmojis` methods, delegating the responsibility to the Assets system.
- Updated `CMakeLists.txt` to set `DEFAULT_EMOJI_COLLECTION` to `otto-gif` for both boards.
- Enhanced `build_default_assets.py` to support alias mapping for Otto GIF emojis.
- Updated `idf_component.yml` to bump `otto-emoji-gif-component` version to `^1.0.5` for improved functionality.
2026-01-31 18:13:15 +08:00
Xiaoxia
aad2f60b87 fix: reset esp-box-3 display to lvgl (#1715) 2026-01-31 03:14:50 +08:00
Copilot
5b874bc3ad Fix LVGL object deletion crash and thread safety issues in SetChatMessage (#1708)
* Initial plan

* Fix LVGL object deletion issue in SetChatMessage

- Refresh child_count after deleting first_child to avoid using stale count
- Add lv_obj_is_valid() checks before accessing deleted objects
- Prevent duplicate deletion by refreshing child_count before system message deletion
- Ensures last_child validation before scrolling to it

Co-authored-by: 78 <4488133+78@users.noreply.github.com>

* Address code review feedback on LVGL object deletion fix

- Move last_child retrieval after deletion to avoid stale pointer
- Add child_count > 0 check before scrolling to prevent edge case issues
- Add lv_obj_is_valid() check for last_bubble for consistency

Co-authored-by: 78 <4488133+78@users.noreply.github.com>

* Final review: LVGL object deletion fix complete

Co-authored-by: 78 <4488133+78@users.noreply.github.com>

* Remove std::thread wrappers in display callbacks to fix thread safety

Remove detached threads that were calling SetChatMessage from download/upgrade
progress callbacks. The DisplayLockGuard mutex already provides thread safety,
so we can call display methods directly from the callback thread without
spawning additional threads, which were causing race conditions.

Co-authored-by: 78 <4488133+78@users.noreply.github.com>

* Use Application::Schedule() for display updates in callbacks

Instead of calling SetChatMessage directly from download/upgrade progress
callbacks, queue the updates using Application::Schedule(). This prevents
blocking the network receive task and avoids potential UART FIFO overflow
issues with 4G modems. The scheduled callbacks execute in the main task
thread, maintaining proper thread safety.

Co-authored-by: 78 <4488133+78@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: 78 <4488133+78@users.noreply.github.com>
2026-01-31 00:11:42 +08:00
254 changed files with 4138 additions and 2135 deletions

1
.gitignore vendored
View File

@@ -2,6 +2,7 @@ tmp/
components/
managed_components/
build/
dist/
.vscode/
.devcontainer/
sdkconfig.old

View File

@@ -9,5 +9,5 @@ include($ENV{IDF_PATH}/tools/cmake/project.cmake)
# "Trim" the build. Include the minimal set of components, main, and anything it depends on.
idf_build_set_property(MINIMAL_BUILD ON)
set(PROJECT_VER "2.2.1")
set(PROJECT_VER "2.2.2")
project(xiaozhi)

View File

@@ -1,6 +1,7 @@
# Define source files
set(SOURCES "audio/audio_codec.cc"
"audio/audio_service.cc"
"audio/demuxer/ogg_demuxer.cc"
"audio/codecs/no_audio_codec.cc"
"audio/codecs/box_audio_codec.cc"
"audio/codecs/es8311_audio_codec.cc"
@@ -38,7 +39,7 @@ set(SOURCES "audio/audio_codec.cc"
"main.cc"
)
set(INCLUDE_DIRS "." "display" "display/lvgl_display" "display/lvgl_display/jpg" "audio" "protocols")
set(INCLUDE_DIRS "." "display" "display/lvgl_display" "display/lvgl_display/jpg" "audio" "audio/demuxer" "protocols")
# Add board common files
list(APPEND SOURCES
@@ -104,28 +105,28 @@ elseif(CONFIG_BOARD_TYPE_BREAD_COMPACT_ESP32_LCD)
set(BUILTIN_ICON_FONT font_awesome_14_1)
elseif(CONFIG_BOARD_TYPE_DF_K10)
set(BOARD_TYPE "df-k10")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_DF_S3_AI_CAM)
set(BOARD_TYPE "df-s3-ai-cam")
elseif(CONFIG_BOARD_TYPE_ESP_BOX_3)
set(BOARD_TYPE "esp-box-3")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
set(EMOTE_RESOLUTION "320_240")
elseif(CONFIG_BOARD_TYPE_ESP_BOX)
set(BOARD_TYPE "esp-box")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
set(EMOTE_RESOLUTION "320_240")
elseif(CONFIG_BOARD_TYPE_ESP_BOX_LITE)
set(BOARD_TYPE "esp-box-lite")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_KEVIN_BOX_2)
set(BOARD_TYPE "kevin-box-2")
set(BUILTIN_TEXT_FONT font_puhui_basic_14_1)
@@ -134,14 +135,14 @@ elseif(CONFIG_BOARD_TYPE_KEVIN_C3)
set(BOARD_TYPE "kevin-c3")
elseif(CONFIG_BOARD_TYPE_KEVIN_SP_V3_DEV)
set(BOARD_TYPE "kevin-sp-v3-dev")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_KEVIN_SP_V4_DEV)
set(BOARD_TYPE "kevin-sp-v4-dev")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_KEVIN_YUYING_313LCD)
set(BOARD_TYPE "kevin-yuying-313lcd")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
@@ -149,9 +150,9 @@ elseif(CONFIG_BOARD_TYPE_KEVIN_YUYING_313LCD)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_LICHUANG_DEV_S3)
set(BOARD_TYPE "lichuang-dev")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_LICHUANG_DEV_C3)
set(BOARD_TYPE "lichuang-c3-dev")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
@@ -201,6 +202,11 @@ elseif(CONFIG_BOARD_TYPE_M5STACK_ATOM_S3R_CAM_M12_ECHO_BASE)
set(BOARD_TYPE "atoms3r-cam-m12-echo-base")
elseif(CONFIG_BOARD_TYPE_M5STACK_ATOM_ECHOS3R)
set(BOARD_TYPE "atom-echos3r")
elseif(CONFIG_BOARD_TYPE_M5STACK_CARDPUTER_ADV)
set(BOARD_TYPE "m5stack-cardputer-adv")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_32)
elseif(CONFIG_BOARD_TYPE_M5STACK_ATOM_MATRIX_ECHO_BASE)
set(BOARD_TYPE "atommatrix-echo-base")
elseif(CONFIG_BOARD_TYPE_XMINI_C3_V3)
@@ -250,131 +256,181 @@ elseif(CONFIG_BOARD_TYPE_ESP_SENSAIRSHUTTLE)
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_32)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_AUDIO_BOARD)
set(BOARD_TYPE "waveshare-s3-audio-board")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_AUDIO_BOARD)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-audio-board")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_32)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_8)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_1_8)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-amoled-1.8")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_1_8)
set(BOARD_TYPE "waveshare-c6-touch-amoled-1.8")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_AMOLED_1_8)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-c6-touch-amoled-1.8")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_2_06)
set(BOARD_TYPE "waveshare-s3-touch-amoled-2.06")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_2_06)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-amoled-2.06")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_2_06)
set(BOARD_TYPE "waveshare-c6-touch-amoled-2.06")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_AMOLED_2_06)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-c6-touch-amoled-2.06")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_4B)
set(BOARD_TYPE "waveshare-s3-touch-lcd-4b")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_4B)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-4b")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_4_3C)
set(BOARD_TYPE "waveshare-s3-touch-lcd-4.3c")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_4_3C)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-4.3c")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_75)
set(BOARD_TYPE "waveshare-s3-touch-amoled-1.75")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_1_75)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-amoled-1.75")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_83)
set(BOARD_TYPE "waveshare-s3-touch-lcd-1.83")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_83)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-1.83")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_85C)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_85C)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-1.85c")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_85)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_85)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-1.85")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_46)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_46)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-1.46")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_5)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_3_5)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-3.5")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_5B)
set(BOARD_TYPE "waveshare-s3-touch-lcd-3.5b")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_3_5B)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-3.5b")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_32)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_ePaper_1_54)
set(BOARD_TYPE "waveshare-s3-epaper-1.54")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_ePaper_1_54)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-epaper-1.54")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_RLCD_4_2)
set(BOARD_TYPE "waveshare-s3-rlcd-4.2")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_RLCD_4_2)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-rlcd-4.2")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_49)
set(BOARD_TYPE "waveshare-s3-touch-lcd-3.49")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_3_49)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-lcd-3.49")
set(LVGL_TEXT_FONT font_puhui_basic_30_4)
set(LVGL_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_C6_LCD_1_69)
set(BOARD_TYPE "waveshare-c6-lcd-1.69")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_C6_LCD_1_69)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-c6-lcd-1.69")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_C6_TOUCH_LCD_1_83)
set(BOARD_TYPE "waveshare-c6-touch-lcd-1.83")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_LCD_1_83)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-c6-touch-lcd-1.83")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_1_43)
set(BOARD_TYPE "waveshare-c6-touch-amoled-1.43")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_AMOLED_1_43)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-c6-touch-amoled-1.43")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_1_32)
set(BOARD_TYPE "waveshare-c6-touch-amoled-1.32")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_AMOLED_1_32)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-c6-touch-amoled-1.32")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_32)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_32)
set(BOARD_TYPE "waveshare-s3-touch-amoled-1.32")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_1_32)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-s3-touch-amoled-1.32")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_32)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_P4_NANO)
set(BOARD_TYPE "waveshare-p4-nano")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_P4_NANO)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-p4-nano")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_4B)
set(BOARD_TYPE "waveshare-p4-wifi6-touch-lcd-4b")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_4B)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-p4-wifi6-touch-lcd")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_7B)
set(BOARD_TYPE "waveshare-p4-wifi6-touch-lcd-7b")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_7B)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-p4-wifi6-touch-lcd")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_XC)
set(BOARD_TYPE "waveshare-p4-wifi6-touch-lcd-xc")
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_3_4C)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-p4-wifi6-touch-lcd")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_4C)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-p4-wifi6-touch-lcd")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_7)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-p4-wifi6-touch-lcd")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_8)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-p4-wifi6-touch-lcd")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_10_1)
set(MANUFACTURER "waveshare")
set(BOARD_TYPE "esp32-p4-wifi6-touch-lcd")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_30_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
@@ -436,29 +492,29 @@ elseif(CONFIG_BOARD_TYPE_MOVECALL_CUICAN_ESP32S3)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
elseif(CONFIG_BOARD_TYPE_ATK_DNESP32S3)
set(BOARD_TYPE "atk-dnesp32s3")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_ATK_DNESP32S3_BOX)
set(BOARD_TYPE "atk-dnesp32s3-box")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_ATK_DNESP32S3_BOX0)
set(BOARD_TYPE "atk-dnesp32s3-box0")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_ATK_DNESP32S3_BOX2_WIFI)
set(BOARD_TYPE "atk-dnesp32s3-box2-wifi")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_ATK_DNESP32S3_BOX2_4G)
set(BOARD_TYPE "atk-dnesp32s3-box2-4g")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_ATK_DNESP32S3M_WIFI)
set(BOARD_TYPE "atk-dnesp32s3m-wifi")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
@@ -499,24 +555,24 @@ elseif(CONFIG_BOARD_TYPE_XINGZHI_CUBE_0_96OLED_ML307)
set(BUILTIN_ICON_FONT font_awesome_14_1)
elseif(CONFIG_BOARD_TYPE_XINGZHI_CUBE_1_54TFT_WIFI)
set(BOARD_TYPE "xingzhi-cube-1.54tft-wifi")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_XINGZHI_CUBE_1_54TFT_ML307)
set(BOARD_TYPE "xingzhi-cube-1.54tft-ml307")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_XINGZHI_METAL_1_54_WIFI)
set(BOARD_TYPE "xingzhi-metal-1.54-wifi")
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4)
set(BUILTIN_TEXT_FONT font_noto_basic_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_SEEED_STUDIO_SENSECAP_WATCHER)
set(BOARD_TYPE "sensecap-watcher")
set(BUILTIN_TEXT_FONT font_puhui_basic_30_4)
set(BUILTIN_TEXT_FONT font_noto_basic_30_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64)
set(DEFAULT_EMOJI_COLLECTION noto-emoji_128)
elseif(CONFIG_BOARD_TYPE_DOIT_S3_AIBOX)
set(BOARD_TYPE "doit-s3-aibox")
elseif(CONFIG_BOARD_TYPE_MIXGO_NOVA)
@@ -586,10 +642,12 @@ elseif(CONFIG_BOARD_TYPE_OTTO_ROBOT)
set(BOARD_TYPE "otto-robot")
set(BUILTIN_TEXT_FONT font_puhui_16_4)
set(BUILTIN_ICON_FONT font_awesome_16_4)
set(DEFAULT_EMOJI_COLLECTION otto-gif)
elseif(CONFIG_BOARD_TYPE_ELECTRON_BOT)
set(BOARD_TYPE "electron-bot")
set(BUILTIN_TEXT_FONT font_puhui_20_4)
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION otto-gif)
elseif(CONFIG_BOARD_TYPE_BREAD_COMPACT_WIFI_CAM)
set(BOARD_TYPE "bread-compact-wifi-s3cam")
set(BUILTIN_TEXT_FONT font_puhui_basic_16_4)
@@ -637,8 +695,15 @@ elseif(CONFIG_BOARD_TYPE_HU_087)
endif()
file(GLOB BOARD_SOURCES
if(MANUFACTURER)
${CMAKE_CURRENT_SOURCE_DIR}/boards/${MANUFACTURER}/${BOARD_TYPE}/*.cc
${CMAKE_CURRENT_SOURCE_DIR}/boards/${MANUFACTURER}/${BOARD_TYPE}/*.c
else
${CMAKE_CURRENT_SOURCE_DIR}/boards/${BOARD_TYPE}/*.cc
${CMAKE_CURRENT_SOURCE_DIR}/boards/${BOARD_TYPE}/*.c
endif()
)
list(APPEND SOURCES ${BOARD_SOURCES})
@@ -777,6 +842,8 @@ if(CONFIG_IDF_TARGET_ESP32)
"display/lvgl_display/jpg/image_to_jpeg.cpp"
"display/lvgl_display/jpg/jpeg_to_image.c"
"boards/common/nt26_board.cc"
"boards/common/ml307_board.cc"
"boards/common/dual_network_board.cc"
)
endif()

View File

@@ -251,86 +251,101 @@ choice BOARD_TYPE
config BOARD_TYPE_M5STACK_ATOM_ECHOS3R
bool "M5Stack AtomEchoS3R"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_M5STACK_CARDPUTER_ADV
bool "M5Stack Cardputer Adv"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_M5STACK_ATOM_MATRIX_ECHO_BASE
bool "M5Stack AtomMatrix + Echo Base"
depends on IDF_TARGET_ESP32
config BOARD_TYPE_WAVESHARE_S3_AUDIO_BOARD
config BOARD_TYPE_WAVESHARE_ESP32_S3_AUDIO_BOARD
bool "Waveshare ESP32-S3-Audio-Board"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_8
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_1_8
bool "Waveshare ESP32-S3-Touch-AMOLED-1.8"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_2_06
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_2_06
bool "Waveshare ESP32-S3-Touch-AMOLED-2.06"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_2_06
config BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_AMOLED_2_06
bool "Waveshare ESP32-C6-Touch-AMOLED-2.06"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_75
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_1_75
bool "Waveshare ESP32-S3-Touch-AMOLED-1.75"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_83
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_83
bool "Waveshare ESP32-S3-Touch-LCD-1.83"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_4B
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_4B
bool "Waveshare ESP32-S3-Touch-LCD-4B"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_4_3C
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_4_3C
bool "Waveshare ESP32-S3-Touch-LCD-4.3C"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_85C
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_85C
bool "Waveshare ESP32-S3-Touch-LCD-1.85C"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_85
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_85
bool "Waveshare ESP32-S3-Touch-LCD-1.85"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_46
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_46
bool "Waveshare ESP32-S3-Touch-LCD-1.46"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_C6_LCD_1_69
config BOARD_TYPE_WAVESHARE_ESP32_C6_LCD_1_69
bool "Waveshare ESP32-C6-LCD-1.69"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_C6_TOUCH_LCD_1_83
config BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_LCD_1_83
bool "Waveshare ESP32-C6-Touch-LCD-1.83"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_1_43
config BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_AMOLED_1_43
bool "Waveshare ESP32-C6-Touch-AMOLOED-1.43"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_1_32
config BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_AMOLED_1_32
bool "Waveshare ESP32-C6-Touch-AMOLOED-1.32"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_1_8
config BOARD_TYPE_WAVESHARE_ESP32_C6_TOUCH_AMOLED_1_8
bool "Waveshare ESP32-C6-Touch-AMOLED-1.8"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_32
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_1_32
bool "Waveshare ESP32-S3-Touch-AMOLOED-1.32"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_49
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_3_49
bool "Waveshare ESP32-S3-Touch-LCD-3.49"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_5
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_3_5
bool "Waveshare ESP32-S3-Touch-LCD-3.5"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_ePaper_1_54
config BOARD_TYPE_WAVESHARE_ESP32_S3_ePaper_1_54
bool "Waveshare ESP32-S3-ePaper-1.54"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_RLCD_4_2
config BOARD_TYPE_WAVESHARE_ESP32_S3_RLCD_4_2
bool "Waveshare ESP32-S3-RLCD-4.2"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_5B
config BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_3_5B
bool "Waveshare ESP32-S3-Touch-LCD-3.5B"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_P4_NANO
config BOARD_TYPE_WAVESHARE_ESP32_P4_NANO
bool "Waveshare ESP32-P4-NANO"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_4B
config BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_4B
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-4B"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_7B
config BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_7B
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-7B"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_XC
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C or ESP32-P4-WIFI6-Touch-LCD-4C"
config BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_3_4C
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_4C
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-4C"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_7
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-7"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_8
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-8"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_10_1
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-10.1"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_TUDOUZI
bool "土豆子"
@@ -526,7 +541,7 @@ choice DISPLAY_OLED_TYPE
endchoice
choice DISPLAY_LCD_TYPE
depends on BOARD_TYPE_BREAD_COMPACT_WIFI_LCD || BOARD_TYPE_BREAD_COMPACT_ESP32_LCD || BOARD_TYPE_CGC || BOARD_TYPE_WAVESHARE_P4_NANO || BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_XC || BOARD_TYPE_BREAD_COMPACT_WIFI_CAM
depends on BOARD_TYPE_BREAD_COMPACT_WIFI_LCD || BOARD_TYPE_BREAD_COMPACT_ESP32_LCD || BOARD_TYPE_CGC || BOARD_TYPE_BREAD_COMPACT_WIFI_CAM
prompt "LCD Type"
default LCD_ST7789_240X320
help
@@ -561,14 +576,6 @@ choice DISPLAY_LCD_TYPE
bool "ILI9341 240*320, Non-IPS"
config LCD_GC9A01_240X240
bool "GC9A01 240*240 Circle"
config LCD_TYPE_800_1280_10_1_INCH
bool "Waveshare 101M-8001280-IPS-CT-K Display"
config LCD_TYPE_800_1280_10_1_INCH_A
bool "Waveshare 10.1-DSI-TOUCH-A Display"
config LCD_TYPE_800_800_3_4_INCH
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C with 800*800 3.4inch round display"
config LCD_TYPE_720_720_4_INCH
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-4C with 720*720 4inch round display"
config LCD_CUSTOM
bool "Custom LCD (自定义屏幕参数)"
endchoice
@@ -586,7 +593,7 @@ choice DISPLAY_ESP32S3_KORVO2_V3
endchoice
choice DISPLAY_ESP32S3_AUDIO_BOARD
depends on BOARD_TYPE_WAVESHARE_S3_AUDIO_BOARD
depends on BOARD_TYPE_WAVESHARE_ESP32_S3_AUDIO_BOARD
prompt "ESP32S3_AUDIO_BOARD LCD Type"
default AUDIO_BOARD_LCD_JD9853
help
@@ -677,6 +684,16 @@ config SEND_WAKE_WORD_DATA
help
Send wake word data to the server as the first message of the conversation and wait for response
config WAKE_WORD_DETECTION_IN_LISTENING
bool "Enable Wake Word Detection in Listening Mode"
default n
depends on USE_AFE_WAKE_WORD || USE_CUSTOM_WAKE_WORD
help
Enable wake word detection while in listening mode.
When enabled, the device can detect wake word during listening,
which allows interrupting the current conversation.
When disabled (default), wake word detection is turned off during listening.
config USE_AUDIO_PROCESSOR
bool "Enable Audio Noise Reduction"
default y
@@ -688,11 +705,12 @@ config USE_DEVICE_AEC
bool "Enable Device-Side AEC"
default n
depends on USE_AUDIO_PROCESSOR && (BOARD_TYPE_ESP_BOX_3 || BOARD_TYPE_ESP_BOX || BOARD_TYPE_ESP_BOX_LITE \
|| BOARD_TYPE_LICHUANG_DEV_S3 || BOARD_TYPE_ESP_KORVO2_V3 || BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_75 || BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_83\
|| BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_2_06 || BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_4B || BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_4B || BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_7B \
|| BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_XC || BOARD_TYPE_ESP_S3_LCD_EV_Board_2 || BOARD_TYPE_YUNLIAO_S3 \
|| BOARD_TYPE_ECHOEAR || BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_49 || BOARD_TYPE_WAVESHARE_S3_RLCD_4_2 || BOARD_TYPE_ZHENGCHEN_CAM || BOARD_TYPE_ZHENGCHEN_CAM_ML307 \
|| BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_4_3C)
|| BOARD_TYPE_LICHUANG_DEV_S3 || BOARD_TYPE_ESP_KORVO2_V3 || BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_1_75 || BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_1_83\
|| BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_AMOLED_2_06 || BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_4B || BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_4B || BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_7B \
|| BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_3_4C || BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_4C || BOARD_TYPE_ESP_S3_LCD_EV_Board_2 || BOARD_TYPE_YUNLIAO_S3 \
|| BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_7 || BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_8 || BOARD_TYPE_WAVESHARE_ESP32_P4_WIFI6_TOUCH_LCD_10_1 \
|| BOARD_TYPE_ECHOEAR || BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_3_49 || BOARD_TYPE_WAVESHARE_ESP32_S3_RLCD_4_2 || BOARD_TYPE_ZHENGCHEN_CAM || BOARD_TYPE_ZHENGCHEN_CAM_ML307 \
|| BOARD_TYPE_WAVESHARE_ESP32_S3_TOUCH_LCD_4_3C)
help
To work properly, device-side AEC requires a clean output reference path from the speaker signal and physical acoustic isolation between the microphone and speaker.

View File

@@ -64,7 +64,7 @@ void Application::Initialize() {
// Setup the display
auto display = board.GetDisplay();
display->SetupUI();
// Print board name/version info
display->SetChatMessage("system", SystemInfo::GetUserAgent().c_str());
@@ -309,13 +309,15 @@ void Application::HandleActivationDoneEvent() {
display->ShowNotification(message.c_str());
display->SetChatMessage("system", "");
// Play the success sound to indicate the device is ready
audio_service_.PlaySound(Lang::Sounds::OGG_SUCCESS);
// Release OTA object after activation is complete
ota_.reset();
auto& board = Board::GetInstance();
board.SetPowerSaveLevel(PowerSaveLevel::LOW_POWER);
Schedule([this]() {
// Play the success sound to indicate the device is ready
audio_service_.PlaySound(Lang::Sounds::OGG_SUCCESS);
});
}
void Application::ActivationTask() {
@@ -691,14 +693,16 @@ void Application::HandleToggleChatEvent() {
}
if (state == kDeviceStateIdle) {
ListeningMode mode = GetDefaultListeningMode();
if (!protocol_->IsAudioChannelOpened()) {
SetDeviceState(kDeviceStateConnecting);
if (!protocol_->OpenAudioChannel()) {
return;
}
// Schedule to let the state change be processed first (UI update)
Schedule([this, mode]() {
ContinueOpenAudioChannel(mode);
});
return;
}
SetListeningMode(aec_mode_ == kAecOff ? kListeningModeAutoStop : kListeningModeRealtime);
SetListeningMode(mode);
} else if (state == kDeviceStateSpeaking) {
AbortSpeaking(kAbortReasonNone);
} else if (state == kDeviceStateListening) {
@@ -706,6 +710,21 @@ void Application::HandleToggleChatEvent() {
}
}
void Application::ContinueOpenAudioChannel(ListeningMode mode) {
// Check state again in case it was changed during scheduling
if (GetDeviceState() != kDeviceStateConnecting) {
return;
}
if (!protocol_->IsAudioChannelOpened()) {
if (!protocol_->OpenAudioChannel()) {
return;
}
}
SetListeningMode(mode);
}
void Application::HandleStartListeningEvent() {
auto state = GetDeviceState();
@@ -726,11 +745,12 @@ void Application::HandleStartListeningEvent() {
if (state == kDeviceStateIdle) {
if (!protocol_->IsAudioChannelOpened()) {
SetDeviceState(kDeviceStateConnecting);
if (!protocol_->OpenAudioChannel()) {
return;
}
// Schedule to let the state change be processed first (UI update)
Schedule([this]() {
ContinueOpenAudioChannel(kListeningModeManualStop);
});
return;
}
SetListeningMode(kListeningModeManualStop);
} else if (state == kDeviceStateSpeaking) {
AbortSpeaking(kAbortReasonNone);
@@ -759,42 +779,79 @@ void Application::HandleWakeWordDetectedEvent() {
}
auto state = GetDeviceState();
auto wake_word = audio_service_.GetLastWakeWord();
ESP_LOGI(TAG, "Wake word detected: %s (state: %d)", wake_word.c_str(), (int)state);
if (state == kDeviceStateIdle) {
audio_service_.EncodeWakeWord();
auto wake_word = audio_service_.GetLastWakeWord();
if (!protocol_->IsAudioChannelOpened()) {
SetDeviceState(kDeviceStateConnecting);
if (!protocol_->OpenAudioChannel()) {
audio_service_.EnableWakeWordDetection(true);
return;
}
// Schedule to let the state change be processed first (UI update),
// then continue with OpenAudioChannel which may block for ~1 second
Schedule([this, wake_word]() {
ContinueWakeWordInvoke(wake_word);
});
return;
}
auto wake_word = audio_service_.GetLastWakeWord();
ESP_LOGI(TAG, "Wake word detected: %s", wake_word.c_str());
#if CONFIG_SEND_WAKE_WORD_DATA
// Encode and send the wake word data to the server
while (auto packet = audio_service_.PopWakeWordPacket()) {
protocol_->SendAudio(std::move(packet));
}
// Set the chat state to wake word detected
protocol_->SendWakeWordDetected(wake_word);
SetListeningMode(aec_mode_ == kAecOff ? kListeningModeAutoStop : kListeningModeRealtime);
#else
// Set flag to play popup sound after state changes to listening
// (PlaySound here would be cleared by ResetDecoder in EnableVoiceProcessing)
play_popup_on_listening_ = true;
SetListeningMode(aec_mode_ == kAecOff ? kListeningModeAutoStop : kListeningModeRealtime);
#endif
} else if (state == kDeviceStateSpeaking) {
// Channel already opened, continue directly
ContinueWakeWordInvoke(wake_word);
} else if (state == kDeviceStateSpeaking || state == kDeviceStateListening) {
AbortSpeaking(kAbortReasonWakeWordDetected);
// Clear send queue to avoid sending residues to server
while (audio_service_.PopPacketFromSendQueue());
if (state == kDeviceStateListening) {
protocol_->SendStartListening(GetDefaultListeningMode());
audio_service_.ResetDecoder();
audio_service_.PlaySound(Lang::Sounds::OGG_POPUP);
// Re-enable wake word detection as it was stopped by the detection itself
audio_service_.EnableWakeWordDetection(true);
} else {
// Play popup sound and start listening again
play_popup_on_listening_ = true;
SetListeningMode(GetDefaultListeningMode());
}
} else if (state == kDeviceStateActivating) {
// Restart the activation check if the wake word is detected during activation
SetDeviceState(kDeviceStateIdle);
}
}
void Application::ContinueWakeWordInvoke(const std::string& wake_word) {
// Check state again in case it was changed during scheduling
if (GetDeviceState() != kDeviceStateConnecting) {
return;
}
if (!protocol_->IsAudioChannelOpened()) {
if (!protocol_->OpenAudioChannel()) {
audio_service_.EnableWakeWordDetection(true);
return;
}
}
ESP_LOGI(TAG, "Wake word detected: %s", wake_word.c_str());
#if CONFIG_SEND_WAKE_WORD_DATA
// Encode and send the wake word data to the server
while (auto packet = audio_service_.PopWakeWordPacket()) {
protocol_->SendAudio(std::move(packet));
}
// Set the chat state to wake word detected
protocol_->SendWakeWordDetected(wake_word);
// Set flag to play popup sound after state changes to listening
play_popup_on_listening_ = true;
SetListeningMode(GetDefaultListeningMode());
#else
// Set flag to play popup sound after state changes to listening
// (PlaySound here would be cleared by ResetDecoder in EnableVoiceProcessing)
play_popup_on_listening_ = true;
SetListeningMode(GetDefaultListeningMode());
#endif
}
void Application::HandleStateChangedEvent() {
DeviceState new_state = state_machine_.GetState();
clock_ticks_ = 0;
@@ -808,7 +865,8 @@ void Application::HandleStateChangedEvent() {
case kDeviceStateUnknown:
case kDeviceStateIdle:
display->SetStatus(Lang::Strings::STANDBY);
display->SetEmotion("neutral");
display->ClearChatMessages(); // Clear messages first
display->SetEmotion("neutral"); // Then set emotion (wechat mode checks child count)
audio_service_.EnableVoiceProcessing(false);
audio_service_.EnableWakeWordDetection(true);
break;
@@ -822,7 +880,7 @@ void Application::HandleStateChangedEvent() {
display->SetEmotion("neutral");
// Make sure the audio processor is running
if (!audio_service_.IsAudioProcessorRunning()) {
if (play_popup_on_listening_ || !audio_service_.IsAudioProcessorRunning()) {
// For auto mode, wait for playback queue to be empty before enabling voice processing
// This prevents audio truncation when STOP arrives late due to network jitter
if (listening_mode_ == kListeningModeAutoStop) {
@@ -832,9 +890,16 @@ void Application::HandleStateChangedEvent() {
// Send the start listening command
protocol_->SendStartListening(listening_mode_);
audio_service_.EnableVoiceProcessing(true);
audio_service_.EnableWakeWordDetection(false);
}
#ifdef CONFIG_WAKE_WORD_DETECTION_IN_LISTENING
// Enable wake word detection in listening mode (configured via Kconfig)
audio_service_.EnableWakeWordDetection(audio_service_.IsAfeWakeWord());
#else
// Disable wake word detection in listening mode
audio_service_.EnableWakeWordDetection(false);
#endif
// Play popup sound after ResetDecoder (in EnableVoiceProcessing) has been called
if (play_popup_on_listening_) {
play_popup_on_listening_ = false;
@@ -882,6 +947,10 @@ void Application::SetListeningMode(ListeningMode mode) {
SetDeviceState(kDeviceStateListening);
}
ListeningMode Application::GetDefaultListeningMode() const {
return aec_mode_ == kAecOff ? kListeningModeAutoStop : kListeningModeRealtime;
}
void Application::Reboot() {
ESP_LOGI(TAG, "Rebooting...");
// Disconnect the audio channel
@@ -959,27 +1028,14 @@ void Application::WakeWordInvoke(const std::string& wake_word) {
if (!protocol_->IsAudioChannelOpened()) {
SetDeviceState(kDeviceStateConnecting);
if (!protocol_->OpenAudioChannel()) {
audio_service_.EnableWakeWordDetection(true);
return;
}
// Schedule to let the state change be processed first (UI update)
Schedule([this, wake_word]() {
ContinueWakeWordInvoke(wake_word);
});
return;
}
ESP_LOGI(TAG, "Wake word detected: %s", wake_word.c_str());
#if CONFIG_USE_AFE_WAKE_WORD || CONFIG_USE_CUSTOM_WAKE_WORD
// Encode and send the wake word data to the server
while (auto packet = audio_service_.PopWakeWordPacket()) {
protocol_->SendAudio(std::move(packet));
}
// Set the chat state to wake word detected
protocol_->SendWakeWordDetected(wake_word);
SetListeningMode(aec_mode_ == kAecOff ? kListeningModeAutoStop : kListeningModeRealtime);
#else
// Set flag to play popup sound after state changes to listening
// (PlaySound here would be cleared by ResetDecoder in EnableVoiceProcessing)
play_popup_on_listening_ = true;
SetListeningMode(aec_mode_ == kAecOff ? kListeningModeAutoStop : kListeningModeRealtime);
#endif
// Channel already opened, continue directly
ContinueWakeWordInvoke(wake_word);
} else if (state == kDeviceStateSpeaking) {
Schedule([this]() {
AbortSpeaking(kAbortReasonNone);

View File

@@ -153,6 +153,8 @@ private:
void HandleNetworkDisconnectedEvent();
void HandleActivationDoneEvent();
void HandleWakeWordDetectedEvent();
void ContinueOpenAudioChannel(ListeningMode mode);
void ContinueWakeWordInvoke(const std::string& wake_word);
// Activation task (runs in background)
void ActivationTask();
@@ -163,6 +165,7 @@ private:
void InitializeProtocol();
void ShowActivationCode(const std::string& code, const std::string& message);
void SetListeningMode(ListeningMode mode);
ListeningMode GetDefaultListeningMode() const;
// State change handler called by state machine
void OnStateChanged(DeviceState old_state, DeviceState new_state);

View File

@@ -12,6 +12,7 @@
#include <esp_log.h>
#include <esp_timer.h>
#include <esp_heap_caps.h>
#include <cbin_font.h>
@@ -464,16 +465,21 @@ bool Assets::Download(std::string url, std::function<void(int progress, size_t s
SECTOR_SIZE, content_length, sectors_to_erase, total_erase_size);
// 写入新的资源文件到分区一边erase一边写入
char buffer[512];
char* buffer = (char*)heap_caps_malloc(SECTOR_SIZE, MALLOC_CAP_INTERNAL);
if (buffer == nullptr) {
ESP_LOGE(TAG, "Failed to allocate buffer");
return false;
}
size_t total_written = 0;
size_t recent_written = 0;
size_t current_sector = 0;
auto last_calc_time = esp_timer_get_time();
while (true) {
int ret = http->Read(buffer, sizeof(buffer));
int ret = http->Read(buffer, SECTOR_SIZE);
if (ret < 0) {
ESP_LOGE(TAG, "Failed to read HTTP data: %s", esp_err_to_name(ret));
heap_caps_free(buffer);
return false;
}
@@ -493,6 +499,7 @@ bool Assets::Download(std::string url, std::function<void(int progress, size_t s
// 确保擦除范围不超过分区大小
if (sector_end > partition_->size) {
ESP_LOGE(TAG, "Sector end (%u) exceeds partition size (%lu)", sector_end, partition_->size);
heap_caps_free(buffer);
return false;
}
@@ -500,6 +507,7 @@ bool Assets::Download(std::string url, std::function<void(int progress, size_t s
esp_err_t err = esp_partition_erase_range(partition_, sector_start, SECTOR_SIZE);
if (err != ESP_OK) {
ESP_LOGE(TAG, "Failed to erase sector %u at offset %u: %s", current_sector, sector_start, esp_err_to_name(err));
heap_caps_free(buffer);
return false;
}
@@ -510,6 +518,7 @@ bool Assets::Download(std::string url, std::function<void(int progress, size_t s
esp_err_t err = esp_partition_write(partition_, total_written, buffer, ret);
if (err != ESP_OK) {
ESP_LOGE(TAG, "Failed to write to assets partition at offset %u: %s", total_written, esp_err_to_name(err));
heap_caps_free(buffer);
return false;
}
@@ -531,6 +540,7 @@ bool Assets::Download(std::string url, std::function<void(int progress, size_t s
}
http->Close();
heap_caps_free(buffer);
if (total_written != content_length) {
ESP_LOGE(TAG, "Downloaded size (%u) does not match expected size (%u)", total_written, content_length);

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "جاري تحميل الموارد...",
"PLEASE_WAIT": "يرجى الانتظار...",
"FOUND_NEW_ASSETS": "تم العثور على موارد جديدة: %s",
"HELLO_MY_FRIEND": "مرحباً، صديقي!"
"HELLO_MY_FRIEND": "مرحباً، صديقي!",
"CONNECTION_SUCCESSFUL": "تم الاتصال بنجاح",
"FLIGHT_MODE_OFF": "وضع الطيران معطل",
"FLIGHT_MODE_ON": "وضع الطيران قيد التشغيل",
"MODEM_INIT_ERROR": "فشل تهيئة المودم"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Намерени нови ресурси: %s",
"DOWNLOAD_ASSETS_FAILED": "Неуспешно изтегляне на ресурси",
"LOADING_ASSETS": "Зареждане на ресурси...",
"HELLO_MY_FRIEND": "Здравей, мой приятел!"
"HELLO_MY_FRIEND": "Здравей, мой приятел!",
"FLIGHT_MODE_OFF": "Режим на самолет е изключен",
"FLIGHT_MODE_ON": "Режим на самолет е включен",
"MODEM_INIT_ERROR": "Неуспешна инициализация на модема"
}
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "S'han trobat nous recursos: %s",
"DOWNLOAD_ASSETS_FAILED": "No s'han pogut descarregar els recursos",
"LOADING_ASSETS": "Carregant recursos...",
"HELLO_MY_FRIEND": "Hola, amic meu!"
"HELLO_MY_FRIEND": "Hola, amic meu!",
"FLIGHT_MODE_OFF": "El mode avió està desactivat",
"FLIGHT_MODE_ON": "El mode avió està activat",
"MODEM_INIT_ERROR": "Error d'inicialització del mòdem"
}
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Načítání prostředků...",
"PLEASE_WAIT": "Prosím čekejte...",
"FOUND_NEW_ASSETS": "Nalezeny nové prostředky: %s",
"HELLO_MY_FRIEND": "Ahoj, můj příteli!"
"HELLO_MY_FRIEND": "Ahoj, můj příteli!",
"CONNECTION_SUCCESSFUL": "Připojení úspěšné",
"FLIGHT_MODE_OFF": "Letecký režim je vypnutý",
"FLIGHT_MODE_ON": "Letecký režim je zapnutý",
"MODEM_INIT_ERROR": "Chyba inicializace modemu"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Fandt nye ressourcer: %s",
"DOWNLOAD_ASSETS_FAILED": "Download af ressourcer mislykkedes",
"LOADING_ASSETS": "Indlæser ressourcer...",
"HELLO_MY_FRIEND": "Hej, min ven!"
"HELLO_MY_FRIEND": "Hej, min ven!",
"FLIGHT_MODE_OFF": "Flytilstand er slukket",
"FLIGHT_MODE_ON": "Flytilstand er tændt",
"MODEM_INIT_ERROR": "Modeminitialisering mislykkedes"
}
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Ressourcen werden geladen...",
"PLEASE_WAIT": "Bitte warten...",
"FOUND_NEW_ASSETS": "Neue Ressourcen gefunden: %s",
"HELLO_MY_FRIEND": "Hallo, mein Freund!"
"HELLO_MY_FRIEND": "Hallo, mein Freund!",
"CONNECTION_SUCCESSFUL": "Verbindung erfolgreich",
"FLIGHT_MODE_OFF": "Flugmodus ist deaktiviert",
"FLIGHT_MODE_ON": "Flugmodus ist aktiviert",
"MODEM_INIT_ERROR": "Modem-Initialisierung fehlgeschlagen"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Βρέθηκαν νέοι πόροι: %s",
"DOWNLOAD_ASSETS_FAILED": "Αποτυχία λήψης πόρων",
"LOADING_ASSETS": "Φόρτωση πόρων...",
"HELLO_MY_FRIEND": "Γεια σου, φίλε μου!"
"HELLO_MY_FRIEND": "Γεια σου, φίλε μου!",
"FLIGHT_MODE_OFF": "Η λειτουργία πτήσης είναι απενεργοποιημένη",
"FLIGHT_MODE_ON": "Η λειτουργία πτήσης είναι ενεργή",
"MODEM_INIT_ERROR": "Αποτυχία αρχικοποίησης modem"
}
}
}

View File

@@ -13,6 +13,8 @@
"REG_ERROR": "Unable to access network, please check SIM card status",
"MODEM_INIT_ERROR": "Modem initialization failed",
"DETECTING_MODULE": "Detecting module...",
"FLIGHT_MODE_ON": "Flight mode is on",
"FLIGHT_MODE_OFF": "Flight mode is off",
"REGISTERING_NETWORK": "Waiting for network...",
"CHECKING_NEW_VERSION": "Checking for new version...",
"CHECK_NEW_VERSION_FAILED": "Check for new version failed, will retry in %d seconds: %s",

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Cargando recursos...",
"PLEASE_WAIT": "Por favor espere...",
"FOUND_NEW_ASSETS": "Encontrados nuevos recursos: %s",
"HELLO_MY_FRIEND": "¡Hola, mi amigo!"
"HELLO_MY_FRIEND": "¡Hola, mi amigo!",
"CONNECTION_SUCCESSFUL": "Conexión exitosa",
"FLIGHT_MODE_OFF": "El modo avión está desactivado",
"FLIGHT_MODE_ON": "El modo avión está activado",
"MODEM_INIT_ERROR": "Error de inicialización del módem"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "منابع جدید یافت شد: %s",
"DOWNLOAD_ASSETS_FAILED": "دانلود منابع ناموفق بود",
"LOADING_ASSETS": "بارگذاری منابع...",
"HELLO_MY_FRIEND": "سلام، دوست من!"
"HELLO_MY_FRIEND": "سلام، دوست من!",
"FLIGHT_MODE_OFF": "حالت پرواز خاموش است",
"FLIGHT_MODE_ON": "حالت پرواز روشن است",
"MODEM_INIT_ERROR": "خطا در راه‌اندازی مودم"
}
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Ladataan resursseja...",
"PLEASE_WAIT": "Odota hetki...",
"FOUND_NEW_ASSETS": "Löydetty uusia resursseja: %s",
"HELLO_MY_FRIEND": "Hei, ystäväni!"
"HELLO_MY_FRIEND": "Hei, ystäväni!",
"CONNECTION_SUCCESSFUL": "Yhteys onnistui",
"FLIGHT_MODE_OFF": "Lentotila on pois päältä",
"FLIGHT_MODE_ON": "Lentotila on päällä",
"MODEM_INIT_ERROR": "Modeemin alustus epäonnistui"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Nakahanap ng mga bagong assets: %s",
"DOWNLOAD_ASSETS_FAILED": "Nabigo ang pag-download ng mga assets",
"LOADING_ASSETS": "Nilo-load ang mga assets...",
"HELLO_MY_FRIEND": "Kumusta, kaibigan ko!"
"HELLO_MY_FRIEND": "Kumusta, kaibigan ko!",
"FLIGHT_MODE_OFF": "Naka-off ang flight mode",
"FLIGHT_MODE_ON": "Naka-on ang flight mode",
"MODEM_INIT_ERROR": "Nabigo ang pag-initialize ng modem"
}
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Chargement des ressources...",
"PLEASE_WAIT": "Veuillez patienter...",
"FOUND_NEW_ASSETS": "Nouvelles ressources trouvées: %s",
"HELLO_MY_FRIEND": "Bonjour, mon ami !"
"HELLO_MY_FRIEND": "Bonjour, mon ami !",
"CONNECTION_SUCCESSFUL": "Connexion réussie",
"FLIGHT_MODE_OFF": "Le mode avion est désactivé",
"FLIGHT_MODE_ON": "Le mode avion est activé",
"MODEM_INIT_ERROR": "Échec de l'initialisation du modem"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "נמצאו משאבים חדשים: %s",
"DOWNLOAD_ASSETS_FAILED": "הורדת משאבים נכשלה",
"LOADING_ASSETS": "טוען משאבים...",
"HELLO_MY_FRIEND": "שלום, ידידי!"
"HELLO_MY_FRIEND": "שלום, ידידי!",
"FLIGHT_MODE_OFF": "מצב טיסה כבוי",
"FLIGHT_MODE_ON": "מצב טיסה מופעל",
"MODEM_INIT_ERROR": "אתחול המודם נכשל"
}
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "संसाधन लोड हो रहे हैं...",
"PLEASE_WAIT": "कृपया प्रतीक्षा करें...",
"FOUND_NEW_ASSETS": "नए संसाधन मिले: %s",
"HELLO_MY_FRIEND": "नमस्ते, मेरे दोस्त!"
"HELLO_MY_FRIEND": "नमस्ते, मेरे दोस्त!",
"CONNECTION_SUCCESSFUL": "कनेक्शन सफल",
"FLIGHT_MODE_OFF": "फ़्लाइट मोड बंद है",
"FLIGHT_MODE_ON": "फ़्लाइट मोड चालू है",
"MODEM_INIT_ERROR": "मॉडेम आरंभीकरण विफल"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Pronađeni novi resursi: %s",
"DOWNLOAD_ASSETS_FAILED": "Preuzimanje resursa nije uspjelo",
"LOADING_ASSETS": "Učitavanje resursa...",
"HELLO_MY_FRIEND": "Bok, moj prijatelju!"
"HELLO_MY_FRIEND": "Bok, moj prijatelju!",
"FLIGHT_MODE_OFF": "Način rada u zrakoplovu je isključen",
"FLIGHT_MODE_ON": "Način rada u zrakoplovu je uključen",
"MODEM_INIT_ERROR": "Neuspjela inicijalizacija modema"
}
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Új erőforrások találva: %s",
"DOWNLOAD_ASSETS_FAILED": "Az erőforrások letöltése sikertelen",
"LOADING_ASSETS": "Erőforrások betöltése...",
"HELLO_MY_FRIEND": "Helló, barátom!"
"HELLO_MY_FRIEND": "Helló, barátom!",
"FLIGHT_MODE_OFF": "A repülési mód ki van kapcsolva",
"FLIGHT_MODE_ON": "A repülési mód be van kapcsolva",
"MODEM_INIT_ERROR": "A modem inicializálása sikertelen"
}
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Memuat aset...",
"PLEASE_WAIT": "Mohon tunggu...",
"FOUND_NEW_ASSETS": "Ditemukan aset baru: %s",
"HELLO_MY_FRIEND": "Halo, teman saya!"
"HELLO_MY_FRIEND": "Halo, teman saya!",
"CONNECTION_SUCCESSFUL": "Koneksi berhasil",
"FLIGHT_MODE_OFF": "Mode pesawat nonaktif",
"FLIGHT_MODE_ON": "Mode pesawat aktif",
"MODEM_INIT_ERROR": "Gagal menginisialisasi modem"
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Caricamento risorse...",
"PLEASE_WAIT": "Attendere prego...",
"FOUND_NEW_ASSETS": "Trovate nuove risorse: %s",
"HELLO_MY_FRIEND": "Ciao, amico mio!"
"HELLO_MY_FRIEND": "Ciao, amico mio!",
"CONNECTION_SUCCESSFUL": "Connessione riuscita",
"FLIGHT_MODE_OFF": "La modalità aereo è disattivata",
"FLIGHT_MODE_ON": "La modalità aereo è attiva",
"MODEM_INIT_ERROR": "Inizializzazione modem non riuscita"
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "アセットを読み込み中...",
"PLEASE_WAIT": "お待ちください...",
"FOUND_NEW_ASSETS": "新しいアセットが見つかりました: %s",
"HELLO_MY_FRIEND": "こんにちは、友達!"
"HELLO_MY_FRIEND": "こんにちは、友達!",
"CONNECTION_SUCCESSFUL": "接続成功",
"FLIGHT_MODE_OFF": "機内モードがオフです",
"FLIGHT_MODE_ON": "機内モードがオンです",
"MODEM_INIT_ERROR": "モデムの初期化に失敗しました"
}
}

View File

@@ -51,6 +51,9 @@
"LOADING_ASSETS": "에셋 로딩 중...",
"PLEASE_WAIT": "잠시 기다려 주세요...",
"FOUND_NEW_ASSETS": "새로운 에셋을 발견했습니다: %s",
"HELLO_MY_FRIEND": "안녕하세요, 친구!"
"HELLO_MY_FRIEND": "안녕하세요, 친구!",
"FLIGHT_MODE_OFF": "비행기 모드가 꺼져 있습니다",
"FLIGHT_MODE_ON": "비행기 모드가 켜져 있습니다",
"MODEM_INIT_ERROR": "모뎀 초기화 실패"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Menemui aset baharu: %s",
"DOWNLOAD_ASSETS_FAILED": "Gagal memuat turun aset",
"LOADING_ASSETS": "Memuatkan aset...",
"HELLO_MY_FRIEND": "Hai, kawan saya!"
"HELLO_MY_FRIEND": "Hai, kawan saya!",
"FLIGHT_MODE_OFF": "Mod penerbangan dimatikan",
"FLIGHT_MODE_ON": "Mod penerbangan dihidupkan",
"MODEM_INIT_ERROR": "Modem gagal dimulakan"
}
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Fant nye ressurser: %s",
"DOWNLOAD_ASSETS_FAILED": "Nedlasting av ressurser mislyktes",
"LOADING_ASSETS": "Laster ressurser...",
"HELLO_MY_FRIEND": "Hei, min venn!"
"HELLO_MY_FRIEND": "Hei, min venn!",
"FLIGHT_MODE_OFF": "Flymodus er av",
"FLIGHT_MODE_ON": "Flymodus er på",
"MODEM_INIT_ERROR": "Modeminitialisering mislyktes"
}
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Nieuwe bronnen gevonden: %s",
"DOWNLOAD_ASSETS_FAILED": "Downloaden van bronnen mislukt",
"LOADING_ASSETS": "Bronnen laden...",
"HELLO_MY_FRIEND": "Hallo, mijn vriend!"
"HELLO_MY_FRIEND": "Hallo, mijn vriend!",
"FLIGHT_MODE_OFF": "Vliegtuigmodus is uitgeschakeld",
"FLIGHT_MODE_ON": "Vliegtuigmodus is ingeschakeld",
"MODEM_INIT_ERROR": "Modeminitialisatie mislukt"
}
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Ładowanie zasobów...",
"PLEASE_WAIT": "Proszę czekać...",
"FOUND_NEW_ASSETS": "Znaleziono nowe zasoby: %s",
"HELLO_MY_FRIEND": "Cześć, mój przyjacielu!"
"HELLO_MY_FRIEND": "Cześć, mój przyjacielu!",
"CONNECTION_SUCCESSFUL": "Połączenie udane",
"FLIGHT_MODE_OFF": "Tryb samolotowy jest wyłączony",
"FLIGHT_MODE_ON": "Tryb samolotowy jest włączony",
"MODEM_INIT_ERROR": "Inicjalizacja modemu nie powiodła się"
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "A carregar recursos...",
"PLEASE_WAIT": "Por favor aguarde...",
"FOUND_NEW_ASSETS": "Encontrados novos recursos: %s",
"HELLO_MY_FRIEND": "Olá, meu amigo!"
"HELLO_MY_FRIEND": "Olá, meu amigo!",
"CONNECTION_SUCCESSFUL": "Ligação bem-sucedida",
"FLIGHT_MODE_OFF": "O modo avião está desativado",
"FLIGHT_MODE_ON": "O modo avião está ativado",
"MODEM_INIT_ERROR": "Falha na inicialização do modem"
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Se încarcă resursele...",
"PLEASE_WAIT": "Vă rugăm să așteptați...",
"FOUND_NEW_ASSETS": "S-au găsit resurse noi: %s",
"HELLO_MY_FRIEND": "Salut, prietenul meu!"
"HELLO_MY_FRIEND": "Salut, prietenul meu!",
"CONNECTION_SUCCESSFUL": "Conexiune reușită",
"FLIGHT_MODE_OFF": "Modul avion este dezactivat",
"FLIGHT_MODE_ON": "Modul avion este activat",
"MODEM_INIT_ERROR": "Inițializarea modemului a eșuat"
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Загрузка ресурсов...",
"PLEASE_WAIT": "Пожалуйста, подождите...",
"FOUND_NEW_ASSETS": "Найдены новые ресурсы: %s",
"HELLO_MY_FRIEND": "Привет, мой друг!"
"HELLO_MY_FRIEND": "Привет, мой друг!",
"CONNECTION_SUCCESSFUL": "Подключение успешно",
"FLIGHT_MODE_OFF": "Режим полета выключен",
"FLIGHT_MODE_ON": "Режим полета включен",
"MODEM_INIT_ERROR": "Ошибка инициализации модема"
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Nájdené nové zdroje: %s",
"DOWNLOAD_ASSETS_FAILED": "Sťahovanie zdrojov zlyhalo",
"LOADING_ASSETS": "Načítavanie zdrojov...",
"HELLO_MY_FRIEND": "Ahoj, môj priateľ!"
"HELLO_MY_FRIEND": "Ahoj, môj priateľ!",
"FLIGHT_MODE_OFF": "Letecký režim je vypnutý",
"FLIGHT_MODE_ON": "Letecký režim je zapnutý",
"MODEM_INIT_ERROR": "Chyba inicializácie modemu"
}
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Najdeni novi viri: %s",
"DOWNLOAD_ASSETS_FAILED": "Prenos virov ni uspel",
"LOADING_ASSETS": "Nalaganje virov...",
"HELLO_MY_FRIEND": "Pozdravljeni, moj prijatelj!"
"HELLO_MY_FRIEND": "Pozdravljeni, moj prijatelj!",
"FLIGHT_MODE_OFF": "Način leta je izklopljen",
"FLIGHT_MODE_ON": "Način leta je vklopljen",
"MODEM_INIT_ERROR": "Inicializacija modema ni uspela"
}
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Пронађени нови ресурси: %s",
"DOWNLOAD_ASSETS_FAILED": "Преузимање ресурса није успело",
"LOADING_ASSETS": "Учитавање ресурса...",
"HELLO_MY_FRIEND": "Здраво, пријатељу!"
"HELLO_MY_FRIEND": "Здраво, пријатељу!",
"FLIGHT_MODE_OFF": "Режим лета је искључен",
"FLIGHT_MODE_ON": "Режим лета је укључен",
"MODEM_INIT_ERROR": "Иницијализација модема није успела"
}
}
}

View File

@@ -51,7 +51,9 @@
"FOUND_NEW_ASSETS": "Hittade nya resurser: %s",
"DOWNLOAD_ASSETS_FAILED": "Nedladdning av resurser misslyckades",
"LOADING_ASSETS": "Laddar resurser...",
"HELLO_MY_FRIEND": "Hej, min vän!"
"HELLO_MY_FRIEND": "Hej, min vän!",
"FLIGHT_MODE_OFF": "Flygläge är av",
"FLIGHT_MODE_ON": "Flygläge är på",
"MODEM_INIT_ERROR": "Modeminitiering misslyckades"
}
}
}

View File

@@ -51,6 +51,9 @@
"LOADING_ASSETS": "กำลังโหลดทรัพยากร...",
"PLEASE_WAIT": "กรุณารอสักครู่...",
"FOUND_NEW_ASSETS": "พบทรัพยากรใหม่: %s",
"HELLO_MY_FRIEND": "สวัสดี เพื่อนของฉัน!"
"HELLO_MY_FRIEND": "สวัสดี เพื่อนของฉัน!",
"FLIGHT_MODE_OFF": "โหมดเครื่องบินปิดอยู่",
"FLIGHT_MODE_ON": "โหมดเครื่องบินเปิดอยู่",
"MODEM_INIT_ERROR": "การเริ่มต้นโมเด็มล้มเหลว"
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Varlıklar yükleniyor...",
"PLEASE_WAIT": "Lütfen bekleyin...",
"FOUND_NEW_ASSETS": "Yeni varlıklar bulundu: %s",
"HELLO_MY_FRIEND": "Merhaba, arkadaşım!"
"HELLO_MY_FRIEND": "Merhaba, arkadaşım!",
"CONNECTION_SUCCESSFUL": "Bağlantı başarılı",
"FLIGHT_MODE_OFF": "Uçak modu kapalı",
"FLIGHT_MODE_ON": "Uçak modu açık",
"MODEM_INIT_ERROR": "Modem başlatma hatası"
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "Завантаження ресурсів...",
"PLEASE_WAIT": "Будь ласка, зачекайте...",
"FOUND_NEW_ASSETS": "Знайдено нові ресурси: %s",
"HELLO_MY_FRIEND": "Привіт, мій друже!"
"HELLO_MY_FRIEND": "Привіт, мій друже!",
"CONNECTION_SUCCESSFUL": "Підключення успішне",
"FLIGHT_MODE_OFF": "Режим польоту вимкнено",
"FLIGHT_MODE_ON": "Режим польоту увімкнено",
"MODEM_INIT_ERROR": "Помилка ініціалізації модему"
}
}

View File

@@ -51,6 +51,9 @@
"LOADING_ASSETS": "Đang tải tài nguyên...",
"PLEASE_WAIT": "Vui lòng đợi...",
"FOUND_NEW_ASSETS": "Tìm thấy tài nguyên mới: %s",
"HELLO_MY_FRIEND": "Xin chào, bạn của tôi!"
"HELLO_MY_FRIEND": "Xin chào, bạn của tôi!",
"FLIGHT_MODE_OFF": "Chế độ máy bay đang tắt",
"FLIGHT_MODE_ON": "Chế độ máy bay đang bật",
"MODEM_INIT_ERROR": "Khởi tạo modem thất bại"
}
}

View File

@@ -51,6 +51,9 @@
"LOADING_ASSETS": "加载资源...",
"PLEASE_WAIT": "请稍候...",
"FOUND_NEW_ASSETS": "发现新资源: %s",
"HELLO_MY_FRIEND": "你好,我的朋友!"
"HELLO_MY_FRIEND": "你好,我的朋友!",
"CONNECTION_SUCCESSFUL": "连接成功",
"FLIGHT_MODE_OFF": "飞行模式已关闭",
"FLIGHT_MODE_ON": "飞行模式已开启"
}
}

View File

@@ -50,6 +50,10 @@
"LOADING_ASSETS": "載入資源...",
"PLEASE_WAIT": "請稍候...",
"FOUND_NEW_ASSETS": "發現新資源: %s",
"HELLO_MY_FRIEND": "你好,我的朋友!"
"HELLO_MY_FRIEND": "你好,我的朋友!",
"CONNECTION_SUCCESSFUL": "連線成功",
"FLIGHT_MODE_OFF": "飛航模式已關閉",
"FLIGHT_MODE_ON": "飛航模式已開啟",
"MODEM_INIT_ERROR": "模組初始化失敗"
}
}

View File

@@ -34,16 +34,6 @@ void AudioCodec::Start() {
output_volume_ = 10;
}
if (tx_handle_ != nullptr) {
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
}
if (rx_handle_ != nullptr) {
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
}
EnableInput(true);
EnableOutput(true);
ESP_LOGI(TAG, "Audio codec started");
}

View File

@@ -265,27 +265,18 @@ void AudioService::AudioInputTask() {
}
}
/* Feed the wake word */
if (bits & AS_EVENT_WAKE_WORD_RUNNING) {
/* Feed the wake word and/or audio processor */
if (bits & (AS_EVENT_WAKE_WORD_RUNNING | AS_EVENT_AUDIO_PROCESSOR_RUNNING)) {
int samples = 160; // 10ms
std::vector<int16_t> data;
int samples = wake_word_->GetFeedSize();
if (samples > 0) {
if (ReadAudioData(data, 16000, samples)) {
if (ReadAudioData(data, 16000, samples)) {
if (bits & AS_EVENT_WAKE_WORD_RUNNING) {
wake_word_->Feed(data);
continue;
}
}
}
/* Feed the audio processor */
if (bits & AS_EVENT_AUDIO_PROCESSOR_RUNNING) {
std::vector<int16_t> data;
int samples = audio_processor_->GetFeedSize();
if (samples > 0) {
if (ReadAudioData(data, 16000, samples)) {
if (bits & AS_EVENT_AUDIO_PROCESSOR_RUNNING) {
audio_processor_->Feed(std::move(data));
continue;
}
continue;
}
}
@@ -314,6 +305,7 @@ void AudioService::AudioOutputTask() {
esp_timer_start_periodic(audio_power_timer_, AUDIO_POWER_CHECK_INTERVAL_MS * 1000);
codec_->EnableOutput(true);
}
codec_->OutputData(task->pcm);
/* Update the last output time */
@@ -645,94 +637,20 @@ void AudioService::PlaySound(const std::string_view& ogg) {
codec_->EnableOutput(true);
}
const uint8_t* buf = reinterpret_cast<const uint8_t*>(ogg.data());
const auto* buf = reinterpret_cast<const uint8_t*>(ogg.data());
size_t size = ogg.size();
size_t offset = 0;
auto find_page = [&](size_t start)->size_t {
for (size_t i = start; i + 4 <= size; ++i) {
if (buf[i] == 'O' && buf[i+1] == 'g' && buf[i+2] == 'g' && buf[i+3] == 'S') return i;
}
return static_cast<size_t>(-1);
};
bool seen_head = false;
bool seen_tags = false;
int sample_rate = 16000; // 默认值
while (true) {
size_t pos = find_page(offset);
if (pos == static_cast<size_t>(-1)) break;
offset = pos;
if (offset + 27 > size) break;
const uint8_t* page = buf + offset;
uint8_t page_segments = page[26];
size_t seg_table_off = offset + 27;
if (seg_table_off + page_segments > size) break;
size_t body_size = 0;
for (size_t i = 0; i < page_segments; ++i) body_size += page[27 + i];
size_t body_off = seg_table_off + page_segments;
if (body_off + body_size > size) break;
// Parse packets using lacing
size_t cur = body_off;
size_t seg_idx = 0;
while (seg_idx < page_segments) {
size_t pkt_len = 0;
size_t pkt_start = cur;
bool continued = false;
do {
uint8_t l = page[27 + seg_idx++];
pkt_len += l;
cur += l;
continued = (l == 255);
} while (continued && seg_idx < page_segments);
if (pkt_len == 0) continue;
const uint8_t* pkt_ptr = buf + pkt_start;
if (!seen_head) {
// 解析OpusHead包
if (pkt_len >= 19 && std::memcmp(pkt_ptr, "OpusHead", 8) == 0) {
seen_head = true;
// OpusHead结构[0-7] "OpusHead", [8] version, [9] channel_count, [10-11] pre_skip
// [12-15] input_sample_rate, [16-17] output_gain, [18] mapping_family
if (pkt_len >= 12) {
uint8_t version = pkt_ptr[8];
uint8_t channel_count = pkt_ptr[9];
if (pkt_len >= 16) {
// 读取输入采样率 (little-endian)
sample_rate = pkt_ptr[12] | (pkt_ptr[13] << 8) |
(pkt_ptr[14] << 16) | (pkt_ptr[15] << 24);
ESP_LOGI(TAG, "OpusHead: version=%d, channels=%d, sample_rate=%d",
version, channel_count, sample_rate);
}
}
}
continue;
}
if (!seen_tags) {
// Expect OpusTags in second packet
if (pkt_len >= 8 && std::memcmp(pkt_ptr, "OpusTags", 8) == 0) {
seen_tags = true;
}
continue;
}
// Audio packet (Opus)
auto packet = std::make_unique<AudioStreamPacket>();
packet->sample_rate = sample_rate;
packet->frame_duration = 60;
packet->payload.resize(pkt_len);
std::memcpy(packet->payload.data(), pkt_ptr, pkt_len);
PushPacketToDecodeQueue(std::move(packet), true);
}
offset = body_off + body_size;
}
auto demuxer = std::make_unique<OggDemuxer>();
demuxer->OnDemuxerFinished([this](const uint8_t* data, int sample_rate, size_t size){
auto packet = std::make_unique<AudioStreamPacket>();
packet->sample_rate = sample_rate;
packet->frame_duration = 60;
packet->payload.resize(size);
std::memcpy(packet->payload.data(), data, size);
PushPacketToDecodeQueue(std::move(packet), true);
});
demuxer->Reset();
demuxer->Process(buf, size);
}
bool AudioService::IsIdle() {

View File

@@ -23,7 +23,7 @@
#include "processors/audio_debugger.h"
#include "wake_word.h"
#include "protocol.h"
#include "ogg_demuxer.h"
/*
* There are two types of audio data flow:

View File

@@ -176,6 +176,8 @@ void BoxAudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gpio_
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_tdm_mode(rx_handle_, &tdm_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -150,6 +150,8 @@ void Es8311AudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gp
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_std_mode(rx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -126,6 +126,8 @@ void Es8374AudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gp
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_std_mode(rx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -131,6 +131,8 @@ void Es8388AudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gp
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_std_mode(rx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}
@@ -186,9 +188,6 @@ void Es8388AudioCodec::EnableOutput(bool enable) {
// Set analog output volume to 0dB, default is -45dB
uint8_t reg_val = 30; // 0dB
if(input_reference_){
reg_val = 27;
}
uint8_t regs[] = { 46, 47, 48, 49 }; // HP_LVOL, HP_RVOL, SPK_LVOL, SPK_RVOL
for (uint8_t reg : regs) {
ctrl_if_->write_reg(ctrl_if_, reg, 1, &reg_val, 1);

View File

@@ -132,6 +132,8 @@ void Es8389AudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gp
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_std_mode(rx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -254,6 +254,32 @@ int NoAudioCodec::Read(int16_t* dest, int samples) {
return samples;
}
void NoAudioCodec::EnableInput(bool enable) {
std::lock_guard<std::mutex> lock(data_if_mutex_);
if (enable == input_enabled_) {
return;
}
if (enable) {
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
} else {
ESP_ERROR_CHECK(i2s_channel_disable(rx_handle_));
}
AudioCodec::EnableInput(enable);
}
void NoAudioCodec::EnableOutput(bool enable) {
std::lock_guard<std::mutex> lock(data_if_mutex_);
if (enable == output_enabled_) {
return;
}
if (enable) {
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
} else {
ESP_ERROR_CHECK(i2s_channel_disable(tx_handle_));
}
AudioCodec::EnableOutput(enable);
}
// Delegating constructor: calls the main constructor with default slot mask
NoAudioCodecSimplexPdm::NoAudioCodecSimplexPdm(int input_sample_rate, int output_sample_rate, gpio_num_t spk_bclk, gpio_num_t spk_ws, gpio_num_t spk_dout, gpio_num_t mic_sck, gpio_num_t mic_din)
: NoAudioCodecSimplexPdm(input_sample_rate, output_sample_rate, spk_bclk, spk_ws, spk_dout, I2S_STD_SLOT_LEFT, mic_sck, mic_din) {

View File

@@ -13,6 +13,8 @@ protected:
virtual int Write(const int16_t* data, int samples) override;
virtual int Read(int16_t* dest, int samples) override;
virtual void EnableInput(bool enable) override;
virtual void EnableOutput(bool enable) override;
public:
virtual ~NoAudioCodec();

View File

@@ -0,0 +1,311 @@
#include "ogg_demuxer.h"
#include "esp_log.h"
#define TAG "OggDemuxer"
/// @brief 重置解封器
void OggDemuxer::Reset()
{
opus_info_ = {
.head_seen = false,
.tags_seen = false,
.sample_rate = 48000
};
state_ = ParseState::FIND_PAGE;
ctx_.packet_len = 0;
ctx_.seg_count = 0;
ctx_.seg_index = 0;
ctx_.data_offset = 0;
ctx_.bytes_needed = 4; // 需要4字节"OggS"
ctx_.seg_remaining = 0;
ctx_.body_size = 0;
ctx_.body_offset = 0;
ctx_.packet_continued = false;
// 清空缓冲区数据
memset(ctx_.header, 0, sizeof(ctx_.header));
memset(ctx_.seg_table, 0, sizeof(ctx_.seg_table));
memset(ctx_.packet_buf, 0, sizeof(ctx_.packet_buf));
}
/// @brief 处理数据块
/// @param data 输入数据
/// @param size 输入数据大小
/// @return 已处理的字节数
size_t OggDemuxer::Process(const uint8_t* data, size_t size)
{
size_t processed = 0; // 已处理的字节数
while (processed < size) {
switch (state_) {
case ParseState::FIND_PAGE: {
// 寻找页头"OggS"
if (ctx_.bytes_needed < 4) {
// 处理不完整的"OggS"匹配(跨数据块)
size_t to_copy = std::min(size - processed, ctx_.bytes_needed);
memcpy(ctx_.header + (4 - ctx_.bytes_needed), data + processed, to_copy);
processed += to_copy;
ctx_.bytes_needed -= to_copy;
if (ctx_.bytes_needed == 0) {
// 检查是否匹配"OggS"
if (memcmp(ctx_.header, "OggS", 4) == 0) {
state_ = ParseState::PARSE_HEADER;
ctx_.data_offset = 4;
ctx_.bytes_needed = 27 - 4; // 还需要23字节完成页头
} else {
// 匹配失败滑动1字节继续匹配
memmove(ctx_.header, ctx_.header + 1, 3);
ctx_.bytes_needed = 1;
}
} else {
// 数据不足,等待更多数据
return processed;
}
} else if (ctx_.bytes_needed == 4) {
// 在数据块中查找完整的"OggS"
bool found = false;
size_t i = 0;
size_t remaining = size - processed;
// 搜索"OggS"
for (; i + 4 <= remaining; i++) {
if (memcmp(data + processed + i, "OggS", 4) == 0) {
found = true;
break;
}
}
if (found) {
// 找到"OggS",跳过已搜索的字节
processed += i;
// 不记录找到的"OggS",无必要
// memcpy(ctx_.header, data + processed, 4);
processed += 4;
state_ = ParseState::PARSE_HEADER;
ctx_.data_offset = 4;
ctx_.bytes_needed = 27 - 4; // 还需要23字节
} else {
// 没有找到完整"OggS",保存可能的部分匹配
size_t partial_len = remaining - i;
if (partial_len > 0) {
memcpy(ctx_.header, data + processed + i, partial_len);
ctx_.bytes_needed = 4 - partial_len;
processed += i + partial_len;
} else {
processed += i; // 已搜索所有字节
}
return processed; // 返回已处理的字节数
}
} else {
ESP_LOGE(TAG, "OggDemuxer run in error state: bytes_needed=%zu", ctx_.bytes_needed);
Reset();
return processed;
}
break;
}
case ParseState::PARSE_HEADER: {
size_t available = size - processed;
if (available < ctx_.bytes_needed) {
// 数据不足,复制可用的部分
memcpy(ctx_.header + ctx_.data_offset,
data + processed, available);
ctx_.data_offset += available;
ctx_.bytes_needed -= available;
processed += available;
return processed; // 等待更多数据
} else {
// 有足够的数据完成页头
size_t to_copy = ctx_.bytes_needed;
memcpy(ctx_.header + ctx_.data_offset,
data + processed, to_copy);
processed += to_copy;
ctx_.data_offset += to_copy;
ctx_.bytes_needed = 0;
// 验证页头
if (ctx_.header[4] != 0) {
ESP_LOGE(TAG, "无效的Ogg版本: %d", ctx_.header[4]);
state_ = ParseState::FIND_PAGE;
ctx_.bytes_needed = 4;
ctx_.data_offset = 0;
break;
}
ctx_.seg_count = ctx_.header[26];
if (ctx_.seg_count > 0 && ctx_.seg_count <= 255) {
state_ = ParseState::PARSE_SEGMENTS;
ctx_.bytes_needed = ctx_.seg_count;
ctx_.data_offset = 0;
} else if (ctx_.seg_count == 0) {
// 没有段,直接跳到下一个页面
state_ = ParseState::FIND_PAGE;
ctx_.bytes_needed = 4;
ctx_.data_offset = 0;
} else {
ESP_LOGE(TAG, "无效的段数: %u", ctx_.seg_count);
state_ = ParseState::FIND_PAGE;
ctx_.bytes_needed = 4;
ctx_.data_offset = 0;
}
}
break;
}
case ParseState::PARSE_SEGMENTS: {
size_t available = size - processed;
if (available < ctx_.bytes_needed) {
memcpy(ctx_.seg_table + ctx_.data_offset,
data + processed, available);
ctx_.data_offset += available;
ctx_.bytes_needed -= available;
processed += available;
return processed; // 等待更多数据
} else {
size_t to_copy = ctx_.bytes_needed;
memcpy(ctx_.seg_table + ctx_.data_offset,
data + processed, to_copy);
processed += to_copy;
ctx_.data_offset += to_copy;
ctx_.bytes_needed = 0;
state_ = ParseState::PARSE_DATA;
ctx_.seg_index = 0;
ctx_.data_offset = 0;
// 计算数据体总大小
ctx_.body_size = 0;
for (size_t i = 0; i < ctx_.seg_count; ++i) {
ctx_.body_size += ctx_.seg_table[i];
}
ctx_.body_offset = 0;
ctx_.seg_remaining = 0;
}
break;
}
case ParseState::PARSE_DATA: {
while (ctx_.seg_index < ctx_.seg_count && processed < size) {
uint8_t seg_len = ctx_.seg_table[ctx_.seg_index];
// 检查段数据是否已经部分读取
if (ctx_.seg_remaining > 0) {
seg_len = ctx_.seg_remaining;
} else {
ctx_.seg_remaining = seg_len;
}
// 检查缓冲区是否足够
if (ctx_.packet_len + seg_len > sizeof(ctx_.packet_buf)) {
ESP_LOGE(TAG, "包缓冲区溢出: %zu + %u > %zu", ctx_.packet_len, seg_len, sizeof(ctx_.packet_buf));
state_ = ParseState::FIND_PAGE;
ctx_.packet_len = 0;
ctx_.packet_continued = false;
ctx_.seg_remaining = 0;
ctx_.bytes_needed = 4;
return processed;
}
// 复制数据
size_t to_copy = std::min(size - processed, (size_t)seg_len);
memcpy(ctx_.packet_buf + ctx_.packet_len, data + processed, to_copy);
processed += to_copy;
ctx_.packet_len += to_copy;
ctx_.body_offset += to_copy;
ctx_.seg_remaining -= to_copy;
// 检查段是否完整
if (ctx_.seg_remaining > 0) {
// 段不完整,等待更多数据
return processed;
}
// 段完整
bool seg_continued = (ctx_.seg_table[ctx_.seg_index] == 255);
if (!seg_continued) {
// 包结束
if (ctx_.packet_len) {
if (!opus_info_.head_seen) {
if (ctx_.packet_len >=8 && memcmp(ctx_.packet_buf, "OpusHead", 8) == 0) {
opus_info_.head_seen = true;
if (ctx_.packet_len >= 19) {
opus_info_.sample_rate = ctx_.packet_buf[12] |
(ctx_.packet_buf[13] << 8) |
(ctx_.packet_buf[14] << 16) |
(ctx_.packet_buf[15] << 24);
ESP_LOGI(TAG, "OpusHead found, sample_rate=%d", opus_info_.sample_rate);
}
ctx_.packet_len = 0;
ctx_.packet_continued = false;
ctx_.seg_index++;
ctx_.seg_remaining = 0;
continue;
}
}
if (!opus_info_.tags_seen) {
if (ctx_.packet_len >= 8 && memcmp(ctx_.packet_buf, "OpusTags", 8) == 0) {
opus_info_.tags_seen = true;
ESP_LOGI(TAG, "OpusTags found.");
ctx_.packet_len = 0;
ctx_.packet_continued = false;
ctx_.seg_index++;
ctx_.seg_remaining = 0;
continue;
}
}
if (opus_info_.head_seen && opus_info_.tags_seen) {
if (on_demuxer_finished_) {
on_demuxer_finished_(ctx_.packet_buf, opus_info_.sample_rate, ctx_.packet_len);
}
} else {
ESP_LOGW(TAG, "当前Ogg容器未解析到OpusHead/OpusTags丢弃");
}
}
ctx_.packet_len = 0;
ctx_.packet_continued = false;
} else {
ctx_.packet_continued = true;
}
ctx_.seg_index++;
ctx_.seg_remaining = 0;
}
if (ctx_.seg_index == ctx_.seg_count) {
// 检查是否所有数据体都已读取
if (ctx_.body_offset < ctx_.body_size) {
ESP_LOGW(TAG, "数据体不完整: %zu/%zu",
ctx_.body_offset, ctx_.body_size);
}
// 如果包跨页保持packet_len和packet_continued
if (!ctx_.packet_continued) {
ctx_.packet_len = 0;
}
// 进入下一页面
state_ = ParseState::FIND_PAGE;
ctx_.bytes_needed = 4;
ctx_.data_offset = 0;
}
break;
}
}
}
return processed;
}

View File

@@ -0,0 +1,63 @@
#ifndef OGG_DEMUXER_H_
#define OGG_DEMUXER_H_
#include <functional>
#include <cstdint>
#include <cstring>
#include <vector>
class OggDemuxer {
private:
enum ParseState : int8_t {
FIND_PAGE,
PARSE_HEADER,
PARSE_SEGMENTS,
PARSE_DATA
};
struct Opus_t {
bool head_seen{false};
bool tags_seen{false};
int sample_rate{48000};
};
// 使用固定大小的缓冲区避免动态分配
struct context_t {
bool packet_continued{false}; // 当前包是否跨多个段
uint8_t header[27]; // Ogg页头
uint8_t seg_table[255]; // 当前存储的段表
uint8_t packet_buf[8192]; // 8KB包缓冲区
size_t packet_len = 0; // 缓冲区中累计的数据长度
size_t seg_count = 0; // 当前页段数
size_t seg_index = 0; // 当前处理的段索引
size_t data_offset = 0; // 解析当前阶段已读取的字节数
size_t bytes_needed = 0; // 解析当前字段还需要读取的字节数
size_t seg_remaining = 0; // 当前段剩余需要读取的字节数
size_t body_size = 0; // 数据体总大小
size_t body_offset = 0; // 数据体已读取的字节数
};
public:
OggDemuxer() {
Reset();
}
void Reset();
size_t Process(const uint8_t* data, size_t size);
/// @brief 设置解封装完毕后回调处理函数
/// @param on_demuxer_finished
void OnDemuxerFinished(std::function<void(const uint8_t* data, int sample_rate, size_t len)> on_demuxer_finished) {
on_demuxer_finished_ = on_demuxer_finished;
}
private:
ParseState state_ = ParseState::FIND_PAGE;
context_t ctx_;
Opus_t opus_info_;
std::function<void(const uint8_t*, int, size_t)> on_demuxer_finished_;
};
#endif

View File

@@ -92,7 +92,18 @@ void AfeAudioProcessor::Feed(std::vector<int16_t>&& data) {
if (afe_data_ == nullptr) {
return;
}
afe_iface_->feed(afe_data_, data.data());
std::lock_guard<std::mutex> lock(input_buffer_mutex_);
// Check running state inside lock to avoid TOCTOU race with Stop()
if (!IsRunning()) {
return;
}
input_buffer_.insert(input_buffer_.end(), data.begin(), data.end());
size_t chunk_size = afe_iface_->get_feed_chunksize(afe_data_) * codec_->input_channels();
while (input_buffer_.size() >= chunk_size) {
afe_iface_->feed(afe_data_, input_buffer_.data());
input_buffer_.erase(input_buffer_.begin(), input_buffer_.begin() + chunk_size);
}
}
void AfeAudioProcessor::Start() {
@@ -101,9 +112,12 @@ void AfeAudioProcessor::Start() {
void AfeAudioProcessor::Stop() {
xEventGroupClearBits(event_group_, PROCESSOR_RUNNING);
std::lock_guard<std::mutex> lock(input_buffer_mutex_);
if (afe_data_ != nullptr) {
afe_iface_->reset_buffer(afe_data_);
}
input_buffer_.clear();
}
bool AfeAudioProcessor::IsRunning() {

View File

@@ -9,6 +9,7 @@
#include <string>
#include <vector>
#include <functional>
#include <mutex>
#include "audio_processor.h"
#include "audio_codec.h"
@@ -37,6 +38,8 @@ private:
AudioCodec* codec_ = nullptr;
int frame_samples_ = 0;
bool is_speaking_ = false;
std::vector<int16_t> input_buffer_;
std::mutex input_buffer_mutex_;
std::vector<int16_t> output_buffer_;
void AudioProcessorTask();

View File

@@ -3,6 +3,7 @@
#include <vector>
#include <functional>
#include <atomic>
#include "audio_processor.h"
#include "audio_codec.h"
@@ -27,7 +28,7 @@ private:
int frame_samples_ = 0;
std::function<void(std::vector<int16_t>&& data)> output_callback_;
std::function<void(bool speaking)> vad_state_change_callback_;
bool is_running_ = false;
std::atomic<bool> is_running_ = false;
};
#endif

View File

@@ -99,16 +99,30 @@ void AfeWakeWord::Start() {
void AfeWakeWord::Stop() {
xEventGroupClearBits(event_group_, DETECTION_RUNNING_EVENT);
std::lock_guard<std::mutex> lock(input_buffer_mutex_);
if (afe_data_ != nullptr) {
afe_iface_->reset_buffer(afe_data_);
}
input_buffer_.clear();
}
void AfeWakeWord::Feed(const std::vector<int16_t>& data) {
if (afe_data_ == nullptr) {
return;
}
afe_iface_->feed(afe_data_, data.data());
std::lock_guard<std::mutex> lock(input_buffer_mutex_);
// Check running state inside lock to avoid TOCTOU race with Stop()
if (!(xEventGroupGetBits(event_group_) & DETECTION_RUNNING_EVENT)) {
return;
}
input_buffer_.insert(input_buffer_.end(), data.begin(), data.end());
size_t chunk_size = afe_iface_->get_feed_chunksize(afe_data_) * codec_->input_channels();
while (input_buffer_.size() >= chunk_size) {
afe_iface_->feed(afe_data_, input_buffer_.data());
input_buffer_.erase(input_buffer_.begin(), input_buffer_.begin() + chunk_size);
}
}
size_t AfeWakeWord::GetFeedSize() {

View File

@@ -44,6 +44,8 @@ private:
std::function<void(const std::string& wake_word)> wake_word_detected_callback_;
AudioCodec* codec_ = nullptr;
std::string last_detected_wake_word_;
std::vector<int16_t> input_buffer_;
std::mutex input_buffer_mutex_;
TaskHandle_t wake_word_encode_task_ = nullptr;
StaticTask_t* wake_word_encode_task_buffer_ = nullptr;

View File

@@ -138,49 +138,64 @@ void CustomWakeWord::Start() {
void CustomWakeWord::Stop() {
running_ = false;
std::lock_guard<std::mutex> lock(input_buffer_mutex_);
input_buffer_.clear();
}
void CustomWakeWord::Feed(const std::vector<int16_t>& data) {
if (multinet_model_data_ == nullptr || !running_) {
if (multinet_model_data_ == nullptr) {
return;
}
std::lock_guard<std::mutex> lock(input_buffer_mutex_);
// Check running state inside lock to avoid TOCTOU race with Stop()
if (!running_) {
return;
}
esp_mn_state_t mn_state;
// If input channels is 2, we need to fetch the left channel data
if (codec_->input_channels() == 2) {
auto mono_data = std::vector<int16_t>(data.size() / 2);
for (size_t i = 0, j = 0; i < mono_data.size(); ++i, j += 2) {
mono_data[i] = data[j];
for (size_t i = 0; i < data.size(); i += 2) {
input_buffer_.push_back(data[i]);
}
StoreWakeWordData(mono_data);
mn_state = multinet_->detect(multinet_model_data_, const_cast<int16_t*>(mono_data.data()));
} else {
StoreWakeWordData(data);
mn_state = multinet_->detect(multinet_model_data_, const_cast<int16_t*>(data.data()));
input_buffer_.insert(input_buffer_.end(), data.begin(), data.end());
}
if (mn_state == ESP_MN_STATE_DETECTING) {
return;
} else if (mn_state == ESP_MN_STATE_DETECTED) {
esp_mn_results_t *mn_result = multinet_->get_results(multinet_model_data_);
for (int i = 0; i < mn_result->num && running_; i++) {
ESP_LOGI(TAG, "Custom wake word detected: command_id=%d, string=%s, prob=%f",
mn_result->command_id[i], mn_result->string, mn_result->prob[i]);
auto& command = commands_[mn_result->command_id[i] - 1];
if (command.action == "wake") {
last_detected_wake_word_ = command.text;
running_ = false;
if (wake_word_detected_callback_) {
wake_word_detected_callback_(last_detected_wake_word_);
int chunksize = multinet_->get_samp_chunksize(multinet_model_data_);
while (input_buffer_.size() >= chunksize) {
std::vector<int16_t> chunk(input_buffer_.begin(), input_buffer_.begin() + chunksize);
StoreWakeWordData(chunk);
esp_mn_state_t mn_state = multinet_->detect(multinet_model_data_, chunk.data());
if (mn_state == ESP_MN_STATE_DETECTED) {
esp_mn_results_t *mn_result = multinet_->get_results(multinet_model_data_);
for (int i = 0; i < mn_result->num && running_; i++) {
ESP_LOGI(TAG, "Custom wake word detected: command_id=%d, string=%s, prob=%f",
mn_result->command_id[i], mn_result->string, mn_result->prob[i]);
auto& command = commands_[mn_result->command_id[i] - 1];
if (command.action == "wake") {
last_detected_wake_word_ = command.text;
running_ = false;
input_buffer_.clear();
if (wake_word_detected_callback_) {
wake_word_detected_callback_(last_detected_wake_word_);
}
}
}
multinet_->clean(multinet_model_data_);
} else if (mn_state == ESP_MN_STATE_TIMEOUT) {
ESP_LOGD(TAG, "Command word detection timeout, cleaning state");
multinet_->clean(multinet_model_data_);
}
multinet_->clean(multinet_model_data_);
} else if (mn_state == ESP_MN_STATE_TIMEOUT) {
ESP_LOGD(TAG, "Command word detection timeout, cleaning state");
multinet_->clean(multinet_model_data_);
if (!running_) {
break;
}
input_buffer_.erase(input_buffer_.begin(), input_buffer_.begin() + chunksize);
}
}

View File

@@ -53,6 +53,8 @@ private:
AudioCodec* codec_ = nullptr;
std::string last_detected_wake_word_;
std::atomic<bool> running_ = false;
std::vector<int16_t> input_buffer_;
std::mutex input_buffer_mutex_;
TaskHandle_t wake_word_encode_task_ = nullptr;
StaticTask_t* wake_word_encode_task_buffer_ = nullptr;

View File

@@ -54,21 +54,44 @@ void EspWakeWord::Start() {
void EspWakeWord::Stop() {
running_ = false;
std::lock_guard<std::mutex> lock(input_buffer_mutex_);
input_buffer_.clear();
}
void EspWakeWord::Feed(const std::vector<int16_t>& data) {
if (wakenet_data_ == nullptr || !running_) {
if (wakenet_data_ == nullptr) {
return;
}
int res = wakenet_iface_->detect(wakenet_data_, (int16_t *)data.data());
if (res > 0) {
last_detected_wake_word_ = wakenet_iface_->get_word_name(wakenet_data_, res);
running_ = false;
std::lock_guard<std::mutex> lock(input_buffer_mutex_);
// Check running state inside lock to avoid TOCTOU race with Stop()
if (!running_) {
return;
}
if (wake_word_detected_callback_) {
wake_word_detected_callback_(last_detected_wake_word_);
if (codec_->input_channels() == 2) {
for (size_t i = 0; i < data.size(); i += 2) {
input_buffer_.push_back(data[i]);
}
} else {
input_buffer_.insert(input_buffer_.end(), data.begin(), data.end());
}
int chunksize = wakenet_iface_->get_samp_chunksize(wakenet_data_);
while (input_buffer_.size() >= chunksize) {
int res = wakenet_iface_->detect(wakenet_data_, input_buffer_.data());
if (res > 0) {
last_detected_wake_word_ = wakenet_iface_->get_word_name(wakenet_data_, res);
running_ = false;
input_buffer_.clear();
if (wake_word_detected_callback_) {
wake_word_detected_callback_(last_detected_wake_word_);
}
break;
}
input_buffer_.erase(input_buffer_.begin(), input_buffer_.begin() + chunksize);
}
}

View File

@@ -9,6 +9,7 @@
#include <vector>
#include <functional>
#include <atomic>
#include <mutex>
#include "audio_codec.h"
#include "wake_word.h"
@@ -37,6 +38,8 @@ private:
std::function<void(const std::string& wake_word)> wake_word_detected_callback_;
std::string last_detected_wake_word_;
std::vector<int16_t> input_buffer_;
std::mutex input_buffer_mutex_;
};
#endif

View File

@@ -106,6 +106,10 @@ private:
InitializeGc9107Display();
InitializeButtons();
GetBacklight()->SetBrightness(100);
// Ensure UI is set up before displaying error
display_->SetupUI();
display_->SetStatus(Lang::Strings::ERROR);
display_->SetEmotion("triangle_exclamation");
display_->SetChatMessage("system", "Echo Base\nnot connected");

View File

@@ -173,6 +173,10 @@ private:
InitializeGc9107Display();
InitializeButtons();
GetBacklight()->SetBrightness(100);
// Ensure UI is set up before displaying error
display_->SetupUI();
display_->SetStatus(Lang::Strings::ERROR);
display_->SetEmotion("triangle_exclamation");
display_->SetChatMessage("system", "Echo Base\nnot connected");

View File

@@ -1,4 +1,4 @@
#include "dual_network_board.h"
#include "wifi_board.h"
#include "codecs/no_audio_codec.h"
#include "display/lcd_display.h"
#include "system_reset.h"
@@ -57,7 +57,7 @@ static const gc9a01_lcd_init_cmd_t gc9107_lcd_init_cmds[] = {
#define TAG "ESP32-LCD-MarsbearSupport"
class CompactWifiBoardLCD : public DualNetworkBoard {
class CompactWifiBoardLCD : public WifiBoard {
private:
Button boot_button_;
Button touch_button_;
@@ -136,26 +136,14 @@ private:
boot_button_.OnClick([this]() {
auto& app = Application::GetInstance();
if (GetNetworkType() == NetworkType::WIFI) {
if (app.GetDeviceState() == kDeviceStateStarting) {
// cast to WifiBoard
auto& wifi_board = static_cast<WifiBoard&>(GetCurrentBoard());
wifi_board.EnterWifiConfigMode();
return;
}
if (app.GetDeviceState() == kDeviceStateStarting) {
EnterWifiConfigMode();
return;
}
gpio_set_level(BUILTIN_LED_GPIO, 1);
app.ToggleChatState();
});
boot_button_.OnDoubleClick([this]() {
auto& app = Application::GetInstance();
if (app.GetDeviceState() == kDeviceStateStarting || app.GetDeviceState() == kDeviceStateWifiConfiguring) {
SwitchNetworkType();
}
});
asr_button_.OnClick([this]() {
std::string wake_word="你好小智";
Application::GetInstance().WakeWordInvoke(wake_word);
@@ -174,8 +162,7 @@ private:
}
public:
CompactWifiBoardLCD() :
DualNetworkBoard(ML307_TX_PIN, ML307_RX_PIN),
CompactWifiBoardLCD() : WifiBoard(),
boot_button_(BOOT_BUTTON_GPIO), touch_button_(TOUCH_BUTTON_GPIO), asr_button_(ASR_BUTTON_GPIO) {
InitializeSpi();
InitializeLcdDisplay();

View File

@@ -1,4 +1,4 @@
#include "dual_network_board.h"
#include "wifi_board.h"
#include "codecs/no_audio_codec.h"
#include "system_reset.h"
#include "application.h"
@@ -16,7 +16,7 @@
#define TAG "ESP32-MarsbearSupport"
class CompactWifiBoard : public DualNetworkBoard {
class CompactWifiBoard : public WifiBoard {
private:
Button boot_button_;
Button touch_button_;
@@ -104,26 +104,14 @@ private:
boot_button_.OnClick([this]() {
auto& app = Application::GetInstance();
if (GetNetworkType() == NetworkType::WIFI) {
if (app.GetDeviceState() == kDeviceStateStarting) {
// cast to WifiBoard
auto& wifi_board = static_cast<WifiBoard&>(GetCurrentBoard());
wifi_board.EnterWifiConfigMode();
return;
}
if (app.GetDeviceState() == kDeviceStateStarting) {
EnterWifiConfigMode();
return;
}
gpio_set_level(BUILTIN_LED_GPIO, 1);
app.ToggleChatState();
});
boot_button_.OnDoubleClick([this]() {
auto& app = Application::GetInstance();
if (app.GetDeviceState() == kDeviceStateStarting || app.GetDeviceState() == kDeviceStateWifiConfiguring) {
SwitchNetworkType();
}
});
asr_button_.OnClick([this]() {
std::string wake_word="你好小智";
Application::GetInstance().WakeWordInvoke(wake_word);
@@ -145,7 +133,7 @@ private:
}
public:
CompactWifiBoard() : DualNetworkBoard(ML307_TX_PIN, ML307_RX_PIN), boot_button_(BOOT_BUTTON_GPIO), touch_button_(TOUCH_BUTTON_GPIO), asr_button_(ASR_BUTTON_GPIO)
CompactWifiBoard() : WifiBoard(), boot_button_(BOOT_BUTTON_GPIO), touch_button_(TOUCH_BUTTON_GPIO), asr_button_(ASR_BUTTON_GPIO)
{
InitializeDisplayI2c();
InitializeSsd1306Display();

View File

@@ -24,26 +24,6 @@ idf.py menuconfig
Xiaozhi Assistant -> Board Type ->面包板新版接线WiFi+ LCD + Camera
```
**配置摄像头传感器:**
> **注意:** 确认摄像头传感器型号,确定型号在 esp_cam_sensor 支持的范围内。当前板子用的是 OV2640是符合支持范围。
在 menuconfig 中按以下步骤启用对应型号的支持:
1. **导航到传感器配置:**
```
(Top) → Component config → Espressif Camera Sensors Configurations → Camera Sensor Configuration → Select and Set Camera Sensor
```
2. **选择传感器型号:**
- 选中所需的传感器型号OV2640
3. **配置传感器参数:**
- 按 → 进入传感器详细设置
- 启用 **Auto detect**
- 推荐将 **default output format** 调整为 **YUV422** 及合适的分辨率大小
- (目前支持 YUV422、RGB565YUV422 更节省内存空间)
**编译烧入:**
```bash

View File

@@ -8,7 +8,7 @@
#include "mcp_server.h"
#include "lamp_controller.h"
#include "led/single_led.h"
#include "esp_video.h"
#include "esp32_camera.h"
#include <esp_log.h>
#include <driver/i2c_master.h>
@@ -65,7 +65,7 @@ private:
Button boot_button_;
LcdDisplay* display_;
EspVideo* camera_;
Esp32Camera* camera_;
void InitializeSpi() {
spi_bus_config_t buscfg = {};
@@ -125,47 +125,32 @@ private:
}
void InitializeCamera() {
static esp_cam_ctlr_dvp_pin_config_t dvp_pin_config = {
.data_width = CAM_CTLR_DATA_WIDTH_8,
.data_io = {
[0] = CAMERA_PIN_D0,
[1] = CAMERA_PIN_D1,
[2] = CAMERA_PIN_D2,
[3] = CAMERA_PIN_D3,
[4] = CAMERA_PIN_D4,
[5] = CAMERA_PIN_D5,
[6] = CAMERA_PIN_D6,
[7] = CAMERA_PIN_D7,
},
.vsync_io = CAMERA_PIN_VSYNC,
.de_io = CAMERA_PIN_HREF,
.pclk_io = CAMERA_PIN_PCLK,
.xclk_io = CAMERA_PIN_XCLK,
};
esp_video_init_sccb_config_t sccb_config = {
.init_sccb = true,
.i2c_config = {
.port = 0,
.scl_pin = CAMERA_PIN_SIOC,
.sda_pin = CAMERA_PIN_SIOD,
},
.freq = 100000,
};
esp_video_init_dvp_config_t dvp_config = {
.sccb_config = sccb_config,
.reset_pin = CAMERA_PIN_RESET,
.pwdn_pin = CAMERA_PIN_PWDN,
.dvp_pin = dvp_pin_config,
.xclk_freq = XCLK_FREQ_HZ,
};
esp_video_init_config_t video_config = {
.dvp = &dvp_config,
};
camera_ = new EspVideo(video_config);
camera_config_t config = {};
config.pin_d0 = CAMERA_PIN_D0;
config.pin_d1 = CAMERA_PIN_D1;
config.pin_d2 = CAMERA_PIN_D2;
config.pin_d3 = CAMERA_PIN_D3;
config.pin_d4 = CAMERA_PIN_D4;
config.pin_d5 = CAMERA_PIN_D5;
config.pin_d6 = CAMERA_PIN_D6;
config.pin_d7 = CAMERA_PIN_D7;
config.pin_xclk = CAMERA_PIN_XCLK;
config.pin_pclk = CAMERA_PIN_PCLK;
config.pin_vsync = CAMERA_PIN_VSYNC;
config.pin_href = CAMERA_PIN_HREF;
config.pin_sccb_sda = CAMERA_PIN_SIOD;
config.pin_sccb_scl = CAMERA_PIN_SIOC;
config.sccb_i2c_port = 0;
config.pin_pwdn = CAMERA_PIN_PWDN;
config.pin_reset = CAMERA_PIN_RESET;
config.xclk_freq_hz = XCLK_FREQ_HZ;
config.pixel_format = PIXFORMAT_RGB565;
config.frame_size = FRAMESIZE_VGA;
config.jpeg_quality = 12;
config.fb_count = 1;
config.fb_location = CAMERA_FB_IN_PSRAM;
config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
camera_ = new Esp32Camera(config);
camera_->SetHMirror(false);
}

View File

@@ -9,6 +9,7 @@ public:
virtual bool Capture() = 0;
virtual bool SetHMirror(bool enabled) = 0;
virtual bool SetVFlip(bool enabled) = 0;
virtual bool SetSwapBytes(bool enabled) { return false; } // Optional, default no-op
virtual std::string Explain(const std::string& question) = 0;
};

View File

@@ -41,6 +41,11 @@ Esp32Camera::~Esp32Camera() {
esp_camera_fb_return(current_fb_);
current_fb_ = nullptr;
}
if (encode_buf_) {
heap_caps_free(encode_buf_);
encode_buf_ = nullptr;
encode_buf_size_ = 0;
}
esp_camera_deinit();
streaming_on_ = false;
}
@@ -72,30 +77,46 @@ bool Esp32Camera::Capture() {
}
}
// Perform byte swapping for RGB565 format and prepare preview image
// Prepare encode buffer for RGB565 format (with optional byte swapping)
if (current_fb_->format == PIXFORMAT_RGB565) {
size_t pixel_count = current_fb_->width * current_fb_->height;
size_t data_size = pixel_count * 2;
uint8_t *preview_data = (uint8_t *)heap_caps_malloc(data_size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if (preview_data == nullptr) {
ESP_LOGE(TAG, "Failed to allocate memory for preview image");
return false;
// Allocate or reallocate encode buffer if needed
if (encode_buf_size_ < data_size) {
if (encode_buf_) {
heap_caps_free(encode_buf_);
}
encode_buf_ = (uint8_t *)heap_caps_malloc(data_size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if (encode_buf_ == nullptr) {
ESP_LOGE(TAG, "Failed to allocate memory for encode buffer");
encode_buf_size_ = 0;
return false;
}
encode_buf_size_ = data_size;
}
// Copy data to encode buffer with optional byte swapping
uint16_t *src = (uint16_t *)current_fb_->buf;
uint16_t *dst = (uint16_t *)preview_data;
for (size_t i = 0; i < pixel_count; i++) {
// Copy data from driver buffer to preview buffer with byte swapping
dst[i] = __builtin_bswap16(src[i]);
uint16_t *dst = (uint16_t *)encode_buf_;
if (swap_bytes_enabled_) {
for (size_t i = 0; i < pixel_count; i++) {
dst[i] = __builtin_bswap16(src[i]);
}
} else {
memcpy(encode_buf_, current_fb_->buf, data_size);
}
// Display preview image
auto display = dynamic_cast<LvglDisplay *>(Board::GetInstance().GetDisplay());
if (display != nullptr) {
display->SetPreviewImage(std::make_unique<LvglAllocatedImage>(preview_data, data_size, current_fb_->width, current_fb_->height, current_fb_->width * 2, LV_COLOR_FORMAT_RGB565));
} else {
heap_caps_free(preview_data);
// Allocate separate buffer for preview display
uint8_t *preview_data = (uint8_t *)heap_caps_malloc(data_size, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
if (preview_data != nullptr) {
memcpy(preview_data, encode_buf_, data_size);
auto display = dynamic_cast<LvglDisplay *>(Board::GetInstance().GetDisplay());
if (display != nullptr) {
display->SetPreviewImage(std::make_unique<LvglAllocatedImage>(preview_data, data_size, current_fb_->width, current_fb_->height, current_fb_->width * 2, LV_COLOR_FORMAT_RGB565));
} else {
heap_caps_free(preview_data);
}
}
} else if (current_fb_->format == PIXFORMAT_JPEG) {
// JPEG format preview usually requires decoding, skip preview display for now, just log
@@ -126,6 +147,11 @@ bool Esp32Camera::SetVFlip(bool enabled) {
return true;
}
bool Esp32Camera::SetSwapBytes(bool enabled) {
swap_bytes_enabled_ = enabled;
return true;
}
std::string Esp32Camera::Explain(const std::string &question) {
if (explain_url_.empty()) {
throw std::runtime_error("Image explain URL or token is not set");
@@ -172,7 +198,15 @@ std::string Esp32Camera::Explain(const std::string &question) {
return;
}
bool ok = image_to_jpeg_cb(current_fb_->buf, current_fb_->len, w, h, enc_fmt, 80,
// Use encode buffer for RGB565, otherwise use original frame buffer
uint8_t *jpeg_src_buf = current_fb_->buf;
size_t jpeg_src_len = current_fb_->len;
if (current_fb_->format == PIXFORMAT_RGB565 && encode_buf_ != nullptr) {
jpeg_src_buf = encode_buf_;
jpeg_src_len = encode_buf_size_;
}
bool ok = image_to_jpeg_cb(jpeg_src_buf, jpeg_src_len, w, h, enc_fmt, 80,
[](void* arg, size_t index, const void* data, size_t len) -> size_t {
auto jpeg_queue = static_cast<QueueHandle_t>(arg);
JpegChunk chunk = {.data = nullptr, .len = len};

View File

@@ -23,10 +23,13 @@ class Esp32Camera : public Camera
{
private:
bool streaming_on_ = false;
bool swap_bytes_enabled_ = true; // Swap pixel byte order for RGB565, enabled by default
std::string explain_url_;
std::string explain_token_;
std::thread encoder_thread_;
camera_fb_t *current_fb_ = nullptr;
uint8_t *encode_buf_ = nullptr; // Buffer for JPEG encoding (with optional byte swap)
size_t encode_buf_size_ = 0;
public:
Esp32Camera(const camera_config_t &config);
@@ -36,5 +39,6 @@ public:
virtual bool Capture() override;
virtual bool SetHMirror(bool enabled) override;
virtual bool SetVFlip(bool enabled) override;
virtual bool SetSwapBytes(bool enabled) override;
virtual std::string Explain(const std::string &question) override;
};

View File

@@ -107,6 +107,9 @@ void Nt26Board::StartNetwork() {
ScheduleAsyncStop();
OnNetworkEvent(NetworkEvent::ModemErrorInitFailed);
break;
case UartEthModem::UartEthModemEvent::InFlightMode:
ESP_LOGW(TAG, "Modem in flight mode");
break;
}
});

View File

@@ -152,6 +152,8 @@ void K10AudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gpio_
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_tdm_mode(rx_handle_, &tdm_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -3,80 +3,40 @@
#include <esp_log.h>
#include <cstring>
#include <vector>
#include "assets.h"
#include "assets/lang_config.h"
#include "display/lvgl_display/emoji_collection.h"
#include "display/lvgl_display/lvgl_image.h"
#include "display/lvgl_display/lvgl_theme.h"
#include "otto_emoji_gif.h"
#define TAG "ElectronEmojiDisplay"
ElectronEmojiDisplay::ElectronEmojiDisplay(esp_lcd_panel_io_handle_t panel_io, esp_lcd_panel_handle_t panel, int width, int height, int offset_x, int offset_y, bool mirror_x, bool mirror_y,
bool swap_xy)
: SpiLcdDisplay(panel_io, panel, width, height, offset_x, offset_y, mirror_x, mirror_y, swap_xy) {
InitializeElectronEmojis();
SetupChatLabel();
}
void ElectronEmojiDisplay::InitializeElectronEmojis() {
ESP_LOGI(TAG, "初始化Electron GIF表情");
auto otto_emoji_collection = std::make_shared<EmojiCollection>();
// 中性/平静类表情 -> staticstate
otto_emoji_collection->AddEmoji("staticstate", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
otto_emoji_collection->AddEmoji("neutral", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
otto_emoji_collection->AddEmoji("relaxed", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
otto_emoji_collection->AddEmoji("sleepy", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
otto_emoji_collection->AddEmoji("idle", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
// 积极/开心类表情 -> happy
otto_emoji_collection->AddEmoji("happy", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("laughing", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("funny", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("loving", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("confident", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("winking", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("cool", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("delicious", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("kissy", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("silly", new LvglRawImage((void*)happy.data, happy.data_size));
// 悲伤类表情 -> sad
otto_emoji_collection->AddEmoji("sad", new LvglRawImage((void*)sad.data, sad.data_size));
otto_emoji_collection->AddEmoji("crying", new LvglRawImage((void*)sad.data, sad.data_size));
// 愤怒类表情 -> anger
otto_emoji_collection->AddEmoji("anger", new LvglRawImage((void*)anger.data, anger.data_size));
otto_emoji_collection->AddEmoji("angry", new LvglRawImage((void*)anger.data, anger.data_size));
// 惊讶类表情 -> scare
otto_emoji_collection->AddEmoji("scare", new LvglRawImage((void*)scare.data, scare.data_size));
otto_emoji_collection->AddEmoji("surprised", new LvglRawImage((void*)scare.data, scare.data_size));
otto_emoji_collection->AddEmoji("shocked", new LvglRawImage((void*)scare.data, scare.data_size));
// 思考/困惑类表情 -> buxue
otto_emoji_collection->AddEmoji("buxue", new LvglRawImage((void*)buxue.data, buxue.data_size));
otto_emoji_collection->AddEmoji("thinking", new LvglRawImage((void*)buxue.data, buxue.data_size));
otto_emoji_collection->AddEmoji("confused", new LvglRawImage((void*)buxue.data, buxue.data_size));
otto_emoji_collection->AddEmoji("embarrassed", new LvglRawImage((void*)buxue.data, buxue.data_size));
// 将表情集合添加到主题中
auto& theme_manager = LvglThemeManager::GetInstance();
auto light_theme = theme_manager.GetTheme("light");
auto dark_theme = theme_manager.GetTheme("dark");
if (light_theme != nullptr) {
light_theme->set_emoji_collection(otto_emoji_collection);
void ElectronEmojiDisplay::SetupUI() {
// Prevent duplicate calls - parent SetupUI() will also check, but check here for early return
if (setup_ui_called_) {
ESP_LOGW(TAG, "SetupUI() called multiple times, skipping duplicate call");
return;
}
if (dark_theme != nullptr) {
dark_theme->set_emoji_collection(otto_emoji_collection);
}
// 设置默认表情为staticstate
// Call parent SetupUI() first to create all lvgl objects
SpiLcdDisplay::SetupUI();
// Set default emotion after UI is initialized
SetEmotion("staticstate");
}
ESP_LOGI(TAG, "Electron GIF表情初始化完成");
void ElectronEmojiDisplay::InitializeElectronEmojis() {
ESP_LOGI(TAG, "Electron表情初始化将由Assets系统处理");
// 表情初始化已移至assets系统,通过DEFAULT_EMOJI_COLLECTION=otto-gif配置
// assets.cc会从assets分区加载GIF表情并设置到theme
// Note: Default emotion is now set in SetupUI() after LVGL objects are created
}
void ElectronEmojiDisplay::SetupChatLabel() {

View File

@@ -15,6 +15,7 @@ class ElectronEmojiDisplay : public SpiLcdDisplay {
virtual ~ElectronEmojiDisplay() = default;
virtual void SetStatus(const char* status) override;
virtual void SetupUI() override;
private:
void InitializeElectronEmojis();

View File

@@ -4,10 +4,7 @@
{
"name": "esp-box-3",
"sdkconfig_append": [
"CONFIG_USE_DEVICE_AEC=y",
"CONFIG_USE_EMOTE_MESSAGE_STYLE=y",
"CONFIG_FLASH_CUSTOM_ASSETS=y",
"CONFIG_CUSTOM_ASSETS_FILE=\"https://dl.espressif.com/AE/wn9_nihaoxiaozhi_tts-font_puhui_common_20_4-esp-box-3.bin\""
"CONFIG_USE_DEVICE_AEC=y"
]
}
]

View File

@@ -165,6 +165,8 @@ void BoxAudioCodecLite::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, g
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_tdm_mode(rx_handle_, &tdm_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -81,6 +81,7 @@ AdcPdmAudioCodec::AdcPdmAudioCodec(int input_sample_rate, int output_sample_rate
const i2s_pdm_tx_config_t *p_i2s_cfg = &pdm_cfg_default;
ESP_ERROR_CHECK(i2s_channel_init_pdm_tx_mode(tx_handle_, p_i2s_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
audio_codec_i2s_cfg_t i2s_cfg = {
.port = I2S_NUM_0,

View File

@@ -81,6 +81,7 @@ AdcPdmAudioCodec::AdcPdmAudioCodec(int input_sample_rate, int output_sample_rate
const i2s_pdm_tx_config_t *p_i2s_cfg = &pdm_cfg_default;
ESP_ERROR_CHECK(i2s_channel_init_pdm_tx_mode(tx_handle_, p_i2s_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
audio_codec_i2s_cfg_t i2s_cfg = {
.port = I2S_NUM_0,

View File

@@ -21,4 +21,19 @@
2. 关闭设备电源后,长按电源键不松手;
3. 在烧录工具中选择对应的串口COM Port
4. 点击烧录按钮,选择 UART 模式;
5. 烧录完成前请勿松开电源键。
5. 烧录完成前请勿松开电源键。
## 引脚
- 1-9
- 1. DAT2 NC
- 2. CD/DAT3 片选,低电平有效。(未知)
- 3. CMD IO48Command/Response Line主机通过此线向TF卡发送命令和数据
- 4. VDD 供电
- 5. CLX IO47时钟由主机产生同步数据通信
- 6. VSS GND
- 7. DAT0 IO21SPI_MISOTF卡通过此线向主机返回响应和数据
- 8. DAT1 NC
依次为从右向左为1-9

View File

@@ -39,10 +39,23 @@ public:
bool swap_xy)
: SpiLcdDisplay(io_handle, panel_handle, width, height, offset_x, offset_y, mirror_x, mirror_y, swap_xy)
{
// Note: UI customization should be done in SetupUI(), not in constructor
// to ensure lvgl objects are created before accessing them
}
virtual void SetupUI() override {
// Call parent SetupUI() first to create all lvgl objects
SpiLcdDisplay::SetupUI();
DisplayLockGuard lock(this);
lv_obj_set_style_pad_left(status_bar_, LV_HOR_RES * 0.167, 0);
lv_obj_set_style_pad_right(status_bar_, LV_HOR_RES * 0.167, 0);
// 状态栏容器适配
lv_obj_set_style_pad_left(top_bar_, LV_HOR_RES * 0.12, 0); // 左侧填充12%
lv_obj_set_style_pad_right(top_bar_, LV_HOR_RES * 0.12, 0); // 右侧填充12%
// 表情容器上移适配
lv_obj_align(emoji_box_, LV_ALIGN_CENTER, 0, -50); // 向上偏移50
// 消息栏适配
lv_obj_align(bottom_bar_, LV_ALIGN_BOTTOM_MID, 0, -40); // 向上偏移40
}
};

View File

@@ -230,7 +230,7 @@ private:
config.pin_reset = CAMERA_PIN_RESET;
config.xclk_freq_hz = XCLK_FREQ_HZ;
config.pixel_format = PIXFORMAT_RGB565;
config.frame_size = FRAMESIZE_VGA;
config.frame_size = FRAMESIZE_QVGA;
config.jpeg_quality = 12;
config.fb_count = 1;
config.fb_location = CAMERA_FB_IN_PSRAM;

View File

@@ -0,0 +1,48 @@
# M5Stack Cardputer Adv
M5Stack Cardputer Adv 是一款基于 ESP32-S3FN8 (Stamp-S3A) 的卡片式电脑。
## 硬件规格
| 组件 | 规格 |
|------|------|
| MCU | ESP32-S3FN8 @ 240MHz |
| Flash | 8MB |
| 显示屏 | ST7789V2 1.14" 240x135 |
| 音频编解码 | ES8311 |
| 功放 | NS4150B |
| 麦克风 | MEMS |
| 键盘 | 56键 (TCA8418) |
| IMU | BMI270 |
| 电池 | 1750mAh |
## 引脚定义
### 显示屏 (ST7789V2)
| 功能 | GPIO |
|------|------|
| MOSI | GPIO35 |
| SCLK | GPIO36 |
| CS | GPIO37 |
| DC | GPIO34 |
| RST | GPIO33 |
| BL | GPIO38 |
### 音频 (ES8311)
| 功能 | GPIO |
|------|------|
| I2C SDA | GPIO8 |
| I2C SCL | GPIO9 |
| I2S BCLK | GPIO41 |
| I2S LRCK | GPIO43 |
| I2S DOUT | GPIO46 |
| I2S DIN | GPIO42 |
## 使用方法
1. 按下 BOOT 按钮进入配网模式
2. 连接 WiFi 后即可使用语音助手功能
## 参考链接
- [M5Stack Cardputer Adv 官方文档](https://docs.m5stack.com/en/core/Cardputer-Adv)

View File

@@ -0,0 +1,58 @@
#ifndef _BOARD_CONFIG_H_
#define _BOARD_CONFIG_H_
// M5Stack Cardputer Adv Board configuration
// MCU: ESP32-S3FN8 (Stamp-S3A)
// Display: ST7789V2 1.14" 240x135
// Audio: ES8311 + NS4150B
#include <driver/gpio.h>
// Audio settings
#define AUDIO_INPUT_SAMPLE_RATE 24000
#define AUDIO_OUTPUT_SAMPLE_RATE 24000
// I2S Audio pins (ES8311)
#define AUDIO_I2S_GPIO_MCLK GPIO_NUM_NC
#define AUDIO_I2S_GPIO_BCLK GPIO_NUM_41 // SCLK
#define AUDIO_I2S_GPIO_WS GPIO_NUM_43 // LRCK
#define AUDIO_I2S_GPIO_DOUT GPIO_NUM_42 // DSDIN (MCU -> ES8311)
#define AUDIO_I2S_GPIO_DIN GPIO_NUM_46 // ASDOUT (ES8311 -> MCU)
// I2C pins (shared for ES8311, TCA8418, BMI270)
#define AUDIO_CODEC_I2C_SDA_PIN GPIO_NUM_8
#define AUDIO_CODEC_I2C_SCL_PIN GPIO_NUM_9
#define AUDIO_CODEC_ES8311_ADDR ES8311_CODEC_DEFAULT_ADDR
#define AUDIO_CODEC_PA_PIN GPIO_NUM_NC // NS4150B is always on
// Button
#define BOOT_BUTTON_GPIO GPIO_NUM_0
#define BUILTIN_LED_GPIO GPIO_NUM_NC
#define VOLUME_UP_BUTTON_GPIO GPIO_NUM_NC
#define VOLUME_DOWN_BUTTON_GPIO GPIO_NUM_NC
// Display ST7789V2 (SPI)
#define DISPLAY_WIDTH 240
#define DISPLAY_HEIGHT 135
#define DISPLAY_MIRROR_X true
#define DISPLAY_MIRROR_Y false
#define DISPLAY_SWAP_XY true
#define DISPLAY_OFFSET_X 40
#define DISPLAY_OFFSET_Y 52
#define DISPLAY_SPI_MOSI_PIN GPIO_NUM_35
#define DISPLAY_SPI_SCLK_PIN GPIO_NUM_36
#define DISPLAY_SPI_CS_PIN GPIO_NUM_37
#define DISPLAY_DC_PIN GPIO_NUM_34
#define DISPLAY_RST_PIN GPIO_NUM_33
#define DISPLAY_BACKLIGHT_PIN GPIO_NUM_38
#define DISPLAY_BACKLIGHT_OUTPUT_INVERT false
// Keyboard TCA8418 I2C address
#define KEYBOARD_TCA8418_ADDR 0x34
// IMU BMI270 I2C address
#define IMU_BMI270_ADDR 0x68
#endif // _BOARD_CONFIG_H_

View File

@@ -2,11 +2,12 @@
"target": "esp32s3",
"builds": [
{
"name": "waveshare-s3-epaper-1.54",
"name": "m5stack-cardputer-adv",
"sdkconfig_append": [
"CONFIG_SPIRAM=n",
"CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y",
"CONFIG_PARTITION_TABLE_CUSTOM_FILENAME=\"partitions/v2/8m.csv\""
]
}
]
}
}

View File

@@ -0,0 +1,158 @@
#include "wifi_board.h"
#include "codecs/es8311_audio_codec.h"
#include "display/lcd_display.h"
#include "application.h"
#include "button.h"
#include "config.h"
#include "i2c_device.h"
#include <esp_log.h>
#include <driver/i2c_master.h>
#include <driver/spi_common.h>
#include <esp_lcd_panel_io.h>
#include <esp_lcd_panel_ops.h>
#include <esp_lcd_panel_vendor.h>
#define TAG "CardputerAdv"
class M5StackCardputerAdvBoard : public WifiBoard {
private:
i2c_master_bus_handle_t i2c_bus_;
LcdDisplay* display_;
Button boot_button_;
esp_lcd_panel_io_handle_t panel_io_ = nullptr;
esp_lcd_panel_handle_t panel_ = nullptr;
void InitializeI2c() {
ESP_LOGI(TAG, "Initialize I2C bus");
i2c_master_bus_config_t i2c_bus_cfg = {
.i2c_port = I2C_NUM_0,
.sda_io_num = AUDIO_CODEC_I2C_SDA_PIN,
.scl_io_num = AUDIO_CODEC_I2C_SCL_PIN,
.clk_source = I2C_CLK_SRC_DEFAULT,
.glitch_ignore_cnt = 7,
.intr_priority = 0,
.trans_queue_depth = 0,
.flags = {
.enable_internal_pullup = 1,
},
};
ESP_ERROR_CHECK(i2c_new_master_bus(&i2c_bus_cfg, &i2c_bus_));
}
void I2cDetect() {
uint8_t address;
ESP_LOGI(TAG, "I2C device scan:");
printf(" 0 1 2 3 4 5 6 7 8 9 a b c d e f\r\n");
for (int i = 0; i < 128; i += 16) {
printf("%02x: ", i);
for (int j = 0; j < 16; j++) {
fflush(stdout);
address = i + j;
esp_err_t ret = i2c_master_probe(i2c_bus_, address, pdMS_TO_TICKS(200));
if (ret == ESP_OK) {
printf("%02x ", address);
} else if (ret == ESP_ERR_TIMEOUT) {
printf("UU ");
} else {
printf("-- ");
}
}
printf("\r\n");
}
}
void InitializeSpi() {
ESP_LOGI(TAG, "Initialize SPI bus");
spi_bus_config_t buscfg = {};
buscfg.mosi_io_num = DISPLAY_SPI_MOSI_PIN;
buscfg.miso_io_num = GPIO_NUM_NC;
buscfg.sclk_io_num = DISPLAY_SPI_SCLK_PIN;
buscfg.quadwp_io_num = GPIO_NUM_NC;
buscfg.quadhd_io_num = GPIO_NUM_NC;
buscfg.max_transfer_sz = DISPLAY_WIDTH * DISPLAY_HEIGHT * sizeof(uint16_t);
ESP_ERROR_CHECK(spi_bus_initialize(SPI3_HOST, &buscfg, SPI_DMA_CH_AUTO));
}
void InitializeSt7789Display() {
ESP_LOGI(TAG, "Initialize ST7789V2 display");
esp_lcd_panel_io_spi_config_t io_config = {};
io_config.cs_gpio_num = DISPLAY_SPI_CS_PIN;
io_config.dc_gpio_num = DISPLAY_DC_PIN;
io_config.spi_mode = 0;
io_config.pclk_hz = 40 * 1000 * 1000;
io_config.trans_queue_depth = 10;
io_config.lcd_cmd_bits = 8;
io_config.lcd_param_bits = 8;
io_config.flags.sio_mode = 1; // 3-wire SPI mode (M5GFX uses spi_3wire = true)
ESP_ERROR_CHECK(esp_lcd_new_panel_io_spi(SPI3_HOST, &io_config, &panel_io_));
ESP_LOGI(TAG, "Install ST7789 panel driver");
esp_lcd_panel_dev_config_t panel_config = {};
panel_config.reset_gpio_num = DISPLAY_RST_PIN;
panel_config.rgb_ele_order = LCD_RGB_ELEMENT_ORDER_RGB;
panel_config.bits_per_pixel = 16;
ESP_ERROR_CHECK(esp_lcd_new_panel_st7789(panel_io_, &panel_config, &panel_));
ESP_ERROR_CHECK(esp_lcd_panel_reset(panel_));
ESP_ERROR_CHECK(esp_lcd_panel_init(panel_));
ESP_ERROR_CHECK(esp_lcd_panel_invert_color(panel_, true));
ESP_ERROR_CHECK(esp_lcd_panel_swap_xy(panel_, DISPLAY_SWAP_XY));
ESP_ERROR_CHECK(esp_lcd_panel_mirror(panel_, DISPLAY_MIRROR_X, DISPLAY_MIRROR_Y));
display_ = new SpiLcdDisplay(panel_io_, panel_,
DISPLAY_WIDTH, DISPLAY_HEIGHT, DISPLAY_OFFSET_X, DISPLAY_OFFSET_Y,
DISPLAY_MIRROR_X, DISPLAY_MIRROR_Y, DISPLAY_SWAP_XY);
}
void InitializeButtons() {
boot_button_.OnClick([this]() {
auto& app = Application::GetInstance();
if (app.GetDeviceState() == kDeviceStateStarting) {
EnterWifiConfigMode();
return;
}
app.ToggleChatState();
});
}
public:
M5StackCardputerAdvBoard() : boot_button_(BOOT_BUTTON_GPIO) {
InitializeI2c();
I2cDetect();
InitializeSpi();
InitializeSt7789Display();
InitializeButtons();
GetBacklight()->RestoreBrightness();
}
virtual AudioCodec* GetAudioCodec() override {
static Es8311AudioCodec audio_codec(
i2c_bus_,
I2C_NUM_0,
AUDIO_INPUT_SAMPLE_RATE,
AUDIO_OUTPUT_SAMPLE_RATE,
AUDIO_I2S_GPIO_MCLK,
AUDIO_I2S_GPIO_BCLK,
AUDIO_I2S_GPIO_WS,
AUDIO_I2S_GPIO_DOUT,
AUDIO_I2S_GPIO_DIN,
AUDIO_CODEC_PA_PIN,
AUDIO_CODEC_ES8311_ADDR,
false); // use_mclk = false, Cardputer Adv has no MCLK pin
return &audio_codec;
}
virtual Display* GetDisplay() override {
return display_;
}
virtual Backlight* GetBacklight() override {
// M5GFX uses 256Hz PWM frequency for Cardputer backlight
static PwmBacklight backlight(DISPLAY_BACKLIGHT_PIN, DISPLAY_BACKLIGHT_OUTPUT_INVERT, 256);
return &backlight;
}
};
DECLARE_BOARD(M5StackCardputerAdvBoard);

View File

@@ -177,6 +177,8 @@ void CoreS3AudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gp
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_tdm_mode(rx_handle_, &tdm_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -176,6 +176,8 @@ void Tab5AudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gpio
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_init_tdm_mode(rx_handle_, &tdm_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -31,6 +31,13 @@ public:
bool mirror_y,
bool swap_xy)
: SpiLcdDisplay(io_handle, panel_handle, width, height, offset_x, offset_y, mirror_x, mirror_y, swap_xy) {
// Note: UI customization should be done in SetupUI(), not in constructor
// to ensure lvgl objects are created before accessing them
}
virtual void SetupUI() override {
// Call parent SetupUI() first to create all lvgl objects
SpiLcdDisplay::SetupUI();
DisplayLockGuard lock(this);
// 由于屏幕是圆的,所以状态栏需要增加左右内边距

View File

@@ -1,44 +1,63 @@
#ifndef _BOARD_CONFIG_H_
#define _BOARD_CONFIG_H_
#include <driver/gpio.h>
#include <driver/adc.h>
#include <driver/gpio.h>
#define OTTO_VERSION_AUTO 0
#define OTTO_VERSION_CAMERA 1
#define OTTO_VERSION_NO_CAMERA 2
#ifndef OTTO_HARDWARE_VERSION
#define OTTO_HARDWARE_VERSION OTTO_VERSION_AUTO
#endif
enum OttoCameraType {
OTTO_CAMERA_NONE = 0,
OTTO_CAMERA_OV2640 = 1,
OTTO_CAMERA_OV3660 = 2,
OTTO_CAMERA_UNKNOWN = 99,
};
#define OV2640_PID_1 0x2640
#define OV2640_PID_2 0x2626
#define OV3660_PID 0x3660
struct HardwareConfig {
gpio_num_t power_charge_detect_pin;
adc_unit_t power_adc_unit;
adc_channel_t power_adc_channel;
gpio_num_t right_leg_pin;
gpio_num_t right_foot_pin;
gpio_num_t left_leg_pin;
gpio_num_t left_foot_pin;
gpio_num_t left_hand_pin;
gpio_num_t right_hand_pin;
int audio_input_sample_rate;
int audio_output_sample_rate;
bool audio_use_simplex;
gpio_num_t audio_i2s_gpio_ws;
gpio_num_t audio_i2s_gpio_bclk;
gpio_num_t audio_i2s_gpio_din;
gpio_num_t audio_i2s_gpio_dout;
gpio_num_t audio_i2s_mic_gpio_ws;
gpio_num_t audio_i2s_mic_gpio_sck;
gpio_num_t audio_i2s_mic_gpio_din;
gpio_num_t audio_i2s_spk_gpio_dout;
gpio_num_t audio_i2s_spk_gpio_bclk;
gpio_num_t audio_i2s_spk_gpio_lrck;
gpio_num_t display_backlight_pin;
gpio_num_t display_mosi_pin;
gpio_num_t display_clk_pin;
gpio_num_t display_dc_pin;
gpio_num_t display_rst_pin;
gpio_num_t display_cs_pin;
gpio_num_t i2c_sda_pin;
gpio_num_t i2c_scl_pin;
};
@@ -47,37 +66,37 @@ constexpr HardwareConfig CAMERA_VERSION_CONFIG = {
.power_charge_detect_pin = GPIO_NUM_NC,
.power_adc_unit = ADC_UNIT_1,
.power_adc_channel = ADC_CHANNEL_1,
.right_leg_pin = GPIO_NUM_43,
.right_foot_pin = GPIO_NUM_44,
.left_leg_pin = GPIO_NUM_5,
.left_foot_pin = GPIO_NUM_6,
.left_hand_pin = GPIO_NUM_4,
.right_hand_pin = GPIO_NUM_7,
.audio_input_sample_rate = 16000,
.audio_output_sample_rate = 16000,
.audio_use_simplex = false,
.audio_i2s_gpio_ws = GPIO_NUM_40,
.audio_i2s_gpio_bclk = GPIO_NUM_42,
.audio_i2s_gpio_din = GPIO_NUM_41,
.audio_i2s_gpio_dout = GPIO_NUM_39,
.audio_i2s_mic_gpio_ws = GPIO_NUM_NC,
.audio_i2s_mic_gpio_sck = GPIO_NUM_NC,
.audio_i2s_mic_gpio_din = GPIO_NUM_NC,
.audio_i2s_spk_gpio_dout = GPIO_NUM_NC,
.audio_i2s_spk_gpio_bclk = GPIO_NUM_NC,
.audio_i2s_spk_gpio_lrck = GPIO_NUM_NC,
.display_backlight_pin = GPIO_NUM_38,
.display_mosi_pin = GPIO_NUM_45,
.display_clk_pin = GPIO_NUM_48,
.display_dc_pin = GPIO_NUM_47,
.display_rst_pin = GPIO_NUM_1,
.display_cs_pin = GPIO_NUM_NC,
.i2c_sda_pin = GPIO_NUM_15,
.i2c_scl_pin = GPIO_NUM_16,
};
@@ -86,37 +105,37 @@ constexpr HardwareConfig NON_CAMERA_VERSION_CONFIG = {
.power_charge_detect_pin = GPIO_NUM_21,
.power_adc_unit = ADC_UNIT_2,
.power_adc_channel = ADC_CHANNEL_3,
.right_leg_pin = GPIO_NUM_39,
.right_foot_pin = GPIO_NUM_38,
.left_leg_pin = GPIO_NUM_17,
.left_foot_pin = GPIO_NUM_18,
.left_hand_pin = GPIO_NUM_8,
.right_hand_pin = GPIO_NUM_12,
.audio_input_sample_rate = 16000,
.audio_output_sample_rate = 24000,
.audio_use_simplex = true,
.audio_i2s_gpio_ws = GPIO_NUM_NC,
.audio_i2s_gpio_bclk = GPIO_NUM_NC,
.audio_i2s_gpio_din = GPIO_NUM_NC,
.audio_i2s_gpio_dout = GPIO_NUM_NC,
.audio_i2s_mic_gpio_ws = GPIO_NUM_4,
.audio_i2s_mic_gpio_sck = GPIO_NUM_5,
.audio_i2s_mic_gpio_din = GPIO_NUM_6,
.audio_i2s_spk_gpio_dout = GPIO_NUM_7,
.audio_i2s_spk_gpio_bclk = GPIO_NUM_15,
.audio_i2s_spk_gpio_lrck = GPIO_NUM_16,
.display_backlight_pin = GPIO_NUM_3,
.display_mosi_pin = GPIO_NUM_10,
.display_clk_pin = GPIO_NUM_9,
.display_dc_pin = GPIO_NUM_46,
.display_rst_pin = GPIO_NUM_11,
.display_cs_pin = GPIO_NUM_12,
.i2c_sda_pin = GPIO_NUM_NC,
.i2c_scl_pin = GPIO_NUM_NC,
};

View File

@@ -7,7 +7,10 @@
"CONFIG_HTTPD_WS_SUPPORT=y",
"CONFIG_CAMERA_OV2640=y",
"CONFIG_CAMERA_OV2640_AUTO_DETECT_DVP_INTERFACE_SENSOR=y",
"CONFIG_CAMERA_OV2640_DVP_YUV422_240X240_25FPS=y"
"CONFIG_CAMERA_OV2640_DVP_YUV422_240X240_25FPS=y",
"CONFIG_CAMERA_OV3660=y",
"CONFIG_CAMERA_OV3660_AUTO_DETECT_DVP_INTERFACE_SENSOR=y",
"CONFIG_CAMERA_OV3660_DVP_YUV422_240X240_24FPS=y"
]
}
]

View File

@@ -3,85 +3,52 @@
#include <esp_log.h>
#include <cstring>
#include <vector>
#include "assets.h"
#include "assets/lang_config.h"
#include "display/lvgl_display/emoji_collection.h"
#include "display/lvgl_display/lvgl_image.h"
#include "display/lvgl_display/lvgl_theme.h"
#include "otto_emoji_gif.h"
#define TAG "OttoEmojiDisplay"
OttoEmojiDisplay::OttoEmojiDisplay(esp_lcd_panel_io_handle_t panel_io, esp_lcd_panel_handle_t panel, int width, int height, int offset_x, int offset_y, bool mirror_x, bool mirror_y, bool swap_xy)
: SpiLcdDisplay(panel_io, panel, width, height, offset_x, offset_y, mirror_x, mirror_y, swap_xy) {
InitializeOttoEmojis();
SetupPreviewImage();
SetTheme(LvglThemeManager::GetInstance().GetTheme("dark"));
}
void OttoEmojiDisplay::SetupUI() {
// Prevent duplicate calls - parent SetupUI() will also check, but check here for early return
if (setup_ui_called_) {
ESP_LOGW(TAG, "SetupUI() called multiple times, skipping duplicate call");
return;
}
// Call parent SetupUI() first to create all lvgl objects
SpiLcdDisplay::SetupUI();
// Setup preview image after UI is initialized
DisplayLockGuard lock(this);
lv_obj_set_size(preview_image_, width_ , height_ );
// Set default emotion after UI is initialized
SetEmotion("staticstate");
}
void OttoEmojiDisplay::SetupPreviewImage() {
DisplayLockGuard lock(this);
if (preview_image_ == nullptr) {
ESP_LOGW(TAG, "SetupPreviewImage called but preview_image_ is nullptr (UI not initialized yet)");
return;
}
lv_obj_set_size(preview_image_, width_ , height_ );
}
void OttoEmojiDisplay::InitializeOttoEmojis() {
ESP_LOGI(TAG, "初始化Otto GIF表情");
auto otto_emoji_collection = std::make_shared<EmojiCollection>();
// 中性/平静类表情 -> staticstate
otto_emoji_collection->AddEmoji("staticstate", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
otto_emoji_collection->AddEmoji("neutral", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
otto_emoji_collection->AddEmoji("relaxed", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
otto_emoji_collection->AddEmoji("sleepy", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
otto_emoji_collection->AddEmoji("idle", new LvglRawImage((void*)staticstate.data, staticstate.data_size));
// 积极/开心类表情 -> happy
otto_emoji_collection->AddEmoji("happy", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("laughing", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("funny", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("loving", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("confident", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("winking", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("cool", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("delicious", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("kissy", new LvglRawImage((void*)happy.data, happy.data_size));
otto_emoji_collection->AddEmoji("silly", new LvglRawImage((void*)happy.data, happy.data_size));
// 悲伤类表情 -> sad
otto_emoji_collection->AddEmoji("sad", new LvglRawImage((void*)sad.data, sad.data_size));
otto_emoji_collection->AddEmoji("crying", new LvglRawImage((void*)sad.data, sad.data_size));
// 愤怒类表情 -> anger
otto_emoji_collection->AddEmoji("anger", new LvglRawImage((void*)anger.data, anger.data_size));
otto_emoji_collection->AddEmoji("angry", new LvglRawImage((void*)anger.data, anger.data_size));
// 惊讶类表情 -> scare
otto_emoji_collection->AddEmoji("scare", new LvglRawImage((void*)scare.data, scare.data_size));
otto_emoji_collection->AddEmoji("surprised", new LvglRawImage((void*)scare.data, scare.data_size));
otto_emoji_collection->AddEmoji("shocked", new LvglRawImage((void*)scare.data, scare.data_size));
// 思考/困惑类表情 -> buxue
otto_emoji_collection->AddEmoji("buxue", new LvglRawImage((void*)buxue.data, buxue.data_size));
otto_emoji_collection->AddEmoji("thinking", new LvglRawImage((void*)buxue.data, buxue.data_size));
otto_emoji_collection->AddEmoji("confused", new LvglRawImage((void*)buxue.data, buxue.data_size));
otto_emoji_collection->AddEmoji("embarrassed", new LvglRawImage((void*)buxue.data, buxue.data_size));
// 将表情集合添加到主题中
auto& theme_manager = LvglThemeManager::GetInstance();
auto light_theme = theme_manager.GetTheme("light");
auto dark_theme = theme_manager.GetTheme("dark");
if (light_theme != nullptr) {
light_theme->set_emoji_collection(otto_emoji_collection);
}
if (dark_theme != nullptr) {
dark_theme->set_emoji_collection(otto_emoji_collection);
}
// 设置默认表情为staticstate
SetEmotion("staticstate");
ESP_LOGI(TAG, "Otto GIF表情初始化完成");
ESP_LOGI(TAG, "Otto表情初始化将由Assets系统处理");
// 表情初始化已移至assets系统,通过DEFAULT_EMOJI_COLLECTION=otto-gif配置
// assets.cc会从assets分区加载GIF表情并设置到theme
// Note: Default emotion is now set in SetupUI() after LVGL objects are created
}
LV_FONT_DECLARE(OTTO_ICON_FONT);
@@ -148,7 +115,7 @@ void OttoEmojiDisplay::SetPreviewImage(std::unique_ptr<LvglImage> image) {
auto img_dsc = preview_image_cached_->image_dsc();
// 设置图片源并显示预览图片
lv_image_set_src(preview_image_, img_dsc);
lv_image_set_rotation(preview_image_, -900);
lv_image_set_rotation(preview_image_, 900);
if (img_dsc->header.w > 0 && img_dsc->header.h > 0) {
// zoom factor 1.0
lv_image_set_scale(preview_image_, 256 * width_ / img_dsc->header.w);

View File

@@ -16,6 +16,7 @@ class OttoEmojiDisplay : public SpiLcdDisplay {
virtual ~OttoEmojiDisplay() = default;
virtual void SetStatus(const char* status) override;
virtual void SetPreviewImage(std::unique_ptr<LvglImage> image) override;
virtual void SetupUI() override;
private:
void InitializeOttoEmojis();

View File

@@ -1,25 +1,25 @@
#include <driver/i2c_master.h>
#include <driver/spi_common.h>
#include <driver/ledc.h>
#include <driver/spi_common.h>
#include <esp_lcd_panel_io.h>
#include <esp_lcd_panel_ops.h>
#include <esp_lcd_panel_vendor.h>
#include <esp_log.h>
#include "application.h"
#include "codecs/no_audio_codec.h"
#include "button.h"
#include "codecs/no_audio_codec.h"
#include "config.h"
#include "display/lcd_display.h"
#include "esp_video.h"
#include "lamp_controller.h"
#include "led/single_led.h"
#include "mcp_server.h"
#include "otto_emoji_display.h"
#include "power_manager.h"
#include "system_reset.h"
#include "wifi_board.h"
#include "esp_video.h"
#include "websocket_control_server.h"
#include "wifi_board.h"
#define TAG "OttoRobot"
@@ -34,9 +34,10 @@ private:
HardwareConfig hw_config_;
AudioCodec* audio_codec_;
i2c_master_bus_handle_t i2c_bus_;
EspVideo *camera_;
EspVideo* camera_;
bool has_camera_;
OttoCameraType camera_type_;
bool DetectHardwareVersion() {
ledc_timer_config_t ledc_timer = {
.speed_mode = LEDC_LOW_SPEED_MODE,
@@ -49,7 +50,7 @@ private:
if (ret != ESP_OK) {
return false;
}
ledc_channel_config_t ledc_channel = {
.gpio_num = CAMERA_XCLK,
.speed_mode = LEDC_LOW_SPEED_MODE,
@@ -63,7 +64,7 @@ private:
if (ret != ESP_OK) {
return false;
}
vTaskDelay(pdMS_TO_TICKS(100));
i2c_master_bus_config_t i2c_bus_cfg = {
.i2c_port = I2C_NUM_0,
@@ -73,11 +74,12 @@ private:
.glitch_ignore_cnt = 7,
.intr_priority = 0,
.trans_queue_depth = 0,
.flags = {
.enable_internal_pullup = 1,
},
.flags =
{
.enable_internal_pullup = 1,
},
};
ret = i2c_new_master_bus(&i2c_bus_cfg, &i2c_bus_);
if (ret != ESP_OK) {
ledc_stop(LEDC_LOW_SPEED_MODE, LEDC_CHANNEL, 0);
@@ -85,7 +87,8 @@ private:
}
const uint8_t camera_addresses[] = {0x30, 0x3C, 0x21, 0x60};
bool camera_found = false;
uint16_t detected_pid = 0;
for (size_t i = 0; i < sizeof(camera_addresses); i++) {
uint8_t addr = camera_addresses[i];
i2c_device_config_t dev_cfg = {
@@ -93,36 +96,71 @@ private:
.device_address = addr,
.scl_speed_hz = 100000,
};
i2c_master_dev_handle_t dev_handle;
ret = i2c_master_bus_add_device(i2c_bus_, &dev_cfg, &dev_handle);
if (ret == ESP_OK) {
uint8_t reg_addr = 0x0A;
uint8_t data[2];
ret = i2c_master_transmit_receive(dev_handle, &reg_addr, 1, data, 2, 200);
if (ret == ESP_OK) {
uint8_t data[2] = {0, 0};
uint8_t reg_addr_8bit = 0x0A;
ret = i2c_master_transmit_receive(dev_handle, &reg_addr_8bit, 1, data, 2, 200);
if (ret == ESP_OK && (data[0] != 0 || data[1] != 0)) {
detected_pid = (data[0] << 8) | data[1];
ESP_LOGI(TAG, "检测到摄像头 (OV2640方式) PID=0x%04X (地址=0x%02X)",
detected_pid, addr);
camera_found = true;
i2c_master_bus_rm_device(dev_handle);
break;
}
uint8_t reg_addr_high[2] = {0x30, 0x0A};
uint8_t reg_addr_low[2] = {0x30, 0x0B};
uint8_t pid_high = 0, pid_low = 0;
ret = i2c_master_transmit_receive(dev_handle, reg_addr_high, 2, &pid_high, 1, 200);
if (ret == ESP_OK) {
ret =
i2c_master_transmit_receive(dev_handle, reg_addr_low, 2, &pid_low, 1, 200);
if (ret == ESP_OK) {
detected_pid = (pid_high << 8) | pid_low;
if (detected_pid != 0) {
ESP_LOGI(TAG, "检测到摄像头 (OV3660方式) PID=0x%04X (地址=0x%02X)",
detected_pid, addr);
camera_found = true;
i2c_master_bus_rm_device(dev_handle);
break;
}
}
}
i2c_master_bus_rm_device(dev_handle);
}
}
if (!camera_found) {
i2c_del_master_bus(i2c_bus_);
i2c_bus_ = nullptr;
ledc_stop(LEDC_LOW_SPEED_MODE, LEDC_CHANNEL, 0);
camera_type_ = OTTO_CAMERA_NONE;
} else {
// 根据 PID 判断摄像头类型
if (detected_pid == OV2640_PID_1 || detected_pid == OV2640_PID_2) {
camera_type_ = OTTO_CAMERA_OV2640;
ESP_LOGI(TAG, "摄像头类型: OV2640 (PID=0x%04X)", detected_pid);
} else if (detected_pid == OV3660_PID) {
camera_type_ = OTTO_CAMERA_OV3660;
ESP_LOGI(TAG, "摄像头类型: OV3660 (PID=0x%04X)", detected_pid);
} else {
camera_type_ = OTTO_CAMERA_UNKNOWN;
ESP_LOGW(TAG, "未知摄像头类型PID=0x%04X", detected_pid);
}
}
return camera_found;
}
void InitializePowerManager() {
power_manager_ = new PowerManager(
hw_config_.power_charge_detect_pin,
hw_config_.power_adc_unit,
hw_config_.power_adc_channel
);
power_manager_ = new PowerManager(hw_config_.power_charge_detect_pin,
hw_config_.power_adc_unit, hw_config_.power_adc_channel);
}
void InitializeSpi() {
@@ -163,9 +201,9 @@ private:
esp_lcd_panel_swap_xy(panel, DISPLAY_SWAP_XY);
esp_lcd_panel_mirror(panel, DISPLAY_MIRROR_X, DISPLAY_MIRROR_Y);
display_ = new OttoEmojiDisplay(
panel_io, panel, DISPLAY_WIDTH, DISPLAY_HEIGHT, DISPLAY_OFFSET_X, DISPLAY_OFFSET_Y,
DISPLAY_MIRROR_X, DISPLAY_MIRROR_Y, DISPLAY_SWAP_XY);
display_ = new OttoEmojiDisplay(panel_io, panel, DISPLAY_WIDTH, DISPLAY_HEIGHT,
DISPLAY_OFFSET_X, DISPLAY_OFFSET_Y, DISPLAY_MIRROR_X,
DISPLAY_MIRROR_Y, DISPLAY_SWAP_XY);
}
void InitializeButtons() {
@@ -179,17 +217,14 @@ private:
});
}
void InitializeOttoController() {
::InitializeOttoController(hw_config_);
}
public:
const HardwareConfig& GetHardwareConfig() const {
return hw_config_;
}
private:
void InitializeOttoController() { ::InitializeOttoController(hw_config_); }
public:
const HardwareConfig& GetHardwareConfig() const { return hw_config_; }
OttoCameraType GetCameraType() const { return camera_type_; }
private:
void InitializeWebSocketControlServer() {
ws_control_server_ = new WebSocketControlServer();
if (!ws_control_server_->Start(8080)) {
@@ -201,7 +236,7 @@ private:
void StartNetwork() override {
WifiBoard::StartNetwork();
vTaskDelay(pdMS_TO_TICKS(1000));
InitializeWebSocketControlServer();
}
@@ -209,20 +244,21 @@ private:
if (!has_camera_ || i2c_bus_ == nullptr) {
return false;
}
try {
static esp_cam_ctlr_dvp_pin_config_t dvp_pin_config = {
.data_width = CAM_CTLR_DATA_WIDTH_8,
.data_io = {
[0] = CAMERA_D0,
[1] = CAMERA_D1,
[2] = CAMERA_D2,
[3] = CAMERA_D3,
[4] = CAMERA_D4,
[5] = CAMERA_D5,
[6] = CAMERA_D6,
[7] = CAMERA_D7,
},
.data_io =
{
[0] = CAMERA_D0,
[1] = CAMERA_D1,
[2] = CAMERA_D2,
[3] = CAMERA_D3,
[4] = CAMERA_D4,
[5] = CAMERA_D5,
[6] = CAMERA_D6,
[7] = CAMERA_D7,
},
.vsync_io = CAMERA_VSYNC,
.de_io = CAMERA_HSYNC,
.pclk_io = CAMERA_PCLK,
@@ -248,86 +284,126 @@ private:
};
camera_ = new EspVideo(video_config);
camera_->SetVFlip(true);
// 根据摄像头类型设置不同的翻转参数
switch (camera_type_) {
case OTTO_CAMERA_OV3660:
camera_->SetVFlip(true);
camera_->SetHMirror(true);
ESP_LOGI(TAG, "OV3660: 设置 VFlip=true, HMirror=true");
break;
case OTTO_CAMERA_OV2640:
default:
camera_->SetVFlip(true);
camera_->SetHMirror(false);
ESP_LOGI(TAG, "OV2640: 设置 VFlip=true, HMirror=false");
break;
}
return true;
} catch (...) {
camera_ = nullptr;
return false;
}
}
void InitializeAudioCodec() {
if (hw_config_.audio_use_simplex) {
audio_codec_ = new NoAudioCodecSimplex(
hw_config_.audio_input_sample_rate,
hw_config_.audio_output_sample_rate,
hw_config_.audio_i2s_spk_gpio_bclk,
hw_config_.audio_i2s_spk_gpio_lrck,
hw_config_.audio_i2s_spk_gpio_dout,
hw_config_.audio_i2s_mic_gpio_sck,
hw_config_.audio_i2s_mic_gpio_ws,
hw_config_.audio_i2s_mic_gpio_din
);
hw_config_.audio_input_sample_rate, hw_config_.audio_output_sample_rate,
hw_config_.audio_i2s_spk_gpio_bclk, hw_config_.audio_i2s_spk_gpio_lrck,
hw_config_.audio_i2s_spk_gpio_dout, hw_config_.audio_i2s_mic_gpio_sck,
hw_config_.audio_i2s_mic_gpio_ws, hw_config_.audio_i2s_mic_gpio_din);
} else {
audio_codec_ = new NoAudioCodecDuplex(
hw_config_.audio_input_sample_rate,
hw_config_.audio_output_sample_rate,
hw_config_.audio_i2s_gpio_bclk,
hw_config_.audio_i2s_gpio_ws,
hw_config_.audio_i2s_gpio_dout,
hw_config_.audio_i2s_gpio_din
);
hw_config_.audio_input_sample_rate, hw_config_.audio_output_sample_rate,
hw_config_.audio_i2s_gpio_bclk, hw_config_.audio_i2s_gpio_ws,
hw_config_.audio_i2s_gpio_dout, hw_config_.audio_i2s_gpio_din);
}
}
public:
OttoRobot() : boot_button_(BOOT_BUTTON_GPIO),
audio_codec_(nullptr),
i2c_bus_(nullptr),
camera_(nullptr),
has_camera_(false) {
OttoRobot()
: boot_button_(BOOT_BUTTON_GPIO),
audio_codec_(nullptr),
i2c_bus_(nullptr),
camera_(nullptr),
has_camera_(false),
camera_type_(OTTO_CAMERA_NONE) {
#if OTTO_HARDWARE_VERSION == OTTO_VERSION_AUTO
// 自动检测硬件版本(同时检测摄像头类型)
has_camera_ = DetectHardwareVersion();
if (has_camera_)
ESP_LOGI(TAG, "自动检测硬件版本: %s", has_camera_ ? "摄像头版" : "无摄像头版");
#elif OTTO_HARDWARE_VERSION == OTTO_VERSION_CAMERA
// 强制使用摄像头版本,但仍检测具体摄像头类型
has_camera_ = DetectHardwareVersion();
if (!has_camera_) {
// 检测失败时仍使用摄像头配置,但不知道具体类型
has_camera_ = true;
camera_type_ = OTTO_CAMERA_UNKNOWN;
ESP_LOGW(TAG, "强制使用摄像头版本配置,但未能检测到摄像头类型");
// 初始化 I2C 总线用于摄像头
i2c_master_bus_config_t i2c_bus_cfg = {
.i2c_port = I2C_NUM_0,
.sda_io_num = CAMERA_VERSION_CONFIG.i2c_sda_pin,
.scl_io_num = CAMERA_VERSION_CONFIG.i2c_scl_pin,
.clk_source = I2C_CLK_SRC_DEFAULT,
.glitch_ignore_cnt = 7,
.intr_priority = 0,
.trans_queue_depth = 0,
.flags =
{
.enable_internal_pullup = 1,
},
};
i2c_new_master_bus(&i2c_bus_cfg, &i2c_bus_);
} else {
ESP_LOGI(TAG, "强制使用摄像头版本配置");
}
#elif OTTO_HARDWARE_VERSION == OTTO_VERSION_NO_CAMERA
// 强制使用无摄像头版本
has_camera_ = false;
camera_type_ = OTTO_CAMERA_NONE;
ESP_LOGI(TAG, "强制使用无摄像头版本配置");
#else
#error \
"OTTO_HARDWARE_VERSION 设置无效,请使用 OTTO_VERSION_AUTO, OTTO_VERSION_CAMERA 或 OTTO_VERSION_NO_CAMERA"
#endif
if (has_camera_)
hw_config_ = CAMERA_VERSION_CONFIG;
else
else
hw_config_ = NON_CAMERA_VERSION_CONFIG;
InitializeSpi();
InitializeLcdDisplay();
InitializeButtons();
InitializePowerManager();
InitializeAudioCodec();
if (has_camera_) {
if (!InitializeCamera()) {
has_camera_ = false;
}
}
InitializeOttoController();
ws_control_server_ = nullptr;
GetBacklight()->RestoreBrightness();
}
virtual AudioCodec *GetAudioCodec() override {
return audio_codec_;
}
virtual AudioCodec* GetAudioCodec() override { return audio_codec_; }
virtual Display* GetDisplay() override {
return display_;
}
virtual Display* GetDisplay() override { return display_; }
virtual Backlight* GetBacklight() override {
static PwmBacklight* backlight = nullptr;
if (backlight == nullptr) {
backlight = new PwmBacklight(hw_config_.display_backlight_pin, DISPLAY_BACKLIGHT_OUTPUT_INVERT);
backlight =
new PwmBacklight(hw_config_.display_backlight_pin, DISPLAY_BACKLIGHT_OUTPUT_INVERT);
}
return backlight;
}
virtual bool GetBatteryLevel(int& level, bool& charging, bool& discharging) override {
charging = power_manager_->IsCharging();
discharging = !charging;
@@ -335,9 +411,7 @@ public:
return true;
}
virtual Camera *GetCamera() override {
return has_camera_ ? camera_ : nullptr;
}
virtual Camera* GetCamera() override { return has_camera_ ? camera_ : nullptr; }
};
DECLARE_BOARD(OttoRobot);

View File

@@ -143,6 +143,8 @@ void SensecapAudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk,
std_cfg.slot_cfg.slot_mask = I2S_STD_SLOT_RIGHT;
ESP_ERROR_CHECK(i2s_channel_init_std_mode(rx_handle_, &std_cfg));
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
ESP_LOGI(TAG, "Duplex channels created");
}

View File

@@ -45,7 +45,14 @@ class CustomLcdDisplay : public SpiLcdDisplay {
bool mirror_y,
bool swap_xy)
: SpiLcdDisplay(io_handle, panel_handle, width, height, offset_x, offset_y, mirror_x, mirror_y, swap_xy) {
// Note: UI customization should be done in SetupUI(), not in constructor
// to ensure lvgl objects are created before accessing them
}
virtual void SetupUI() override {
// Call parent SetupUI() first to create all lvgl objects
SpiLcdDisplay::SetupUI();
DisplayLockGuard lock(this);
auto lvgl_theme = static_cast<LvglTheme*>(current_theme_);
auto text_font = lvgl_theme->text_font()->font();

Some files were not shown because too many files have changed in this diff Show More