Compare commits

...

208 Commits

Author SHA1 Message Date
Terrence
737333e1a6 Add bed-operator board 2026-01-06 20:42:52 +08:00
Terrence
89a06eda81 fix: esp_psram_get_size not found in c3 2026-01-06 15:11:31 +08:00
Terrence
846d7afb80 use MAIN_EVENT_CLOCK_TICK to avoid audio glitches 2026-01-06 15:11:31 +08:00
Terrence
855e8c09b4 Add image cache 2026-01-06 15:11:31 +08:00
laride
be88719932 feat: Add ESP-SensairShuttle (#1620)
* feat: Add ESP-SensairShuttle

* fix: fix board name
2026-01-02 12:19:46 +08:00
Wang is proud
213117ded2 BLUFI network configuration supports encryption. (#1603) 2025-12-29 15:04:29 +08:00
Wang is proud
76ff1cf0dc Fixed the issue where voice wake-up causes panic when DEBUG is enabled. (#1606) 2025-12-29 15:03:02 +08:00
zczc365
5d44633687 feat: 小智云聊增加两个语音指令 (#1596) 2025-12-26 03:01:18 +08:00
Kevincoooool
5113a5f4bb Add power management for ESP32S3-Korvo2-V3 board (#1591)
Introduces PowerManager and integrates it into the ESP32S3-Korvo2-V3 board class to monitor battery level, charging status, and manage power save mode. Adds power_manager.h with battery ADC reading, calibration, and event callbacks. Updates board initialization to support power management and power save timer functionality.
2025-12-22 20:33:02 +08:00
pfangzhi
f501a5f440 add boards waveshare-c6-touch-lcd-1.83 (#1575)
* Update CMakeLists.txt 

WAVESHARE_C6_TOUCH_LCD_1_83

* Update Kconfig.projbuild

WAVESHARE_C6_TOUCH_LCD_1_83

* Create README.md

waveshare-c6-touch-lcd-1.83

* Add files via upload
2025-12-22 14:11:18 +08:00
MOV
e9649cfc58 feat: add support for Movecall Moji2 board (ESP32-C5) (#1589)
* add: Moji 2 has built-in ESP32-C5 dual band Wi-Fi

* feat: PowerSaveTimer 160 >> 240
2025-12-21 18:57:53 +08:00
Denis Ryabchikov
ee5587019b Fix issue "Missing spaces" (#1582)
Co-authored-by: xuxin <xuxin@espressif.com>
2025-12-18 20:01:00 +08:00
Denis Ryabchikov
ebdd58748a echoear: improved quality of audio assets for Russian language (#1584)
Co-authored-by: Рябчиков Денис <DRyabchikov_od@inno.tech>
2025-12-18 07:02:55 +08:00
Wang is proud
99c32d9331 Fixed the bug where BLUFI would output error logs even when the network connection was normal. (#1567) 2025-12-17 06:18:41 +08:00
Kevincoooool
cccaf71c3e Enable camera configs and update button/camera logic (#1561)
Added camera-related sdkconfig options to esp32s3-korvo2-v3 and kevin-sp-v4-dev config.json files. Updated esp32s3_korvo2_v3_board.cc to implement button actions for WiFi config and chat state toggling. Refactored kevin-sp-v4_board.cc to unify I2C bus usage for codec and camera, and adjusted SCCB initialization logic.
2025-12-15 20:16:56 +08:00
Xiaoxia
564018c762 Enhance audio feedback mechanism by introducing a flag to play a popup (#1560)
* Enhance audio feedback mechanism by introducing a flag to play a popup sound after transitioning to listening mode. Update Schedule method to accept rvalue references for callbacks. Bump esp-ml307 component version to 3.5.3. Adjust signal strength thresholds in Ml307Board for better accuracy.

* Update esp-wifi-connect component version to 3.0.2 in idf_component.yml

* Adjust Wi-Fi signal strength thresholds in WifiBoard for improved accuracy in network state icon representation.
2025-12-15 12:54:17 +08:00
神奇bug在哪里
0ccdc082b5 feat: BluFi Wi-Fi configuration (#1321)
Signed-off-by: WhereAreBugs <wherearebugs@icloud.com>
2025-12-13 01:43:32 +08:00
laride
4b582f8074 feat: support USB Camera (#1519) 2025-12-11 07:10:12 +08:00
Tomato Me
1f0d2e993b fix incorrect json configuartion (#1541) 2025-12-10 21:07:56 +08:00
Xiaoxia
b7db68457c v2.1.0: Upgrade esp-wifi-connect to 3.0; New device state machine (#1528)
* Upgrade component version

* update fonts component version

* Handle OTA error code

* Update project version to 2.1.0 and add device state machine implementation

- Upgrade  esp-wifi-connect to 3.0.0, allowing reconfiguring wifi without rebooting
- Introduce device state machine with state change notification in new files
- Remove obsolete device state event files
- Update application logic to utilize new state machine
- Minor adjustments in various board implementations for state handling

* fix compile errors

* Refactor power saving mode implementation to use PowerSaveLevel enumeration

- Updated Application class to replace SetPowerSaveMode with SetPowerSaveLevel, allowing for LOW_POWER and PERFORMANCE settings.
- Modified various board implementations to align with the new power save level structure.
- Ensured consistent handling of power save levels across different board files, enhancing code maintainability and clarity.

* Refactor power save level checks across multiple board implementations

- Updated the condition for power save level checks in various board files to ensure that the power save timer only wakes up when the level is not set to LOW_POWER.
- Improved consistency in handling power save levels, enhancing code clarity and maintainability.

* Refactor EnterWifiConfigMode calls in board implementations

- Updated calls to EnterWifiConfigMode to use the appropriate instance reference (self or board) across multiple board files.
- Improved code consistency and clarity in handling device state during WiFi configuration mode entry.

* Add cellular modem event handling and improve network status updates

- Introduced new network events for cellular modem operations, including detecting, registration errors, and timeouts.
- Enhanced the Application class to handle different network states and update the display status accordingly.
- Refactored Ml307Board to implement a callback mechanism for network events, improving modularity and responsiveness.
- Updated dual_network_board and board headers to support new network event callbacks, ensuring consistent handling across board implementations.

* update esp-wifi-connect version

* Update WiFi configuration tool messages across multiple board implementations to clarify user actions
2025-12-09 09:24:56 +08:00
小鹏
11c79a7003 Refactor Otto Robot configuration and initialization (#1534)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters

* Update Otto controller parameters for oscillation settings

- Changed default oscillation period from 500ms to 300ms.
- Increased default steps from 5.0 to 8.0.
- Updated default amplitude from 20 degrees to 0 degrees.
- Enhanced documentation with new examples for oscillation modes and sequences.

* Fix default amplitude initialization in Otto controller to use a single zero instead of two digits.

* chore: update txp666/otto-emoji-gif-component version to 1.0.3 in idf_component.yml

* Refactor Otto controller
- Consolidated movement actions into a unified tool for the Otto robot, allowing for a single action command with various parameters.
- Removed individual movement tools (walk, turn, jump, etc.) and replaced them with a more flexible action system.

* Enhance Otto robot functionality by adding WebSocket control server and IP address retrieval feature. Updated config to support WebSocket, and revised README to include new control options and usage examples.

* Add camera support for Otto Robot board

- Introduced configuration option to enable the Otto Robot camera in Kconfig.
- Updated config.h to define camera-related GPIO pins and settings.
- Modified config.json to include camera configuration.
- Enhanced otto_robot.cc to initialize I2C and camera components when the camera is enabled.
- Adjusted power_manager.h to manage battery updates during camera operations.
- Removed unused SetupChatLabel method from OttoEmojiDisplay class.

* Refactor Otto Robot configuration and initialization

- Removed the camera configuration option from Kconfig and related code.
- Introduced a new HardwareConfig struct to encapsulate hardware pin definitions and settings.
- Updated config.h to define camera and non-camera configurations using the new struct.
- Refactored otto_controller.cc and otto_robot.cc to utilize the HardwareConfig struct for initialization.
- Enhanced camera detection and initialization logic based on hardware version.
- Improved audio codec initialization based on configuration settings.
2025-12-08 20:55:23 +08:00
小鹏
f9de29519b Add camera support for Otto Robot board (#1520)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters

* Update Otto controller parameters for oscillation settings

- Changed default oscillation period from 500ms to 300ms.
- Increased default steps from 5.0 to 8.0.
- Updated default amplitude from 20 degrees to 0 degrees.
- Enhanced documentation with new examples for oscillation modes and sequences.

* Fix default amplitude initialization in Otto controller to use a single zero instead of two digits.

* chore: update txp666/otto-emoji-gif-component version to 1.0.3 in idf_component.yml

* Refactor Otto controller
- Consolidated movement actions into a unified tool for the Otto robot, allowing for a single action command with various parameters.
- Removed individual movement tools (walk, turn, jump, etc.) and replaced them with a more flexible action system.

* Enhance Otto robot functionality by adding WebSocket control server and IP address retrieval feature. Updated config to support WebSocket, and revised README to include new control options and usage examples.

* Add camera support for Otto Robot board

- Introduced configuration option to enable the Otto Robot camera in Kconfig.
- Updated config.h to define camera-related GPIO pins and settings.
- Modified config.json to include camera configuration.
- Enhanced otto_robot.cc to initialize I2C and camera components when the camera is enabled.
- Adjusted power_manager.h to manage battery updates during camera operations.
- Removed unused SetupChatLabel method from OttoEmojiDisplay class.
2025-12-05 23:05:44 +08:00
NologoTech
d7c1aef77a 增加无名科技星智铝合金 wifi版本 的board支持 (#1352)
* 增加无名科技星智铝合金 wifi版本 的board支持

* 删除多余头文件

* 星智铝合金:触摸单击时间改为200ms

---------

Co-authored-by: Limh2017 <wenwen19115>
2025-12-01 19:38:14 +08:00
Piat Jonathan
33c2fe90a8 Support for DIY AI Watch HU-087 kit (#1445)
* Added new board for hu-087 smart AI watch

* Added a README with link to prouct on Aliexpress

* Trying to control chatbot mode instead of listening state

* Removed the esp32-s3 prefix for the board

* Changed board name to match folder naming

* Fixed formating and long URL

* Fixed typo in README.md
2025-12-01 19:30:40 +08:00
Denis Ryabchikov
e8f68a331f Fix: Build fails for esp32s3 Echoear (#1505) (#1507)
Co-authored-by: Рябчиков Денис <DRyabchikov_od@inno.tech>
2025-11-30 12:44:23 +08:00
tojoevan
2d15bef298 Add GetBatteryLevel for Waveshare ESP32-S3-ePaper-1.54 board (#1506) 2025-11-30 11:44:07 +08:00
virgil
28db4bd60a Fix sensecap watcher inference (#1501)
* feat: add keepalive check for Himax client and handle restart

* fix: adjust layout and positioning for top and bottom bars

* fix: fix other param restoring default ​​when modifying one param .
2025-11-30 10:48:14 +08:00
Y1hsiaochunnn
01a12b325f Fix the touch initialization issue. (#1497) 2025-11-28 06:44:59 +08:00
Y1hsiaochunnn
c87b1eabf4 Add Camera support for Waveshare's ESP32-P4 series development board (#1459)
* Add Camera support to Waveshare's ESP32-P4 series development board

* Fix i2c handle error
2025-11-22 15:37:22 +08:00
laride
908c9d5708 feat: add esp-spot c5 (#1462)
* feat: add esp-spot c5

* fix: fix table custom filename
2025-11-20 15:52:49 +08:00
laride
860d12a12c feat: support JPEG input (#1455) 2025-11-18 20:34:22 +08:00
Xiaoxia
511349a7bd Change LCD display layout from grids to layers (#1438)
* Upgrade component version

* update fonts component version

* change lcd display layout from grids to layers

* Update English README as default

* Handle OTA error code
2025-11-17 22:38:31 +08:00
小鹏
764f6e3349 Add wechat-mini-program support for otto-robot (#1444)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters

* Update Otto controller parameters for oscillation settings

- Changed default oscillation period from 500ms to 300ms.
- Increased default steps from 5.0 to 8.0.
- Updated default amplitude from 20 degrees to 0 degrees.
- Enhanced documentation with new examples for oscillation modes and sequences.

* Fix default amplitude initialization in Otto controller to use a single zero instead of two digits.

* chore: update txp666/otto-emoji-gif-component version to 1.0.3 in idf_component.yml

* Refactor Otto controller
- Consolidated movement actions into a unified tool for the Otto robot, allowing for a single action command with various parameters.
- Removed individual movement tools (walk, turn, jump, etc.) and replaced them with a more flexible action system.

* Enhance Otto robot functionality by adding WebSocket control server and IP address retrieval feature. Updated config to support WebSocket, and revised README to include new control options and usage examples.
2025-11-17 22:26:58 +08:00
laride
59d08c7612 feat: add camera support for M5Stack Tab5 (#1442) 2025-11-17 22:20:25 +08:00
laride
ebbb2fa319 fix: avoid SRAM overflow in software JPEG encode (#1441) 2025-11-17 15:20:37 +08:00
香草味的纳西妲喵
d984cda486 Revise README for ogg_converter clarity and instructions (#1432)
Updated README to clarify the purpose of the OGG converter and provide clearer instructions for downloading and installing FFmpeg.
2025-11-15 13:11:35 +08:00
Y1hsiaochunnn
52f3134f30 Add support for the Waveshare ESP32-C6-Touch-AMOLED-2.06 third-party board (#1342)
* Add support for the Waveshare ESP32-C6-Touch-AMOLED-2.06 third-party board

* Fix the known issues

* Fix the name

* Update screen brightness

* readme content error correction
2025-11-15 13:07:46 +08:00
Tomato Me
7f332f120c Add support for Waveshare ESP32-S3-ePaper-1.54 board (#1375)
* Add support for Waveshare ESP32-S3-ePaper-1.54 board

* 根据审核,做对应的修改

* fix:优化格式,主要是对头文件引用进行修改

---------

Co-authored-by: Tomato Me <55608753+wurongmin@users.noreply.github.com>
2025-11-15 13:01:12 +08:00
Y1hsiaochunnn
ba10b2a2d2 fix MIPI_DSI_PHY_CLK_SRC_DEFAULT compile error (#1430)
* Update the default wake-up word configuration for the ESP32-P4 chip

* Refactor DSI bus configuration for LCD panel

* Update MIPI DSI bus configuration for LCD

* Update MIPI DSI bus configuration structure

* Cancel redundant configuration

* Cancel redundant configuration
2025-11-15 12:56:25 +08:00
laride
92de37e182 fix: fix esp_afe_sr_iface_t compile error (#1423)
* fix: fix esp_afe_sr_iface_t compile error

* fix: fix `MIPI_DSI_PHY_CLK_SRC_DEFAULT` compile error in IDF 5.5.2
2025-11-13 20:05:42 +08:00
Create123
30970abd1f board/m5stack-tab5: Add new display st7123 support. (#1409) 2025-11-12 18:32:06 +08:00
so95
06da25fd26 ESP-P4-Function-EV-Board add SD card, camera, and font initialization (#1403)
* feat: add support for ESP32-P4-Function-EV-Board with configuration and display handling

* detect wake word model from index.json

* update wait time before entering wifi configure mode

* feat: Enhance ESP32-P4 Function EV Board support with LCD and touch initialization

* feat: Update ESP32-P4 Function EV Board configuration for improved touch and SD card support

* feat: add touch I2C configuration and improve initialization structure

* Remove ESP hosted configuration from defaults

Removed ESP hosted configuration options.

* chore: update documentation for improved clarity

* refactor: remove obsolete files for ESP32-P4-Function-EV-Board and add updated configurations

* refactor: reintroduce ESP32-P4-Function-EV-Board implementation with updated configurations

* refactor: restore esp32_p4_function_ev_board dependency with updated version

* Remove ESP-P4-Function-EV-Board configuration and update README with detailed features; add SD card, camera, and font initialization in esp-p4-function-ev-board.cc

* Update main/boards/esp-p4-function-ev-board/esp-p4-function-ev-board.cc

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update main/boards/esp-p4-function-ev-board/esp-p4-function-ev-board.cc

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Enhance camera initialization in ESP-P4-Function-EV-Board; add fallback configuration for direct camera setup and update README for clarity

* Update clock source for MIPI DSI configuration in ESP-P4-Function-EV-Board

---------

Co-authored-by: n2flowjs-bot <n2flowjs@gmail.com>
Co-authored-by: Terrence <terrence@tenclass.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-11-10 21:41:59 +08:00
Spotpear
cdb025dd90 sp-esp32-s3-1.28-box (#1397)
* 修复屏幕条纹

* 增加充电标识

---------

Co-authored-by: Spotpear <services01@spotpear.cn>
2025-11-08 07:07:53 +08:00
Tomato Me
ce72f196b7 Add support for Waveshare ESP32-C6-Touch-AMOLED-1.32 board (#1376)
* Add support for Waveshare ESP32-C6-Touch-AMOLED-1.32 board

* Modify the config.json to comply with the rules
2025-11-06 15:53:54 +08:00
小鹏
4aa47cc591 Refactor Otto controller (#1388)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters

* Update Otto controller parameters for oscillation settings

- Changed default oscillation period from 500ms to 300ms.
- Increased default steps from 5.0 to 8.0.
- Updated default amplitude from 20 degrees to 0 degrees.
- Enhanced documentation with new examples for oscillation modes and sequences.

* Fix default amplitude initialization in Otto controller to use a single zero instead of two digits.

* chore: update txp666/otto-emoji-gif-component version to 1.0.3 in idf_component.yml

* Refactor Otto controller
- Consolidated movement actions into a unified tool for the Otto robot, allowing for a single action command with various parameters.
- Removed individual movement tools (walk, turn, jump, etc.) and replaced them with a more flexible action system.
2025-11-06 15:52:25 +08:00
Tomato Me
1664fda6e4 Add support for Waveshare ESP32-S3-Touch-AMOLED-1.32 board (#1378)
* Add support for Waveshare ESP32-S3-Touch-AMOLED-1.32 board

* Modify the config.json to comply with the rules
2025-11-05 17:28:46 +08:00
laoshanxi
ccee790c0d build fix for image_player (#1387)
* build fix for image_player

* Update image_player version to 1.1.1

---------

Co-authored-by: Xiaoxia <terrence@tenclass.com>
2025-11-05 17:24:14 +08:00
Xiaoxia
c6815c7ad5 Update component version (#1382)
* Add download_github_runs.py

* Update esp-ml307 version

* Update esp-sr to 2.2.0
2025-11-04 18:40:54 +08:00
小鹏
0e51c4c94c chore: update txp666/otto-emoji-gif-component version to 1.0.3 in idf_component.yml (#1381)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters

* Update Otto controller parameters for oscillation settings

- Changed default oscillation period from 500ms to 300ms.
- Increased default steps from 5.0 to 8.0.
- Updated default amplitude from 20 degrees to 0 degrees.
- Enhanced documentation with new examples for oscillation modes and sequences.

* Fix default amplitude initialization in Otto controller to use a single zero instead of two digits.

* chore: update txp666/otto-emoji-gif-component version to 1.0.3 in idf_component.yml
2025-11-04 16:16:54 +08:00
laride
221d4e7f69 feat: support camera image rotation (#1370) 2025-11-04 05:14:00 +08:00
小鹏
bcddbbde26 Update Otto controller parameters for oscillation settings (#1373)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters

* Update Otto controller parameters for oscillation settings

- Changed default oscillation period from 500ms to 300ms.
- Increased default steps from 5.0 to 8.0.
- Updated default amplitude from 20 degrees to 0 degrees.
- Enhanced documentation with new examples for oscillation modes and sequences.

* Fix default amplitude initialization in Otto controller to use a single zero instead of two digits.
2025-11-04 04:07:10 +08:00
小鹏
e39a46c1a0 otto新增动作和AI自定义编程动作MCP工具 (#1365)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* refactor: Update Otto emoji display configurations and functionalities

- Changed chat label text mode to circular scrolling for both Otto and Electron emoji displays.
- Bumped Otto robot version to 2.0.5 in the configuration file.
- Added new actions for Otto robot including Sit, WhirlwindLeg, Fitness, Greeting, Shy, RadioCalisthenics, MagicCircle, and Showcase.
- Enhanced servo sequence handling and added support for executing custom servo sequences.
- Improved logging and error handling for servo sequence execution.

* refactor: Update chat label long mode for Electron and Otto emoji displays

- Changed chat label text mode from wrap to circular scrolling for both Electron and Otto emoji displays.
- Improved consistency in chat label setup across both implementations.

* Update Otto robot README with new actions and parameters
2025-11-02 18:04:06 +08:00
Clark Scheff
1f602fa3a0 Add support for the Xorigin AiPi Lite (#1330)
* Add support for the Xorigin AiPi Lite

* aipi-lite: correction to README_en.md

* aipi-lite: use 20MHz display clock

Highger speeds might work sometimes but best not to push
it and use a sane speed.

* aipi-lite: change English name and remove Chinese TTS
2025-10-31 22:10:32 +08:00
小鹏
a9413e2d45 otto更新V2分区使用官方gif管理器 (#1359)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug

* refactor: Update Electron and Otto emoji display implementations

- Removed GIF selection from Kconfig for Electron and Otto boards.
- Updated Electron and Otto bot versions to 2.0.4 in their respective config files.
- Refactored emoji display classes to utilize EmojiCollection for managing emojis.
- Enhanced chat label setup and status display functionality in both classes.
- Cleaned up unused code and improved initialization logging for emoji displays.

* Rename OTTO_ICON_FONT.c to otto_icon_font.c

* Rename OTTO_ICON_FONT.c to otto_icon_font.c
2025-10-31 22:07:42 +08:00
virgil
6fbc60fa9f fix: fix memory leak,fix jpeg memory 16-bit alignment (#1350) 2025-10-30 10:44:18 +08:00
espressif2022
43ef2f4e7f feat: Add emote support for lichuang-dev (#1337) 2025-10-28 14:33:05 +08:00
Xiaoxia
3f54330ac9 Set minimal IDF version to 5.4.0 (#1339)
* version.py: not upload directory

* Downgrade IDF version to 5.4.0

---------

Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
2025-10-28 07:48:47 +08:00
Terrence
99a4fc94bf Normalize board type macros 2025-10-25 22:34:21 +08:00
Terrence
c052903ade Bump to 2.0.4 2025-10-25 04:21:46 +08:00
so95
6a4ec9dbbd feat: add support for ESP32-P4-Function-EV-Board (#1210)
* feat: add support for ESP32-P4-Function-EV-Board with configuration and display handling

* detect wake word model from index.json

* update wait time before entering wifi configure mode

* feat: Enhance ESP32-P4 Function EV Board support with LCD and touch initialization

* feat: Update ESP32-P4 Function EV Board configuration for improved touch and SD card support

* feat: add touch I2C configuration and improve initialization structure

* Remove ESP hosted configuration from defaults

Removed ESP hosted configuration options.

* chore: update documentation for improved clarity

* refactor: remove obsolete files for ESP32-P4-Function-EV-Board and add updated configurations

* refactor: reintroduce ESP32-P4-Function-EV-Board implementation with updated configurations

* refactor: restore esp32_p4_function_ev_board dependency with updated version

---------

Co-authored-by: n2flowjs-bot <n2flowjs@gmail.com>
Co-authored-by: Terrence <terrence@tenclass.com>
2025-10-25 03:41:00 +08:00
Y1hsiaochunnn
7a85430465 New Waveshare ESP32-S3-Touch-LCD-1.83 third-party board is available (#1323) 2025-10-25 02:28:02 +08:00
konglingboy
e8ed16bb42 修复charging_pin为NC充电时Battery Level不更新的问题 (#1316)
Co-authored-by: Yuv Zhao <admin@yuvcloud.com>
2025-10-21 13:03:26 +08:00
Y1hsiaochunnn
3176a45ebe 同步esp-video组件驱动摄像头的配置(waveshare s3-3.5、s3-3.5b、s3-audio-board) (#1291)
* 同步esp-video组件驱动摄像头的配置

* 修改Waveshare ESP32-S3-AUDIO-Board 摄像头配置。

* Modify the XCLK parameters of the camera, and modify the config.txt to be compatible with 2640 and 5640.

* Update the esp_video component version to 1.3.0

* Downgrade the esp_video component version. There are unfixed bugs.

* Update component version
2025-10-20 21:20:42 +08:00
virgil
a601a5cbc1 feat: sensecap watcher add inference (#1312)
* feat: Wake up when a person is detected

* fix: Solve the problem of no sound when using WakeWordInvoke

* fix: Solve the problem of triggering dialogue when the person has not left

* feat(vision): 优化视觉检测逻辑并增加配置接口

本次提交旨在优化视觉检测功能,使其行为更自然、更智能,并为用户提供灵活的配置选项。

主要更新包括:

1. 引入了更精细的检测状态机:

   - IDLE: 空闲状态,等待检测目标。

   - VALIDATING: 验证状态,在检测到目标后,持续一段时间(可配置)以确认其存在,防止误触发。

   - COOLDOWN: 冷却状态,在一次成功交互后进入,避免过于频繁的打扰。

2. 新增了用于配置视觉检测的 MCP 工具:

   - self.vision.get_detection_config: 获取当前的检测参数(阈值、冷却间隔、验证时长、目标类型)。

   - self.vision.set_detection_config: 允许用户动态修改这些参数,以适应不同场景。

3. 性能优化:

   - 增加了配置参数的内存缓存,避免了在检测循环中对 NVS 的频繁访问。

* feat: Inference using Model 4

* feat: default inference disable

* feat: version cmd change to output json

* fix: fix image display

* Fix include directives for esp_check and esp_app_desc

---------

Co-authored-by: Spencer <love4yzp@gmail.com>
Co-authored-by: Xiaoxia <terrence@tenclass.com>
2025-10-20 21:18:44 +08:00
Xiaoxia
ac03f8097d Add 16 more languages to locales (#1297)
* Add 16 more languages to locales

* Update esp-wifi-connect component version
2025-10-16 23:22:01 +08:00
yusuhua
92238612b6 Fix LilyGo-T-CameraPlus-S3 Camera (#1295)
* 修改摄像头配置

* Fix lilygo-t-cameraplus-s3 camera

---------

Co-authored-by: yusuhua <yusuhua@baidu.com>
2025-10-15 15:48:41 +08:00
Xiaoxia
c62ab7b239 Fix lichuang-dev camera (#1290)
* Fixatk-dnesp32s3 camera

* Update bread s3cam README

* Fix  lichuang-dev camera
2025-10-14 20:44:44 +08:00
DeepStart
992be9a636 将太极派新款和旧款分离 (#1287)
* 将太极派新款和旧款分离

* 新增太极派OTA配置

* 新增OTA配置
2025-10-14 18:52:18 +08:00
laride
60ad1c5afc Refactor: Use esp_video component (#1245)
* refactor: migrate camera module to esp-video library

* refactor: migrate boards to esp-video API (1/2)

* refactor: migrate boards to esp-video API (2/2)

* fix: use ESP-IDF 5.5

* refactor: migrate the JPEG encoder to `esp_new_jpeg`

* feat: add YUV422 support

* feat: improve pixelformat and device selection process

* feat: use ESP32-P4 Hardware JPEG Encoder
2025-10-14 10:44:45 +08:00
Linar Yusupov
4854bda302 add support for Wireless-Tag WTP4C5MP07S ESP32P4 board + 7 inch MIPI DSI LCD display combo (#1281)
* add support for Wireless-Tag WTP4C5MP07S ESP32P4 + 7 inch MIPI DSI LCD display combo

* a minor update of README

* update of the device name/folder

* exclude USE_DEVICE_AEC from the options list
2025-10-13 14:13:48 +08:00
konglingboy
1ccd5108c3 Optimize AdcBatteryMonitor to work without charge detection pin. (#1276)
Co-authored-by: Yuv Zhao <admin@yuvcloud.com>
2025-10-11 21:11:30 +08:00
junchao98
1d1f9ff1ff enable rrti for m5stack-tab5 (#1268) 2025-10-10 10:58:47 +08:00
LILYGO_L
8373a26f85 Add LILYGO T-Display-P4 board adaptation (#1269)
* Adapt for LilyGO-T-Circle-S3 device

* Adapt for LilyGO-T-Circle-S3 device

* Remove comments and modify the size of the lilygo-t-circle-s3 image

* Modify the code style and format to Google C++

* Modify the code style and format to Google C++

* Fixed bugs in the LILYGO T-Circle-S3 board and added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Merge branch 'main' of https://github.com/Llgok/xiaozhi-esp32

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Fix the color display issue for T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Update T-CameraPlus-S3_V1.2 Version Xiaozhi Example

* Resolve the issue where the camera on the T-CameraPlus-S3_V1.2 board cannot be used normally.

* Enhance microphone reception volume

* fix the issue where voice wake-up is not working

* fix the issue where voice wake-up is not working

* Add LILYGO T-Display-P4 board adaptation
2025-10-10 10:57:59 +08:00
Y1hsiaochunnn
6a28a98c8b Added Waveshare ESP32-P4-WIFI6-Touch-LCD-7B third party board (#1243)
* Modify the MIPILCDDISPLAY class

* Added Waveshare ESP32-P4-WIFI6-Touch-LCD-7B third party board

* Add config,json for Waveshare ESP32-P4 series boards

* Fix some errors
2025-10-09 12:00:22 +08:00
Xiaoxia
549399b795 Add esp32c5 configuration (#1258)
* Add esp32c5 configuration

* update idf_component.yml
2025-10-05 09:59:28 +08:00
masc2008
3ee2195340 Change the button array to ADC buttons as in the board for esp32s3-korv2 (#1256)
* Change the button array to ADC buttons as in the board for esp32s3-korv2

* Add MuteVol function to control audio volume
2025-10-03 11:01:00 +08:00
Xiaoxia
2fd0ff0b2a Add SetInputGain(float gain) to AudioCodec (#1252) 2025-10-02 09:55:45 +08:00
Xiaoxia
911fee2d85 Update README (#1251)
* Upgrade to the latest ml307 component

* update README
2025-10-02 03:29:16 +08:00
Toby
68b0e225a2 OTTO 左右腿反了 (#1239) 2025-09-27 05:20:37 +08:00
Xiaoxia
dd45d0de26 Bump to 2.0.3 (#1241)
* fix c6 compilation error

* remove afe preferred core

* fix release.py missing board type config

* Bump to 2.0.3

* remove duplicated depends
2025-09-26 08:52:43 +08:00
Xiaoxia
916ea39fad fix emote display errors (#1240) 2025-09-26 06:18:57 +08:00
zczc365
983d86a334 fix:小智云聊some bugfix (#1238) 2025-09-26 05:12:35 +08:00
laride
e7fc9ed489 ci: support multiple variants per board (#1036) 2025-09-26 05:09:54 +08:00
DeepStart
e329fcc6b8 添加太极派双声道配置 (#1235)
* Add Guition Boards

* Add Product Links

* 适配新版太极派

* 适配新版太极派

* 添加太极派双声道配置

* 添加太极派双声道配置

* 添加太极派双声道配置

* 太极派添加双声道输出配置
2025-09-26 05:04:12 +08:00
Xiaoxia
d3e7fee828 fix multiple wakenet words and custom wake word (#1226)
* fix multiple wakenet words and custom wake word

* fix idf_component.yml
2025-09-22 10:49:08 +08:00
Tomato Me
96e39bea1b 添加 Waveshare ESP32-S3-Touch-LCD-3.49 (#1227) 2025-09-22 10:46:33 +08:00
Y1hsiaochunnn
a8687f3736 New Waveshare ESP32-S3-Touch-LCD-4B third party board, 86 box form. (#1199)
Co-authored-by: Xiaoxia <terrence@tenclass.com>
2025-09-19 22:42:30 +08:00
espressif2022
8d58bdb21b feat: add emote style for v2 (#1217)
* feat: add emote style for v2

* feat: delete asset probe apply
2025-09-19 14:14:43 +08:00
ooxxU
4616fa3486 ESP32 Wifi And 4G Merge In All (#1219) 2025-09-19 00:33:33 +08:00
Xiaoxia
f11f84dfbd fix: Add function to handle local asset file paths 2025-09-17 08:36:39 +08:00
Xiaoxia
f418c16b2c Detect wake word model from index.json (#1211)
* detect wake word model from index.json

* update wait time before entering wifi configure mode
2025-09-17 08:31:51 +08:00
ZhouShaoYuan
a1e1f73886 fix: Corrected the inverted touch screen parameter configuration of lichuang_S3_dev, which caused touch offset. (#1209) 2025-09-17 01:18:22 +08:00
Xiaoxia
d2e99bae34 fix multinet model for v2 (#1208) 2025-09-16 19:05:28 +08:00
Xiaoxia
d188415949 fix: ESP-HI audio sampling problem (#1207) 2025-09-16 17:23:52 +08:00
Xiaoxia
5018f6c03a feat: build default assets instead of downloading and v2 tables for esp-hi, echoear (#1203)
* fix: call flush ready on io ready

* eachear: update to v2 partition table but disable class Assets currently

* stop gif if previewing an image

* feat: build default assets instead of downloading

* version updates

* fix None error

* delay 1s before enter wifi config mode

* fix compiling with v1 partition table

---------

Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
2025-09-16 15:24:24 +08:00
Xiaoxia
b413e3ec03 regenerate jpeg encoder (#1198)
* regenerate jpeg encoder

* add README to gif/jpeg

* 开机启动显示开发板信息,提前启动event loop
2025-09-16 05:00:02 +08:00
Terrence
fe7ae99a4d Bump to 2.0.1 2025-09-14 18:25:22 +08:00
Xiaoxia
147d71b9f1 feat: add snapshot mcp tool (#1196)
* use main task to execute tool calls

* feat: add snapshot mcp tool

* fix compiling errors

* 取消 audio input 的 pin core,core1留给显示,可能会对aec性能有影响

* update ml307 version

* remove v1 theme colors
2025-09-14 15:16:49 +08:00
zczc365
384da9fd0f fix:小智云聊compile error (#1195) 2025-09-14 13:51:19 +08:00
Xiaoxia
ae40f72a39 fix: crash if GIF version is 87a (#1194)
* fix: no tool call reply after self.reboot

* fix: 87a gif error

* append display info to board info
2025-09-13 16:53:10 +08:00
zczc365
d0ba3a923c 添加小智云聊-S3并修改8388支持AEC (#1179) 2025-09-13 14:38:16 +08:00
Terrence
76c19a0f2d fix: compiling errors 2025-09-12 15:55:55 +08:00
Terrence
f79506d58b feat: add manual upgrade firmware 2025-09-12 08:45:29 +08:00
Terrence
469ee2d92a rename withSystemTools to withUserTools 2025-09-11 04:22:33 +08:00
Xiaoxia
57c2c64047 feat: Add gif support (#1183)
* feat: Add gif support

* fix: compiling errors

* fix remove bg image
2025-09-11 03:53:12 +08:00
Xiaoxia
4048647ef8 feat: Add lvgl display theme control (#1180)
* feat: Add lvgl display theme control

* fix: compiling errors

* move light/dark themes to lcd display

* fix compile errors

---------

Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
2025-09-10 18:43:47 +08:00
Terrence
bce662d135 feat: default to flash non assets 2025-09-10 01:02:23 +08:00
Xiaoxia
d04b08133f Move fonts / assets definition from main/boards to CMakeLists.txt (#1174)
* update surfer-c3-1.14tft font size

* Move fonts / assets definition from main/boards to CMakeLists.txt

* fix c3 compiling errors

---------

Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
2025-09-08 17:30:18 +08:00
Terrence
897239e997 fix: c3 lcd stack protected error 2025-09-05 07:55:36 +08:00
Xiaoxia
83f6f8c703 Switch to 2.0 branch (#1152)
* Adapt boards to v2 partition tables

* fix esp log error

* fix display style

* reset emotion after download assets

* fix compiling

* update assets default url

* Add user only tools

* Add image cache

* smaller cache and buffer, more heap

* use MAIN_EVENT_CLOCK_TICK to avoid audio glitches

* bump to 2.0.0

* fix compiling errors

---------

Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
2025-09-04 15:41:28 +08:00
Johnson
3a3dfc003e Fix build error with esp_emote_gfx (#1159)
'bool' is defined in header '<stdbool.h>'; this is probably fixable by adding '#include <stdbool.h>'
   11 | #include "decoder/gfx_aaf_format.h"
2025-09-04 15:18:22 +08:00
Xiaoxia
fc355605f5 fix compiling errors (#1161) 2025-09-04 15:17:32 +08:00
Xiaoxia
5d3f597137 Bump to v1.9.0 (#1157)
* update v2 partition table readme

* feat: Add user only tool

* Add image cache

* smaller cache and buffer, more heap

* use MAIN_EVENT_CLOCK_TICK to avoid audio glitches

* fix: esp_psram_get_size not found in c3

* Bump to 1.9.0
2025-09-04 12:30:26 +08:00
ggc121238
3e37551923 add waveshare-s3-audio-board (#1139)
* add waveshare-s3-audio-board

* Modify the product link in the readme of the waveshare esp32s3-audio-board

* Modify reset time
2025-09-01 10:50:39 +08:00
Xiaoxia
d09537ed5c Add V2 parition tables (#1137)
* v1.9.0: update font icons, add mqtt reconnect

* Add v2 parition tables
2025-08-29 09:04:23 +08:00
Create123
86921f4862 Add M5Stack AtomEchoS3R Board. (#1123)
Signed-off-by: hlym123 <lwylwt@qq.com>
2025-08-26 14:05:10 +08:00
Terrence
7af366b7b2 fix: ES7120_SEL_MIC1 => ES7210_SEL_MIC1 2025-08-23 16:05:49 +08:00
Xiaoxia
ddbb24942d v1.8.9: Upgrade component versions (#1118) 2025-08-23 07:12:14 +08:00
Ben
610a4a0703 Update README.md (#1115)
delete '的'
2025-08-22 18:49:26 +08:00
香草味的纳西妲喵
7cd37427b2 feat: 添加批量转换OGG音频的相关脚本,移动声波配网HTML文件到scripts文件夹下 (#1107)
* feat: 添加批量转换OGG音频的相关脚本,移动声波配网HTML文件到scripts文件夹下

* Rename

* moved README.md
2025-08-22 00:53:18 +08:00
laride
2d772dad68 fix: resolve some audio issues on esp-hi (#1027)
* fix: resolve crash when closing codec dev on esp-hi

* fix: fix incorrect status display in non-zh-CN languages

* fix: reduce noise when not in Speaking state
2025-08-19 11:50:00 +08:00
Terrence
156eb15f58 fix: dual mic without afe 2025-08-16 03:08:00 +08:00
Xiaoxia
c59c515706 v1.8.8: release with esp-sr==2.1.4 and without font placeholder (#1086) 2025-08-15 04:50:33 +08:00
Terrence
44b8d5e4c1 fix: c3 wakeword not working with esp-sr 2.1.5 2025-08-15 01:07:00 +08:00
Xiaoxia
cc07ef447e Revert "camera 优化:在原有的RGB565处理下,容易超时改为JPEG格式 (#1029)" (#1085)
This reverts commit d6b1414967.
2025-08-14 22:23:29 +08:00
Ky1eYang
cf4afde88e add: 添加声音检测的可视化以及声波demod的准确度 (#1077)
Co-authored-by: yangkaiyue <yangkaiyue1@tenclass.com>
2025-08-14 22:11:56 +08:00
Dong Ning
d6b1414967 camera 优化:在原有的RGB565处理下,容易超时改为JPEG格式 (#1029)
* camera 优化

feat(camera): 修改摄像头配置为JPEG格式并优化图像处理逻辑

将摄像头输出格式从RGB565改为JPEG以提高传输效率,同时调整JPEG质量为10
重构预览图像处理逻辑,支持直接处理JPEG格式并自动转换为RGB565
优化Explain方法中的JPEG队列处理,减少内存分配和拷贝操作

* 修复代码缩进

调整代码缩进格式以提升可读性

* fix(esp32_camera): 修复RGB565格式预览图像的字节序问题

添加字节交换处理,将大端序转换为小端序,确保预览图像显示正确

* 使用旧的处理方式

* refactor(esp32_camera): 移除preview_buffer_直接使用preview_image_.data
2025-08-14 22:11:15 +08:00
Terrence
00dd89079b Bump to 1.8.7 2025-08-13 11:50:27 +08:00
vritser
cfb635d870 feat: 适配 ESP-S3-EV-Board-2 开发板 (#1043)
* feat: 适配 ESP-S3-EV-Board-2 开发板

* fix: config name

* fix: ev_board type choice

* fix: remove version config

---------

Co-authored-by: vritser <vritser@gmail.com>
2025-08-12 20:21:18 +08:00
Xiaoxia
9ae34d3b45 feat: Add locales with OGG sounds (#1057)
* fix jiuchuan build problem

* feat: Add locales with OGG sounds

* fix building echoear

* Support ogg files frame duration <= 60
2025-08-12 18:41:00 +08:00
Terrence
08b8b04c6c remove data mutex from read/write 2025-08-12 17:11:57 +08:00
Terrence
f890acfc7c fix i2s race condition error 2025-08-12 14:54:16 +08:00
Terrence
a4fe4d8d99 fix firmware image size 2025-08-12 14:53:17 +08:00
Xiaoxia
593b495139 xmin-c3 share MCP tool and sleep mode can be disabled (#1054) 2025-08-09 03:08:42 +08:00
Xiaoxia
2a02dd65be fix ReadAudioData frame duration (#1051) 2025-08-08 21:50:28 +08:00
virgil
845b760db3 fix: Delete the esp_jpeg_simd component to resolve function conflicts (#1049) 2025-08-08 21:00:34 +08:00
Xiaoxia
f86637cf1c fix: codec unexpectedly closed by timer after open (#1046)
Co-authored-by: Xiaoxia <terrence.huang@tenclass.com>
2025-08-08 15:39:12 +08:00
Xiaoxia
363073658a fix: OTA buffer overflow caused by slow SetChatMessage (#1031) 2025-08-06 01:16:52 +08:00
laride
da228f2582 fix: ci idf.py not found (#1030) 2025-08-05 22:18:28 +08:00
laride
36476f05cd feat: add build CI (#1028) 2025-08-05 20:03:37 +08:00
flying1425
90602d3802 waveshare-c6-lcd-1.69 新增电池电量显示功能及电池供电状态下的PWR按键开关机功能。 (#1020)
* 新增电池电量显示功能及电池供电状态下的PWR按键开关机功能。

* 代码风格优化

---------

Co-authored-by: flyingtjy <flyingtjy@gmail.com>
2025-08-04 22:38:13 +08:00
Zxczzzzzzz
b2e1c5bb5c 添加了一个声波配网的HTML界面 (#996)
* Add HTML page for sonic-based Wi-Fi provisioning

* Add HTML page for sonic-based Wi-Fi provisioning

* Add soundwave provisioning UI and fix bugs

* Add soundwave provisioning UI and fix bugs

* Move sonic_wifi_config.html to the docs folder

---------

Co-authored-by: luyuhan <xiaolumylove@icloud.com>
2025-08-02 14:13:35 +08:00
Terrence
d58d55cfe7 Bump to 1.8.5 2025-08-02 01:27:14 +08:00
espressif2022
cd23e0f155 feat: add emote_gfx UI for EchoEar (#1022)
* feat: add emote_gfx UI for EchoEar

* feat: delete local assets
2025-08-01 18:07:13 +08:00
Xiaoxia
26d9ff283f Fix custom wakeword for dual mic (#1018) 2025-08-01 13:30:17 +08:00
Xiaoxia
fb85019c3c change bread-compact-wifi-s3cam test gpio num (#1017) 2025-08-01 05:28:27 +08:00
HonestQiao
4859d57fea 修复esp32-p4配网客户端无法连接、连接获取不到ip或者无法打开配置页面的问题 (#1012) 2025-07-31 05:14:17 +08:00
Xiaoxia
03394fe38d update the sleep time of xmini-c3 with wake word to 300s (#1007) 2025-07-30 15:25:39 +08:00
jake12355
e0e12450c5 jiuchuan-s3修改按键定义取消不对话自动关机修复屏幕显示不全 (#997)
* jiuchuan-s3修改按键定义取消不对话自动关机修复屏幕显示不全

* jiuchuan-s3修改按键定义取消不对话自动关机修复屏幕显示不全
2025-07-29 17:28:40 +08:00
Xiaoxia
e5ac40aac8 fix audio pm (#1004) 2025-07-29 15:25:40 +08:00
Xiaoxia
345c8be467 Add custom wakeword threshold option (#1003) 2025-07-29 10:56:52 +08:00
香草味的纳西妲喵
def6301292 增加其他分辨率表情转换选项 (#987)
* Create requirements.txt

* Update README.md

* add: 增加其他尺寸表情选项
2025-07-26 02:31:09 +08:00
Xiaoxia
df7cbdfcb6 fix esp-hi crashing in esp_codec_dev_close() (#984) 2025-07-25 10:36:56 +08:00
Terrence
d38763d5ef Reduce SRAM usage of audio tasks 2025-07-25 08:27:12 +08:00
LILYGO_L
e90e540933 增强T-CameraPlus-S3麦克风接收音量 (#958)
* Adapt for LilyGO-T-Circle-S3 device

* Adapt for LilyGO-T-Circle-S3 device

* Remove comments and modify the size of the lilygo-t-circle-s3 image

* Modify the code style and format to Google C++

* Modify the code style and format to Google C++

* Fixed bugs in the LILYGO T-Circle-S3 board and added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Merge branch 'main' of https://github.com/Llgok/xiaozhi-esp32

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Added support for two new boards: LILYGO T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Fix the color display issue for T-Display-S3-Pro-MVSRLora and LILYGO T-Display-S3-Pro-MVSRLora_NO_BATTERY.

* Update T-CameraPlus-S3_V1.2 Version Xiaozhi Example

* Resolve the issue where the camera on the T-CameraPlus-S3_V1.2 board cannot be used normally.

* Enhance microphone reception volume

* fix the issue where voice wake-up is not working

* fix the issue where voice wake-up is not working
2025-07-23 23:02:45 +08:00
Ky1eYang
656bf3c7fa FIX: 修复双声道声波配网失效, 添加屏幕打印SSID/密码 (#971)
* debug: 添加声波配网的log打印点display

* fix: 修复双声道下声波配网失效的问题

* fix: codec可能为nullptr的问题(需要从单例board获取)

* Update afsk_demod.cc

fix coding style

---------

Co-authored-by: yangkaiyue <yangkaiyue1@tenclass.com>
Co-authored-by: Xiaoxia <terrence@tenclass.com>
2025-07-23 22:59:07 +08:00
Y1hsiaochunnn
ca35b0761b Update README.md (#968)
The README text description is incorrect. It needs to be corrected.
2025-07-23 22:47:19 +08:00
Terrence
b031a829c0 Bump to 1.8.2 2025-07-22 22:43:41 +08:00
Xiaoxia
3c11cceb43 增加自定义唤醒词启动失败的提升 (#965) 2025-07-22 18:57:25 +08:00
Y1hsiaochunnn
15f233e773 Add compatibility for Waveshare ESP32-S3-Touch-AMOLED-2.06 (#960)
* Add Waveshare ESP32-S3-Touch-AMOLED-2.06

* Update some configuration settings

* Add configuration to the configuration file

* Fix the abnormal areas
2025-07-22 18:02:31 +08:00
Forairaaaaa
721b58f8c7 Fix atoms3 backlight control (#959) 2025-07-22 17:55:33 +08:00
Xiaoxia
eb0bba2c89 Update LICENSE 2025-07-22 10:06:11 +08:00
Surfer
0d45c636a8 Surfer-C3-1.14tft 支持默认离线语音唤醒配置等 (#951)
* 添加 Surfer-C3-1.14TFT开发板支持

添加 Surfer-C3-1.14TFT开发板,基于C3的1.14寸LCD开发板,支持GPIO2  ADC 电池电量检测,应用于wifi Modem-Sleep 和低电量提醒。

* 修改surfer-c3-1.14tff电池电量适配值

修改surfer-c3-1.14tff电池电量适配值,电池容量400ma

* 修改支持默认离线语音唤醒配置等

1. 增加 config.json 文件默认支持离线语音唤醒配置和USB_SERIAL_JTAG
2. 微调电池电量值显示范围参数
2025-07-21 22:13:56 +08:00
Wiking_Xu
3e2bc9ee74 新增正点原子BOX2板子,有WIFI版本和4G版本 (#954) 2025-07-21 22:13:12 +08:00
Terrence
6a7a403117 修复C3唤醒提示音 2025-07-21 14:40:35 +08:00
Terrence
d1c047d060 Update to esp-ml307@3.1.1 2025-07-21 05:46:05 +08:00
Terrence
a35a344f42 Fix frame samples for server AEC 2025-07-20 07:53:52 +08:00
Terrence
efc6f238e7 Fix Server AEC 2025-07-20 03:57:36 +08:00
Xiaoxia
3c71558a5f v1.8.0: Audio 代码重构与低功耗优化 (#943)
* Reconstruct Audio Code

* Remove old IoT implementation

* Add MQTT-UDP documentation

* OTA升级失败时,可以继续使用
2025-07-19 22:45:22 +08:00
laride
0621578f55 fix: fix image player compile error (#933) 2025-07-18 14:08:04 +08:00
laride
5c8707075f feat: add state change events and callbacks (#798) 2025-07-18 01:35:31 +08:00
jake12355
c68c959e9b [优化]-jiuchuan-s3-优化电源管理架构 (#875)
* 添加 jiuchuang-s3 开发板支持

* 增加编译指南

* 优开发板支持包文件目录,更新README.md

删除了多余板载文件
[* ]README.md -> 更新了编译指南和烧录指南

* 修改板级支持包名

* 使用乐鑫提供的电源监测

* 修复部分代码格式问题

* 解决合并冲突

* 解决部分合并内容

* 完善合并内容

* 修复电量映射表错误

* 优化电源管理架构-使用rtc-io电源保持

---------

Co-authored-by: unknown <jake12355>
Co-authored-by: Hangon66 <2630612613@qq.com>
2025-07-18 01:29:30 +08:00
ZhouKe
1aea59a472 multinet wakeword (#927)
Co-authored-by: zk <982145@qq.com>
2025-07-18 01:18:40 +08:00
laride
be46cf1731 feat: upgrade component versions for esp-hi (#931) 2025-07-18 01:09:40 +08:00
Ky1eYang
0c971f76e3 feat: 添加自定义消息接收并打印在display或串口上 (#921)
* feat: 添加声波配网, 需调整application的ReadAudio公有, 需添加条件编译, 位于'afsk_demod.h'内定义参数

* mod: afsk的重构,旨在提高代码可读性并遵循Google C++代码风格指南

* mod: 更新依赖esp-wifi-connect需求版号

* feat: 添加声波配网, 需调整application的ReadAudio公有, 需添加条件编译, 位于'afsk_demod.h'内定义参数

* mod: afsk的重构,旨在提高代码可读性并遵循Google C++代码风格指南

* mod: 更新依赖esp-wifi-connect需求版号

* mod: 添加判断只有在WiFi配置模式下才会调用ReadAudio, 否则delay(联网成功重启后该任务不会被启动)

* add: 添加USE_ACOUSTIC_WIFI_PROVISIONING进MENU开关声波配网功能

* feat: 添加mqtt自定义消息接收, 修复menu的声波配网config

* 简化接收custom消息的处理, 只打印或处理payload字段

* fix: 修复payload在执行时指向被释放的线程不安全问题

---------

Co-authored-by: yangkaiyue <yangkaiyue1@tenclass.com>
2025-07-18 01:07:57 +08:00
Xiaoxia
6bb95073a4 v1.7.7: 支持 EC801E 模组,增加 xmini-c3-v3, xmini-c3-4g (#930) 2025-07-17 20:18:21 +08:00
diskman88
ea8769e1fc add labplus-Ledong-v2 and labplus-mpython-v3 board: (#897)
* add labplus-Ledong-v2 and labplus-mpython-v3 board:
	modified:   main/CMakeLists.txt
	modified:   main/Kconfig.projbuild
	new file:   main/boards/labplus-Ledong-v2/README.md
	new file:   main/boards/labplus-Ledong-v2/config.h
	new file:   main/boards/labplus-Ledong-v2/config.json
	new file:   main/boards/labplus-Ledong-v2/labplus_Ledong_v2.cc
	new file:   main/boards/labplus-mpython-v3/README.md
	new file:   main/boards/labplus-mpython-v3/config.h
	new file:   main/boards/labplus-mpython-v3/config.json
	new file:   main/boards/labplus-mpython-v3/mpython_pro.cc

* Changes to be committed:
	new file:   .clangd

* 修改板名:labplus-Ledong-v2 -> labplus-ledong-v2
	modified:   main/CMakeLists.txt
	modified:   main/Kconfig.projbuild
	renamed:    main/boards/labplus-Ledong-v2/README.md -> main/boards/labplus-ledong-v2/README.md
	renamed:    main/boards/labplus-Ledong-v2/config.h -> main/boards/labplus-ledong-v2/config.h
	renamed:    main/boards/labplus-Ledong-v2/config.json -> main/boards/labplus-ledong-v2/config.json
	renamed:    main/boards/labplus-Ledong-v2/labplus_Ledong_v2.cc -> main/boards/labplus-ledong-v2/labplus_ledong_v2.cc
2025-07-17 00:12:53 +08:00
wurongmin
0b0e1391f0 添加微雪电子 esp32-c6-touch-amoled-1.43 和 esp32-c6-touch-amoled-1.43-b的支持 (#907)
* 添加微雪电子 esp32-c6-touch-amoled-1.43 和 esp32-c6-touch-amoled-1.43-b的支持

* Fix: 根据PR评论优化代码格式
2025-07-15 00:27:30 +08:00
Surfer
0b364a83b3 添加 Surfer-C3-1.14TFT开发板支持 (#914)
* 添加 Surfer-C3-1.14TFT开发板支持

添加 Surfer-C3-1.14TFT开发板,基于C3的1.14寸LCD开发板,支持GPIO2  ADC 电池电量检测,应用于wifi Modem-Sleep 和低电量提醒。

* 修改surfer-c3-1.14tff电池电量适配值

修改surfer-c3-1.14tff电池电量适配值,电池容量400ma
2025-07-15 00:26:20 +08:00
DeepStart
5da837541b 增加新版太极小派适配 (#902)
* Add Guition Boards

* Add Product Links

* 适配新版太极派

* 适配新版太极派
2025-07-11 05:46:49 +08:00
Terrence
cec0b5b273 防止误刷xmini-c3固件导致永久性损害 2025-07-10 15:53:09 +08:00
virgil
8e0f76c0ef fix: fix wakenet model index (#893)
* fix: fix wakenet model index

* fix:Fix warning

* feat: Use jarvis and sophia as wakeup words
2025-07-09 03:19:08 +08:00
小林同志
f556e72604 规范函数名,修复一个重写错误 (#894)
Co-authored-by: smalllin0 <aslinqf@163.com>
2025-07-09 03:14:20 +08:00
ning yuxiang
2cdec25162 Add support for EchoEar V1.2 version PCB. (#891)
Co-authored-by: NingYuxiang <ningyuxiang@espressif.com>
2025-07-09 03:10:43 +08:00
flying1425
f002a1185b 添加对微雪电子 ESP32-S3-Touch-LCD-3.5B 开发板的支持 (#849)
* 添加对微雪电子 ESP32-S3-Touch-LCD-3.5B 开发板的支持

* 更改config.json的错误

* 修改板子的id以waveshare开头

* 更改config.json

* 修改config.json的 name

---------

Co-authored-by: flyingtjy <flyingtjy@gmail.com>
2025-07-05 18:09:23 +08:00
小林同志
1314ccfc0f 使用原始字符串字面量代替转义,提高可读性 (#861)
* 使用原始字符串字面量代替转义,提高可读性

* 使用原始字符串字面量代替转义,提高可读性

* 增加一个使用ESP-IDF Monitor作为输出显示内容的类

* 修改代码风格
2025-07-05 15:11:02 +08:00
laride
b7c1989a34 fix: resolve potential crash on touch isr (#885) 2025-07-05 14:56:11 +08:00
Ky1eYang
10016a3ea5 feat: 添加声波配网, 需调整application的ReadAudio公有, 需添加条件编译, 位于'afsk_demod.h'内定义参数 (#852)
* feat: 添加声波配网, 需调整application的ReadAudio公有, 需添加条件编译, 位于'afsk_demod.h'内定义参数

* mod: afsk的重构,旨在提高代码可读性并遵循Google C++代码风格指南

* mod: 更新依赖esp-wifi-connect需求版号

* feat: 添加声波配网, 需调整application的ReadAudio公有, 需添加条件编译, 位于'afsk_demod.h'内定义参数

* mod: afsk的重构,旨在提高代码可读性并遵循Google C++代码风格指南

* mod: 更新依赖esp-wifi-connect需求版号

* mod: 添加判断只有在WiFi配置模式下才会调用ReadAudio, 否则delay(联网成功重启后该任务不会被启动)

* add: 添加USE_ACOUSTIC_WIFI_PROVISIONING进MENU开关声波配网功能

---------

Co-authored-by: yangkaiyue <yangkaiyue1@tenclass.com>
2025-07-05 14:45:48 +08:00
0x1abin
f744c9bf3e 添加编译优化配置以减小生成的固件大小 (#864)
先合并进来看看大家的反馈。
2025-07-02 06:05:58 +08:00
laride
ad2c187433 添加 EchoEar 喵伴 (#869)
* add EchoEar

Co-authored-by: Qin Feng <qingfeng@espressif.com>

* feat: use new display backlight API

* feat: use MCP protocol

* remove yuying lcd

* add echoear README.md

---------

Co-authored-by: NingYuxiang <ningyuxiang@espressif.com>
Co-authored-by: Qin Feng <qingfeng@espressif.com>
2025-06-30 23:01:17 +08:00
virgil
8a0ec8b657 fix: fixed the issue of slow startup for devices without cameras (#866) 2025-06-30 22:41:42 +08:00
Terrence
d1c586239c fix CPU usage of esp32c3 with ml307 2025-06-29 06:01:41 +08:00
Hangon66
f2f54ba1fd 添加 jiuchuan-s3 开发板支持 (#775)
* 添加 jiuchuang-s3 开发板支持

* 增加编译指南

* 优开发板支持包文件目录,更新README.md

删除了多余板载文件
[* ]README.md -> 更新了编译指南和烧录指南

* 修改板级支持包名

* 使用乐鑫提供的电源监测

* 修复部分代码格式问题

* 解决合并冲突

* 解决部分合并内容

* 完善合并内容

* 修复电量映射表错误

---------

Co-authored-by: unknown <jake12355>
2025-06-29 05:41:42 +08:00
virgil
f1277934d1 feat: Support choosing different config.json (#848) 2025-06-26 02:55:45 +08:00
Terrence
c17bd15baa Bump to 1.7.6 2025-06-24 10:19:06 +08:00
Terrence
b3ab3d0920 在S3芯片上使用更多的PSRAM,解决立创开发板拍照时可能出现内存不足的问题 2025-06-24 04:59:00 +08:00
Terrence
2b0362a812 Fix: misconfigured target of atommatrix-echo-base 2025-06-19 14:52:53 +08:00
Terrence
d3367d6b92 Update esp-ml307 to 2.2.1 2025-06-18 05:24:51 +08:00
Terrence
116234a147 Bump to 1.7.5 2025-06-16 20:55:57 +08:00
Xiaoxia
f29c1a11d9 ml307: Add sleep mode (#826) 2025-06-16 19:05:31 +08:00
laride
f98ffdbb5c fix: optimize MCP commands for ESP-Hi (#825) 2025-06-16 16:03:21 +08:00
Xiaoxia
89f10365b1 xmini-c3: 休眠时关闭es8311可减少20mA电流 (#822)
* Correct class member name

* xmini-c3: 休眠时关闭es8311可减少20mA电流
2025-06-16 12:51:27 +08:00
ZhouKe
e9f23ea231 -增加带面包板的摄像头功能 (#815)
* -增加带面包板的摄像头功能

* rename board

---------

Co-authored-by: zk <982145@qq.com>
2025-06-13 21:03:13 +08:00
小鹏
7435c98609 1.增加robot舵机初始位置校准 2.fix(mcp_sever) 超出范围异常捕获类型 bug (#817)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug

* 1.增加robot舵机初始位置校准
2.fix(mcp_sever) 超出范围异常捕获类型  bug
2025-06-13 21:02:03 +08:00
Terrence
bf125446b3 feat: Use BOOT button to enter audio testing state when Wi-Fi configuring 2025-06-13 19:57:07 +08:00
flying1425
dfad6a5b2c 添加微雪电子esp32-c6-lcd-1.69、esp32-c6-Touch-lcd-1.69的支持 (#816)
* 添加了摄像头和触控支持

* 添加微雪电子esp32-c6-lcd-1.69、esp32-c6-Touch-lcd-1.69的支持

---------

Co-authored-by: flyingtjy <flyingtjy@gmail.com>
2025-06-13 18:20:22 +08:00
YeezB
d460af8426 feat: Add df-k10 MCP control on board RGB LED (#810)
* feat: Add dfk10 MCP blink

* fix: delete iot protocol related part
2025-06-13 18:08:08 +08:00
小鹏
7bb17f7539 fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug (#812)
* otto v1.4.0 MCP

1.使用MCP协议控制机器人
2.gif继承lcdDisplay,避免修改lcdDisplay

* otto v1.4.1 gif as components

gif as components

* electronBot v1.1.0 mcp

1.增加electronBot支持
2.mcp协议
3.gif 作为组件
4.display子类

* 规范代码

1.规范代码
2.修复切换主题死机bug

* fix(ota): 修复 ottoRobot和electronBot OTA 升级崩溃问题 bug
2025-06-13 13:46:14 +08:00
1258 changed files with 49995 additions and 15814 deletions

View File

@@ -1,32 +1,111 @@
name: Build and Test
name: Build Boards
on:
push:
branches:
- main
- ci/* # for ci test
pull_request:
branches:
- main
permissions:
contents: read
jobs:
build:
prepare:
name: Determine variants to build
runs-on: ubuntu-latest
outputs:
variants: ${{ steps.select.outputs.variants }}
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Espressif IoT Development Framework (ESP-IDF)
# You may pin to the exact commit or the version.
# uses: espressif/esp-idf-ci-action@8cd22ae10042fadc37890e81e9988a9113e7b506
uses: espressif/esp-idf-ci-action@v1.1.0
- name: Checkout
uses: actions/checkout@v4
with:
# Relative path under $GITHUB_WORKSPACE to place the repository
#path: # optional, default is
# Version of ESP-IDF docker image to use
esp_idf_version: release-v5.4
# ESP32 variant to build for
target: esp32s3
# Command to run inside the docker container (default: builds the project)
# command: # optional, default is idf.py build
fetch-depth: 0
- name: Install jq
run: sudo apt-get update && sudo apt-get install -y jq
- id: list
name: Get all variant list
run: |
echo "all_variants=$(python scripts/release.py --list-boards --json)" >> $GITHUB_OUTPUT
- id: select
name: Select variants based on changes
env:
ALL_VARIANTS: ${{ steps.list.outputs.all_variants }}
run: |
EVENT_NAME="${{ github.event_name }}"
# push 到 main 分支,编译全部变体
if [[ "$EVENT_NAME" == "push" ]]; then
echo "variants=$ALL_VARIANTS" >> $GITHUB_OUTPUT
exit 0
fi
# pull_request 场景
BASE_SHA="${{ github.event.pull_request.base.sha }}"
HEAD_SHA="${{ github.event.pull_request.head.sha }}"
echo "Base: $BASE_SHA, Head: $HEAD_SHA"
CHANGED=$(git diff --name-only $BASE_SHA $HEAD_SHA || true)
echo -e "Changed files:\n$CHANGED"
NEED_ALL=0
declare -A AFFECTED
while IFS= read -r file; do
if [[ "$file" == main/* && "$file" != main/boards/* ]]; then
NEED_ALL=1
fi
if [[ "$file" == main/boards/common/* ]]; then
NEED_ALL=1
fi
if [[ "$file" == main/boards/* ]]; then
board=$(echo "$file" | cut -d '/' -f3)
AFFECTED[$board]=1
fi
done <<< "$CHANGED"
if [[ "$NEED_ALL" -eq 1 ]]; then
echo "variants=$ALL_VARIANTS" >> $GITHUB_OUTPUT
else
if [[ ${#AFFECTED[@]} -eq 0 ]]; then
echo "variants=[]" >> $GITHUB_OUTPUT
else
BOARDS_JSON=$(printf '%s\n' "${!AFFECTED[@]}" | sort -u | jq -R -s -c 'split("\n")[:-1]')
FILTERED=$(echo "$ALL_VARIANTS" | jq -c --argjson boards "$BOARDS_JSON" 'map(select(.board as $b | $boards | index($b)))')
echo "variants=$FILTERED" >> $GITHUB_OUTPUT
fi
fi
build:
name: Build ${{ matrix.name }}
needs: prepare
if: ${{ needs.prepare.outputs.variants != '[]' }}
strategy:
fail-fast: false # 单个变体失败不影响其它变体
matrix:
include: ${{ fromJson(needs.prepare.outputs.variants) }}
runs-on: ubuntu-latest
container:
image: espressif/idf:release-v5.5
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Build current variant
shell: bash
run: |
source $IDF_PATH/export.sh
python scripts/release.py ${{ matrix.board }} --name ${{ matrix.name }}
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: xiaozhi_${{ matrix.name }}_${{ github.sha }}.bin
path: build/merged-binary.bin
if-no-files-found: error

5
.gitignore vendored
View File

@@ -13,4 +13,7 @@ main/assets/lang_config.h
main/mmap_generate_emoji.h
.DS_Store
.cache
main/mmap_generate_emoji.h
*.pyc
*.bin
mmap_generate_*.h
.clangd

View File

@@ -4,7 +4,7 @@
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.16)
set(PROJECT_VER "1.7.4")
set(PROJECT_VER "2.1.0")
# Add this line to disable the specific warning
add_compile_options(-Wno-missing-field-initializers)

View File

@@ -1,6 +1,7 @@
MIT License
Copyright (c) 2024 Xiaoxia
Copyright (c) 2025 Shenzhen Xinzhi Future Technology Co., Ltd.
Copyright (c) 2025 Project Contributors
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

166
README.md
View File

@@ -1,74 +1,73 @@
# An MCP-based Chatbot | 一个基于 MCP 的聊天机器人
# An MCP-based Chatbot
(中文 | [English](README_en.md) | [日本語](README_ja.md)
(English | [中文](README_zh.md) | [日本語](README_ja.md))
## 视频
## Introduction
👉 [人类:给 AI 装摄像头 vs AI当场发现主人三天没洗头【bilibili】](https://www.bilibili.com/video/BV1bpjgzKEhd/)
👉 [Human: Give AI a camera vs AI: Instantly finds out the owner hasn't washed hair for three days【bilibili】](https://www.bilibili.com/video/BV1bpjgzKEhd/)
👉 [手工打造你的 AI 女友,新手入门教程【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/)
👉 [Handcraft your AI girlfriend, beginner's guide【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/)
## 介绍
As a voice interaction entry, the XiaoZhi AI chatbot leverages the AI capabilities of large models like Qwen / DeepSeek, and achieves multi-terminal control via the MCP protocol.
这是一个由虾哥开源的 ESP32 项目,以 MIT 许可证发布,允许任何人免费使用,或用于商业用途。
<img src="docs/mcp-based-graph.jpg" alt="Control everything via MCP" width="320">
我们希望通过这个项目,能够帮助大家了解 AI 硬件开发,将当下飞速发展的大语言模型应用到实际的硬件设备中。
## Version Notes
如果你有任何想法或建议,请随时提出 Issues 或加入 QQ 群575180511
The current v2 version is incompatible with the v1 partition table, so it is not possible to upgrade from v1 to v2 via OTA. For partition table details, see [partitions/v2/README.md](partitions/v2/README.md).
### 基于 MCP 控制万物
All hardware running v1 can be upgraded to v2 by manually flashing the firmware.
小智 AI 聊天机器人作为一个语音交互入口,利用 Qwen / DeepSeek 等大模型的 AI 能力,通过 MCP 协议实现多端控制。
The stable version of v1 is 1.9.2. You can switch to v1 by running `git checkout v1`. The v1 branch will be maintained until February 2026.
![通过MCP控制万物](docs/mcp-based-graph.jpg)
### 已实现功能
### Features Implemented
- Wi-Fi / ML307 Cat.1 4G
- 离线语音唤醒 [ESP-SR](https://github.com/espressif/esp-sr)
- 支持两种通信协议([Websocket](docs/websocket.md) MQTT+UDP
- 采用 OPUS 音频编解码
- 基于流式 ASR + LLM + TTS 架构的语音交互
- 声纹识别,识别当前说话人的身份 [3D Speaker](https://github.com/modelscope/3D-Speaker)
- OLED / LCD 显示屏,支持表情显示
- 电量显示与电源管理
- 支持多语言(中文、英文、日文)
- 支持 ESP32-C3ESP32-S3ESP32-P4 芯片平台
- 通过设备端 MCP 实现设备控制音量、灯光、电机、GPIO 等)
- 通过云端 MCP 扩展大模型能力智能家居控制、PC桌面操作、知识搜索、邮件收发等
- Offline voice wake-up [ESP-SR](https://github.com/espressif/esp-sr)
- Supports two communication protocols ([Websocket](docs/websocket.md) or MQTT+UDP)
- Uses OPUS audio codec
- Voice interaction based on streaming ASR + LLM + TTS architecture
- Speaker recognition, identifies the current speaker [3D Speaker](https://github.com/modelscope/3D-Speaker)
- OLED / LCD display, supports emoji display
- Battery display and power management
- Multi-language support (Chinese, English, Japanese)
- Supports ESP32-C3, ESP32-S3, ESP32-P4 chip platforms
- Device-side MCP for device control (Speaker, LED, Servo, GPIO, etc.)
- Cloud-side MCP to extend large model capabilities (smart home control, PC desktop operation, knowledge search, email, etc.)
- Customizable wake words, fonts, emojis, and chat backgrounds with online web-based editing ([Custom Assets Generator](https://github.com/78/xiaozhi-assets-generator))
## 硬件
## Hardware
### 面包板手工制作实践
### Breadboard DIY Practice
详见飞书文档教程:
See the Feishu document tutorial:
👉 [《小智 AI 聊天机器人百科全书》](https://ccnphfhqs21z.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb?from=from_copylink)
👉 ["XiaoZhi AI Chatbot Encyclopedia"](https://ccnphfhqs21z.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb?from=from_copylink)
面包板效果图如下:
Breadboard demo:
![面包板效果图](docs/v1/wiring2.jpg)
![Breadboard Demo](docs/v1/wiring2.jpg)
### 支持 70 多个开源硬件(仅展示部分)
### Supports 70+ Open Source Hardware (Partial List)
- <a href="https://oshwhub.com/li-chuang-kai-fa-ban/li-chuang-shi-zhan-pai-esp32-s3-kai-fa-ban" target="_blank" title="立创·实战派 ESP32-S3 开发板">立创·实战派 ESP32-S3 开发板</a>
- <a href="https://github.com/espressif/esp-box" target="_blank" title="乐鑫 ESP32-S3-BOX3">乐鑫 ESP32-S3-BOX3</a>
- <a href="https://oshwhub.com/li-chuang-kai-fa-ban/li-chuang-shi-zhan-pai-esp32-s3-kai-fa-ban" target="_blank" title="LiChuang ESP32-S3 Development Board">LiChuang ESP32-S3 Development Board</a>
- <a href="https://github.com/espressif/esp-box" target="_blank" title="Espressif ESP32-S3-BOX3">Espressif ESP32-S3-BOX3</a>
- <a href="https://docs.m5stack.com/zh_CN/core/CoreS3" target="_blank" title="M5Stack CoreS3">M5Stack CoreS3</a>
- <a href="https://docs.m5stack.com/en/atom/Atomic%20Echo%20Base" target="_blank" title="AtomS3R + Echo Base">M5Stack AtomS3R + Echo Base</a>
- <a href="https://gf.bilibili.com/item/detail/1108782064" target="_blank" title="神奇按钮 2.4">神奇按钮 2.4</a>
- <a href="https://www.waveshare.net/shop/ESP32-S3-Touch-AMOLED-1.8.htm" target="_blank" title="微雪电子 ESP32-S3-Touch-AMOLED-1.8">微雪电子 ESP32-S3-Touch-AMOLED-1.8</a>
- <a href="https://gf.bilibili.com/item/detail/1108782064" target="_blank" title="Magic Button 2.4">Magic Button 2.4</a>
- <a href="https://www.waveshare.net/shop/ESP32-S3-Touch-AMOLED-1.8.htm" target="_blank" title="Waveshare ESP32-S3-Touch-AMOLED-1.8">Waveshare ESP32-S3-Touch-AMOLED-1.8</a>
- <a href="https://github.com/Xinyuan-LilyGO/T-Circle-S3" target="_blank" title="LILYGO T-Circle-S3">LILYGO T-Circle-S3</a>
- <a href="https://oshwhub.com/tenclass01/xmini_c3" target="_blank" title="虾哥 Mini C3">虾哥 Mini C3</a>
- <a href="https://oshwhub.com/movecall/cuican-ai-pendant-lights-up-y" target="_blank" title="Movecall CuiCan ESP32S3">璀璨·AI 吊坠</a>
- <a href="https://github.com/WMnologo/xingzhi-ai" target="_blank" title="无名科技Nologo-星智-1.54">无名科技 Nologo-星智-1.54TFT</a>
- <a href="https://oshwhub.com/tenclass01/xmini_c3" target="_blank" title="XiaGe Mini C3">XiaGe Mini C3</a>
- <a href="https://oshwhub.com/movecall/cuican-ai-pendant-lights-up-y" target="_blank" title="Movecall CuiCan ESP32S3">CuiCan AI Pendant</a>
- <a href="https://github.com/WMnologo/xingzhi-ai" target="_blank" title="WMnologo-Xingzhi-1.54">WMnologo-Xingzhi-1.54TFT</a>
- <a href="https://www.seeedstudio.com/SenseCAP-Watcher-W1-A-p-5979.html" target="_blank" title="SenseCAP Watcher">SenseCAP Watcher</a>
- <a href="https://www.bilibili.com/video/BV1BHJtz6E2S/" target="_blank" title="ESP-HI 超低成本机器狗">ESP-HI 超低成本机器狗</a>
- <a href="https://www.bilibili.com/video/BV1BHJtz6E2S/" target="_blank" title="ESP-HI Low Cost Robot Dog">ESP-HI Low Cost Robot Dog</a>
<div style="display: flex; justify-content: space-between;">
<a href="docs/v1/lichuang-s3.jpg" target="_blank" title="立创·实战派 ESP32-S3 开发板">
<a href="docs/v1/lichuang-s3.jpg" target="_blank" title="LiChuang ESP32-S3 Development Board">
<img src="docs/v1/lichuang-s3.jpg" width="240" />
</a>
<a href="docs/v1/espbox3.jpg" target="_blank" title="乐鑫 ESP32-S3-BOX3">
<a href="docs/v1/espbox3.jpg" target="_blank" title="Espressif ESP32-S3-BOX3">
<img src="docs/v1/espbox3.jpg" width="240" />
</a>
<a href="docs/v1/m5cores3.jpg" target="_blank" title="M5Stack CoreS3">
@@ -77,77 +76,90 @@
<a href="docs/v1/atoms3r.jpg" target="_blank" title="AtomS3R + Echo Base">
<img src="docs/v1/atoms3r.jpg" width="240" />
</a>
<a href="docs/v1/magiclick.jpg" target="_blank" title="神奇按钮 2.4">
<a href="docs/v1/magiclick.jpg" target="_blank" title="Magic Button 2.4">
<img src="docs/v1/magiclick.jpg" width="240" />
</a>
<a href="docs/v1/waveshare.jpg" target="_blank" title="微雪电子 ESP32-S3-Touch-AMOLED-1.8">
<a href="docs/v1/waveshare.jpg" target="_blank" title="Waveshare ESP32-S3-Touch-AMOLED-1.8">
<img src="docs/v1/waveshare.jpg" width="240" />
</a>
<a href="docs/v1/lilygo-t-circle-s3.jpg" target="_blank" title="LILYGO T-Circle-S3">
<img src="docs/v1/lilygo-t-circle-s3.jpg" width="240" />
</a>
<a href="docs/v1/xmini-c3.jpg" target="_blank" title="虾哥 Mini C3">
<a href="docs/v1/xmini-c3.jpg" target="_blank" title="XiaGe Mini C3">
<img src="docs/v1/xmini-c3.jpg" width="240" />
</a>
<a href="docs/v1/movecall-cuican-esp32s3.jpg" target="_blank" title="CuiCan">
<img src="docs/v1/movecall-cuican-esp32s3.jpg" width="240" />
</a>
<a href="docs/v1/wmnologo_xingzhi_1.54.jpg" target="_blank" title="无名科技Nologo-星智-1.54">
<a href="docs/v1/wmnologo_xingzhi_1.54.jpg" target="_blank" title="WMnologo-Xingzhi-1.54">
<img src="docs/v1/wmnologo_xingzhi_1.54.jpg" width="240" />
</a>
<a href="docs/v1/sensecap_watcher.jpg" target="_blank" title="SenseCAP Watcher">
<img src="docs/v1/sensecap_watcher.jpg" width="240" />
</a>
<a href="docs/v1/esp-hi.jpg" target="_blank" title="ESP-HI 超低成本机器狗">
<a href="docs/v1/esp-hi.jpg" target="_blank" title="ESP-HI Low Cost Robot Dog">
<img src="docs/v1/esp-hi.jpg" width="240" />
</a>
</div>
## 软件
## Software
### 固件烧录
### Firmware Flashing
新手第一次操作建议先不要搭建开发环境,直接使用免开发环境烧录的固件。
For beginners, it is recommended to use the firmware that can be flashed without setting up a development environment.
固件默认接入 [xiaozhi.me](https://xiaozhi.me) 官方服务器,个人用户注册账号可以免费使用 Qwen 实时模型。
The firmware connects to the official [xiaozhi.me](https://xiaozhi.me) server by default. Personal users can register an account to use the Qwen real-time model for free.
👉 [新手烧录固件教程](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS)
👉 [Beginner's Firmware Flashing Guide](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS)
### 开发环境
### Development Environment
- Cursor VSCode
- 安装 ESP-IDF 插件,选择 SDK 版本 5.4 或以上
- Linux 比 Windows 更好,编译速度快,也免去驱动问题的困扰
- 本项目使用 Google C++ 代码风格,提交代码时请确保符合规范
- Cursor or VSCode
- Install ESP-IDF plugin, select SDK version 5.4 or above
- Linux is better than Windows for faster compilation and fewer driver issues
- This project uses Google C++ code style, please ensure compliance when submitting code
### 开发者文档
### Developer Documentation
- [自定义开发板指南](main/boards/README.md) - 学习如何为小智 AI 创建自定义开发板
- [MCP 协议物联网控制用法说明](docs/mcp-usage.md) - 了解如何通过 MCP 协议控制物联网设备
- [MCP 协议交互流程](docs/mcp-protocol.md) - 设备端 MCP 协议的实现方式
- [一份详细的 WebSocket 通信协议文档](docs/websocket.md)
- [Custom Board Guide](docs/custom-board.md) - Learn how to create custom boards for XiaoZhi AI
- [MCP Protocol IoT Control Usage](docs/mcp-usage.md) - Learn how to control IoT devices via MCP protocol
- [MCP Protocol Interaction Flow](docs/mcp-protocol.md) - Device-side MCP protocol implementation
- [MQTT + UDP Hybrid Communication Protocol Document](docs/mqtt-udp.md)
- [A detailed WebSocket communication protocol document](docs/websocket.md)
## 大模型配置
## Large Model Configuration
如果你已经拥有一个的小智 AI 聊天机器人设备,并且已接入官方服务器,可以登录 [xiaozhi.me](https://xiaozhi.me) 控制台进行配置。
If you already have a XiaoZhi AI chatbot device and have connected to the official server, you can log in to the [xiaozhi.me](https://xiaozhi.me) console for configuration.
👉 [后台操作视频教程(旧版界面)](https://www.bilibili.com/video/BV1jUCUY2EKM/)
👉 [Backend Operation Video Tutorial (Old Interface)](https://www.bilibili.com/video/BV1jUCUY2EKM/)
## 相关开源项目
## Related Open Source Projects
在个人电脑上部署服务器,可以参考以下第三方开源的项目:
For server deployment on personal computers, refer to the following open-source projects:
- [xinnan-tech/xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) Python 服务器
- [joey-zhou/xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) Java 服务器
- [AnimeAIChat/xiaozhi-server-go](https://github.com/AnimeAIChat/xiaozhi-server-go) Golang 服务器
- [xinnan-tech/xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) Python server
- [joey-zhou/xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) Java server
- [AnimeAIChat/xiaozhi-server-go](https://github.com/AnimeAIChat/xiaozhi-server-go) Golang server
使用小智通信协议的第三方客户端项目:
Other client projects using the XiaoZhi communication protocol:
- [huangjunsen0406/py-xiaozhi](https://github.com/huangjunsen0406/py-xiaozhi) Python 客户端
- [TOM88812/xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) Android 客户端
- [100askTeam/xiaozhi-linux](http://github.com/100askTeam/xiaozhi-linux) 百问科技提供的 Linux 客户端
- [78/xiaozhi-sf32](https://github.com/78/xiaozhi-sf32) 思澈科技的蓝牙芯片固件
- [QuecPython/solution-xiaozhiAI](https://github.com/QuecPython/solution-xiaozhiAI) 移远提供的 QuecPython 固件
- [huangjunsen0406/py-xiaozhi](https://github.com/huangjunsen0406/py-xiaozhi) Python client
- [TOM88812/xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) Android client
- [100askTeam/xiaozhi-linux](http://github.com/100askTeam/xiaozhi-linux) Linux client by 100ask
- [78/xiaozhi-sf32](https://github.com/78/xiaozhi-sf32) Bluetooth chip firmware by Sichuan
- [QuecPython/solution-xiaozhiAI](https://github.com/QuecPython/solution-xiaozhiAI) QuecPython firmware by Quectel
Custom Assets Tools:
- [78/xiaozhi-assets-generator](https://github.com/78/xiaozhi-assets-generator) Custom Assets Generator (Wake words, fonts, emojis, backgrounds)
## About the Project
This is an open-source ESP32 project, released under the MIT license, allowing anyone to use it for free, including for commercial purposes.
We hope this project helps everyone understand AI hardware development and apply rapidly evolving large language models to real hardware devices.
If you have any ideas or suggestions, please feel free to raise Issues or join the QQ group: 1011329060
## Star History
@@ -157,4 +169,4 @@
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=78/xiaozhi-esp32&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=78/xiaozhi-esp32&type=Date" />
</picture>
</a>
</a>

View File

@@ -1,157 +0,0 @@
# An MCP-based Chatbot
(English | [中文](README.md) | [日本語](README_ja.md))
## Video
👉 [Human: Give AI a camera vs AI: Instantly finds out the owner hasn't washed hair for three days【bilibili】](https://www.bilibili.com/video/BV1bpjgzKEhd/)
👉 [Handcraft your AI girlfriend, beginner's guide【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/)
## Introduction
This is an open-source ESP32 project, released under the MIT license, allowing anyone to use it for free, including for commercial purposes.
We hope this project helps everyone understand AI hardware development and apply rapidly evolving large language models to real hardware devices.
If you have any ideas or suggestions, please feel free to raise Issues or join the QQ group: 575180511
### Control Everything with MCP
As a voice interaction entry, the XiaoZhi AI chatbot leverages the AI capabilities of large models like Qwen / DeepSeek, and achieves multi-terminal control via the MCP protocol.
![Control everything via MCP](docs/mcp-based-graph.jpg)
### Features Implemented
- Wi-Fi / ML307 Cat.1 4G
- Offline voice wake-up [ESP-SR](https://github.com/espressif/esp-sr)
- Supports two communication protocols ([Websocket](docs/websocket.md) or MQTT+UDP)
- Uses OPUS audio codec
- Voice interaction based on streaming ASR + LLM + TTS architecture
- Speaker recognition, identifies the current speaker [3D Speaker](https://github.com/modelscope/3D-Speaker)
- OLED / LCD display, supports emoji display
- Battery display and power management
- Multi-language support (Chinese, English, Japanese)
- Supports ESP32-C3, ESP32-S3, ESP32-P4 chip platforms
- Device-side MCP for device control (Speaker, LED, Servo, GPIO, etc.)
- Cloud-side MCP to extend large model capabilities (smart home control, PC desktop operation, knowledge search, email, etc.)
## Hardware
### Breadboard DIY Practice
See the Feishu document tutorial:
👉 ["XiaoZhi AI Chatbot Encyclopedia"](https://ccnphfhqs21z.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb?from=from_copylink)
Breadboard demo:
![Breadboard Demo](docs/v1/wiring2.jpg)
### Supports 70+ Open Source Hardware (Partial List)
- <a href="https://oshwhub.com/li-chuang-kai-fa-ban/li-chuang-shi-zhan-pai-esp32-s3-kai-fa-ban" target="_blank" title="LiChuang ESP32-S3 Development Board">LiChuang ESP32-S3 Development Board</a>
- <a href="https://github.com/espressif/esp-box" target="_blank" title="Espressif ESP32-S3-BOX3">Espressif ESP32-S3-BOX3</a>
- <a href="https://docs.m5stack.com/zh_CN/core/CoreS3" target="_blank" title="M5Stack CoreS3">M5Stack CoreS3</a>
- <a href="https://docs.m5stack.com/en/atom/Atomic%20Echo%20Base" target="_blank" title="AtomS3R + Echo Base">M5Stack AtomS3R + Echo Base</a>
- <a href="https://gf.bilibili.com/item/detail/1108782064" target="_blank" title="Magic Button 2.4">Magic Button 2.4</a>
- <a href="https://www.waveshare.net/shop/ESP32-S3-Touch-AMOLED-1.8.htm" target="_blank" title="Waveshare ESP32-S3-Touch-AMOLED-1.8">Waveshare ESP32-S3-Touch-AMOLED-1.8</a>
- <a href="https://github.com/Xinyuan-LilyGO/T-Circle-S3" target="_blank" title="LILYGO T-Circle-S3">LILYGO T-Circle-S3</a>
- <a href="https://oshwhub.com/tenclass01/xmini_c3" target="_blank" title="XiaGe Mini C3">XiaGe Mini C3</a>
- <a href="https://oshwhub.com/movecall/cuican-ai-pendant-lights-up-y" target="_blank" title="Movecall CuiCan ESP32S3">CuiCan AI Pendant</a>
- <a href="https://github.com/WMnologo/xingzhi-ai" target="_blank" title="WMnologo-Xingzhi-1.54">WMnologo-Xingzhi-1.54TFT</a>
- <a href="https://www.seeedstudio.com/SenseCAP-Watcher-W1-A-p-5979.html" target="_blank" title="SenseCAP Watcher">SenseCAP Watcher</a>
- <a href="https://www.bilibili.com/video/BV1BHJtz6E2S/" target="_blank" title="ESP-HI Low Cost Robot Dog">ESP-HI Low Cost Robot Dog</a>
<div style="display: flex; justify-content: space-between;">
<a href="docs/v1/lichuang-s3.jpg" target="_blank" title="LiChuang ESP32-S3 Development Board">
<img src="docs/v1/lichuang-s3.jpg" width="240" />
</a>
<a href="docs/v1/espbox3.jpg" target="_blank" title="Espressif ESP32-S3-BOX3">
<img src="docs/v1/espbox3.jpg" width="240" />
</a>
<a href="docs/v1/m5cores3.jpg" target="_blank" title="M5Stack CoreS3">
<img src="docs/v1/m5cores3.jpg" width="240" />
</a>
<a href="docs/v1/atoms3r.jpg" target="_blank" title="AtomS3R + Echo Base">
<img src="docs/v1/atoms3r.jpg" width="240" />
</a>
<a href="docs/v1/magiclick.jpg" target="_blank" title="Magic Button 2.4">
<img src="docs/v1/magiclick.jpg" width="240" />
</a>
<a href="docs/v1/waveshare.jpg" target="_blank" title="Waveshare ESP32-S3-Touch-AMOLED-1.8">
<img src="docs/v1/waveshare.jpg" width="240" />
</a>
<a href="docs/v1/lilygo-t-circle-s3.jpg" target="_blank" title="LILYGO T-Circle-S3">
<img src="docs/v1/lilygo-t-circle-s3.jpg" width="240" />
</a>
<a href="docs/v1/xmini-c3.jpg" target="_blank" title="XiaGe Mini C3">
<img src="docs/v1/xmini-c3.jpg" width="240" />
</a>
<a href="docs/v1/movecall-cuican-esp32s3.jpg" target="_blank" title="CuiCan">
<img src="docs/v1/movecall-cuican-esp32s3.jpg" width="240" />
</a>
<a href="docs/v1/wmnologo_xingzhi_1.54.jpg" target="_blank" title="WMnologo-Xingzhi-1.54">
<img src="docs/v1/wmnologo_xingzhi_1.54.jpg" width="240" />
</a>
<a href="docs/v1/sensecap_watcher.jpg" target="_blank" title="SenseCAP Watcher">
<img src="docs/v1/sensecap_watcher.jpg" width="240" />
</a>
<a href="docs/v1/esp-hi.jpg" target="_blank" title="ESP-HI Low Cost Robot Dog">
<img src="docs/v1/esp-hi.jpg" width="240" />
</a>
</div>
## Software
### Firmware Flashing
For beginners, it is recommended to use the firmware that can be flashed without setting up a development environment.
The firmware connects to the official [xiaozhi.me](https://xiaozhi.me) server by default. Personal users can register an account to use the Qwen real-time model for free.
👉 [Beginner's Firmware Flashing Guide](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS)
### Development Environment
- Cursor or VSCode
- Install ESP-IDF plugin, select SDK version 5.4 or above
- Linux is better than Windows for faster compilation and fewer driver issues
- This project uses Google C++ code style, please ensure compliance when submitting code
### Developer Documentation
- [Custom Board Guide](main/boards/README.md) - Learn how to create custom boards for XiaoZhi AI
- [MCP Protocol IoT Control Usage](docs/mcp-usage.md) - Learn how to control IoT devices via MCP protocol
- [MCP Protocol Interaction Flow](docs/mcp-protocol.md) - Device-side MCP protocol implementation
- [A detailed WebSocket communication protocol document](docs/websocket.md)
## Large Model Configuration
If you already have a XiaoZhi AI chatbot device and have connected to the official server, you can log in to the [xiaozhi.me](https://xiaozhi.me) console for configuration.
👉 [Backend Operation Video Tutorial (Old Interface)](https://www.bilibili.com/video/BV1jUCUY2EKM/)
## Related Open Source Projects
For server deployment on personal computers, refer to the following open-source projects:
- [xinnan-tech/xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) Python server
- [joey-zhou/xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) Java server
- [AnimeAIChat/xiaozhi-server-go](https://github.com/AnimeAIChat/xiaozhi-server-go) Golang server
Other client projects using the XiaoZhi communication protocol:
- [huangjunsen0406/py-xiaozhi](https://github.com/huangjunsen0406/py-xiaozhi) Python client
- [TOM88812/xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) Android client
## Star History
<a href="https://star-history.com/#78/xiaozhi-esp32&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=78/xiaozhi-esp32&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=78/xiaozhi-esp32&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=78/xiaozhi-esp32&type=Date" />
</picture>
</a>

View File

@@ -1,26 +1,24 @@
# MCP ベースのチャットボット
(日本語 | [中文](README.md) | [English](README_en.md)
(日本語 | [中文](README_zh.md) | [English](README.md)
## 動画
## はじめに
👉 [人間AIにカメラを装着 vs AIその場で飼い主が3日間髪を洗っていないことを発見【bilibili】](https://www.bilibili.com/video/BV1bpjgzKEhd/)
👉 [手作りでAIガールフレンドを作る、初心者入門チュートリアル【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/)
## イントロダクション
これはエビ兄さんがオープンソースで公開しているESP32プロジェクトで、MITライセンスのもと、誰でも無料で、商用利用も可能です。
このプロジェクトを通じて、AIハードウェア開発を理解し、急速に進化する大規模言語モデルを実際のハードウェアデバイスに応用できるようになることを目指しています。
ご意見やご提案があれば、いつでもIssueを提出するか、QQグループ575180511 にご参加ください。
### MCPであらゆるものを制御
シャオジーAIチャットボットは音声インタラクションの入口として、Qwen / DeepSeekなどの大規模モデルのAI能力を活用し、MCPプロトコルを通じてマルチエンド制御を実現します。
![MCPであらゆるものを制御](docs/mcp-based-graph.jpg)
<img src="docs/mcp-based-graph.jpg" alt="MCPであらゆるものを制御" width="320">
## バージョンノート
現在のv2バージョンはv1パーティションテーブルと互換性がないため、v1からv2へOTAでアップグレードすることはできません。パーティションテーブルの詳細については、[partitions/v2/README.md](partitions/v2/README.md)をご参照ください。
v1を実行しているすべてのハードウェアは、ファームウェアを手動で書き込むことでv2にアップグレードできます。
v1の安定版は1.9.2です。`git checkout v1`でv1に切り替えることができます。v1ブランチは2026年2月まで継続的にメンテナンスされます。
### 実装済み機能
@@ -36,6 +34,7 @@
- ESP32-C3、ESP32-S3、ESP32-P4チッププラットフォーム対応
- デバイス側MCPによるデバイス制御音量・明るさ調整、アクション制御など
- クラウド側MCPで大規模モデル能力を拡張スマートホーム制御、PCデスクトップ操作、知識検索、メール送受信など
- カスタマイズ可能なウェイクワード、フォント、絵文字、チャット背景、オンラインWeb編集に対応 ([カスタムアセットジェネレーター](https://github.com/78/xiaozhi-assets-generator))
## ハードウェア
@@ -122,9 +121,10 @@ Feishuドキュメントチュートリアルをご覧ください
### 開発者ドキュメント
- [カスタム開発ボードガイド](main/boards/README.md) - シャオジーAI用のカスタム開発ボード作成方法
- [カスタム開発ボードガイド](docs/custom-board.md) - シャオジーAI用のカスタム開発ボード作成方法
- [MCPプロトコルIoT制御使用法](docs/mcp-usage.md) - MCPプロトコルでIoTデバイスを制御する方法
- [MCPプロトコルインタラクションフロー](docs/mcp-protocol.md) - デバイス側MCPプロトコルの実装方法
- [MQTT + UDP ハイブリッド通信プロトコルドキュメント](docs/mqtt-udp.md)
- [詳細なWebSocket通信プロトコルドキュメント](docs/websocket.md)
## 大規模モデル設定
@@ -145,6 +145,17 @@ Feishuドキュメントチュートリアルをご覧ください
- [huangjunsen0406/py-xiaozhi](https://github.com/huangjunsen0406/py-xiaozhi) Pythonクライアント
- [TOM88812/xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) Androidクライアント
- [100askTeam/xiaozhi-linux](http://github.com/100askTeam/xiaozhi-linux) 百問科技提供のLinuxクライアント
- [78/xiaozhi-sf32](https://github.com/78/xiaozhi-sf32) 思澈科技のBluetoothチップファームウェア
- [QuecPython/solution-xiaozhiAI](https://github.com/QuecPython/solution-xiaozhiAI) 移遠提供のQuecPythonファームウェア
## プロジェクトについて
これはエビ兄さんがオープンソースで公開しているESP32プロジェクトで、MITライセンスのもと、誰でも無料で、商用利用も可能です。
このプロジェクトを通じて、AIハードウェア開発を理解し、急速に進化する大規模言語モデルを実際のハードウェアデバイスに応用できるようになることを目指しています。
ご意見やご提案があれば、いつでもIssueを提出するか、QQグループ1011329060 にご参加ください。
## スター履歴

168
README_zh.md Normal file
View File

@@ -0,0 +1,168 @@
# An MCP-based Chatbot
(中文 | [English](README.md) | [日本語](README_ja.md)
## 介绍
👉 [人类:给 AI 装摄像头 vs AI当场发现主人三天没洗头【bilibili】](https://www.bilibili.com/video/BV1bpjgzKEhd/)
👉 [手工打造你的 AI 女友新手入门教程【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/)
小智 AI 聊天机器人作为一个语音交互入口,利用 Qwen / DeepSeek 等大模型的 AI 能力,通过 MCP 协议实现多端控制。
<img src="docs/mcp-based-graph.jpg" alt="通过MCP控制万物" width="320">
### 版本说明
当前 v2 版本与 v1 版本分区表不兼容,所以无法从 v1 版本通过 OTA 升级到 v2 版本。分区表说明参见 [partitions/v2/README.md](partitions/v2/README.md)。
使用 v1 版本的所有硬件,可以通过手动烧录固件来升级到 v2 版本。
v1 的稳定版本为 1.9.2,可以通过 `git checkout v1` 来切换到 v1 版本,该分支会持续维护到 2026 年 2 月。
### 已实现功能
- Wi-Fi / ML307 Cat.1 4G
- 离线语音唤醒 [ESP-SR](https://github.com/espressif/esp-sr)
- 支持两种通信协议([Websocket](docs/websocket.md) 或 MQTT+UDP
- 采用 OPUS 音频编解码
- 基于流式 ASR + LLM + TTS 架构的语音交互
- 声纹识别,识别当前说话人的身份 [3D Speaker](https://github.com/modelscope/3D-Speaker)
- OLED / LCD 显示屏,支持表情显示
- 电量显示与电源管理
- 支持多语言(中文、英文、日文)
- 支持 ESP32-C3、ESP32-S3、ESP32-P4 芯片平台
- 通过设备端 MCP 实现设备控制音量、灯光、电机、GPIO 等)
- 通过云端 MCP 扩展大模型能力智能家居控制、PC桌面操作、知识搜索、邮件收发等
- 自定义唤醒词、字体、表情与聊天背景,支持网页端在线修改 ([自定义Assets生成器](https://github.com/78/xiaozhi-assets-generator))
## 硬件
### 面包板手工制作实践
详见飞书文档教程:
👉 [《小智 AI 聊天机器人百科全书》](https://ccnphfhqs21z.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb?from=from_copylink)
面包板效果图如下:
![面包板效果图](docs/v1/wiring2.jpg)
### 支持 70 多个开源硬件(仅展示部分)
- <a href="https://oshwhub.com/li-chuang-kai-fa-ban/li-chuang-shi-zhan-pai-esp32-s3-kai-fa-ban" target="_blank" title="立创·实战派 ESP32-S3 开发板">立创·实战派 ESP32-S3 开发板</a>
- <a href="https://github.com/espressif/esp-box" target="_blank" title="乐鑫 ESP32-S3-BOX3">乐鑫 ESP32-S3-BOX3</a>
- <a href="https://docs.m5stack.com/zh_CN/core/CoreS3" target="_blank" title="M5Stack CoreS3">M5Stack CoreS3</a>
- <a href="https://docs.m5stack.com/en/atom/Atomic%20Echo%20Base" target="_blank" title="AtomS3R + Echo Base">M5Stack AtomS3R + Echo Base</a>
- <a href="https://gf.bilibili.com/item/detail/1108782064" target="_blank" title="神奇按钮 2.4">神奇按钮 2.4</a>
- <a href="https://www.waveshare.net/shop/ESP32-S3-Touch-AMOLED-1.8.htm" target="_blank" title="微雪电子 ESP32-S3-Touch-AMOLED-1.8">微雪电子 ESP32-S3-Touch-AMOLED-1.8</a>
- <a href="https://github.com/Xinyuan-LilyGO/T-Circle-S3" target="_blank" title="LILYGO T-Circle-S3">LILYGO T-Circle-S3</a>
- <a href="https://oshwhub.com/tenclass01/xmini_c3" target="_blank" title="虾哥 Mini C3">虾哥 Mini C3</a>
- <a href="https://oshwhub.com/movecall/cuican-ai-pendant-lights-up-y" target="_blank" title="Movecall CuiCan ESP32S3">璀璨·AI 吊坠</a>
- <a href="https://github.com/WMnologo/xingzhi-ai" target="_blank" title="无名科技Nologo-星智-1.54">无名科技 Nologo-星智-1.54TFT</a>
- <a href="https://www.seeedstudio.com/SenseCAP-Watcher-W1-A-p-5979.html" target="_blank" title="SenseCAP Watcher">SenseCAP Watcher</a>
- <a href="https://www.bilibili.com/video/BV1BHJtz6E2S/" target="_blank" title="ESP-HI 超低成本机器狗">ESP-HI 超低成本机器狗</a>
<div style="display: flex; justify-content: space-between;">
<a href="docs/v1/lichuang-s3.jpg" target="_blank" title="立创·实战派 ESP32-S3 开发板">
<img src="docs/v1/lichuang-s3.jpg" width="240" />
</a>
<a href="docs/v1/espbox3.jpg" target="_blank" title="乐鑫 ESP32-S3-BOX3">
<img src="docs/v1/espbox3.jpg" width="240" />
</a>
<a href="docs/v1/m5cores3.jpg" target="_blank" title="M5Stack CoreS3">
<img src="docs/v1/m5cores3.jpg" width="240" />
</a>
<a href="docs/v1/atoms3r.jpg" target="_blank" title="AtomS3R + Echo Base">
<img src="docs/v1/atoms3r.jpg" width="240" />
</a>
<a href="docs/v1/magiclick.jpg" target="_blank" title="神奇按钮 2.4">
<img src="docs/v1/magiclick.jpg" width="240" />
</a>
<a href="docs/v1/waveshare.jpg" target="_blank" title="微雪电子 ESP32-S3-Touch-AMOLED-1.8">
<img src="docs/v1/waveshare.jpg" width="240" />
</a>
<a href="docs/v1/lilygo-t-circle-s3.jpg" target="_blank" title="LILYGO T-Circle-S3">
<img src="docs/v1/lilygo-t-circle-s3.jpg" width="240" />
</a>
<a href="docs/v1/xmini-c3.jpg" target="_blank" title="虾哥 Mini C3">
<img src="docs/v1/xmini-c3.jpg" width="240" />
</a>
<a href="docs/v1/movecall-cuican-esp32s3.jpg" target="_blank" title="CuiCan">
<img src="docs/v1/movecall-cuican-esp32s3.jpg" width="240" />
</a>
<a href="docs/v1/wmnologo_xingzhi_1.54.jpg" target="_blank" title="无名科技Nologo-星智-1.54">
<img src="docs/v1/wmnologo_xingzhi_1.54.jpg" width="240" />
</a>
<a href="docs/v1/sensecap_watcher.jpg" target="_blank" title="SenseCAP Watcher">
<img src="docs/v1/sensecap_watcher.jpg" width="240" />
</a>
<a href="docs/v1/esp-hi.jpg" target="_blank" title="ESP-HI 超低成本机器狗">
<img src="docs/v1/esp-hi.jpg" width="240" />
</a>
</div>
## 软件
### 固件烧录
新手第一次操作建议先不要搭建开发环境,直接使用免开发环境烧录的固件。
固件默认接入 [xiaozhi.me](https://xiaozhi.me) 官方服务器,个人用户注册账号可以免费使用 Qwen 实时模型。
👉 [新手烧录固件教程](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS)
### 开发环境
- Cursor 或 VSCode
- 安装 ESP-IDF 插件,选择 SDK 版本 5.4 或以上
- Linux 比 Windows 更好,编译速度快,也免去驱动问题的困扰
- 本项目使用 Google C++ 代码风格,提交代码时请确保符合规范
### 开发者文档
- [自定义开发板指南](docs/custom-board.md) - 学习如何为小智 AI 创建自定义开发板
- [MCP 协议物联网控制用法说明](docs/mcp-usage.md) - 了解如何通过 MCP 协议控制物联网设备
- [MCP 协议交互流程](docs/mcp-protocol.md) - 设备端 MCP 协议的实现方式
- [MQTT + UDP 混合通信协议文档](docs/mqtt-udp.md)
- [一份详细的 WebSocket 通信协议文档](docs/websocket.md)
## 大模型配置
如果你已经拥有一个小智 AI 聊天机器人设备,并且已接入官方服务器,可以登录 [xiaozhi.me](https://xiaozhi.me) 控制台进行配置。
👉 [后台操作视频教程(旧版界面)](https://www.bilibili.com/video/BV1jUCUY2EKM/)
## 相关开源项目
在个人电脑上部署服务器,可以参考以下第三方开源的项目:
- [xinnan-tech/xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) Python 服务器
- [joey-zhou/xiaozhi-esp32-server-java](https://github.com/joey-zhou/xiaozhi-esp32-server-java) Java 服务器
- [AnimeAIChat/xiaozhi-server-go](https://github.com/AnimeAIChat/xiaozhi-server-go) Golang 服务器
使用小智通信协议的第三方客户端项目:
- [huangjunsen0406/py-xiaozhi](https://github.com/huangjunsen0406/py-xiaozhi) Python 客户端
- [TOM88812/xiaozhi-android-client](https://github.com/TOM88812/xiaozhi-android-client) Android 客户端
- [100askTeam/xiaozhi-linux](http://github.com/100askTeam/xiaozhi-linux) 百问科技提供的 Linux 客户端
- [78/xiaozhi-sf32](https://github.com/78/xiaozhi-sf32) 思澈科技的蓝牙芯片固件
- [QuecPython/solution-xiaozhiAI](https://github.com/QuecPython/solution-xiaozhiAI) 移远提供的 QuecPython 固件
## 关于项目
这是一个由虾哥开源的 ESP32 项目,以 MIT 许可证发布,允许任何人免费使用,修改或用于商业用途。
我们希望通过这个项目,能够帮助大家了解 AI 硬件开发,将当下飞速发展的大语言模型应用到实际的硬件设备中。
如果你有任何想法或建议,请随时提出 Issues 或加入 QQ 群1011329060
## Star History
<a href="https://star-history.com/#78/xiaozhi-esp32&Date">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=78/xiaozhi-esp32&type=Date&theme=dark" />
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=78/xiaozhi-esp32&type=Date" />
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=78/xiaozhi-esp32&type=Date" />
</picture>
</a>

36
docs/blufi.md Normal file
View File

@@ -0,0 +1,36 @@
# BluFi 配网(集成 esp-wifi-connect
本文档说明如何在小智固件中启用和使用 BluFiBLE WiFi 配网),并结合项目内置的 `esp-wifi-connect` 组件完成 WiFi 连接与存储。官方
BluFi
协议说明请参考 [Espressif 文档](https://docs.espressif.com/projects/esp-idf/zh_CN/stable/esp32/api-guides/ble/blufi.html)。
## 前置条件
- 需要支持 BLE 的芯片与固件配置。
-`idf.py menuconfig` 中启用 `WiFi Configuration Method -> Esp Blufi``CONFIG_USE_ESP_BLUFI_WIFI_PROVISIONING=y`
)。如果只想用 BluFi可关闭同一菜单下的 Hotspot/Acoustic 选项。
- 保持默认的 NVS 与事件循环初始化(项目的 `app_main` 已处理)。
- CONFIG_BT_BLUEDROID_ENABLED、CONFIG_BT_NIMBLE_ENABLED这两个宏应二选一不能同时启用。
## 工作流程
1) 手机端通过 BluFi如官方 EspBlufi App 或自研客户端)连接设备,发送 WiFi SSID/密码。
2) 设备侧在 `ESP_BLUFI_EVENT_REQ_CONNECT_TO_AP` 中将凭据写入 `SsidManager`(存储到 NVS属于 `esp-wifi-connect` 组件)。
3) 随后启动 `WifiStation` 扫描并连接;状态通过 BluFi 返回。
4) 配网成功后设备会自动连接新 WiFi失败则返回失败状态。
## 使用步骤
1. 配置:在 menuconfig 开启 `Esp Blufi`。编译并烧录固件。
2. 触发配网:设备首次启动且没有已保存的 WiFi 时会自动进入配网。
3. 手机端操作:打开 EspBlufi App或其他 BluFi 客户端),搜索并连接设备,可以选择是否加密,按提示输入 WiFi SSID/密码并发送。
4. 观察结果:
- 成功BluFi 报告连接成功,设备自动连接 WiFi。
- 失败BluFi 返回失败状态,可重新发送或检查路由器。
## 注意事项
- BluFi 与 Hotspot/声波配网可以同时编译,但会同时启动,增加内存占用。建议在 menuconfig 中只保留一种方式。
- 若多次测试,建议清除或覆盖存储的 SSID`wifi` 命名空间),避免旧配置干扰。
- 如果使用自定义 BluFi 客户端,需遵循官方协议帧格式,参考上文官方文档链接。
- 官方文档中已提供EspBlufi APP下载地址

View File

@@ -21,7 +21,7 @@
### 1. 创建新的开发板目录
首先在`boards/`目录下创建一个新的目录,例如`my-custom-board/`
首先在`boards/`目录下创建一个新的目录,命名方式应使用 `[品牌名]-[开发板类型]` 的形式,例如 `m5stack-tab5`
```bash
mkdir main/boards/my-custom-board
@@ -87,24 +87,51 @@ mkdir main/boards/my-custom-board
#### config.json
`config.json`中定义编译配置:
`config.json`中定义编译配置,这个文件用于 `scripts/release.py` 脚本自动化编译:
```json
{
"target": "esp32s3", // 目标芯片型号: esp32, esp32s3, esp32c3等
"target": "esp32s3", // 目标芯片型号: esp32, esp32s3, esp32c3, esp32c6, esp32p4
"builds": [
{
"name": "my-custom-board", // 开发板名称
"name": "my-custom-board", // 开发板名称,用于生成固件包
"sdkconfig_append": [
// 额外需要的编译配置
// 特别 Flash 大小配置
"CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y",
"CONFIG_PARTITION_TABLE_CUSTOM_FILENAME=\"partitions/v1/8m.csv\""
// 特别分区表配置
"CONFIG_PARTITION_TABLE_CUSTOM_FILENAME=\"partitions/v2/8m.csv\""
]
}
]
}
```
**配置项说明:**
- `target`: 目标芯片型号,必须与硬件匹配
- `name`: 编译输出的固件包名称,建议与目录名一致
- `sdkconfig_append`: 额外的 sdkconfig 配置项数组,会追加到默认配置中
**常用的 sdkconfig_append 配置:**
```json
// Flash 大小
"CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y" // 4MB Flash
"CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y" // 8MB Flash
"CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y" // 16MB Flash
// 分区表
"CONFIG_PARTITION_TABLE_CUSTOM_FILENAME=\"partitions/v2/4m.csv\"" // 4MB 分区表
"CONFIG_PARTITION_TABLE_CUSTOM_FILENAME=\"partitions/v2/8m.csv\"" // 8MB 分区表
"CONFIG_PARTITION_TABLE_CUSTOM_FILENAME=\"partitions/v2/16m.csv\"" // 16MB 分区表
// 语言配置
"CONFIG_LANGUAGE_EN_US=y" // 英语
"CONFIG_LANGUAGE_ZH_CN=y" // 简体中文
// 唤醒词配置
"CONFIG_USE_DEVICE_AEC=y" // 启用设备端 AEC
"CONFIG_WAKE_WORD_DISABLED=y" // 禁用唤醒词
```
### 3. 编写板级初始化代码
创建一个`my_custom_board.cc`文件,实现开发板的所有初始化逻辑。
@@ -118,12 +145,12 @@ mkdir main/boards/my-custom-board
```cpp
#include "wifi_board.h"
#include "audio_codecs/es8311_audio_codec.h"
#include "codecs/es8311_audio_codec.h"
#include "display/lcd_display.h"
#include "application.h"
#include "button.h"
#include "config.h"
#include "iot/thing_manager.h"
#include "mcp_server.h"
#include <esp_log.h>
#include <driver/i2c_master.h>
@@ -131,10 +158,6 @@ mkdir main/boards/my-custom-board
#define TAG "MyCustomBoard"
// 声明字体
LV_FONT_DECLARE(font_puhui_16_4);
LV_FONT_DECLARE(font_awesome_16_4);
class MyCustomBoard : public WifiBoard {
private:
i2c_master_bus_handle_t codec_i2c_bus_;
@@ -174,8 +197,9 @@ private:
void InitializeButtons() {
boot_button_.OnClick([this]() {
auto& app = Application::GetInstance();
if (app.GetDeviceState() == kDeviceStateStarting && !WifiStation::GetInstance().IsConnected()) {
ResetWifiConfiguration();
if (app.GetDeviceState() == kDeviceStateStarting) {
EnterWifiConfigMode();
return;
}
app.ToggleChatState();
});
@@ -212,20 +236,12 @@ private:
display_ = new SpiLcdDisplay(panel_io, panel,
DISPLAY_WIDTH, DISPLAY_HEIGHT,
DISPLAY_OFFSET_X, DISPLAY_OFFSET_Y,
DISPLAY_MIRROR_X, DISPLAY_MIRROR_Y, DISPLAY_SWAP_XY,
{
.text_font = &font_puhui_16_4,
.icon_font = &font_awesome_16_4,
.emoji_font = font_emoji_32_init(),
});
DISPLAY_MIRROR_X, DISPLAY_MIRROR_Y, DISPLAY_SWAP_XY);
}
// IoT设备初始化
void InitializeIot() {
auto& thing_manager = iot::ThingManager::GetInstance();
thing_manager.AddThing(iot::CreateThing("Speaker"));
thing_manager.AddThing(iot::CreateThing("Screen"));
// 可以添加更多IoT设备
// MCP Tools 初始化
void InitializeTools() {
// 参考 MCP 文档
}
public:
@@ -235,7 +251,7 @@ public:
InitializeSpi();
InitializeDisplay();
InitializeButtons();
InitializeIot();
InitializeTools();
GetBacklight()->SetBrightness(100);
}
@@ -272,7 +288,106 @@ public:
DECLARE_BOARD(MyCustomBoard);
```
### 4. 创建README.md
### 4. 添加构建系统配置
#### 在 Kconfig.projbuild 中添加开发板选项
打开 `main/Kconfig.projbuild` 文件,在 `choice BOARD_TYPE` 部分添加新的开发板配置项:
```kconfig
choice BOARD_TYPE
prompt "Board Type"
default BOARD_TYPE_BREAD_COMPACT_WIFI
help
Board type. 开发板类型
# ... 其他开发板选项 ...
config BOARD_TYPE_MY_CUSTOM_BOARD
bool "My Custom Board (我的自定义开发板)"
depends on IDF_TARGET_ESP32S3 # 根据你的目标芯片修改
endchoice
```
**注意事项:**
- `BOARD_TYPE_MY_CUSTOM_BOARD` 是配置项名称,需要全大写,使用下划线分隔
- `depends on` 指定了目标芯片类型(如 `IDF_TARGET_ESP32S3``IDF_TARGET_ESP32C3` 等)
- 描述文字可以使用中英文
#### 在 CMakeLists.txt 中添加开发板配置
打开 `main/CMakeLists.txt` 文件,在开发板类型判断部分添加新的配置:
```cmake
# 在 elseif 链中添加你的开发板配置
elseif(CONFIG_BOARD_TYPE_MY_CUSTOM_BOARD)
set(BOARD_TYPE "my-custom-board") # 与目录名一致
set(BUILTIN_TEXT_FONT font_puhui_basic_20_4) # 根据屏幕大小选择合适的字体
set(BUILTIN_ICON_FONT font_awesome_20_4)
set(DEFAULT_EMOJI_COLLECTION twemoji_64) # 可选,如果需要表情显示
endif()
```
**字体和表情配置说明:**
根据屏幕分辨率选择合适的字体大小:
- 小屏幕128x64 OLED`font_puhui_basic_14_1` / `font_awesome_14_1`
- 中小屏幕240x240`font_puhui_basic_16_4` / `font_awesome_16_4`
- 中等屏幕240x320`font_puhui_basic_20_4` / `font_awesome_20_4`
- 大屏幕480x320+`font_puhui_basic_30_4` / `font_awesome_30_4`
表情集合选项:
- `twemoji_32` - 32x32 像素表情(小屏幕)
- `twemoji_64` - 64x64 像素表情(大屏幕)
### 5. 配置和编译
#### 方法一:使用 idf.py 手动配置
1. **设置目标芯片**(首次配置或更换芯片时):
```bash
# 对于 ESP32-S3
idf.py set-target esp32s3
# 对于 ESP32-C3
idf.py set-target esp32c3
# 对于 ESP32
idf.py set-target esp32
```
2. **清理旧配置**
```bash
idf.py fullclean
```
3. **进入配置菜单**
```bash
idf.py menuconfig
```
在菜单中导航到:`Xiaozhi Assistant` -> `Board Type`,选择你的自定义开发板。
4. **编译和烧录**
```bash
idf.py build
idf.py flash monitor
```
#### 方法二:使用 release.py 脚本(推荐)
如果你的开发板目录下有 `config.json` 文件,可以使用此脚本自动完成配置和编译:
```bash
python scripts/release.py my-custom-board
```
此脚本会自动:
- 读取 `config.json` 中的 `target` 配置并设置目标芯片
- 应用 `sdkconfig_append` 中的编译选项
- 完成编译并打包固件
### 6. 创建README.md
在README.md中说明开发板的特性、硬件要求、编译和烧录步骤

View File

@@ -42,7 +42,7 @@ MCP 的交互主要围绕客户端(后台 API发现和调用设备上的“
- **时机:** 设备启动并成功连接到后台 API 后。
- **发送方:** 设备。
- **消息:** 设备发送基础协议的 "hello" 消息给后台 API消息中包含设备支持的能力列表例如通过 `CONFIG_IOT_PROTOCOL_MCP` 配置来表明支持 MCP 协议 (`"mcp": true`)。
- **消息:** 设备发送基础协议的 "hello" 消息给后台 API消息中包含设备支持的能力列表例如通过支持 MCP 协议 (`"mcp": true`)。
- **示例 (非 MCP 负载,而是基础协议消息):**
```json
{

393
docs/mqtt-udp.md Normal file
View File

@@ -0,0 +1,393 @@
# MQTT + UDP 混合通信协议文档
基于代码实现整理的 MQTT + UDP 混合通信协议文档,概述设备端与服务器之间如何通过 MQTT 进行控制消息传输,通过 UDP 进行音频数据传输的交互方式。
---
## 1. 协议概览
本协议采用混合传输方式:
- **MQTT**用于控制消息、状态同步、JSON 数据交换
- **UDP**:用于实时音频数据传输,支持加密
### 1.1 协议特点
- **双通道设计**:控制与数据分离,确保实时性
- **加密传输**UDP 音频数据使用 AES-CTR 加密
- **序列号保护**:防止数据包重放和乱序
- **自动重连**MQTT 连接断开时自动重连
---
## 2. 总体流程概览
```mermaid
sequenceDiagram
participant Device as ESP32 设备
participant MQTT as MQTT 服务器
participant UDP as UDP 服务器
Note over Device, UDP: 1. 建立 MQTT 连接
Device->>MQTT: MQTT Connect
MQTT->>Device: Connected
Note over Device, UDP: 2. 请求音频通道
Device->>MQTT: Hello Message (type: "hello", transport: "udp")
MQTT->>Device: Hello Response (UDP 连接信息 + 加密密钥)
Note over Device, UDP: 3. 建立 UDP 连接
Device->>UDP: UDP Connect
UDP->>Device: Connected
Note over Device, UDP: 4. 音频数据传输
loop 音频流传输
Device->>UDP: 加密音频数据 (Opus)
UDP->>Device: 加密音频数据 (Opus)
end
Note over Device, UDP: 5. 控制消息交换
par 控制消息
Device->>MQTT: Listen/TTS/MCP 消息
MQTT->>Device: STT/TTS/MCP 响应
end
Note over Device, UDP: 6. 关闭连接
Device->>MQTT: Goodbye Message
Device->>UDP: Disconnect
```
---
## 3. MQTT 控制通道
### 3.1 连接建立
设备通过 MQTT 连接到服务器,连接参数包括:
- **Endpoint**MQTT 服务器地址和端口
- **Client ID**:设备唯一标识符
- **Username/Password**:认证凭据
- **Keep Alive**心跳间隔默认240秒
### 3.2 Hello 消息交换
#### 3.2.1 设备端发送 Hello
```json
{
"type": "hello",
"version": 3,
"transport": "udp",
"features": {
"mcp": true
},
"audio_params": {
"format": "opus",
"sample_rate": 16000,
"channels": 1,
"frame_duration": 60
}
}
```
#### 3.2.2 服务器响应 Hello
```json
{
"type": "hello",
"transport": "udp",
"session_id": "xxx",
"audio_params": {
"format": "opus",
"sample_rate": 24000,
"channels": 1,
"frame_duration": 60
},
"udp": {
"server": "192.168.1.100",
"port": 8888,
"key": "0123456789ABCDEF0123456789ABCDEF",
"nonce": "0123456789ABCDEF0123456789ABCDEF"
}
}
```
**字段说明:**
- `udp.server`UDP 服务器地址
- `udp.port`UDP 服务器端口
- `udp.key`AES 加密密钥(十六进制字符串)
- `udp.nonce`AES 加密随机数(十六进制字符串)
### 3.3 JSON 消息类型
#### 3.3.1 设备端→服务器
1. **Listen 消息**
```json
{
"session_id": "xxx",
"type": "listen",
"state": "start",
"mode": "manual"
}
```
2. **Abort 消息**
```json
{
"session_id": "xxx",
"type": "abort",
"reason": "wake_word_detected"
}
```
3. **MCP 消息**
```json
{
"session_id": "xxx",
"type": "mcp",
"payload": {
"jsonrpc": "2.0",
"id": 1,
"result": {...}
}
}
```
4. **Goodbye 消息**
```json
{
"session_id": "xxx",
"type": "goodbye"
}
```
#### 3.3.2 服务器→设备端
支持的消息类型与 WebSocket 协议一致,包括:
- **STT**:语音识别结果
- **TTS**:语音合成控制
- **LLM**:情感表达控制
- **MCP**:物联网控制
- **System**:系统控制
- **Custom**:自定义消息(可选)
---
## 4. UDP 音频通道
### 4.1 连接建立
设备收到 MQTT Hello 响应后,使用其中的 UDP 连接信息建立音频通道:
1. 解析 UDP 服务器地址和端口
2. 解析加密密钥和随机数
3. 初始化 AES-CTR 加密上下文
4. 建立 UDP 连接
### 4.2 音频数据格式
#### 4.2.1 加密音频包结构
```
|type 1byte|flags 1byte|payload_len 2bytes|ssrc 4bytes|timestamp 4bytes|sequence 4bytes|
|payload payload_len bytes|
```
**字段说明:**
- `type`:数据包类型,固定为 0x01
- `flags`:标志位,当前未使用
- `payload_len`:负载长度(网络字节序)
- `ssrc`:同步源标识符
- `timestamp`:时间戳(网络字节序)
- `sequence`:序列号(网络字节序)
- `payload`:加密的 Opus 音频数据
#### 4.2.2 加密算法
使用 **AES-CTR** 模式加密:
- **密钥**128位由服务器提供
- **随机数**128位由服务器提供
- **计数器**:包含时间戳和序列号信息
### 4.3 序列号管理
- **发送端**`local_sequence_` 单调递增
- **接收端**`remote_sequence_` 验证连续性
- **防重放**:拒绝序列号小于期望值的数据包
- **容错处理**:允许轻微的序列号跳跃,记录警告
### 4.4 错误处理
1. **解密失败**:记录错误,丢弃数据包
2. **序列号异常**:记录警告,但仍处理数据包
3. **数据包格式错误**:记录错误,丢弃数据包
---
## 5. 状态管理
### 5.1 连接状态
```mermaid
stateDiagram
direction TB
[*] --> Disconnected
Disconnected --> MqttConnecting: StartMqttClient()
MqttConnecting --> MqttConnected: MQTT Connected
MqttConnecting --> Disconnected: Connect Failed
MqttConnected --> RequestingChannel: OpenAudioChannel()
RequestingChannel --> ChannelOpened: Hello Exchange Success
RequestingChannel --> MqttConnected: Hello Timeout/Failed
ChannelOpened --> UdpConnected: UDP Connect Success
UdpConnected --> AudioStreaming: Start Audio Transfer
AudioStreaming --> UdpConnected: Stop Audio Transfer
UdpConnected --> ChannelOpened: UDP Disconnect
ChannelOpened --> MqttConnected: CloseAudioChannel()
MqttConnected --> Disconnected: MQTT Disconnect
```
### 5.2 状态检查
设备通过以下条件判断音频通道是否可用:
```cpp
bool IsAudioChannelOpened() const {
return udp_ != nullptr && !error_occurred_ && !IsTimeout();
}
```
---
## 6. 配置参数
### 6.1 MQTT 配置
从设置中读取的配置项:
- `endpoint`MQTT 服务器地址
- `client_id`:客户端标识符
- `username`:用户名
- `password`:密码
- `keepalive`心跳间隔默认240秒
- `publish_topic`:发布主题
### 6.2 音频参数
- **格式**Opus
- **采样率**16000 Hz设备端/ 24000 Hz服务器端
- **声道数**1单声道
- **帧时长**60ms
---
## 7. 错误处理与重连
### 7.1 MQTT 重连机制
- 连接失败时自动重试
- 支持错误上报控制
- 断线时触发清理流程
### 7.2 UDP 连接管理
- 连接失败时不自动重试
- 依赖 MQTT 通道重新协商
- 支持连接状态查询
### 7.3 超时处理
基类 `Protocol` 提供超时检测:
- 默认超时时间120 秒
- 基于最后接收时间计算
- 超时时自动标记为不可用
---
## 8. 安全考虑
### 8.1 传输加密
- **MQTT**:支持 TLS/SSL 加密端口8883
- **UDP**:使用 AES-CTR 加密音频数据
### 8.2 认证机制
- **MQTT**:用户名/密码认证
- **UDP**:通过 MQTT 通道分发密钥
### 8.3 防重放攻击
- 序列号单调递增
- 拒绝过期数据包
- 时间戳验证
---
## 9. 性能优化
### 9.1 并发控制
使用互斥锁保护 UDP 连接:
```cpp
std::lock_guard<std::mutex> lock(channel_mutex_);
```
### 9.2 内存管理
- 动态创建/销毁网络对象
- 智能指针管理音频数据包
- 及时释放加密上下文
### 9.3 网络优化
- UDP 连接复用
- 数据包大小优化
- 序列号连续性检查
---
## 10. 与 WebSocket 协议的比较
| 特性 | MQTT + UDP | WebSocket |
|------|------------|-----------|
| 控制通道 | MQTT | WebSocket |
| 音频通道 | UDP (加密) | WebSocket (二进制) |
| 实时性 | 高 (UDP) | 中等 |
| 可靠性 | 中等 | 高 |
| 复杂度 | 高 | 低 |
| 加密 | AES-CTR | TLS |
| 防火墙友好度 | 低 | 高 |
---
## 11. 部署建议
### 11.1 网络环境
- 确保 UDP 端口可达
- 配置防火墙规则
- 考虑 NAT 穿透
### 11.2 服务器配置
- MQTT Broker 配置
- UDP 服务器部署
- 密钥管理系统
### 11.3 监控指标
- 连接成功率
- 音频传输延迟
- 数据包丢失率
- 解密失败率
---
## 12. 总结
MQTT + UDP 混合协议通过以下设计实现高效的音视频通信:
- **分离式架构**:控制与数据通道分离,各司其职
- **加密保护**AES-CTR 确保音频数据安全传输
- **序列化管理**:防止重放攻击和数据乱序
- **自动恢复**:支持连接断开后的自动重连
- **性能优化**UDP 传输保证音频数据的实时性
该协议适用于对实时性要求较高的语音交互场景,但需要在网络复杂度和传输性能之间做出权衡。

View File

@@ -84,7 +84,7 @@
在建立 WebSocket 连接时,代码示例中设置了以下请求头:
- `Authorization`: 用于存放访问令牌,形如 `"Bearer <token>"`
- `Protocol-Version`: 固定示例中为 `"1"`,与 hello 消息体内的 `version` 字段保持一致
- `Protocol-Version`: 协议版本号,与 hello 消息体内的 `version` 字段保持一致
- `Device-Id`: 设备物理网卡 MAC 地址
- `Client-Id`: 软件生成的 UUID擦除 NVS 或重新烧录完整固件会重置)
@@ -92,11 +92,44 @@
---
## 3. JSON 消息结构
## 3. 二进制协议版本
设备支持多种二进制协议版本,通过配置中的 `version` 字段指定:
### 3.1 版本1默认
直接发送 Opus 音频数据无额外元数据。Websocket 协议会区分 text 与 binary。
### 3.2 版本2
使用 `BinaryProtocol2` 结构:
```c
struct BinaryProtocol2 {
uint16_t version; // 协议版本
uint16_t type; // 消息类型 (0: OPUS, 1: JSON)
uint32_t reserved; // 保留字段
uint32_t timestamp; // 时间戳毫秒用于服务器端AEC
uint32_t payload_size; // 负载大小(字节)
uint8_t payload[]; // 负载数据
} __attribute__((packed));
```
### 3.3 版本3
使用 `BinaryProtocol3` 结构:
```c
struct BinaryProtocol3 {
uint8_t type; // 消息类型
uint8_t reserved; // 保留字段
uint16_t payload_size; // 负载大小
uint8_t payload[]; // 负载数据
} __attribute__((packed));
```
---
## 4. JSON 消息结构
WebSocket 文本帧以 JSON 方式传输,以下为常见的 `"type"` 字段及其对应业务逻辑。若消息里包含未列出的字段,可能为可选或特定实现细节。
### 3.1 设备端→服务器
### 4.1 设备端→服务器
1. **Hello**
- 连接成功后,由设备端发送,告知服务器基本参数。
@@ -183,7 +216,7 @@ WebSocket 文本帧以 JSON 方式传输,以下为常见的 `"type"` 字段及
---
### 3.2 服务器→设备端
### 4.2 服务器→设备端
1. **Hello**
- 服务器端返回的握手确认消息。
@@ -227,17 +260,43 @@ WebSocket 文本帧以 JSON 方式传输,以下为常见的 `"type"` 字段及
}
```
6. **音频数据:二进制帧**
6. **System**
- 系统控制命令,常用于远程升级更新。
- 例:
```json
{
"session_id": "xxx",
"type": "system",
"command": "reboot"
}
```
- 支持的命令:
- `"reboot"`:重启设备
7. **Custom**(可选)
- 自定义消息,当 `CONFIG_RECEIVE_CUSTOM_MESSAGE` 启用时支持。
- 例:
```json
{
"session_id": "xxx",
"type": "custom",
"payload": {
"message": "自定义内容"
}
}
```
8. **音频数据:二进制帧**
- 当服务器发送音频二进制帧Opus 编码)时,设备端解码并播放。
- 若设备端正在处于 "listening" (录音)状态,收到的音频帧会被忽略或清空以防冲突。
---
## 4. 音频编解码
## 5. 音频编解码
1. **设备端发送录音数据**
- 音频输入经过可能的回声消除、降噪或音量增益后,通过 Opus 编码打包为二进制帧发送给服务器。
- 如果设备端每次编码生成的二进制帧大小为 N 字节,则会通过 WebSocket 的 **binary** 消息发送这块数据
- 根据协议版本,可能直接发送 Opus 数据版本1或使用带元数据的二进制协议版本2/3
2. **设备端播放收到的音频**
- 收到服务器的二进制帧时,同样认定是 Opus 数据。
@@ -246,7 +305,7 @@ WebSocket 文本帧以 JSON 方式传输,以下为常见的 `"type"` 字段及
---
## 5. 常见状态流转
## 6. 常见状态流转
以下为常见设备端关键状态流转,与 WebSocket 消息对应:
@@ -307,7 +366,7 @@ stateDiagram
---
## 6. 错误处理
## 7. 错误处理
1. **连接失败**
- 如果 `Connect(url)` 返回失败或在等待服务器 "hello" 消息时超时,触发 `on_network_error_()` 回调。设备会提示"无法连接到服务"或类似错误信息。
@@ -319,7 +378,7 @@ stateDiagram
---
## 7. 其它注意事项
## 8. 其它注意事项
1. **鉴权**
- 设备通过设置 `Authorization: Bearer <token>` 提供鉴权,服务器端需验证是否有效。
@@ -331,17 +390,23 @@ stateDiagram
3. **音频负载**
- 代码里默认使用 Opus 格式,并设置 `sample_rate = 16000`,单声道。帧时长由 `OPUS_FRAME_DURATION_MS` 控制,一般为 60ms。可根据带宽或性能做适当调整。为了获得更好的音乐播放效果服务器下行音频可能使用 24000 采样率。
4. **物联网控制推荐 MCP 协议**
4. **协议版本配置**
- 通过设置中的 `version` 字段配置二进制协议版本1、2 或 3
- 版本1直接发送 Opus 数据
- 版本2使用带时间戳的二进制协议适用于服务器端 AEC
- 版本3使用简化的二进制协议
5. **物联网控制推荐 MCP 协议**
- 设备与服务器之间的物联网能力发现、状态同步、控制指令等,建议全部通过 MCP 协议type: "mcp")实现。原有的 type: "iot" 方案已废弃。
- MCP 协议可在 WebSocket、MQTT 等多种底层协议上传输,具备更好的扩展性和标准化能力。
- 详细用法请参考 [MCP 协议文档](./mcp-protocol.md) 及 [MCP 物联网控制用法](./mcp-usage.md)。
5. **错误或异常 JSON**
6. **错误或异常 JSON**
- 当 JSON 中缺少必要字段,例如 `{"type": ...}`,设备端会记录错误日志(`ESP_LOGE(TAG, "Missing message type, data: %s", data);`),不会执行任何业务。
---
## 8. 消息示例
## 9. 消息示例
下面给出一个典型的双向消息示例(流程简化示意):
@@ -418,13 +483,13 @@ stateDiagram
---
## 9. 总结
## 10. 总结
本协议通过在 WebSocket 上层传输 JSON 文本与二进制音频帧完成功能包括音频流上传、TTS 音频播放、语音识别与状态管理、MCP 指令下发等。其核心特征:
- **握手阶段**:发送 `"type":"hello"`,等待服务器返回。
- **音频通道**:采用 Opus 编码的二进制帧双向传输语音流。
- **JSON 消息**:使用 `"type"` 为核心字段标识不同业务逻辑,包括 TTS、STT、MCP、WakeWord 等。
- **音频通道**:采用 Opus 编码的二进制帧双向传输语音流,支持多种协议版本
- **JSON 消息**:使用 `"type"` 为核心字段标识不同业务逻辑,包括 TTS、STT、MCP、WakeWord、System、Custom 等。
- **扩展性**:可根据实际需求在 JSON 消息中添加字段,或在 headers 里进行额外鉴权。
服务器与设备端需提前约定各类消息的字段含义、时序逻辑以及错误处理规则,方能保证通信顺畅。上述信息可作为基础文档,便于后续对接、开发或扩展。

File diff suppressed because it is too large Load Diff

View File

@@ -6,6 +6,27 @@ config OTA_URL
help
The application will access this URL to check for new firmwares and server address.
choice
prompt "Flash Assets"
default FLASH_DEFAULT_ASSETS
help
Select the assets to flash.
config FLASH_NONE_ASSETS
bool "Do not flash assets"
config FLASH_DEFAULT_ASSETS
bool "Flash Default Assets"
config FLASH_CUSTOM_ASSETS
bool "Flash Custom Assets"
endchoice
config CUSTOM_ASSETS_FILE
depends on FLASH_CUSTOM_ASSETS
string "Custom Assets File"
default "assets.bin"
help
The custom assets file to flash.
It can be a local file relative to the project directory or a remote url.
choice
prompt "Default Language"
@@ -21,6 +42,74 @@ choice
bool "English"
config LANGUAGE_JA_JP
bool "Japanese"
config LANGUAGE_KO_KR
bool "Korean"
config LANGUAGE_VI_VN
bool "Vietnamese"
config LANGUAGE_TH_TH
bool "Thai"
config LANGUAGE_DE_DE
bool "German"
config LANGUAGE_FR_FR
bool "French"
config LANGUAGE_ES_ES
bool "Spanish"
config LANGUAGE_IT_IT
bool "Italian"
config LANGUAGE_RU_RU
bool "Russian"
config LANGUAGE_AR_SA
bool "Arabic"
config LANGUAGE_HI_IN
bool "Hindi"
config LANGUAGE_PT_PT
bool "Portuguese"
config LANGUAGE_PL_PL
bool "Polish"
config LANGUAGE_CS_CZ
bool "Czech"
config LANGUAGE_FI_FI
bool "Finnish"
config LANGUAGE_TR_TR
bool "Turkish"
config LANGUAGE_ID_ID
bool "Indonesian"
config LANGUAGE_UK_UA
bool "Ukrainian"
config LANGUAGE_RO_RO
bool "Romanian"
config LANGUAGE_BG_BG
bool "Bulgarian"
config LANGUAGE_CA_ES
bool "Catalan"
config LANGUAGE_DA_DK
bool "Danish"
config LANGUAGE_EL_GR
bool "Greek"
config LANGUAGE_FA_IR
bool "Persian"
config LANGUAGE_FIL_PH
bool "Filipino"
config LANGUAGE_HE_IL
bool "Hebrew"
config LANGUAGE_HR_HR
bool "Croatian"
config LANGUAGE_HU_HU
bool "Hungarian"
config LANGUAGE_MS_MY
bool "Malay"
config LANGUAGE_NB_NO
bool "Norwegian"
config LANGUAGE_NL_NL
bool "Dutch"
config LANGUAGE_SK_SK
bool "Slovak"
config LANGUAGE_SL_SI
bool "Slovenian"
config LANGUAGE_SV_SE
bool "Swedish"
config LANGUAGE_SR_RS
bool "Serbian"
endchoice
choice BOARD_TYPE
@@ -29,46 +118,64 @@ choice BOARD_TYPE
help
Board type. 开发板类型
config BOARD_TYPE_BREAD_COMPACT_WIFI
bool "面包板新版接线WiFi"
bool "Bread Compact WiFi (面包板)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_BREAD_COMPACT_WIFI_LCD
bool "面包板新版接线(WiFi+ LCD"
bool "Bread Compact WiFi + LCD (面包板)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_BREAD_COMPACT_WIFI_CAM
bool "Bread Compact WiFi + LCD + Camera (面包板)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_BREAD_COMPACT_ML307
bool "面包板新版接线ML307 AT"
bool "Bread Compact ML307/EC801E (面包板 4G)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_BREAD_COMPACT_ESP32
bool "面包板WiFi ESP32 DevKit"
bool "Bread Compact ESP32 DevKit (面包板)"
depends on IDF_TARGET_ESP32
config BOARD_TYPE_BREAD_COMPACT_ESP32_LCD
bool "面包板WiFi+ LCD ESP32 DevKit"
bool "Bread Compact ESP32 DevKit + LCD (面包板)"
depends on IDF_TARGET_ESP32
config BOARD_TYPE_XMINI_C3
bool "虾哥 Mini C3"
config BOARD_TYPE_XMINI_C3_V3
bool "Xmini C3 V3"
depends on IDF_TARGET_ESP32C3
config BOARD_TYPE_ESP32S3_KORVO2_V3
bool "ESP32S3_KORVO2_V3开发板"
config BOARD_TYPE_XMINI_C3_4G
bool "Xmini C3 4G"
depends on IDF_TARGET_ESP32C3
config BOARD_TYPE_XMINI_C3
bool "Xmini C3"
depends on IDF_TARGET_ESP32C3
config BOARD_TYPE_ESP_KORVO2_V3
bool "Espressif Korvo2 V3"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP_SPARKBOT
bool "ESP-SparkBot开发板"
bool "Espressif SparkBot"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP_SENSAIRSHUTTLE
bool "Espressif ESP-SensairShuttle"
depends on IDF_TARGET_ESP32C5
config BOARD_TYPE_ESP_SPOT_S3
bool "ESP-Spot-S3"
bool "Espressif Spot-S3"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP_SPOT_C5
bool "Espressif Spot-C5"
depends on IDF_TARGET_ESP32C5
config BOARD_TYPE_ESP_HI
bool "ESP-HI"
bool "Espressif ESP-HI"
depends on IDF_TARGET_ESP32C3
config BOARD_TYPE_ESP_BOX_3
bool "ESP BOX 3"
bool "Espressif ESP-BOX-3"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP_BOX
bool "ESP BOX"
bool "Espressif ESP-BOX"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP_BOX_LITE
bool "ESP BOX Lite"
bool "Espressif ESP-BOX-Lite"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_KEVIN_BOX_1
bool "Kevin Box 1"
config BOARD_TYPE_ESP_P4_FUNCTION_EV_BOARD
bool "Espressif ESP-P4-Function-EV-Board"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_ECHOEAR
bool "EchoEar"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_KEVIN_BOX_2
bool "Kevin Box 2"
@@ -77,25 +184,28 @@ choice BOARD_TYPE
bool "Kevin C3"
depends on IDF_TARGET_ESP32C3
config BOARD_TYPE_KEVIN_SP_V3_DEV
bool "Kevin SP V3开发板"
bool "Kevin SP V3"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_KEVIN_SP_V4_DEV
bool "Kevin SP V4开发板"
bool "Kevin SP V4"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32_CGC
bool "ESP32 CGC"
depends on IDF_TARGET_ESP32
config BOARD_TYPE_ESP32_CGC_144
bool "ESP32 CGC 144"
depends on IDF_TARGET_ESP32
config BOARD_TYPE_KEVIN_YUYING_313LCD
bool "鱼鹰科技3.13LCD开发板"
bool "鱼鹰科技 3.13LCD"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_LICHUANG_DEV
bool "立创·实战派ESP32-S3开发板"
config BOARD_TYPE_CGC
bool "CGC"
depends on IDF_TARGET_ESP32
config BOARD_TYPE_CGC_144
bool "CGC 144"
depends on IDF_TARGET_ESP32
config BOARD_TYPE_LICHUANG_DEV_S3
bool "立创·实战派 ESP32-S3"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_LICHUANG_C3_DEV
bool "立创·实战派ESP32-C3开发板"
config BOARD_TYPE_LICHUANG_DEV_S3_BED_OPERATOR
bool "立创·实战派ESP32-S3开发板(床控)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_LICHUANG_DEV_C3
bool "立创·实战派 ESP32-C3"
depends on IDF_TARGET_ESP32C3
config BOARD_TYPE_DF_K10
bool "DFRobot 行空板 k10"
@@ -103,10 +213,10 @@ choice BOARD_TYPE
config BOARD_TYPE_DF_S3_AI_CAM
bool "DFRobot ESP32-S3 AI智能摄像头模块"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_MAGICLICK_2P4
config BOARD_TYPE_MAGICLICK_S3_2P4
bool "神奇按钮 Magiclick_2.4"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_MAGICLICK_2P5
config BOARD_TYPE_MAGICLICK_S3_2P5
bool "神奇按钮 Magiclick_2.5"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_MAGICLICK_C3
@@ -121,43 +231,88 @@ choice BOARD_TYPE
config BOARD_TYPE_M5STACK_CORE_TAB5
bool "M5Stack Tab5"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_ATOMS3_ECHO_BASE
bool "AtomS3 + Echo Base"
config BOARD_TYPE_M5STACK_ATOM_S3_ECHO_BASE
bool "M5Stack AtomS3 + Echo Base"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ATOMS3R_ECHO_BASE
bool "AtomS3R + Echo Base"
config BOARD_TYPE_M5STACK_ATOM_S3R_ECHO_BASE
bool "M5Stack AtomS3R + Echo Base"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ATOMS3R_CAM_M12_ECHO_BASE
bool "AtomS3R CAM/M12 + Echo Base"
config BOARD_TYPE_M5STACK_ATOM_S3R_CAM_M12_ECHO_BASE
bool "M5Stack AtomS3R CAM/M12 + Echo Base"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ATOMMATRIX_ECHO_BASE
bool "AtomMatrix + Echo Base"
config BOARD_TYPE_M5STACK_ATOM_ECHOS3R
bool "M5Stack AtomEchoS3R"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32S3_Touch_AMOLED_1_8
config BOARD_TYPE_M5STACK_ATOM_MATRIX_ECHO_BASE
bool "M5Stack AtomMatrix + Echo Base"
depends on IDF_TARGET_ESP32
config BOARD_TYPE_WAVESHARE_S3_AUDIO_BOARD
bool "Waveshare ESP32-S3-Audio-Board"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_8
bool "Waveshare ESP32-S3-Touch-AMOLED-1.8"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32S3_Touch_AMOLED_1_75
config BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_2_06
bool "Waveshare ESP32-S3-Touch-AMOLED-2.06"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_2_06
bool "Waveshare ESP32-C6-Touch-AMOLED-2.06"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_75
bool "Waveshare ESP32-S3-Touch-AMOLED-1.75"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32S3_Touch_LCD_1_85C
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_83
bool "Waveshare ESP32-S3-Touch-LCD-1.83"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_4B
bool "Waveshare ESP32-S3-Touch-LCD-4B"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_85C
bool "Waveshare ESP32-S3-Touch-LCD-1.85C"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32S3_Touch_LCD_1_85
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_85
bool "Waveshare ESP32-S3-Touch-LCD-1.85"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32S3_Touch_LCD_1_46
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_46
bool "Waveshare ESP32-S3-Touch-LCD-1.46"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32S3_Touch_LCD_3_5
config BOARD_TYPE_WAVESHARE_C6_LCD_1_69
bool "Waveshare ESP32-C6-LCD-1.69"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_C6_TOUCH_LCD_1_83
bool "Waveshare ESP32-C6-Touch-LCD-1.83"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_1_43
bool "Waveshare ESP32-C6-Touch-AMOLOED-1.43"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_C6_TOUCH_AMOLED_1_32
bool "Waveshare ESP32-C6-Touch-AMOLOED-1.32"
depends on IDF_TARGET_ESP32C6
config BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_32
bool "Waveshare ESP32-S3-Touch-AMOLOED-1.32"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_49
bool "Waveshare ESP32-S3-Touch-LCD-3.49"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_5
bool "Waveshare ESP32-S3-Touch-LCD-3.5"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32P4_NANO
config BOARD_TYPE_WAVESHARE_S3_ePaper_1_54
bool "Waveshare ESP32-S3-ePaper-1.54"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_5B
bool "Waveshare ESP32-S3-Touch-LCD-3.5B"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WAVESHARE_P4_NANO
bool "Waveshare ESP32-P4-NANO"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_ESP32P4_WIFI6_Touch_LCD_4B
config BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_4B
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-4B"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_ESP32P4_WIFI6_Touch_LCD_XC
config BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_7B
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-7B"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_XC
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-3.4C or ESP32-P4-WIFI6-Touch-LCD-4C"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_TUDOUZI
@@ -178,9 +333,15 @@ choice BOARD_TYPE
config BOARD_TYPE_LILYGO_T_DISPLAY_S3_PRO_MVSRLORA_NO_BATTERY
bool "LILYGO T-Display-S3-Pro-MVSRLora_No_Battery"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_LILYGO_T_DISPLAY_P4
bool "LILYGO T-Display-P4"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_MOVECALL_MOJI_ESP32S3
bool "Movecall Moji 小智AI衍生版"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_MOVECALL_MOJI2_ESP32C5
bool "Movecall Moji2.0 小智AI衍生版"
depends on IDF_TARGET_ESP32C5
config BOARD_TYPE_MOVECALL_CUICAN_ESP32S3
bool "Movecall CuiCan 璀璨·AI吊坠"
depends on IDF_TARGET_ESP32S3
@@ -193,6 +354,12 @@ choice BOARD_TYPE
config BOARD_TYPE_ATK_DNESP32S3_BOX0
bool "正点原子DNESP32S3-BOX0"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ATK_DNESP32S3_BOX2_WIFI
bool "正点原子DNESP32S3-BOX2-WIFI"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ATK_DNESP32S3_BOX2_4G
bool "正点原子DNESP32S3-BOX2-4G"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ATK_DNESP32S3M_WIFI
bool "正点原子DNESP32S3M-WIFI"
depends on IDF_TARGET_ESP32S3
@@ -202,29 +369,32 @@ choice BOARD_TYPE
config BOARD_TYPE_DU_CHATX
bool "嘟嘟开发板CHATX(wifi)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32S3_Taiji_Pi
config BOARD_TYPE_TAIJI_PI_S3
bool "太极小派esp32s3"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_XINGZHI_Cube_0_85TFT_WIFI
config BOARD_TYPE_XINGZHI_CUBE_0_85TFT_WIFI
bool "无名科技星智0.85(WIFI)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_XINGZHI_Cube_0_85TFT_ML307
config BOARD_TYPE_XINGZHI_CUBE_0_85TFT_ML307
bool "无名科技星智0.85(ML307)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_XINGZHI_Cube_0_96OLED_WIFI
config BOARD_TYPE_XINGZHI_CUBE_0_96OLED_WIFI
bool "无名科技星智0.96(WIFI)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_XINGZHI_Cube_0_96OLED_ML307
config BOARD_TYPE_XINGZHI_CUBE_0_96OLED_ML307
bool "无名科技星智0.96(ML307)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_XINGZHI_Cube_1_54TFT_WIFI
config BOARD_TYPE_XINGZHI_CUBE_1_54TFT_WIFI
bool "无名科技星智1.54(WIFI)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_XINGZHI_Cube_1_54TFT_ML307
config BOARD_TYPE_XINGZHI_CUBE_1_54TFT_ML307
bool "无名科技星智1.54(ML307)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_SENSECAP_WATCHER
bool "SenseCAP Watcher"
config BOARD_TYPE_XINGZHI_METAL_1_54_WIFI
bool "无名科技星智1.54 METAL(wifi)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_SEEED_STUDIO_SENSECAP_WATCHER
bool "Seeed Studio SenseCAP Watcher"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_DOIT_S3_AIBOX
bool "四博智联AI陪伴盒子"
@@ -238,6 +408,9 @@ choice BOARD_TYPE
config BOARD_TYPE_ESP_S3_LCD_EV_Board
bool "乐鑫ESP S3 LCD EV Board开发板"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP_S3_LCD_EV_Board_2
bool "乐鑫ESP S3 LCD EV Board 2开发板"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ZHENGCHEN_1_54TFT_WIFI
bool "征辰科技1.54(WIFI)"
depends on IDF_TARGET_ESP32S3
@@ -247,30 +420,68 @@ choice BOARD_TYPE
config BOARD_TYPE_MINSI_K08_DUAL
bool "敏思科技K08(DUAL)"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32_S3_1_54_MUMA
config BOARD_TYPE_SPOTPEAR_ESP32_S3_1_54_MUMA
bool "Spotpear ESP32-S3-1.54-MUMA"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_ESP32_S3_1_28_BOX
config BOARD_TYPE_SPOTPEAR_ESP32_S3_1_28_BOX
bool "Spotpear ESP32-S3-1.28-BOX"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_OTTO_ROBOT
bool "ottoRobot"
depends on IDF_TARGET_ESP32S3
select LV_USE_GIF
select LV_GIF_CACHE_DECODE_DATA
config BOARD_TYPE_ELECTRON_BOT
bool "electronBot"
depends on IDF_TARGET_ESP32S3
select LV_USE_GIF
select LV_GIF_CACHE_DECODE_DATA
config BOARD_TYPE_JIUCHUAN
bool "九川智能"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_LABPLUS_MPYTHON_V3
bool "labplus mpython_v3 board"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_LABPLUS_LEDONG_V2
bool "labplus ledong_v2 board"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_SURFER_C3_1_14TFT
bool "Surfer-C3-1.14TFT"
depends on IDF_TARGET_ESP32C3
config BOARD_TYPE_YUNLIAO_S3
bool "小智云聊-S3"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_WIRELESS_TAG_WTP4C5MP07S
bool "Wireless-Tag WTP4C5MP07S"
depends on IDF_TARGET_ESP32P4
config BOARD_TYPE_AIPI_LITE
bool "AIPI-Lite"
depends on IDF_TARGET_ESP32S3
config BOARD_TYPE_HU_087
bool "HU-087"
depends on IDF_TARGET_ESP32S3
endchoice
choice
depends on BOARD_TYPE_LILYGO_T_DISPLAY_P4
prompt "Select the screen type"
default SCREEN_TYPE_HI8561
config SCREEN_TYPE_HI8561
bool "HI8561"
config SCREEN_TYPE_RM69A10
bool "RM69A10"
endchoice
choice
depends on BOARD_TYPE_LILYGO_T_DISPLAY_P4
prompt "Select the color format of the screen"
default SCREEN_PIXEL_FORMAT_RGB565
config SCREEN_PIXEL_FORMAT_RGB565
bool "RGB565"
config SCREEN_PIXEL_FORMAT_RGB888
bool "RGB888"
endchoice
choice ESP_S3_LCD_EV_Board_Version_TYPE
depends on BOARD_TYPE_ESP_S3_LCD_EV_Board
prompt "EV_BOARD Type"
default ESP_S3_LCD_EV_Board_1p4
help
开发板硬件版本型号选择
config ESP_S3_LCD_EV_Board_1p4
bool "乐鑫ESP32_S3_LCD_EV_Board-MB_V1.4"
config ESP_S3_LCD_EV_Board_1p5
@@ -278,55 +489,55 @@ choice ESP_S3_LCD_EV_Board_Version_TYPE
endchoice
choice DISPLAY_OLED_TYPE
depends on BOARD_TYPE_BREAD_COMPACT_WIFI || BOARD_TYPE_BREAD_COMPACT_ML307 || BOARD_TYPE_BREAD_COMPACT_ESP32
depends on BOARD_TYPE_BREAD_COMPACT_WIFI || BOARD_TYPE_BREAD_COMPACT_ML307 || BOARD_TYPE_BREAD_COMPACT_ESP32 || BOARD_TYPE_HU_087
prompt "OLED Type"
default OLED_SSD1306_128X32
help
OLED 屏幕类型选择
OLED Monochrome Display Type
config OLED_SSD1306_128X32
bool "SSD1306, 分辨率128*32"
bool "SSD1306 128*32"
config OLED_SSD1306_128X64
bool "SSD1306, 分辨率128*64"
bool "SSD1306 128*64"
config OLED_SH1106_128X64
bool "SH1106, 分辨率128*64"
bool "SH1106 128*64"
endchoice
choice DISPLAY_LCD_TYPE
depends on BOARD_TYPE_BREAD_COMPACT_WIFI_LCD || BOARD_TYPE_BREAD_COMPACT_ESP32_LCD || BOARD_TYPE_ESP32_CGC || BOARD_TYPE_ESP32P4_NANO || BOARD_TYPE_ESP32P4_WIFI6_Touch_LCD_XC
depends on BOARD_TYPE_BREAD_COMPACT_WIFI_LCD || BOARD_TYPE_BREAD_COMPACT_ESP32_LCD || BOARD_TYPE_CGC || BOARD_TYPE_WAVESHARE_P4_NANO || BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_XC || BOARD_TYPE_BREAD_COMPACT_WIFI_CAM
prompt "LCD Type"
default LCD_ST7789_240X320
help
屏幕类型选择
LCD Display Type
config LCD_ST7789_240X320
bool "ST7789, 分辨率240*320, IPS"
bool "ST7789 240*320, IPS"
config LCD_ST7789_240X320_NO_IPS
bool "ST7789, 分辨率240*320, IPS"
bool "ST7789 240*320, Non-IPS"
config LCD_ST7789_170X320
bool "ST7789, 分辨率170*320"
bool "ST7789 170*320"
config LCD_ST7789_172X320
bool "ST7789, 分辨率172*320"
bool "ST7789 172*320"
config LCD_ST7789_240X280
bool "ST7789, 分辨率240*280"
bool "ST7789 240*280"
config LCD_ST7789_240X240
bool "ST7789, 分辨率240*240"
bool "ST7789 240*240"
config LCD_ST7789_240X240_7PIN
bool "ST7789, 分辨率240*240, 7PIN"
bool "ST7789 240*240, 7PIN"
config LCD_ST7789_240X135
bool "ST7789, 分辨率240*135"
bool "ST7789 240*135"
config LCD_ST7735_128X160
bool "ST7735, 分辨率128*160"
bool "ST7735 128*160"
config LCD_ST7735_128X128
bool "ST7735, 分辨率128*128"
bool "ST7735 128*128"
config LCD_ST7796_320X480
bool "ST7796, 分辨率320*480 IPS"
bool "ST7796 320*480 IPS"
config LCD_ST7796_320X480_NO_IPS
bool "ST7796, 分辨率320*480, IPS"
bool "ST7796 320*480, Non-IPS"
config LCD_ILI9341_240X320
bool "ILI9341, 分辨率240*320"
bool "ILI9341 240*320"
config LCD_ILI9341_240X320_NO_IPS
bool "ILI9341, 分辨率240*320, IPS"
bool "ILI9341 240*320, Non-IPS"
config LCD_GC9A01_240X240
bool "GC9A01, 分辨率240*240, 圆屏"
bool "GC9A01 240*240 Circle"
config LCD_TYPE_800_1280_10_1_INCH
bool "Waveshare 101M-8001280-IPS-CT-K Display"
config LCD_TYPE_800_1280_10_1_INCH_A
@@ -336,84 +547,274 @@ choice DISPLAY_LCD_TYPE
config LCD_TYPE_720_720_4_INCH
bool "Waveshare ESP32-P4-WIFI6-Touch-LCD-4C with 720*720 4inch round display"
config LCD_CUSTOM
bool "自定义屏幕参数"
bool "Custom LCD (自定义屏幕参数)"
endchoice
choice DISPLAY_ESP32S3_KORVO2_V3
depends on BOARD_TYPE_ESP32S3_KORVO2_V3
depends on BOARD_TYPE_ESP_KORVO2_V3
prompt "ESP32S3_KORVO2_V3 LCD Type"
default LCD_ST7789
default ESP32S3_KORVO2_V3_LCD_ST7789
help
屏幕类型选择
config LCD_ST7789
bool "ST7789, 分辨率240*280"
config LCD_ILI9341
bool "ILI9341, 分辨率240*320"
LCD Display Type
config ESP32S3_KORVO2_V3_LCD_ST7789
bool "ST7789 240*280"
config ESP32S3_KORVO2_V3_LCD_ILI9341
bool "ILI9341 240*320"
endchoice
config USE_WECHAT_MESSAGE_STYLE
bool "Enable WeChat Message Style"
default n
choice DISPLAY_ESP32S3_AUDIO_BOARD
depends on BOARD_TYPE_WAVESHARE_S3_AUDIO_BOARD
prompt "ESP32S3_AUDIO_BOARD LCD Type"
default AUDIO_BOARD_LCD_JD9853
help
使用微信聊天界面风格
LCD Display Type
config AUDIO_BOARD_LCD_JD9853
bool "JD9853 320*172"
config AUDIO_BOARD_LCD_ST7789
bool "ST7789 240*320"
endchoice
config USE_ESP_WAKE_WORD
bool "Enable Wake Word Detection (without AFE)"
default n
depends on IDF_TARGET_ESP32C3 || IDF_TARGET_ESP32C5 || IDF_TARGET_ESP32
choice DISPLAY_STYLE
prompt "Select display style"
default USE_DEFAULT_MESSAGE_STYLE
help
支持 ESP32 C3 与 ESP32 C5增加ESP32支持需要开启PSRAM
Select display style for Xiaozhi device
config USE_AFE_WAKE_WORD
bool "Enable Wake Word Detection (AFE)"
config USE_DEFAULT_MESSAGE_STYLE
bool "Enable default message style"
config USE_WECHAT_MESSAGE_STYLE
bool "Enable WeChat Message Style"
config USE_EMOTE_MESSAGE_STYLE
bool "Emote animation style"
depends on BOARD_TYPE_ESP_BOX_3 || BOARD_TYPE_ECHOEAR || BOARD_TYPE_LICHUANG_DEV_S3 || BOARD_TYPE_ESP_SENSAIRSHUTTLE
endchoice
choice WAKE_WORD_TYPE
prompt "Wake Word Implementation Type"
default USE_AFE_WAKE_WORD if (IDF_TARGET_ESP32S3 || IDF_TARGET_ESP32P4) && SPIRAM
default WAKE_WORD_DISABLED
help
Choose the type of wake word implementation to use
config WAKE_WORD_DISABLED
bool "Disabled"
help
Disable wake word detection
config USE_ESP_WAKE_WORD
bool "Wakenet model without AFE"
depends on IDF_TARGET_ESP32C3 || IDF_TARGET_ESP32C5 || IDF_TARGET_ESP32C6 || (IDF_TARGET_ESP32 && SPIRAM)
help
Support ESP32 C3、ESP32 C5 与 ESP32 C6, and (ESP32 with PSRAM)
config USE_AFE_WAKE_WORD
bool "Wakenet model with AFE"
depends on (IDF_TARGET_ESP32S3 || IDF_TARGET_ESP32P4) && SPIRAM
help
Support AEC if available, requires ESP32 S3 and PSRAM
config USE_CUSTOM_WAKE_WORD
bool "Multinet model (Custom Wake Word)"
depends on (IDF_TARGET_ESP32S3 || IDF_TARGET_ESP32P4) && SPIRAM
help
Requires ESP32 S3 and PSRAM
endchoice
config CUSTOM_WAKE_WORD
string "Custom Wake Word"
default "xiao tu dou"
depends on USE_CUSTOM_WAKE_WORD
help
Custom Wake Word, use pinyin for Chinese, separated by spaces
config CUSTOM_WAKE_WORD_DISPLAY
string "Custom Wake Word Display"
default "小土豆"
depends on USE_CUSTOM_WAKE_WORD
help
Greeting sent to the server after wake word detection
config CUSTOM_WAKE_WORD_THRESHOLD
int "Custom Wake Word Threshold (%)"
default 20
range 1 99
depends on USE_CUSTOM_WAKE_WORD
help
Custom Wake Word Threshold, range 1-99, the smaller the more sensitive, default 20
config SEND_WAKE_WORD_DATA
bool "Send Wake Word Data"
default y
depends on (IDF_TARGET_ESP32S3 || IDF_TARGET_ESP32P4) && SPIRAM
depends on USE_AFE_WAKE_WORD || USE_CUSTOM_WAKE_WORD
help
需要 ESP32 S3 与 PSRAM 支持
Send wake word data to the server as the first message of the conversation and wait for response
config USE_AUDIO_PROCESSOR
bool "Enable Audio Noise Reduction"
default y
depends on (IDF_TARGET_ESP32S3 || IDF_TARGET_ESP32P4) && SPIRAM
help
需要 ESP32 S3 PSRAM 支持
Requires ESP32 S3 and PSRAM
config USE_DEVICE_AEC
bool "Enable Device-Side AEC"
default n
depends on USE_AUDIO_PROCESSOR && (BOARD_TYPE_ESP_BOX_3 || BOARD_TYPE_ESP_BOX || BOARD_TYPE_ESP_BOX_LITE || BOARD_TYPE_LICHUANG_DEV || BOARD_TYPE_ESP32S3_KORVO2_V3 || BOARD_TYPE_ESP32S3_Touch_AMOLED_1_75 || BOARD_TYPE_ESP32P4_WIFI6_Touch_LCD_4B || BOARD_TYPE_ESP32P4_WIFI6_Touch_LCD_XC)
depends on USE_AUDIO_PROCESSOR && (BOARD_TYPE_ESP_BOX_3 || BOARD_TYPE_ESP_BOX || BOARD_TYPE_ESP_BOX_LITE \
|| BOARD_TYPE_LICHUANG_DEV_S3 || BOARD_TYPE_LICHUANG_DEV_S3_BED_OPERATOR || BOARD_TYPE_ESP_KORVO2_V3 || BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_1_75 || BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_1_83\
|| BOARD_TYPE_WAVESHARE_S3_TOUCH_AMOLED_2_06 || BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_4B || BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_4B || BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_7B \
|| BOARD_TYPE_WAVESHARE_P4_WIFI6_TOUCH_LCD_XC || BOARD_TYPE_ESP_S3_LCD_EV_Board_2 || BOARD_TYPE_YUNLIAO_S3 \
|| BOARD_TYPE_ECHOEAR || BOARD_TYPE_WAVESHARE_S3_TOUCH_LCD_3_49)
help
因为性能不够,不建议和微信聊天界面风格同时开启
To work properly, device-side AEC requires a clean output reference path from the speaker signal and physical acoustic isolation between the microphone and speaker.
config USE_SERVER_AEC
bool "Enable Server-Side AEC (Unstable)"
default n
depends on USE_AUDIO_PROCESSOR
help
启用服务器端 AEC需要服务器支持
To work perperly, server-side AEC requires server support
config USE_AUDIO_DEBUGGER
bool "Enable Audio Debugger"
default n
help
启用音频调试功能通过UDP发送音频数据
Enable audio debugger, send audio data through UDP to the host machine
menu "WiFi Configuration Method"
help
WiFi Configuration Method Selection
config USE_HOTSPOT_WIFI_PROVISIONING
bool "Hotspot"
default y
help
Use WiFi Hotspot to transmit WiFi configuration data
config USE_ACOUSTIC_WIFI_PROVISIONING
bool "Acoustic"
help
Use audio signal to transmit WiFi configuration data
config USE_ESP_BLUFI_WIFI_PROVISIONING
bool "Esp Blufi"
help
Use esp blufi protocol to transmit WiFi configuration data
select BT_ENABLED
select BT_BLE_42_FEATURES_SUPPORTED
select BT_BLE_BLUFI_ENABLE
select MBEDTLS_DHM_C
endmenu
config AUDIO_DEBUG_UDP_SERVER
string "Audio Debug UDP Server Address"
default "192.168.2.100:8000"
depends on USE_AUDIO_DEBUGGER
help
UDP服务器地址,格式: IP:PORT用于接收音频调试数据
UDP server address, format: IP:PORT, used to receive audio debugging data
choice IOT_PROTOCOL
prompt "IoT Protocol"
default IOT_PROTOCOL_MCP
config RECEIVE_CUSTOM_MESSAGE
bool "Enable Custom Message Reception"
default n
help
IoT 协议,用于获取设备状态与发送控制指令
config IOT_PROTOCOL_MCP
bool "MCP 2024-11-05"
config IOT_PROTOCOL_XIAOZHI
bool "Xiaozhi IoT 1.0 (Deprecated)"
endchoice
Enable custom message reception, allow the device to receive custom messages from the server (preferably through the MQTT protocol)
menu "Camera Configuration"
depends on !IDF_TARGET_ESP32
comment "Warning: Please read the help text before modifying these settings."
config XIAOZHI_CAMERA_ALLOW_JPEG_INPUT
bool "Allow JPEG Input"
default n
help
Allow JPEG Input format for the camera.
This option may need to be enabled when using a USB camera.
Not currently supported when used simultaneously with XIAOZHI_ENABLE_ROTATE_CAMERA_IMAGE.
config XIAOZHI_ENABLE_HARDWARE_JPEG_ENCODER
bool "Enable Hardware JPEG Encoder"
default y
depends on SOC_JPEG_ENCODE_SUPPORTED
help
Use hardware JPEG encoder on ESP32-P4 to encode image to JPEG.
See https://docs.espressif.com/projects/esp-idf/en/stable/esp32p4/api-reference/peripherals/jpeg.html for more details.
config XIAOZHI_ENABLE_HARDWARE_JPEG_DECODER
bool "Enable Hardware JPEG Decoder"
default n
depends on SOC_JPEG_DECODE_SUPPORTED && XIAOZHI_CAMERA_ALLOW_JPEG_INPUT
help
Use hardware JPEG decoder on ESP32-P4 to decode JPEG to image.
See https://docs.espressif.com/projects/esp-idf/en/stable/esp32p4/api-reference/peripherals/jpeg.html for more details.
config XIAOZHI_ENABLE_CAMERA_DEBUG_MODE
bool "Enable Camera Debug Mode"
default n
help
Enable camera debug mode, print camera debug information to the console.
Only works on boards that support camera.
config XIAOZHI_ENABLE_CAMERA_ENDIANNESS_SWAP
bool "Enable software camera buffer endianness swapping"
default n
depends on !CAMERA_SENSOR_SWAP_PIXEL_BYTE_ORDER
help
This option treats the camera buffer as a uint16_t[] array and performs byte-swapping (endianness conversion) on each element.
Should only be modified by development board integration engineers.
**Incorrect usage may result in incorrect image colors!**
ATTENTION: If the option CAMERA_SENSOR_SWAP_PIXEL_BYTE_ORDER is available for your sensor, please use that instead.
menuconfig XIAOZHI_ENABLE_ROTATE_CAMERA_IMAGE
bool "Enable Camera Image Rotation"
default n
depends on !XIAOZHI_CAMERA_ALLOW_JPEG_INPUT
help
Enable camera image rotation, rotate the camera image to the correct orientation.
- On ESP32-P4, rotation is handled by PPA hardware.
- On other chips, rotation is done in software with performance cost.
- For 180° rotation, use HFlip + VFlip instead of this option.
Not currently supported when used simultaneously with XIAOZHI_CAMERA_ALLOW_JPEG_INPUT.
if XIAOZHI_ENABLE_ROTATE_CAMERA_IMAGE
choice XIAOZHI_CAMERA_IMAGE_ROTATION_ANGLE
prompt "Camera Image Rotation Angle (clockwise)"
default XIAOZHI_CAMERA_IMAGE_ROTATION_ANGLE_90
help
Camera image rotation angle.
config XIAOZHI_CAMERA_IMAGE_ROTATION_ANGLE_90
bool "90°"
config XIAOZHI_CAMERA_IMAGE_ROTATION_ANGLE_270
bool "270°"
comment "For 180° rotation, use HFlip + VFlip instead of this option"
endchoice
endif
endmenu
menu "TAIJIPAI_S3_CONFIG"
depends on BOARD_TYPE_TAIJI_PI_S3
choice I2S_TYPE_TAIJIPI_S3
prompt "taiji-pi-S3 I2S Type"
default TAIJIPAI_I2S_TYPE_STD
help
I2S 类型选择
config TAIJIPAI_I2S_TYPE_STD
bool "I2S Type STD"
config TAIJIPAI_I2S_TYPE_PDM
bool "I2S Type PDM"
endchoice
config I2S_USE_2SLOT
bool "Enable I2S 2 Slot"
default y
help
启动双声道
endmenu
endmenu

File diff suppressed because it is too large Load Diff

View File

@@ -8,25 +8,30 @@
#include <string>
#include <mutex>
#include <list>
#include <vector>
#include <condition_variable>
#include <deque>
#include <memory>
#include <opus_encoder.h>
#include <opus_decoder.h>
#include <opus_resampler.h>
#include "protocol.h"
#include "ota.h"
#include "background_task.h"
#include "audio_processor.h"
#include "wake_word.h"
#include "audio_debugger.h"
#include "audio_service.h"
#include "device_state.h"
#include "device_state_machine.h"
// Main event bits
#define MAIN_EVENT_SCHEDULE (1 << 0)
#define MAIN_EVENT_SEND_AUDIO (1 << 1)
#define MAIN_EVENT_WAKE_WORD_DETECTED (1 << 2)
#define MAIN_EVENT_VAD_CHANGE (1 << 3)
#define MAIN_EVENT_ERROR (1 << 4)
#define MAIN_EVENT_ACTIVATION_DONE (1 << 5)
#define MAIN_EVENT_CLOCK_TICK (1 << 6)
#define MAIN_EVENT_NETWORK_CONNECTED (1 << 7)
#define MAIN_EVENT_NETWORK_DISCONNECTED (1 << 8)
#define MAIN_EVENT_TOGGLE_CHAT (1 << 9)
#define MAIN_EVENT_START_LISTENING (1 << 10)
#define MAIN_EVENT_STOP_LISTENING (1 << 11)
#define MAIN_EVENT_STATE_CHANGED (1 << 12)
#define SCHEDULE_EVENT (1 << 0)
#define SEND_AUDIO_EVENT (1 << 1)
#define CHECK_NEW_VERSION_DONE_EVENT (1 << 2)
enum AecMode {
kAecOff,
@@ -34,106 +39,148 @@ enum AecMode {
kAecOnServerSide,
};
enum DeviceState {
kDeviceStateUnknown,
kDeviceStateStarting,
kDeviceStateWifiConfiguring,
kDeviceStateIdle,
kDeviceStateConnecting,
kDeviceStateListening,
kDeviceStateSpeaking,
kDeviceStateUpgrading,
kDeviceStateActivating,
kDeviceStateFatalError
};
#define OPUS_FRAME_DURATION_MS 60
#define MAX_AUDIO_PACKETS_IN_QUEUE (2400 / OPUS_FRAME_DURATION_MS)
class Application {
public:
static Application& GetInstance() {
static Application instance;
return instance;
}
// 删除拷贝构造函数和赋值运算符
// Delete copy constructor and assignment operator
Application(const Application&) = delete;
Application& operator=(const Application&) = delete;
void Start();
DeviceState GetDeviceState() const { return device_state_; }
bool IsVoiceDetected() const { return voice_detected_; }
void Schedule(std::function<void()> callback);
void SetDeviceState(DeviceState state);
/**
* Initialize the application
* This sets up display, audio, network callbacks, etc.
* Network connection starts asynchronously.
*/
void Initialize();
/**
* Run the main event loop
* This function runs in the main task and never returns.
* It handles all events including network, state changes, and user interactions.
*/
void Run();
DeviceState GetDeviceState() const { return state_machine_.GetState(); }
bool IsVoiceDetected() const { return audio_service_.IsVoiceDetected(); }
/**
* Request state transition
* Returns true if transition was successful
*/
bool SetDeviceState(DeviceState state);
/**
* Schedule a callback to be executed in the main task
*/
void Schedule(std::function<void()>&& callback);
/**
* Alert with status, message, emotion and optional sound
*/
void Alert(const char* status, const char* message, const char* emotion = "", const std::string_view& sound = "");
void DismissAlert();
void AbortSpeaking(AbortReason reason);
/**
* Toggle chat state (event-based, thread-safe)
* Sends MAIN_EVENT_TOGGLE_CHAT to be handled in Run()
*/
void ToggleChatState();
/**
* Start listening (event-based, thread-safe)
* Sends MAIN_EVENT_START_LISTENING to be handled in Run()
*/
void StartListening();
/**
* Stop listening (event-based, thread-safe)
* Sends MAIN_EVENT_STOP_LISTENING to be handled in Run()
*/
void StopListening();
void UpdateIotStates();
void Reboot();
void WakeWordInvoke(const std::string& wake_word);
void PlaySound(const std::string_view& sound);
bool UpgradeFirmware(const std::string& url, const std::string& version = "");
bool CanEnterSleepMode();
void SendMcpMessage(const std::string& payload);
void SetAecMode(AecMode mode);
AecMode GetAecMode() const { return aec_mode_; }
BackgroundTask* GetBackgroundTask() const { return background_task_; }
void PlaySound(const std::string_view& sound);
AudioService& GetAudioService() { return audio_service_; }
/**
* Reset protocol resources (thread-safe)
* Can be called from any task to release resources allocated after network connected
* This includes closing audio channel, resetting protocol and ota objects
*/
void ResetProtocol();
private:
Application();
~Application();
std::unique_ptr<WakeWord> wake_word_;
std::unique_ptr<AudioProcessor> audio_processor_;
std::unique_ptr<AudioDebugger> audio_debugger_;
Ota ota_;
std::mutex mutex_;
std::list<std::function<void()>> main_tasks_;
std::deque<std::function<void()>> main_tasks_;
std::unique_ptr<Protocol> protocol_;
EventGroupHandle_t event_group_ = nullptr;
esp_timer_handle_t clock_timer_handle_ = nullptr;
volatile DeviceState device_state_ = kDeviceStateUnknown;
DeviceStateMachine state_machine_;
ListeningMode listening_mode_ = kListeningModeAutoStop;
AecMode aec_mode_ = kAecOff;
std::string last_error_message_;
AudioService audio_service_;
std::unique_ptr<Ota> ota_;
bool has_server_time_ = false;
bool aborted_ = false;
bool voice_detected_ = false;
bool busy_decoding_audio_ = false;
bool assets_version_checked_ = false;
bool play_popup_on_listening_ = false; // Flag to play popup sound after state changes to listening
int clock_ticks_ = 0;
TaskHandle_t check_new_version_task_handle_ = nullptr;
TaskHandle_t activation_task_handle_ = nullptr;
// Audio encode / decode
TaskHandle_t audio_loop_task_handle_ = nullptr;
BackgroundTask* background_task_ = nullptr;
std::chrono::steady_clock::time_point last_output_time_;
std::list<AudioStreamPacket> audio_send_queue_;
std::list<AudioStreamPacket> audio_decode_queue_;
std::condition_variable audio_decode_cv_;
// 新增用于维护音频包的timestamp队列
std::list<uint32_t> timestamp_queue_;
std::mutex timestamp_mutex_;
// Event handlers
void HandleStateChangedEvent();
void HandleToggleChatEvent();
void HandleStartListeningEvent();
void HandleStopListeningEvent();
void HandleNetworkConnectedEvent();
void HandleNetworkDisconnectedEvent();
void HandleActivationDoneEvent();
void HandleWakeWordDetectedEvent();
std::unique_ptr<OpusEncoderWrapper> opus_encoder_;
std::unique_ptr<OpusDecoderWrapper> opus_decoder_;
// Activation task (runs in background)
void ActivationTask();
OpusResampler input_resampler_;
OpusResampler reference_resampler_;
OpusResampler output_resampler_;
void MainEventLoop();
void OnAudioInput();
void OnAudioOutput();
bool ReadAudio(std::vector<int16_t>& data, int sample_rate, int samples);
void ResetDecoder();
void SetDecodeSampleRate(int sample_rate, int frame_duration);
// Helper methods
void CheckAssetsVersion();
void CheckNewVersion();
void ShowActivationCode();
void OnClockTimer();
void InitializeProtocol();
void ShowActivationCode(const std::string& code, const std::string& message);
void SetListeningMode(ListeningMode mode);
void AudioLoop();
// State change handler called by state machine
void OnStateChanged(DeviceState old_state, DeviceState new_state);
};
class TaskPriorityReset {
public:
TaskPriorityReset(BaseType_t priority) {
original_priority_ = uxTaskPriorityGet(NULL);
vTaskPrioritySet(NULL, priority);
}
~TaskPriorityReset() {
vTaskPrioritySet(NULL, original_priority_);
}
private:
BaseType_t original_priority_;
};
#endif // _APPLICATION_H_

532
main/assets.cc Normal file
View File

@@ -0,0 +1,532 @@
#include "assets.h"
#include "board.h"
#include "display.h"
#include "application.h"
#include "lvgl_theme.h"
#include "emote_display.h"
#ifdef HAVE_LVGL
#include "display/lcd_display.h"
#endif
#include <esp_log.h>
#include <spi_flash_mmap.h>
#include <esp_timer.h>
#include <cbin_font.h>
#define TAG "Assets"
struct mmap_assets_table {
char asset_name[32]; /*!< Name of the asset */
uint32_t asset_size; /*!< Size of the asset */
uint32_t asset_offset; /*!< Offset of the asset */
uint16_t asset_width; /*!< Width of the asset */
uint16_t asset_height; /*!< Height of the asset */
};
Assets::Assets() {
// Initialize the partition
InitializePartition();
}
Assets::~Assets() {
if (mmap_handle_ != 0) {
esp_partition_munmap(mmap_handle_);
}
}
uint32_t Assets::CalculateChecksum(const char* data, uint32_t length) {
uint32_t checksum = 0;
for (uint32_t i = 0; i < length; i++) {
checksum += data[i];
}
return checksum & 0xFFFF;
}
bool Assets::InitializePartition() {
partition_valid_ = false;
checksum_valid_ = false;
assets_.clear();
partition_ = esp_partition_find_first(ESP_PARTITION_TYPE_ANY, ESP_PARTITION_SUBTYPE_ANY, "assets");
if (partition_ == nullptr) {
ESP_LOGI(TAG, "No assets partition found");
return false;
}
int free_pages = spi_flash_mmap_get_free_pages(SPI_FLASH_MMAP_DATA);
uint32_t storage_size = free_pages * 64 * 1024;
ESP_LOGI(TAG, "The storage free size is %ld KB", storage_size / 1024);
ESP_LOGI(TAG, "The partition size is %ld KB", partition_->size / 1024);
if (storage_size < partition_->size) {
ESP_LOGE(TAG, "The free size %ld KB is less than assets partition required %ld KB", storage_size / 1024, partition_->size / 1024);
return false;
}
esp_err_t err = esp_partition_mmap(partition_, 0, partition_->size, ESP_PARTITION_MMAP_DATA, (const void**)&mmap_root_, &mmap_handle_);
if (err != ESP_OK) {
ESP_LOGE(TAG, "Failed to mmap assets partition: %s", esp_err_to_name(err));
return false;
}
partition_valid_ = true;
uint32_t stored_files = *(uint32_t*)(mmap_root_ + 0);
uint32_t stored_chksum = *(uint32_t*)(mmap_root_ + 4);
uint32_t stored_len = *(uint32_t*)(mmap_root_ + 8);
if (stored_len > partition_->size - 12) {
ESP_LOGD(TAG, "The stored_len (0x%lx) is greater than the partition size (0x%lx) - 12", stored_len, partition_->size);
return false;
}
auto start_time = esp_timer_get_time();
uint32_t calculated_checksum = CalculateChecksum(mmap_root_ + 12, stored_len);
auto end_time = esp_timer_get_time();
ESP_LOGI(TAG, "The checksum calculation time is %d ms", int((end_time - start_time) / 1000));
if (calculated_checksum != stored_chksum) {
ESP_LOGE(TAG, "The calculated checksum (0x%lx) does not match the stored checksum (0x%lx)", calculated_checksum, stored_chksum);
return false;
}
checksum_valid_ = true;
for (uint32_t i = 0; i < stored_files; i++) {
auto item = (const mmap_assets_table*)(mmap_root_ + 12 + i * sizeof(mmap_assets_table));
auto asset = Asset{
.size = static_cast<size_t>(item->asset_size),
.offset = static_cast<size_t>(12 + sizeof(mmap_assets_table) * stored_files + item->asset_offset)
};
assets_[item->asset_name] = asset;
}
return checksum_valid_;
}
bool Assets::Apply() {
void* ptr = nullptr;
size_t size = 0;
if (!GetAssetData("index.json", ptr, size)) {
ESP_LOGE(TAG, "The index.json file is not found");
return false;
}
cJSON* root = cJSON_ParseWithLength(static_cast<char*>(ptr), size);
if (root == nullptr) {
ESP_LOGE(TAG, "The index.json file is not valid");
return false;
}
cJSON* version = cJSON_GetObjectItem(root, "version");
if (cJSON_IsNumber(version)) {
if (version->valuedouble > 1) {
ESP_LOGE(TAG, "The assets version %d is not supported, please upgrade the firmware", version->valueint);
return false;
}
}
cJSON* srmodels = cJSON_GetObjectItem(root, "srmodels");
if (cJSON_IsString(srmodels)) {
std::string srmodels_file = srmodels->valuestring;
if (GetAssetData(srmodels_file, ptr, size)) {
if (models_list_ != nullptr) {
esp_srmodel_deinit(models_list_);
models_list_ = nullptr;
}
models_list_ = srmodel_load(static_cast<uint8_t*>(ptr));
if (models_list_ != nullptr) {
auto& app = Application::GetInstance();
app.GetAudioService().SetModelsList(models_list_);
} else {
ESP_LOGE(TAG, "Failed to load srmodels.bin");
}
} else {
ESP_LOGE(TAG, "The srmodels file %s is not found", srmodels_file.c_str());
}
}
#ifdef HAVE_LVGL
auto& theme_manager = LvglThemeManager::GetInstance();
auto light_theme = theme_manager.GetTheme("light");
auto dark_theme = theme_manager.GetTheme("dark");
cJSON* font = cJSON_GetObjectItem(root, "text_font");
if (cJSON_IsString(font)) {
std::string fonts_text_file = font->valuestring;
if (GetAssetData(fonts_text_file, ptr, size)) {
auto text_font = std::make_shared<LvglCBinFont>(ptr);
if (text_font->font() == nullptr) {
ESP_LOGE(TAG, "Failed to load fonts.bin");
return false;
}
if (light_theme != nullptr) {
light_theme->set_text_font(text_font);
}
if (dark_theme != nullptr) {
dark_theme->set_text_font(text_font);
}
} else {
ESP_LOGE(TAG, "The font file %s is not found", fonts_text_file.c_str());
}
}
cJSON* emoji_collection = cJSON_GetObjectItem(root, "emoji_collection");
if (cJSON_IsArray(emoji_collection)) {
auto custom_emoji_collection = std::make_shared<EmojiCollection>();
int emoji_count = cJSON_GetArraySize(emoji_collection);
for (int i = 0; i < emoji_count; i++) {
cJSON* emoji = cJSON_GetArrayItem(emoji_collection, i);
if (cJSON_IsObject(emoji)) {
cJSON* name = cJSON_GetObjectItem(emoji, "name");
cJSON* file = cJSON_GetObjectItem(emoji, "file");
cJSON* eaf = cJSON_GetObjectItem(emoji, "eaf");
if (cJSON_IsString(name) && cJSON_IsString(file) && (NULL== eaf)) {
if (!GetAssetData(file->valuestring, ptr, size)) {
ESP_LOGE(TAG, "Emoji %s image file %s is not found", name->valuestring, file->valuestring);
continue;
}
custom_emoji_collection->AddEmoji(name->valuestring, new LvglRawImage(ptr, size));
}
}
}
if (light_theme != nullptr) {
light_theme->set_emoji_collection(custom_emoji_collection);
}
if (dark_theme != nullptr) {
dark_theme->set_emoji_collection(custom_emoji_collection);
}
}
cJSON* skin = cJSON_GetObjectItem(root, "skin");
if (cJSON_IsObject(skin)) {
cJSON* light_skin = cJSON_GetObjectItem(skin, "light");
if (cJSON_IsObject(light_skin) && light_theme != nullptr) {
cJSON* text_color = cJSON_GetObjectItem(light_skin, "text_color");
cJSON* background_color = cJSON_GetObjectItem(light_skin, "background_color");
cJSON* background_image = cJSON_GetObjectItem(light_skin, "background_image");
if (cJSON_IsString(text_color)) {
light_theme->set_text_color(LvglTheme::ParseColor(text_color->valuestring));
}
if (cJSON_IsString(background_color)) {
light_theme->set_background_color(LvglTheme::ParseColor(background_color->valuestring));
light_theme->set_chat_background_color(LvglTheme::ParseColor(background_color->valuestring));
}
if (cJSON_IsString(background_image)) {
if (!GetAssetData(background_image->valuestring, ptr, size)) {
ESP_LOGE(TAG, "The background image file %s is not found", background_image->valuestring);
return false;
}
auto background_image = std::make_shared<LvglCBinImage>(ptr);
light_theme->set_background_image(background_image);
}
}
cJSON* dark_skin = cJSON_GetObjectItem(skin, "dark");
if (cJSON_IsObject(dark_skin) && dark_theme != nullptr) {
cJSON* text_color = cJSON_GetObjectItem(dark_skin, "text_color");
cJSON* background_color = cJSON_GetObjectItem(dark_skin, "background_color");
cJSON* background_image = cJSON_GetObjectItem(dark_skin, "background_image");
if (cJSON_IsString(text_color)) {
dark_theme->set_text_color(LvglTheme::ParseColor(text_color->valuestring));
}
if (cJSON_IsString(background_color)) {
dark_theme->set_background_color(LvglTheme::ParseColor(background_color->valuestring));
dark_theme->set_chat_background_color(LvglTheme::ParseColor(background_color->valuestring));
}
if (cJSON_IsString(background_image)) {
if (!GetAssetData(background_image->valuestring, ptr, size)) {
ESP_LOGE(TAG, "The background image file %s is not found", background_image->valuestring);
return false;
}
auto background_image = std::make_shared<LvglCBinImage>(ptr);
dark_theme->set_background_image(background_image);
}
}
}
auto display = Board::GetInstance().GetDisplay();
ESP_LOGI(TAG, "Refreshing display theme...");
auto current_theme = display->GetTheme();
if (current_theme != nullptr) {
display->SetTheme(current_theme);
}
// Parse hide_subtitle configuration
cJSON* hide_subtitle = cJSON_GetObjectItem(root, "hide_subtitle");
if (cJSON_IsBool(hide_subtitle)) {
bool hide = cJSON_IsTrue(hide_subtitle);
auto lcd_display = dynamic_cast<LcdDisplay*>(display);
if (lcd_display != nullptr) {
lcd_display->SetHideSubtitle(hide);
ESP_LOGI(TAG, "Set hide_subtitle to %s", hide ? "true" : "false");
}
}
#elif defined(CONFIG_USE_EMOTE_MESSAGE_STYLE)
auto &board = Board::GetInstance();
auto display = board.GetDisplay();
auto emote_display = dynamic_cast<emote::EmoteDisplay*>(display);
cJSON* font = cJSON_GetObjectItem(root, "text_font");
if (cJSON_IsString(font)) {
std::string fonts_text_file = font->valuestring;
if (GetAssetData(fonts_text_file, ptr, size)) {
auto text_font = std::make_shared<LvglCBinFont>(ptr);
if (text_font->font() == nullptr) {
ESP_LOGE(TAG, "Failed to load fonts.bin");
return false;
}
if (emote_display) {
emote_display->AddTextFont(text_font);
}
} else {
ESP_LOGE(TAG, "The font file %s is not found", fonts_text_file.c_str());
}
}
cJSON* emoji_collection = cJSON_GetObjectItem(root, "emoji_collection");
if (cJSON_IsArray(emoji_collection)) {
int emoji_count = cJSON_GetArraySize(emoji_collection);
if (emote_display) {
for (int i = 0; i < emoji_count; i++) {
cJSON* icon = cJSON_GetArrayItem(emoji_collection, i);
if (cJSON_IsObject(icon)) {
cJSON* name = cJSON_GetObjectItem(icon, "name");
cJSON* file = cJSON_GetObjectItem(icon, "file");
if (cJSON_IsString(name) && cJSON_IsString(file)) {
if (GetAssetData(file->valuestring, ptr, size)) {
cJSON* eaf = cJSON_GetObjectItem(icon, "eaf");
bool lack_value = false;
bool loop_value = false;
int fps_value = 0;
if (cJSON_IsObject(eaf)) {
cJSON* lack = cJSON_GetObjectItem(eaf, "lack");
cJSON* loop = cJSON_GetObjectItem(eaf, "loop");
cJSON* fps = cJSON_GetObjectItem(eaf, "fps");
lack_value = lack ? cJSON_IsTrue(lack) : false;
loop_value = loop ? cJSON_IsTrue(loop) : false;
fps_value = fps ? fps->valueint : 0;
emote_display->AddEmojiData(name->valuestring, ptr, size,
static_cast<uint8_t>(fps_value),
loop_value, lack_value);
}
} else {
ESP_LOGE(TAG, "Emoji \"%10s\" image file %s is not found", name->valuestring, file->valuestring);
}
}
}
}
}
}
cJSON* icon_collection = cJSON_GetObjectItem(root, "icon_collection");
if (cJSON_IsArray(icon_collection)) {
if (emote_display) {
int icon_count = cJSON_GetArraySize(icon_collection);
for (int i = 0; i < icon_count; i++) {
cJSON* icon = cJSON_GetArrayItem(icon_collection, i);
if (cJSON_IsObject(icon)) {
cJSON* name = cJSON_GetObjectItem(icon, "name");
cJSON* file = cJSON_GetObjectItem(icon, "file");
if (cJSON_IsString(name) && cJSON_IsString(file)) {
if (GetAssetData(file->valuestring, ptr, size)) {
emote_display->AddIconData(name->valuestring, ptr, size);
} else {
ESP_LOGE(TAG, "Icon \"%10s\" image file %s is not found", name->valuestring, file->valuestring);
}
}
}
}
}
}
cJSON* layout_json = cJSON_GetObjectItem(root, "layout");
if (cJSON_IsArray(layout_json)) {
int layout_count = cJSON_GetArraySize(layout_json);
for (int i = 0; i < layout_count; i++) {
cJSON* layout_item = cJSON_GetArrayItem(layout_json, i);
if (cJSON_IsObject(layout_item)) {
cJSON* name = cJSON_GetObjectItem(layout_item, "name");
cJSON* align = cJSON_GetObjectItem(layout_item, "align");
cJSON* x = cJSON_GetObjectItem(layout_item, "x");
cJSON* y = cJSON_GetObjectItem(layout_item, "y");
cJSON* width = cJSON_GetObjectItem(layout_item, "width");
cJSON* height = cJSON_GetObjectItem(layout_item, "height");
if (cJSON_IsString(name) && cJSON_IsString(align) && cJSON_IsNumber(x) && cJSON_IsNumber(y)) {
int width_val = cJSON_IsNumber(width) ? width->valueint : 0;
int height_val = cJSON_IsNumber(height) ? height->valueint : 0;
if (emote_display) {
emote_display->AddLayoutData(name->valuestring, align->valuestring,
x->valueint, y->valueint, width_val, height_val);
}
} else {
ESP_LOGW(TAG, "Invalid layout item %d: missing required fields", i);
}
}
}
}
#endif
cJSON_Delete(root);
return true;
}
bool Assets::Download(std::string url, std::function<void(int progress, size_t speed)> progress_callback) {
ESP_LOGI(TAG, "Downloading new version of assets from %s", url.c_str());
// 取消当前资源分区的内存映射
if (mmap_handle_ != 0) {
esp_partition_munmap(mmap_handle_);
mmap_handle_ = 0;
mmap_root_ = nullptr;
}
checksum_valid_ = false;
assets_.clear();
// 下载新的资源文件
auto network = Board::GetInstance().GetNetwork();
auto http = network->CreateHttp(0);
if (!http->Open("GET", url)) {
ESP_LOGE(TAG, "Failed to open HTTP connection");
return false;
}
if (http->GetStatusCode() != 200) {
ESP_LOGE(TAG, "Failed to get assets, status code: %d", http->GetStatusCode());
return false;
}
size_t content_length = http->GetBodyLength();
if (content_length == 0) {
ESP_LOGE(TAG, "Failed to get content length");
return false;
}
if (content_length > partition_->size) {
ESP_LOGE(TAG, "Assets file size (%u) is larger than partition size (%lu)", content_length, partition_->size);
return false;
}
// 定义扇区大小为4KBESP32的标准扇区大小
const size_t SECTOR_SIZE = esp_partition_get_main_flash_sector_size();
// 计算需要擦除的扇区数量
size_t sectors_to_erase = (content_length + SECTOR_SIZE - 1) / SECTOR_SIZE; // 向上取整
size_t total_erase_size = sectors_to_erase * SECTOR_SIZE;
ESP_LOGI(TAG, "Sector size: %u, content length: %u, sectors to erase: %u, total erase size: %u",
SECTOR_SIZE, content_length, sectors_to_erase, total_erase_size);
// 写入新的资源文件到分区一边erase一边写入
char buffer[512];
size_t total_written = 0;
size_t recent_written = 0;
size_t current_sector = 0;
auto last_calc_time = esp_timer_get_time();
while (true) {
int ret = http->Read(buffer, sizeof(buffer));
if (ret < 0) {
ESP_LOGE(TAG, "Failed to read HTTP data: %s", esp_err_to_name(ret));
return false;
}
if (ret == 0) {
break;
}
// 检查是否需要擦除新的扇区
size_t write_end_offset = total_written + ret;
size_t needed_sectors = (write_end_offset + SECTOR_SIZE - 1) / SECTOR_SIZE;
// 擦除需要的新扇区
while (current_sector < needed_sectors) {
size_t sector_start = current_sector * SECTOR_SIZE;
size_t sector_end = (current_sector + 1) * SECTOR_SIZE;
// 确保擦除范围不超过分区大小
if (sector_end > partition_->size) {
ESP_LOGE(TAG, "Sector end (%u) exceeds partition size (%lu)", sector_end, partition_->size);
return false;
}
ESP_LOGD(TAG, "Erasing sector %u (offset: %u, size: %u)", current_sector, sector_start, SECTOR_SIZE);
esp_err_t err = esp_partition_erase_range(partition_, sector_start, SECTOR_SIZE);
if (err != ESP_OK) {
ESP_LOGE(TAG, "Failed to erase sector %u at offset %u: %s", current_sector, sector_start, esp_err_to_name(err));
return false;
}
current_sector++;
}
// 写入数据到分区
esp_err_t err = esp_partition_write(partition_, total_written, buffer, ret);
if (err != ESP_OK) {
ESP_LOGE(TAG, "Failed to write to assets partition at offset %u: %s", total_written, esp_err_to_name(err));
return false;
}
total_written += ret;
recent_written += ret;
// 计算进度和速度
if (esp_timer_get_time() - last_calc_time >= 1000000 || total_written == content_length || ret == 0) {
size_t progress = total_written * 100 / content_length;
size_t speed = recent_written; // 每秒的字节数
ESP_LOGI(TAG, "Progress: %u%% (%u/%u), Speed: %u B/s, Sectors erased: %u",
progress, total_written, content_length, speed, current_sector);
if (progress_callback) {
progress_callback(progress, speed);
}
last_calc_time = esp_timer_get_time();
recent_written = 0; // 重置最近写入的字节数
}
}
http->Close();
if (total_written != content_length) {
ESP_LOGE(TAG, "Downloaded size (%u) does not match expected size (%u)", total_written, content_length);
return false;
}
ESP_LOGI(TAG, "Assets download completed, total written: %u bytes, total sectors erased: %u",
total_written, current_sector);
// 重新初始化资源分区
if (!InitializePartition()) {
ESP_LOGE(TAG, "Failed to re-initialize assets partition");
return false;
}
return true;
}
bool Assets::GetAssetData(const std::string& name, void*& ptr, size_t& size) {
auto asset = assets_.find(name);
if (asset == assets_.end()) {
return false;
}
auto data = (const char*)(mmap_root_ + asset->second.offset);
if (data[0] != 'Z' || data[1] != 'Z') {
ESP_LOGE(TAG, "The asset %s is not valid with magic %02x%02x", name.c_str(), data[0], data[1]);
return false;
}
ptr = static_cast<void*>(const_cast<char*>(data + 2));
size = asset->second.size;
return true;
}

52
main/assets.h Normal file
View File

@@ -0,0 +1,52 @@
#ifndef ASSETS_H
#define ASSETS_H
#include <map>
#include <string>
#include <functional>
#include <cJSON.h>
#include <esp_partition.h>
#include <model_path.h>
struct Asset {
size_t size;
size_t offset;
};
class Assets {
public:
static Assets& GetInstance() {
static Assets instance;
return instance;
}
~Assets();
bool Download(std::string url, std::function<void(int progress, size_t speed)> progress_callback);
bool Apply();
bool GetAssetData(const std::string& name, void*& ptr, size_t& size);
inline bool partition_valid() const { return partition_valid_; }
inline bool checksum_valid() const { return checksum_valid_; }
inline std::string default_assets_url() const { return default_assets_url_; }
private:
Assets();
Assets(const Assets&) = delete;
Assets& operator=(const Assets&) = delete;
bool InitializePartition();
uint32_t CalculateChecksum(const char* data, uint32_t length);
const esp_partition_t* partition_ = nullptr;
esp_partition_mmap_handle_t mmap_handle_ = 0;
const char* mmap_root_ = nullptr;
bool partition_valid_ = false;
bool checksum_valid_ = false;
std::string default_assets_url_;
srmodel_list_t* models_list_ = nullptr;
std::map<std::string, Asset> assets_;
};
#endif

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,55 @@
{
"language": {
"type": "ar-SA"
},
"strings": {
"WARNING": "تحذير",
"INFO": "معلومات",
"ERROR": "خطأ",
"VERSION": "الإصدار ",
"LOADING_PROTOCOL": "الاتصال بالخادم...",
"INITIALIZING": "التهيئة...",
"PIN_ERROR": "يرجى إدخال بطاقة SIM",
"REG_ERROR": "لا يمكن الوصول إلى الشبكة، يرجى التحقق من حالة بطاقة البيانات",
"DETECTING_MODULE": "اكتشاف الوحدة...",
"REGISTERING_NETWORK": "انتظار الشبكة...",
"CHECKING_NEW_VERSION": "فحص الإصدار الجديد...",
"CHECK_NEW_VERSION_FAILED": "فشل فحص الإصدار الجديد، سيتم المحاولة خلال %d ثانية: %s",
"SWITCH_TO_WIFI_NETWORK": "التبديل إلى Wi-Fi...",
"SWITCH_TO_4G_NETWORK": "التبديل إلى 4G...",
"STANDBY": "في الانتظار",
"CONNECT_TO": "الاتصال بـ ",
"CONNECTING": "جاري الاتصال...",
"CONNECTED_TO": "متصل بـ ",
"LISTENING": "الاستماع...",
"SPEAKING": "التحدث...",
"SERVER_NOT_FOUND": "البحث عن خدمة متاحة",
"SERVER_NOT_CONNECTED": "لا يمكن الاتصال بالخدمة، يرجى المحاولة لاحقاً",
"SERVER_TIMEOUT": "انتهت مهلة الاستجابة",
"SERVER_ERROR": "فشل الإرسال، يرجى التحقق من الشبكة",
"CONNECT_TO_HOTSPOT": "اتصل الهاتف بنقطة الاتصال ",
"ACCESS_VIA_BROWSER": "،الوصول عبر المتصفح ",
"WIFI_CONFIG_MODE": "وضع تكوين الشبكة",
"ENTERING_WIFI_CONFIG_MODE": "الدخول في وضع تكوين الشبكة...",
"SCANNING_WIFI": "فحص Wi-Fi...",
"NEW_VERSION": "إصدار جديد ",
"OTA_UPGRADE": "تحديث OTA",
"UPGRADING": "تحديث النظام...",
"UPGRADE_FAILED": "فشل التحديث",
"ACTIVATION": "تفعيل الجهاز",
"BATTERY_LOW": "البطارية منخفضة",
"BATTERY_CHARGING": "جاري الشحن",
"BATTERY_FULL": "البطارية ممتلئة",
"BATTERY_NEED_CHARGE": "البطارية منخفضة، يرجى الشحن",
"VOLUME": "الصوت ",
"MUTED": "صامت",
"MAX_VOLUME": "أقصى صوت",
"RTC_MODE_OFF": "AEC مُوقف",
"RTC_MODE_ON": "AEC مُشغل",
"DOWNLOAD_ASSETS_FAILED": "فشل في تنزيل الموارد",
"LOADING_ASSETS": "جاري تحميل الموارد...",
"PLEASE_WAIT": "يرجى الانتظار...",
"FOUND_NEW_ASSETS": "تم العثور على موارد جديدة: %s",
"HELLO_MY_FRIEND": "مرحباً، صديقي!"
}
}

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@@ -0,0 +1,57 @@
{
"language": {
"type": "bg-BG"
},
"strings": {
"WARNING": "Предупреждение",
"INFO": "Информация",
"ERROR": "Грешка",
"VERSION": "Версия ",
"LOADING_PROTOCOL": "Влизане в системата...",
"INITIALIZING": "Инициализация...",
"PIN_ERROR": "Моля, поставете SIM карта",
"REG_ERROR": "Не може да се осъществи достъп до мрежата, моля проверете статуса на SIM картата",
"DETECTING_MODULE": "Откриване на модул...",
"REGISTERING_NETWORK": "Изчакване на мрежата...",
"CHECKING_NEW_VERSION": "Проверка за нова версия...",
"CHECK_NEW_VERSION_FAILED": "Проверката за нова версия е неуспешна, ще се опита отново след %d секунди: %s",
"SWITCH_TO_WIFI_NETWORK": "Превключване към Wi-Fi...",
"SWITCH_TO_4G_NETWORK": "Превключване към 4G...",
"STANDBY": "Режим на готовност",
"CONNECT_TO": "Свързване към ",
"CONNECTING": "Свързване...",
"CONNECTION_SUCCESSFUL": "Успешно свързване",
"CONNECTED_TO": "Свързан към ",
"LISTENING": "Слушане...",
"SPEAKING": "Говорене...",
"SERVER_NOT_FOUND": "Търсене на налична услуга",
"SERVER_NOT_CONNECTED": "Не може да се свърже с услугата, моля опитайте по-късно",
"SERVER_TIMEOUT": "Времето за изчакване на отговор изтече",
"SERVER_ERROR": "Неуспешно изпращане, моля проверете мрежата",
"CONNECT_TO_HOTSPOT": орещa точка: ",
"ACCESS_VIA_BROWSER": " Конфигурационен URL: ",
"WIFI_CONFIG_MODE": "Режим на конфигуриране на Wi-Fi",
"ENTERING_WIFI_CONFIG_MODE": "Влизане в режим на конфигуриране на Wi-Fi...",
"SCANNING_WIFI": "Сканиране на Wi-Fi...",
"NEW_VERSION": "Нова версия ",
"OTA_UPGRADE": "OTA надстройка",
"UPGRADING": "Системата се надстройва...",
"UPGRADE_FAILED": "Надстройката е неуспешна",
"ACTIVATION": "Активация",
"BATTERY_LOW": "Слаба батерия",
"BATTERY_CHARGING": "Зарядна",
"BATTERY_FULL": "Батерията е пълна",
"BATTERY_NEED_CHARGE": "Слаба батерия, моля заредете",
"VOLUME": "Сила на звука ",
"MUTED": "Заглушено",
"MAX_VOLUME": "Максимална сила на звука",
"RTC_MODE_OFF": "AEC изключен",
"RTC_MODE_ON": "AEC включен",
"PLEASE_WAIT": "Моля, изчакайте...",
"FOUND_NEW_ASSETS": "Намерени нови ресурси: %s",
"DOWNLOAD_ASSETS_FAILED": "Неуспешно изтегляне на ресурси",
"LOADING_ASSETS": "Зареждане на ресурси...",
"HELLO_MY_FRIEND": "Здравей, мой приятел!"
}
}

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Some files were not shown because too many files have changed in this diff Show More