fix v0.4.x board name

update to 0.4.1
fix audio data overflow
2024-10-25 12:46:33 +08:00 · 2024-10-25 12:29:23 +08:00 · 2024-10-25 12:25:22 +08:00 · 2024-10-25 10:05:01 +08:00 · 2024-10-24 09:53:08 +08:00 · 2024-10-17 18:34:51 +08:00
36 changed files with 5452 additions and 925 deletions
--- a/.gitignore
+++ b/.gitignore
@@ -8,3 +8,4 @@ sdkconfig.old
 sdkconfig
 dependencies.lock
 .env
+releases/
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -4,7 +4,7 @@
 # CMakeLists in this exact order for cmake to work correctly
 cmake_minimum_required(VERSION 3.16)

-set(PROJECT_VER "0.2.0")
+set(PROJECT_VER "0.4.1")

 include($ENV{IDF_PATH}/tools/cmake/project.cmake)
 project(xiaozhi)
--- a/README.md
+++ b/README.md
@@ -1,102 +1,65 @@
 # 小智 AI 聊天机器人

-BiliBili 视频介绍 [【ESP32+SenseVoice+Qwen72B打造你的AI聊天伴侣！】](https://www.bilibili.com/video/BV11msTenEH3/?share_source=copy_web&vd_source=ee1aafe19d6e60cf22e60a93881faeba)
+这是虾哥的第一个硬件作品。
+
+[ESP32+SenseVoice+Qwen72B打造你的AI聊天伴侣！【bilibili】](https://www.bilibili.com/video/BV11msTenEH3/?share_source=copy_web&vd_source=ee1aafe19d6e60cf22e60a93881faeba)
+
+[手工打造你的 AI 女友，新手入门教程【bilibili】](https://www.bilibili.com/video/BV1XnmFYLEJN/)
+
+## 项目目的
+
+本项目基于乐鑫的 ESP-IDF 进行开发。
+
+本项目是一个开源项目，主要用于教学目的。我们希望通过这个项目，能够帮助更多人入门 AI 硬件开发,了解如何将当下飞速发展的大语言模型应用到实际的硬件设备中。无论你是对 AI 感兴趣的学生，还是想要探索新技术的开发者，都可以通过这个项目获得宝贵的学习经验。
+
+欢迎所有人参与到项目的开发和改进中来。如果你有任何想法或建议，请随时提出 issue 或加入群聊。

 学习交流 QQ 群：946599635

+## 已实现功能
+
+- Wi-Fi 配网
+- 支持 BOOT 键唤醒和打断
+- 离线语音唤醒（乐鑫方案）
+- 流式语音对话（WebSocket 协议）
+- 支持国语、粤语、英语、日语、韩语 5 种语言识别（SenseVoice 方案）
+- 声纹识别（识别是谁在喊 AI 的名字，[3D Speaker 项目](https://github.com/modelscope/3D-Speaker)）
+- 使用大模型 TTS（火山引擎与 CosyVoice 方案）
+- 支持可配置的提示词和音色（自定义角色）
+- Qwen2.5 72B 或 豆包 API
+- 支持每轮对话后自我总结，生成记忆体
+- 扩展液晶显示屏，显示信号强弱
+- 支持 ML307 Cat.1 4G 模块
+
 ## 硬件部分

-### DIY 所需硬件
+为方便协作，目前所有硬件资料都放在飞书文档中：

- 开发板：ESP32-S3-DevKitC-1
- 麦克风：INMP441
- 功放：MAX98357
- 喇叭：8Ω 3W
- 400 孔面包板 2 块
- 导线若干
+[《小智 AI 聊天机器人百科全书》](https://ccnphfhqs21z.feishu.cn/wiki/F5krwD16viZoF0kKkvDcrZNYnhb?from=from_copylink)

+面包板接线图如下：

-### GPIO 接线指引
-
-以下是默认接线方案，如果你的接线跟默认不一样，请在项目配置中同步修改。
-
-![接线图](./docs/wiring.jpg)
-
-注意，MAX98357 的 GND 和 VIN 接线隐藏在元件下方。INMP441 的 VDD 和 GND 不能接反，否则会烧毁麦克风。
-
-#### MAX98357 功放
-
-```
-LRC -> GPIO 4
-BCLK -> GPIO 5
-DIN -> GPIO 6
-GAIN -> GND（如果音量太大，请将 GAIN 接到 3.3V）
-SD -> 3.3V
-GND -> GND
-VIN -> 3.3V 或 5V（如果你的喇叭需要 5V，应该将 VIN 接到 5V）
-```
-
-#### INMP441 麦克风
-
-```
-L/R -> GND
-WS -> GPIO 10
-SCK -> GPIO 11
-SD -> GPIO 3
-VDD -> 3.3V
-GND -> GND
-```
+![面包板接线图](docs/wiring2.jpg)

 ## 固件部分

 ### 免开发环境烧录

-新手第一次操作建议先不要搭建开发环境，直接使用免开发环境烧录的固件。
+新手第一次操作建议先不要搭建开发环境，直接使用免开发环境烧录的固件。固件使用的是作者友情提供的测试服，目前开放免费使用，请勿用于商业用途。

-点击 [这里](https://github.com/78/xiaozhi-esp32/releases) 下载最新版固件。
+[Flash烧录固件（无IDF开发环境）](https://ccnphfhqs21z.feishu.cn/wiki/Zpz4wXBtdimBrLk25WdcXzxcnNS) 

-固件使用的是作者友情提供的测试服，目前开放免费使用，请勿用于商业用途。

-### 搭建开发环境
+### 开发环境

 - Cursor 或 VSCode
 - 安装 ESP-IDF 插件，选择 SDK 版本 5.3 或以上
 - Ubuntu 比 Windows 更好，编译速度快，也免去驱动问题的困扰

-### 配置项目与编译固件

- 目前只支持 ESP32 S3，Flash 至少 8MB, PSRAM 至少 2MB（注意：默认配置只兼容 8MB PSRAM，如果你使用 2MB PSRAM，需要修改配置，否则无法识别）
- 配置 OTA Version URL 为 `https://api.tenclass.net/xiaozhi/ota/`
- 配置 WebSocket URL 为 `wss://api.tenclass.net/xiaozhi/v1/`
- 配置 WebSocket Access Token 为 `test-token`
- 如果 INMP441 和 MAX98357 接线跟默认配置不一样，需要修改 GPIO 配置
- 配置完成后，编译固件
+## AI 角色配置

+如果你已经拥有一个小智 AI 聊天机器人，可以参考 [后台操作视频教程](https://www.bilibili.com/video/BV1jUCUY2EKM/)

-## 配置 Wi-Fi
-
-按照上述接线，烧录固件，设备上电后，开发板上的 RGB 会闪烁蓝灯（部分开发板需要焊接 RGB 灯的开关才会亮），进入配网状态。
-
-打开手机 Wi-Fi，连接上设备热点 `Xiaozhi-xxxx` 后，使用浏览器访问 `http://192.168.4.1`，进入配网页面。
-
-选择你的路由器 WiFi，输入密码，点击连接，设备会在 3 秒后自动重启，之后设备会自动连接到路由器。
-
-## 测试设备是否连接成功
-
-设备连接上路由器后，闪烁一下绿灯。此时，喊一声“你好，小智”，设备会先亮蓝灯（表示连接服务器），然后再亮绿灯，播放语音。
-
-如果没有亮蓝灯，说明麦克风有问题，请检查接线是否正确。
-
-如果没有亮绿灯，或者蓝灯常亮，说明设备没有连接到服务器，请检查 WiFi 连接是否正常。
-
-如果设备已经连接 Wi-Fi，但是没有声音，请检查是否接线正确。
-
-在 v0.2.1 版本之后的固件，也可以按下连接 GPIO 1 的按钮（低电平有效），进行录音测试。
-
-## 配置设备
-
-如果上述步骤测试成功，设备会播报你的设备 ID，你需要到 [小智测试服的控制面板](https://xiaozhi.tenclass.net/) 页面，添加设备。
-
-详细的使用说明以及测试服的注意事项，请参考 [小智测试服的帮助说明](https://xiaozhi.tenclass.net/help)。
-
+详细的使用说明以及测试服的注意事项，请参考 [小智测试服的帮助说明](https://xiaozhi.me/help)。

--- a/docs/wiring2.jpg
+++ b/docs/wiring2.jpg
--- a/main/Application.cc
+++ b/main/Application.cc
--- a/main/Application.h
+++ b/main/Application.h
@@ -1,25 +1,61 @@
 #ifndef _APPLICATION_H_
 #define _APPLICATION_H_

-#include "AudioDevice.h"
-#include "OpusEncoder.h"
-#include "OpusResampler.h"
-#include "WebSocketClient.h"
-#include "FirmwareUpgrade.h"
+#include <OpusEncoder.h>
+#include <OpusResampler.h>
+#include <WebSocket.h>
+#include <Ml307AtModem.h>
+#include <Ml307Http.h>
+#include <EspHttp.h>

-#include "opus.h"
-#include "resampler_structs.h"
-#include "freertos/event_groups.h"
-#include "freertos/queue.h"
-#include "freertos/task.h"
-#include "esp_afe_sr_models.h"
-#include "esp_nsn_models.h"
+#include <opus.h>
+#include <resampler_structs.h>
+#include <freertos/event_groups.h>
+#include <freertos/task.h>
 #include <mutex>
 #include <list>
+#include <condition_variable>
+
+#include "BoxAudioDevice.h"
+#include "Display.h"
+#include "FirmwareUpgrade.h"
+
+#ifdef CONFIG_USE_AFE_SR
+#include "WakeWordDetect.h"
+#include "AudioProcessor.h"
+#endif
+
+#include "Button.h"

 #define DETECTION_RUNNING 1
 #define COMMUNICATION_RUNNING 2
-#define DETECT_PACKETS_ENCODED 4
+
+#define PROTOCOL_VERSION 2
+struct BinaryProtocol {
+    uint16_t version;
+    uint16_t type;
+    uint32_t reserved;
+    uint32_t timestamp;
+    uint32_t payload_size;
+    uint8_t payload[];
+} __attribute__((packed));
+
+enum AudioPacketType {
+    kAudioPacketTypeUnkonwn = 0,
+    kAudioPacketTypeStart,
+    kAudioPacketTypeStop,
+    kAudioPacketTypeData,
+    kAudioPacketTypeSentenceStart,
+    kAudioPacketTypeSentenceEnd
+};
+
+struct AudioPacket {
+    AudioPacketType type = kAudioPacketTypeUnkonwn;
+    std::string text;
+    std::vector<uint8_t> opus;
+    std::vector<int16_t> pcm;
+    uint32_t timestamp;
+};


 enum ChatState {
@@ -28,7 +64,7 @@ enum ChatState {
    kChatStateListening,
    kChatStateSpeaking,
    kChatStateWakeWordDetected,
-    kChatStateTesting
+    kChatStateUpgrading
 };

 class Application {
@@ -48,60 +84,69 @@ private:
    Application();
    ~Application();

+    Button boot_button_;
+    Button volume_up_button_;
+    Button volume_down_button_;
+#ifdef CONFIG_AUDIO_CODEC_ES8311_ES7210
+    BoxAudioDevice audio_device_;
+#else
    AudioDevice audio_device_;
+#endif
+#ifdef CONFIG_USE_DISPLAY
+    Display display_;
+#endif
+#ifdef CONFIG_USE_AFE_SR
+    WakeWordDetect wake_word_detect_;
+    AudioProcessor audio_processor_;
+#endif
+#ifdef CONFIG_USE_ML307
+    Ml307AtModem ml307_at_modem_;
+    Ml307Http http_;
+#else
+    EspHttp http_;
+#endif
    FirmwareUpgrade firmware_upgrade_;
-
-    std::recursive_mutex mutex_;
-    WebSocketClient* ws_client_ = nullptr;
-    esp_afe_sr_data_t* afe_detection_data_ = nullptr;
-    esp_afe_sr_data_t* afe_communication_data_ = nullptr;
+    std::mutex mutex_;
+    std::condition_variable_any cv_;
+    std::list<std::function<void()>> main_tasks_;
+    WebSocket* ws_client_ = nullptr;
    EventGroupHandle_t event_group_;
-    char* wakenet_model_ = NULL;
-    char* nsnet_model_ = NULL;
    volatile ChatState chat_state_ = kChatStateIdle;
+    volatile bool break_speaking_ = false;
+    bool skip_to_end_ = false;

    // Audio encode / decode
-    TaskHandle_t audio_feed_task_ = nullptr;
+    TaskHandle_t audio_encode_task_ = nullptr;
    StaticTask_t audio_encode_task_buffer_;
    StackType_t* audio_encode_task_stack_ = nullptr;
-    QueueHandle_t audio_encode_queue_ = nullptr;
-
-    TaskHandle_t audio_decode_task_ = nullptr;
-    StaticTask_t audio_decode_task_buffer_;
-    StackType_t* audio_decode_task_stack_ = nullptr;
-    QueueHandle_t audio_decode_queue_ = nullptr;
+    std::list<std::vector<int16_t>> audio_encode_queue_;
+    std::list<AudioPacket*> audio_decode_queue_;
+    std::list<AudioPacket*> audio_play_queue_;

    OpusEncoder opus_encoder_;
    OpusDecoder* opus_decoder_ = nullptr;

    int opus_duration_ms_ = 60;
    int opus_decode_sample_rate_ = CONFIG_AUDIO_OUTPUT_SAMPLE_RATE;
-    OpusResampler opus_resampler_;
-    OpusResampler test_resampler_;
-    std::vector<iovec> test_pcm_;
+    OpusResampler input_resampler_;
+    OpusResampler output_resampler_;

-    TaskHandle_t wake_word_encode_task_ = nullptr;
-    StaticTask_t wake_word_encode_task_buffer_;
-    StackType_t* wake_word_encode_task_stack_ = nullptr;
-    std::list<iovec> wake_word_pcm_;
-    std::vector<iovec> wake_word_opus_;
+    TaskHandle_t check_new_version_task_ = nullptr;
+    StaticTask_t check_new_version_task_buffer_;
+    StackType_t* check_new_version_task_stack_ = nullptr;

+    void MainLoop();
+    void Schedule(std::function<void()> callback);
+    BinaryProtocol* AllocateBinaryProtocol(const uint8_t* payload, size_t payload_size);
    void SetDecodeSampleRate(int sample_rate);
    void SetChatState(ChatState state);
-    void StartDetection();
-    void StartCommunication();
    void StartWebSocketClient();
-    void StoreWakeWordData(uint8_t* data, size_t size);
-    void EncodeWakeWordData();
-    void SendWakeWordData();
-    void CheckTestButton();
-    void PlayTestAudio();
-    
-    void AudioFeedTask();
-    void AudioDetectionTask();
-    void AudioCommunicationTask();
+    void CheckNewVersion();
+    void UpdateDisplay();
+
    void AudioEncodeTask();
-    void AudioDecodeTask();
+    void AudioPlayTask();
+    void HandleAudioPacket(AudioPacket* packet);
 };

 #endif // _APPLICATION_H_
--- a/main/AudioDevice.cc
+++ b/main/AudioDevice.cc
@@ -1,18 +1,17 @@
 #include "AudioDevice.h"
-#include "esp_log.h"
+#include <esp_log.h>
 #include <cstring>
-
+#include <cmath>
 #define TAG "AudioDevice"

-AudioDevice::AudioDevice() {
-    audio_play_queue_ = xQueueCreate(100, sizeof(AudioPacket*));
+AudioDevice::AudioDevice()
+    : input_sample_rate_(CONFIG_AUDIO_INPUT_SAMPLE_RATE),
+      output_sample_rate_(CONFIG_AUDIO_OUTPUT_SAMPLE_RATE) {
 }

 AudioDevice::~AudioDevice() {
-    vQueueDelete(audio_play_queue_);
-
-    if (audio_play_task_ != nullptr) {
-        vTaskDelete(audio_play_task_);
+    if (audio_input_task_ != nullptr) {
+        vTaskDelete(audio_input_task_);
    }
    if (rx_handle_ != nullptr) {
        ESP_ERROR_CHECK(i2s_channel_disable(rx_handle_));
@@ -22,26 +21,16 @@ AudioDevice::~AudioDevice() {
    }
 }

-void AudioDevice::Start(int input_sample_rate, int output_sample_rate) {
-    input_sample_rate_ = input_sample_rate;
-    output_sample_rate_ = output_sample_rate;
-
-#ifdef CONFIG_AUDIO_DEVICE_I2S_SIMPLEX
-        CreateSimplexChannels();
+void AudioDevice::Initialize() {
+#ifdef CONFIG_AUDIO_I2S_METHOD_SIMPLEX
+    CreateSimplexChannels();
 #else
-        CreateDuplexChannels();
+    CreateDuplexChannels();
 #endif
-
-    ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
-    ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
-
-    xTaskCreate([](void* arg) {
-        auto audio_device = (AudioDevice*)arg;
-        audio_device->AudioPlayTask();
-    }, "audio_play", 4096 * 4, this, 5, &audio_play_task_);
 }

 void AudioDevice::CreateDuplexChannels() {
+#ifdef CONFIG_AUDIO_I2S_METHOD_DUPLEX
    duplex_ = true;

    i2s_chan_config_t chan_cfg = {
@@ -77,7 +66,7 @@ void AudioDevice::CreateDuplexChannels() {
        .gpio_cfg = {
            .mclk = I2S_GPIO_UNUSED,
            .bclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_BCLK,
-            .ws = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_WS,
+            .ws = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_LRCK,
            .dout = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_DOUT,
            .din = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_DIN,
            .invert_flags = {
@@ -89,11 +78,14 @@ void AudioDevice::CreateDuplexChannels() {
    };
    ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
    ESP_ERROR_CHECK(i2s_channel_init_std_mode(rx_handle_, &std_cfg));
+    ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
+    ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
    ESP_LOGI(TAG, "Duplex channels created");
+#endif
 }

-#ifdef CONFIG_AUDIO_DEVICE_I2S_SIMPLEX
 void AudioDevice::CreateSimplexChannels() {
+#ifdef CONFIG_AUDIO_I2S_METHOD_SIMPLEX
    // Create a new channel for speaker
    i2s_chan_config_t chan_cfg = {
        .id = I2S_NUM_0,
@@ -127,9 +119,9 @@ void AudioDevice::CreateSimplexChannels() {
        },
        .gpio_cfg = {
            .mclk = I2S_GPIO_UNUSED,
-            .bclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_BCLK,
-            .ws = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_WS,
-            .dout = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_DOUT,
+            .bclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_SPK_GPIO_BCLK,
+            .ws = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_SPK_GPIO_LRCK,
+            .dout = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_SPK_GPIO_DOUT,
            .din = I2S_GPIO_UNUSED,
            .invert_flags = {
                .mclk_inv = false,
@@ -144,23 +136,38 @@ void AudioDevice::CreateSimplexChannels() {
    chan_cfg.id = I2S_NUM_1;
    ESP_ERROR_CHECK(i2s_new_channel(&chan_cfg, nullptr, &rx_handle_));
    std_cfg.clk_cfg.sample_rate_hz = (uint32_t)input_sample_rate_;
-    std_cfg.gpio_cfg.bclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_MIC_GPIO_BCLK;
+    std_cfg.gpio_cfg.bclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_MIC_GPIO_SCK;
    std_cfg.gpio_cfg.ws = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_MIC_GPIO_WS;
    std_cfg.gpio_cfg.dout = I2S_GPIO_UNUSED;
-    std_cfg.gpio_cfg.din = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_DIN;
+    std_cfg.gpio_cfg.din = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_MIC_GPIO_DIN;
    ESP_ERROR_CHECK(i2s_channel_init_std_mode(rx_handle_, &std_cfg));
-    ESP_LOGI(TAG, "Simplex channels created");
-}
-#endif

-void AudioDevice::Write(const int16_t* data, int samples) {
+    ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
+    ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
+    ESP_LOGI(TAG, "Simplex channels created");
+#endif
+}
+
+int AudioDevice::Write(const int16_t* data, int samples) {
    int32_t buffer[samples];
+
+    // output_volume_: 0-100
+    // volume_factor_: 0-65536
+    int32_t volume_factor = pow(double(output_volume_) / 100.0, 2) * 65536;
    for (int i = 0; i < samples; i++) {
-        buffer[i] = int32_t(data[i]) << 15;
+        int64_t temp = int64_t(data[i]) * volume_factor; // 使用 int64_t 进行乘法运算
+        if (temp > INT32_MAX) {
+            buffer[i] = INT32_MAX;
+        } else if (temp < INT32_MIN) {
+            buffer[i] = INT32_MIN;
+        } else {
+            buffer[i] = static_cast<int32_t>(temp);
+        }
    }

    size_t bytes_written;
    ESP_ERROR_CHECK(i2s_channel_write(tx_handle_, buffer, samples * sizeof(int32_t), &bytes_written, portMAX_DELAY));
+    return bytes_written / sizeof(int32_t);
 }

 int AudioDevice::Read(int16_t* dest, int samples) {
@@ -180,57 +187,37 @@ int AudioDevice::Read(int16_t* dest, int samples) {
    return samples;
 }

-void AudioDevice::QueueAudioPacket(AudioPacket* packet) {
-    xQueueSend(audio_play_queue_, &packet, portMAX_DELAY);
-}
+void AudioDevice::OnInputData(std::function<void(std::vector<int16_t>&& data)> callback) {
+    on_input_data_ = callback;

-void AudioDevice::AudioPlayTask() {
-    while (true) {
-        AudioPacket* packet;
-        xQueueReceive(audio_play_queue_, &packet, portMAX_DELAY);
-
-        switch (packet->type)
-        {
-        case kAudioPacketTypeStart:
-            playing_ = true;
-            breaked_ = false;
-            if (on_state_changed_) {
-                on_state_changed_();
-            }
-            break;
-        case kAudioPacketTypeStop:
-            playing_ = false;
-            if (on_state_changed_) {
-                on_state_changed_();
-            }
-            break;
-        case kAudioPacketTypeSentenceStart:
-            ESP_LOGI(TAG, "Playing sentence: %s", packet->text.c_str());
-            break;
-        case kAudioPacketTypeSentenceEnd:
-            if (breaked_) { // Clear the queue
-                AudioPacket* p;
-                while (xQueueReceive(audio_play_queue_, &p, 0) == pdTRUE) {
-                    delete p;
-                }
-                breaked_ = false;
-                playing_ = false;
-            }
-            break;
-        case kAudioPacketTypeData:
-            Write(packet->pcm.data(), packet->pcm.size());
-            break;
-        default:
-            ESP_LOGE(TAG, "Unknown audio packet type: %d", packet->type);
-        }
-        delete packet;
+    // 创建音频输入任务
+    if (audio_input_task_ == nullptr) {
+        xTaskCreate([](void* arg) {
+            auto audio_device = (AudioDevice*)arg;
+            audio_device->InputTask();
+        }, "audio_input", 4096 * 2, this, 3, &audio_input_task_);
    }
 }

-void AudioDevice::OnStateChanged(std::function<void()> callback) {
-    on_state_changed_ = callback;
+void AudioDevice::OutputData(std::vector<int16_t>& data) {
+    Write(data.data(), data.size());
 }

-void AudioDevice::Break() {
-    breaked_ = true;
+void AudioDevice::InputTask() {
+    int duration = 30;
+    int input_frame_size = input_sample_rate_ / 1000 * duration * input_channels_;
+    while (true) {
+        std::vector<int16_t> input_data(input_frame_size);
+        int samples = Read(input_data.data(), input_data.size());
+        if (samples > 0) {
+            if (on_input_data_) {
+                on_input_data_(std::move(input_data));
+            }
+        }
+    }
+}
+
+void AudioDevice::SetOutputVolume(int volume) {
+    output_volume_ = volume;
+    ESP_LOGI(TAG, "Set output volume to %d", output_volume_);
 }
--- a/main/AudioDevice.h
+++ b/main/AudioDevice.h
@@ -1,76 +1,52 @@
 #ifndef _AUDIO_DEVICE_H
 #define _AUDIO_DEVICE_H

-#include "opus.h"
-#include "freertos/FreeRTOS.h"
-#include "freertos/queue.h"
-#include "freertos/event_groups.h"
-#include "driver/i2s_std.h"
+#include <freertos/FreeRTOS.h>
+#include <driver/i2s_std.h>

 #include <vector>
 #include <string>
 #include <functional>

-enum AudioPacketType {
-    kAudioPacketTypeUnkonwn = 0,
-    kAudioPacketTypeStart,
-    kAudioPacketTypeStop,
-    kAudioPacketTypeData,
-    kAudioPacketTypeSentenceStart,
-    kAudioPacketTypeSentenceEnd
-};
-
-struct AudioPacket {
-    AudioPacketType type = kAudioPacketTypeUnkonwn;
-    std::string text;
-    std::vector<uint8_t> opus;
-    std::vector<int16_t> pcm;
-    uint32_t timestamp;
-};
-
-struct AudioDataHeader {
-    uint32_t version;
-    uint32_t reserved;
-    uint32_t timestamp;
-    uint32_t payload_size;
-} __attribute__((packed));
-
 class AudioDevice {
 public:
    AudioDevice();
-    ~AudioDevice();
+    virtual ~AudioDevice();
+    virtual void Initialize();

-    void Start(int input_sample_rate, int output_sample_rate);
-    int Read(int16_t* dest, int samples);
-    void Write(const int16_t* data, int samples);
-    void QueueAudioPacket(AudioPacket* packet);
-    void OnStateChanged(std::function<void()> callback);
-    void Break();
+    void OnInputData(std::function<void(std::vector<int16_t>&& data)> callback);
+    void OutputData(std::vector<int16_t>& data);
+    virtual void SetOutputVolume(int volume);

-    int input_sample_rate() const { return input_sample_rate_; }
-    int output_sample_rate() const { return output_sample_rate_; }
-    bool duplex() const { return duplex_; }
-    bool playing() const { return playing_; }
+    inline bool duplex() const { return duplex_; }
+    inline bool input_reference() const { return input_reference_; }
+    inline int input_sample_rate() const { return input_sample_rate_; }
+    inline int output_sample_rate() const { return output_sample_rate_; }
+    inline int input_channels() const { return input_channels_; }
+    inline int output_channels() const { return output_channels_; }
+    inline int output_volume() const { return output_volume_; }

 private:
-    bool playing_ = false;
-    bool breaked_ = false;
+    TaskHandle_t audio_input_task_ = nullptr;
+    std::function<void(std::vector<int16_t>&& data)> on_input_data_; 
+
+    void InputTask();
+    void CreateSimplexChannels();
+
+protected:
    bool duplex_ = false;
+    bool input_reference_ = false;
    int input_sample_rate_ = 0;
    int output_sample_rate_ = 0;
-
+    int input_channels_ = 1;
+    int output_channels_ = 1;
+    int output_volume_ = 70;
    i2s_chan_handle_t tx_handle_ = nullptr;
    i2s_chan_handle_t rx_handle_ = nullptr;

-    QueueHandle_t audio_play_queue_ = nullptr;
-    TaskHandle_t audio_play_task_ = nullptr;
-    
-    EventGroupHandle_t event_group_;
-    std::function<void()> on_state_changed_;
-
-    void CreateDuplexChannels();
-    void CreateSimplexChannels();
-    void AudioPlayTask();
+    virtual void CreateDuplexChannels();
+    virtual int Read(int16_t* dest, int samples);
+    virtual int Write(const int16_t* data, int samples);
 };

 #endif // _AUDIO_DEVICE_H
--- a/main/AudioProcessor.cc
+++ b/main/AudioProcessor.cc
@@ -0,0 +1,115 @@
+#include "AudioProcessor.h"
+#include <esp_log.h>
+
+#define PROCESSOR_RUNNING 0x01
+
+static const char* TAG = "AudioProcessor";
+
+AudioProcessor::AudioProcessor()
+    : afe_communication_data_(nullptr) {
+    event_group_ = xEventGroupCreate();
+}
+
+void AudioProcessor::Initialize(int channels, bool reference) {
+    channels_ = channels;
+    reference_ = reference;
+    int ref_num = reference_ ? 1 : 0;
+
+    afe_config_t afe_config = {
+        .aec_init = false,
+        .se_init = true,
+        .vad_init = false,
+        .wakenet_init = false,
+        .voice_communication_init = true,
+        .voice_communication_agc_init = true,
+        .voice_communication_agc_gain = 10,
+        .vad_mode = VAD_MODE_3,
+        .wakenet_model_name = NULL,
+        .wakenet_model_name_2 = NULL,
+        .wakenet_mode = DET_MODE_90,
+        .afe_mode = SR_MODE_HIGH_PERF,
+        .afe_perferred_core = 1,
+        .afe_perferred_priority = 1,
+        .afe_ringbuf_size = 50,
+        .memory_alloc_mode = AFE_MEMORY_ALLOC_MORE_PSRAM,
+        .afe_linear_gain = 1.0,
+        .agc_mode = AFE_MN_PEAK_AGC_MODE_2,
+        .pcm_config = {
+            .total_ch_num = channels_,
+            .mic_num = channels_ - ref_num,
+            .ref_num = ref_num,
+            .sample_rate = 16000,
+        },
+        .debug_init = false,
+        .debug_hook = {{ AFE_DEBUG_HOOK_MASE_TASK_IN, NULL }, { AFE_DEBUG_HOOK_FETCH_TASK_IN, NULL }},
+        .afe_ns_mode = NS_MODE_SSP,
+        .afe_ns_model_name = NULL,
+        .fixed_first_channel = true,
+    };
+
+    afe_communication_data_ = esp_afe_vc_v1.create_from_config(&afe_config);
+    
+    xTaskCreate([](void* arg) {
+        auto this_ = (AudioProcessor*)arg;
+        this_->AudioProcessorTask();
+        vTaskDelete(NULL);
+    }, "audio_communication", 4096 * 2, this, 1, NULL);
+}
+
+AudioProcessor::~AudioProcessor() {
+    if (afe_communication_data_ != nullptr) {
+        esp_afe_vc_v1.destroy(afe_communication_data_);
+    }
+    vEventGroupDelete(event_group_);
+}
+
+void AudioProcessor::Input(std::vector<int16_t>& data) {
+    input_buffer_.insert(input_buffer_.end(), data.begin(), data.end());
+
+    auto chunk_size = esp_afe_vc_v1.get_feed_chunksize(afe_communication_data_) * channels_;
+    while (input_buffer_.size() >= chunk_size) {
+        auto chunk = input_buffer_.data();
+        esp_afe_vc_v1.feed(afe_communication_data_, chunk);
+        input_buffer_.erase(input_buffer_.begin(), input_buffer_.begin() + chunk_size);
+    }
+}
+
+void AudioProcessor::Start() {
+    xEventGroupSetBits(event_group_, PROCESSOR_RUNNING);
+}
+
+void AudioProcessor::Stop() {
+    xEventGroupClearBits(event_group_, PROCESSOR_RUNNING);
+}
+
+bool AudioProcessor::IsRunning() {
+    return xEventGroupGetBits(event_group_) & PROCESSOR_RUNNING;
+}
+
+void AudioProcessor::OnOutput(std::function<void(std::vector<int16_t>&& data)> callback) {
+    output_callback_ = callback;
+}
+
+void AudioProcessor::AudioProcessorTask() {
+    int chunk_size = esp_afe_vc_v1.get_fetch_chunksize(afe_communication_data_);
+    ESP_LOGI(TAG, "Audio communication task started, chunk size: %d", chunk_size);
+
+    while (true) {
+        xEventGroupWaitBits(event_group_, PROCESSOR_RUNNING, pdFALSE, pdTRUE, portMAX_DELAY);
+
+        auto res = esp_afe_vc_v1.fetch(afe_communication_data_);
+        if ((xEventGroupGetBits(event_group_) & PROCESSOR_RUNNING) == 0) {
+            continue;
+        }
+        if (res == nullptr || res->ret_value == ESP_FAIL) {
+            if (res != nullptr) {
+                ESP_LOGI(TAG, "Error code: %d", res->ret_value);
+            }
+            continue;
+        }
+
+        if (output_callback_) {
+            output_callback_(std::vector<int16_t>(res->data, res->data + res->data_size / sizeof(int16_t)));
+        }
+    }
+}
--- a/main/AudioProcessor.h
+++ b/main/AudioProcessor.h
@@ -0,0 +1,36 @@
+#ifndef AUDIO_PROCESSOR_H
+#define AUDIO_PROCESSOR_H
+
+#include <esp_afe_sr_models.h>
+#include <freertos/FreeRTOS.h>
+#include <freertos/task.h>
+#include <freertos/event_groups.h>
+
+#include <string>
+#include <vector>
+#include <functional>
+
+class AudioProcessor {
+public:
+    AudioProcessor();
+    ~AudioProcessor();
+
+    void Initialize(int channels, bool reference);
+    void Input(std::vector<int16_t>& data);
+    void Start();
+    void Stop();
+    bool IsRunning();
+    void OnOutput(std::function<void(std::vector<int16_t>&& data)> callback);
+
+private:
+    EventGroupHandle_t event_group_ = nullptr;
+    esp_afe_sr_data_t* afe_communication_data_ = nullptr;
+    std::vector<int16_t> input_buffer_;
+    std::function<void(std::vector<int16_t>&& data)> output_callback_;
+    int channels_;
+    bool reference_;
+
+    void AudioProcessorTask();
+};
+
+#endif
--- a/main/BoxAudioDevice.cc
+++ b/main/BoxAudioDevice.cc
@@ -0,0 +1,232 @@
+#include "BoxAudioDevice.h"
+#include <esp_log.h>
+#include <cassert>
+
+static const char* TAG = "BoxAudioDevice";
+
+BoxAudioDevice::BoxAudioDevice() {
+}
+
+BoxAudioDevice::~BoxAudioDevice() {
+    ESP_ERROR_CHECK(esp_codec_dev_close(output_dev_));
+    esp_codec_dev_delete(output_dev_);
+    ESP_ERROR_CHECK(esp_codec_dev_close(input_dev_));
+    esp_codec_dev_delete(input_dev_);
+
+    audio_codec_delete_codec_if(in_codec_if_);
+    audio_codec_delete_ctrl_if(in_ctrl_if_);
+    audio_codec_delete_codec_if(out_codec_if_);
+    audio_codec_delete_ctrl_if(out_ctrl_if_);
+    audio_codec_delete_gpio_if(gpio_if_);
+    audio_codec_delete_data_if(data_if_);
+
+    ESP_ERROR_CHECK(i2c_del_master_bus(i2c_master_handle_));
+}
+
+void BoxAudioDevice::Initialize() {
+    duplex_ = true; // 是否双工
+    input_reference_ = CONFIG_AUDIO_CODEC_INPUT_REFERENCE; // 是否使用参考输入，实现回声消除
+    input_channels_ = input_reference_ ? 2 : 1; // 输入通道数
+
+    // Initialize I2C peripheral
+    i2c_master_bus_config_t i2c_bus_cfg = {
+        .i2c_port = I2C_NUM_0,
+        .sda_io_num = (gpio_num_t)CONFIG_AUDIO_CODEC_I2C_SDA_PIN,
+        .scl_io_num = (gpio_num_t)CONFIG_AUDIO_CODEC_I2C_SCL_PIN,
+        .clk_source = I2C_CLK_SRC_DEFAULT,
+        .glitch_ignore_cnt = 7,
+        .intr_priority = 0,
+        .trans_queue_depth = 0,
+        .flags = {
+            .enable_internal_pullup = 1,
+        },
+    };
+    ESP_ERROR_CHECK(i2c_new_master_bus(&i2c_bus_cfg, &i2c_master_handle_));
+
+    CreateDuplexChannels();
+
+    // Do initialize of related interface: data_if, ctrl_if and gpio_if
+    audio_codec_i2s_cfg_t i2s_cfg = {
+        .port = I2S_NUM_0,
+        .rx_handle = rx_handle_,
+        .tx_handle = tx_handle_,
+    };
+    data_if_ = audio_codec_new_i2s_data(&i2s_cfg);
+    assert(data_if_ != NULL);
+
+    // Output
+    audio_codec_i2c_cfg_t i2c_cfg = {
+        .port = I2C_NUM_0,
+        .addr = ES8311_CODEC_DEFAULT_ADDR,
+        .bus_handle = i2c_master_handle_,
+    };
+    out_ctrl_if_ = audio_codec_new_i2c_ctrl(&i2c_cfg);
+    assert(out_ctrl_if_ != NULL);
+
+    gpio_if_ = audio_codec_new_gpio();
+    assert(gpio_if_ != NULL);
+
+    es8311_codec_cfg_t es8311_cfg = {};
+    es8311_cfg.ctrl_if = out_ctrl_if_;
+    es8311_cfg.gpio_if = gpio_if_;
+    es8311_cfg.codec_mode = ESP_CODEC_DEV_WORK_MODE_DAC;
+    es8311_cfg.pa_pin = CONFIG_AUDIO_CODEC_PA_PIN;
+    es8311_cfg.use_mclk = true;
+    es8311_cfg.hw_gain.pa_voltage = 5.0;
+    es8311_cfg.hw_gain.codec_dac_voltage = 3.3;
+    out_codec_if_ = es8311_codec_new(&es8311_cfg);
+    assert(out_codec_if_ != NULL);
+
+    esp_codec_dev_cfg_t dev_cfg = {
+        .dev_type = ESP_CODEC_DEV_TYPE_OUT,
+        .codec_if = out_codec_if_,
+        .data_if = data_if_,
+    };
+    output_dev_ = esp_codec_dev_new(&dev_cfg);
+    assert(output_dev_ != NULL);
+
+    ESP_ERROR_CHECK(esp_codec_dev_set_out_vol(output_dev_, output_volume_));
+
+    // Play 16bit 1 channel
+    esp_codec_dev_sample_info_t fs = {
+        .bits_per_sample = 16,
+        .channel = 1,
+        .channel_mask = 0,
+        .sample_rate = (uint32_t)output_sample_rate_,
+        .mclk_multiple = 0,
+    };
+    ESP_ERROR_CHECK(esp_codec_dev_open(output_dev_, &fs));
+
+    // Input
+    i2c_cfg.addr = ES7210_CODEC_DEFAULT_ADDR;
+    in_ctrl_if_ = audio_codec_new_i2c_ctrl(&i2c_cfg);
+    assert(in_ctrl_if_ != NULL);
+
+    es7210_codec_cfg_t es7210_cfg = {};
+    es7210_cfg.ctrl_if = in_ctrl_if_;
+    es7210_cfg.mic_selected = ES7120_SEL_MIC1 | ES7120_SEL_MIC2 | ES7120_SEL_MIC3 | ES7120_SEL_MIC4;
+    in_codec_if_ = es7210_codec_new(&es7210_cfg);
+    assert(in_codec_if_ != NULL);
+
+    dev_cfg.dev_type = ESP_CODEC_DEV_TYPE_IN;
+    dev_cfg.codec_if = in_codec_if_;
+    input_dev_ = esp_codec_dev_new(&dev_cfg);
+    assert(input_dev_ != NULL);
+
+    fs.channel = 4;
+    if (input_channels_ == 1) {
+        fs.channel_mask = ESP_CODEC_DEV_MAKE_CHANNEL_MASK(0);
+    } else {
+        fs.channel_mask = ESP_CODEC_DEV_MAKE_CHANNEL_MASK(0) | ESP_CODEC_DEV_MAKE_CHANNEL_MASK(1);
+    }
+    ESP_ERROR_CHECK(esp_codec_dev_open(input_dev_, &fs));
+
+    ESP_ERROR_CHECK(esp_codec_dev_set_in_channel_gain(input_dev_, ESP_CODEC_DEV_MAKE_CHANNEL_MASK(0), 30.0));
+
+    ESP_LOGI(TAG, "BoxAudioDevice initialized");
+}
+
+void BoxAudioDevice::CreateDuplexChannels() {
+    assert(input_sample_rate_ == output_sample_rate_);
+
+    i2s_chan_config_t chan_cfg = {
+        .id = I2S_NUM_0,
+        .role = I2S_ROLE_MASTER,
+        .dma_desc_num = 6,
+        .dma_frame_num = 240,
+        .auto_clear_after_cb = true,
+        .auto_clear_before_cb = false,
+        .intr_priority = 0,
+    };
+    ESP_ERROR_CHECK(i2s_new_channel(&chan_cfg, &tx_handle_, &rx_handle_));
+
+    i2s_std_config_t std_cfg = {
+        .clk_cfg = {
+            .sample_rate_hz = (uint32_t)output_sample_rate_,
+            .clk_src = I2S_CLK_SRC_DEFAULT,
+            .ext_clk_freq_hz = 0,
+            .mclk_multiple = I2S_MCLK_MULTIPLE_256
+        },
+        .slot_cfg = {
+            .data_bit_width = I2S_DATA_BIT_WIDTH_16BIT,
+            .slot_bit_width = I2S_SLOT_BIT_WIDTH_AUTO,
+            .slot_mode = I2S_SLOT_MODE_STEREO,
+            .slot_mask = I2S_STD_SLOT_BOTH,
+            .ws_width = I2S_DATA_BIT_WIDTH_16BIT,
+            .ws_pol = false,
+            .bit_shift = true,
+            .left_align = true,
+            .big_endian = false,
+            .bit_order_lsb = false
+        },
+        .gpio_cfg = {
+            .mclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_MCLK,
+            .bclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_BCLK,
+            .ws = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_LRCK,
+            .dout = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_DOUT,
+            .din = I2S_GPIO_UNUSED,
+            .invert_flags = {
+                .mclk_inv = false,
+                .bclk_inv = false,
+                .ws_inv = false
+            }
+        }
+    };
+
+    i2s_tdm_config_t tdm_cfg = {
+        .clk_cfg = {
+            .sample_rate_hz = (uint32_t)input_sample_rate_,
+            .clk_src = I2S_CLK_SRC_DEFAULT,
+            .ext_clk_freq_hz = 0,
+            .mclk_multiple = I2S_MCLK_MULTIPLE_256,
+            .bclk_div = 8,
+        },
+        .slot_cfg = {
+            .data_bit_width = I2S_DATA_BIT_WIDTH_16BIT,
+            .slot_bit_width = I2S_SLOT_BIT_WIDTH_AUTO,
+            .slot_mode = I2S_SLOT_MODE_STEREO,
+            .slot_mask = i2s_tdm_slot_mask_t(I2S_TDM_SLOT0 | I2S_TDM_SLOT1 | I2S_TDM_SLOT2 | I2S_TDM_SLOT3),
+            .ws_width = I2S_TDM_AUTO_WS_WIDTH,
+            .ws_pol = false,
+            .bit_shift = true,
+            .left_align = false,
+            .big_endian = false,
+            .bit_order_lsb = false,
+            .skip_mask = false,
+            .total_slot = I2S_TDM_AUTO_SLOT_NUM
+        },
+        .gpio_cfg = {
+            .mclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_MCLK,
+            .bclk = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_BCLK,
+            .ws = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_LRCK,
+            .dout = I2S_GPIO_UNUSED,
+            .din = (gpio_num_t)CONFIG_AUDIO_DEVICE_I2S_GPIO_DIN,
+            .invert_flags = {
+                .mclk_inv = false,
+                .bclk_inv = false,
+                .ws_inv = false
+            }
+        }
+    };
+
+    ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
+    ESP_ERROR_CHECK(i2s_channel_init_tdm_mode(rx_handle_, &tdm_cfg));
+    ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
+    ESP_ERROR_CHECK(i2s_channel_enable(rx_handle_));
+    ESP_LOGI(TAG, "Duplex channels created");
+}
+
+int BoxAudioDevice::Read(int16_t *buffer, int samples) {
+    ESP_ERROR_CHECK(esp_codec_dev_read(input_dev_, (void*)buffer, samples * sizeof(int16_t)));
+    return samples;
+}
+
+int BoxAudioDevice::Write(const int16_t *buffer, int samples) {
+    ESP_ERROR_CHECK(esp_codec_dev_write(output_dev_, (void*)buffer, samples * sizeof(int16_t)));
+    return samples;
+}
+
+void BoxAudioDevice::SetOutputVolume(int volume) {
+    ESP_ERROR_CHECK(esp_codec_dev_set_out_vol(output_dev_, volume));
+    AudioDevice::SetOutputVolume(volume);
+}
--- a/main/BoxAudioDevice.h
+++ b/main/BoxAudioDevice.h
@@ -0,0 +1,36 @@
+#ifndef _BOX_AUDIO_DEVICE_H
+#define _BOX_AUDIO_DEVICE_H
+
+#include "AudioDevice.h"
+#include <driver/i2c_master.h>
+#include <driver/i2s_tdm.h>
+#include <esp_codec_dev.h>
+#include <esp_codec_dev_defaults.h>
+
+
+class BoxAudioDevice : public AudioDevice {
+public:
+    BoxAudioDevice();
+    virtual ~BoxAudioDevice();
+    void Initialize() override;
+    void SetOutputVolume(int volume) override;
+
+private:
+    i2c_master_bus_handle_t i2c_master_handle_ = nullptr;
+
+    const audio_codec_data_if_t* data_if_ = nullptr;
+    const audio_codec_ctrl_if_t* out_ctrl_if_ = nullptr;
+    const audio_codec_if_t* out_codec_if_ = nullptr;
+    const audio_codec_ctrl_if_t* in_ctrl_if_ = nullptr;
+    const audio_codec_if_t* in_codec_if_ = nullptr;
+    const audio_codec_gpio_if_t* gpio_if_ = nullptr;
+
+    esp_codec_dev_handle_t output_dev_ = nullptr;
+    esp_codec_dev_handle_t input_dev_ = nullptr;
+
+    void CreateDuplexChannels() override;
+    int Read(int16_t* dest, int samples) override;
+    int Write(const int16_t* data, int samples) override;
+};
+
+#endif // _BOX_AUDIO_DEVICE_H
--- a/main/Button.cc
+++ b/main/Button.cc
@@ -0,0 +1,67 @@
+#include "Button.h"
+#include <esp_log.h>
+
+static const char* TAG = "Button";
+
+Button::Button(gpio_num_t gpio_num) : gpio_num_(gpio_num) {
+    button_config_t button_config = {
+        .type = BUTTON_TYPE_GPIO,
+        .long_press_time = 1000,
+        .short_press_time = 50,
+        .gpio_button_config = {
+            .gpio_num = gpio_num,
+            .active_level = 0
+        }
+    };
+    button_handle_ = iot_button_create(&button_config);
+    if (button_handle_ == NULL) {
+        ESP_LOGE(TAG, "Failed to create button handle");
+        return;
+    }
+}
+
+Button::~Button() {
+    if (button_handle_ != NULL) {
+        iot_button_delete(button_handle_);
+    }
+}
+
+void Button::OnPress(std::function<void()> callback) {
+    on_press_ = callback;
+    iot_button_register_cb(button_handle_, BUTTON_PRESS_DOWN, [](void* handle, void* usr_data) {
+        Button* button = static_cast<Button*>(usr_data);
+        if (button->on_press_) {
+            button->on_press_();
+        }
+    }, this);
+}
+
+void Button::OnLongPress(std::function<void()> callback) {
+    on_long_press_ = callback;
+    iot_button_register_cb(button_handle_, BUTTON_LONG_PRESS_START, [](void* handle, void* usr_data) {
+        Button* button = static_cast<Button*>(usr_data);
+        if (button->on_long_press_) {
+            button->on_long_press_();
+        }
+    }, this);
+}
+
+void Button::OnClick(std::function<void()> callback) {
+    on_click_ = callback;
+    iot_button_register_cb(button_handle_, BUTTON_SINGLE_CLICK, [](void* handle, void* usr_data) {
+        Button* button = static_cast<Button*>(usr_data);
+        if (button->on_click_) {
+            button->on_click_();
+        }
+    }, this);
+}
+
+void Button::OnDoubleClick(std::function<void()> callback) {
+    on_double_click_ = callback;
+    iot_button_register_cb(button_handle_, BUTTON_DOUBLE_CLICK, [](void* handle, void* usr_data) {
+        Button* button = static_cast<Button*>(usr_data);
+        if (button->on_double_click_) {
+            button->on_double_click_();
+        }
+    }, this);
+}
--- a/main/Button.h
+++ b/main/Button.h
@@ -0,0 +1,28 @@
+#ifndef BUTTON_H_
+#define BUTTON_H_
+
+#include <driver/gpio.h>
+#include <iot_button.h>
+#include <functional>
+
+class Button {
+public:
+    Button(gpio_num_t gpio_num);
+    ~Button();
+
+    void OnPress(std::function<void()> callback);
+    void OnLongPress(std::function<void()> callback);
+    void OnClick(std::function<void()> callback);
+    void OnDoubleClick(std::function<void()> callback);
+private:
+    gpio_num_t gpio_num_;
+    button_handle_t button_handle_;
+
+
+    std::function<void()> on_press_;
+    std::function<void()> on_long_press_;
+    std::function<void()> on_click_;
+    std::function<void()> on_double_click_;
+};
+
+#endif // BUTTON_H_
--- a/main/CMakeLists.txt
+++ b/main/CMakeLists.txt
@@ -1,9 +1,20 @@
 set(SOURCES "AudioDevice.cc"
+            "FirmwareUpgrade.cc"
+            "SystemInfo.cc"
            "SystemReset.cc"
            "Application.cc"
+            "Display.cc"
+            "Button.cc"
            "main.cc"
            )

+if(CONFIG_USE_AFE_SR)
+    list(APPEND SOURCES "AudioProcessor.cc" "WakeWordDetect.cc")
+endif()
+if(CONFIG_AUDIO_CODEC_ES8311_ES7210)
+    list(APPEND SOURCES "BoxAudioDevice.cc")
+endif()
+
 idf_component_register(SRCS ${SOURCES}
                    INCLUDE_DIRS "."
                    )
--- a/main/Display.cc
+++ b/main/Display.cc
@@ -0,0 +1,179 @@
+
+#include "Display.h"
+
+#include <esp_log.h>
+#include <esp_err.h>
+#include <esp_lcd_panel_ops.h>
+#include <esp_lcd_panel_vendor.h>
+#include <esp_lvgl_port.h>
+#include <string>
+#include <cstdlib>
+
+#define TAG "Display"
+
+#ifdef CONFIG_USE_DISPLAY
+
+Display::Display(int sda_pin, int scl_pin) : sda_pin_(sda_pin), scl_pin_(scl_pin) {
+    ESP_LOGI(TAG, "Display Pins: %d, %d", sda_pin_, scl_pin_);
+
+    i2c_master_bus_config_t bus_config = {
+        .i2c_port = I2C_NUM_1,
+        .sda_io_num = (gpio_num_t)sda_pin_,
+        .scl_io_num = (gpio_num_t)scl_pin_,
+        .clk_source = I2C_CLK_SRC_DEFAULT,
+        .glitch_ignore_cnt = 7,
+        .intr_priority = 1,
+        .trans_queue_depth = 0,
+        .flags = {
+            .enable_internal_pullup = 1,
+        },
+    };
+
+    ESP_ERROR_CHECK(i2c_new_master_bus(&bus_config, &i2c_bus_));
+
+    // SSD1306 config
+    esp_lcd_panel_io_i2c_config_t io_config = {
+        .dev_addr = 0x3C,
+        .on_color_trans_done = nullptr,
+        .user_ctx = nullptr,
+        .control_phase_bytes = 1,
+        .dc_bit_offset = 6,
+        .lcd_cmd_bits = 8,
+        .lcd_param_bits = 8,
+        .flags = {
+            .dc_low_on_data = 0,
+            .disable_control_phase = 0,
+        },
+        .scl_speed_hz = 400 * 1000,
+    };
+
+    ESP_ERROR_CHECK(esp_lcd_new_panel_io_i2c_v2(i2c_bus_, &io_config, &panel_io_));
+
+    ESP_LOGI(TAG, "Install SSD1306 driver");
+    esp_lcd_panel_dev_config_t panel_config = {};
+    panel_config.reset_gpio_num = -1;
+    panel_config.bits_per_pixel = 1;
+
+    esp_lcd_panel_ssd1306_config_t ssd1306_config = {
+        .height = CONFIG_DISPLAY_HEIGHT
+    };
+    panel_config.vendor_config = &ssd1306_config;
+
+    ESP_ERROR_CHECK(esp_lcd_new_panel_ssd1306(panel_io_, &panel_config, &panel_));
+    ESP_LOGI(TAG, "SSD1306 driver installed");
+
+    // Reset the display
+    ESP_ERROR_CHECK(esp_lcd_panel_reset(panel_));
+    if (esp_lcd_panel_init(panel_) != ESP_OK) {
+        ESP_LOGE(TAG, "Failed to initialize display");
+        return;
+    }
+
+    ESP_LOGI(TAG, "Initialize LVGL");
+    lvgl_port_cfg_t port_cfg = ESP_LVGL_PORT_INIT_CONFIG();
+    lvgl_port_init(&port_cfg);
+
+    const lvgl_port_display_cfg_t display_cfg = {
+        .io_handle = panel_io_,
+        .panel_handle = panel_,
+        .buffer_size = 128 * CONFIG_DISPLAY_HEIGHT,
+        .double_buffer = true,
+        .hres = 128,
+        .vres = CONFIG_DISPLAY_HEIGHT,
+        .monochrome = true,
+        .rotation = {
+            .swap_xy = 0,
+            .mirror_x = 0,
+            .mirror_y = 0,
+        },
+        .flags = {
+            .buff_dma = 0,
+            .buff_spiram = 0,
+        },
+    };
+    disp_ = lvgl_port_add_disp(&display_cfg);
+    lv_disp_set_rotation(disp_, LV_DISP_ROT_180);
+
+    // Set the display to on
+    ESP_LOGI(TAG, "Turning display on");
+    ESP_ERROR_CHECK(esp_lcd_panel_disp_on_off(panel_, true));
+    
+    ESP_LOGI(TAG, "Display Loading...");
+    if (lvgl_port_lock(0)) {
+        label_ = lv_label_create(lv_disp_get_scr_act(disp_));
+        lv_label_set_text(label_, "Initializing...");
+        lv_obj_set_width(label_, disp_->driver->hor_res);
+        lv_obj_set_height(label_, disp_->driver->ver_res);
+
+        notification_ = lv_label_create(lv_disp_get_scr_act(disp_));
+        lv_label_set_text(notification_, "Notification\nTest");
+        lv_obj_set_width(notification_, disp_->driver->hor_res);
+        lv_obj_set_height(notification_, disp_->driver->ver_res);
+        lv_obj_set_style_opa(notification_, LV_OPA_MIN, 0);
+        lvgl_port_unlock();
+    }
+}
+
+Display::~Display() {
+    if (notification_timer_ != nullptr) {
+        esp_timer_stop(notification_timer_);
+        esp_timer_delete(notification_timer_);
+    }
+
+    lvgl_port_lock(0);
+    if (label_ != nullptr) {
+        lv_obj_del(label_);
+        lv_obj_del(notification_);
+    }
+    lvgl_port_unlock();
+
+    if (disp_ != nullptr) {
+        lvgl_port_deinit();
+        esp_lcd_panel_del(panel_);
+        esp_lcd_panel_io_del(panel_io_);
+        i2c_master_bus_reset(i2c_bus_);
+    }
+}
+
+void Display::SetText(const std::string &text) {
+    if (label_ != nullptr) {
+        text_ = text;
+        lvgl_port_lock(0);
+        // Change the text of the label
+        lv_label_set_text(label_, text_.c_str());
+        lvgl_port_unlock();
+    }
+}
+
+void Display::ShowNotification(const std::string &text) {
+    if (notification_ != nullptr) {
+        lvgl_port_lock(0);
+        lv_label_set_text(notification_, text.c_str());
+        lv_obj_set_style_opa(notification_, LV_OPA_MAX, 0);
+        lv_obj_set_style_opa(label_, LV_OPA_MIN, 0);
+        lvgl_port_unlock();
+
+        if (notification_timer_ != nullptr) {
+            esp_timer_stop(notification_timer_);
+            esp_timer_delete(notification_timer_);
+        }
+
+        esp_timer_create_args_t timer_args = {
+            .callback = [](void *arg) {
+                Display *display = static_cast<Display*>(arg);
+                lvgl_port_lock(0);
+                lv_obj_set_style_opa(display->notification_, LV_OPA_MIN, 0);
+                lv_obj_set_style_opa(display->label_, LV_OPA_MAX, 0);
+                lvgl_port_unlock();
+            },
+            .arg = this,
+            .dispatch_method = ESP_TIMER_TASK,
+            .name = "Notification Timer",
+            .skip_unhandled_events = false,
+        };
+        ESP_ERROR_CHECK(esp_timer_create(&timer_args, &notification_timer_));
+        ESP_ERROR_CHECK(esp_timer_start_once(notification_timer_, 3000000));
+    }
+}
+
+#endif
--- a/main/Display.h
+++ b/main/Display.h
@@ -0,0 +1,36 @@
+#ifndef DISPLAY_H
+#define DISPLAY_H
+
+#include <driver/i2c_master.h>
+#include <esp_lcd_panel_io.h>
+#include <esp_lcd_panel_ops.h>
+#include <lvgl.h>
+#include <esp_timer.h>
+
+#include <string>
+
+class Display {
+public:
+    Display(int sda_pin, int scl_pin);
+    ~Display();
+
+    void SetText(const std::string &text);
+    void ShowNotification(const std::string &text);
+
+private:
+    int sda_pin_;
+    int scl_pin_;
+
+    i2c_master_bus_handle_t i2c_bus_ = nullptr;
+
+    esp_lcd_panel_io_handle_t panel_io_ = nullptr;
+    esp_lcd_panel_handle_t panel_ = nullptr;
+    lv_disp_t *disp_ = nullptr;
+    lv_obj_t *label_ = nullptr;
+    lv_obj_t *notification_ = nullptr;
+    esp_timer_handle_t notification_timer_ = nullptr;
+
+    std::string text_;
+};
+
+#endif
--- a/main/FirmwareUpgrade.cc
+++ b/main/FirmwareUpgrade.cc
@@ -0,0 +1,348 @@
+#include "FirmwareUpgrade.h"
+#include "SystemInfo.h"
+#include <cJSON.h>
+#include <esp_log.h>
+#include <esp_partition.h>
+#include <esp_http_client.h>
+#include <esp_ota_ops.h>
+#include <esp_app_format.h>
+#include <esp_chip_info.h>
+
+#include <vector>
+#include <sstream>
+#include <algorithm>
+
+#define TAG "FirmwareUpgrade"
+
+
+FirmwareUpgrade::FirmwareUpgrade(Http& http) : http_(http) {
+}
+
+FirmwareUpgrade::~FirmwareUpgrade() {
+}
+
+void FirmwareUpgrade::SetCheckVersionUrl(std::string check_version_url) {
+    check_version_url_ = check_version_url;
+}
+
+void FirmwareUpgrade::SetHeader(const std::string& key, const std::string& value) {
+    headers_[key] = value;
+}
+
+void FirmwareUpgrade::CheckVersion() {
+    std::string current_version = esp_app_get_description()->version;
+    ESP_LOGI(TAG, "Current version: %s", current_version.c_str());
+
+    if (check_version_url_.length() < 10) {
+        ESP_LOGE(TAG, "Check version URL is not properly set");
+        return;
+    }
+
+    for (const auto& header : headers_) {
+        http_.SetHeader(header.first, header.second);
+    }
+
+    http_.SetHeader("Content-Type", "application/json");
+    http_.SetContent(GetPostData());
+    http_.Open("POST", check_version_url_);
+
+    auto response = http_.GetBody();
+    http_.Close();
+
+    // Response: { "firmware": { "version": "1.0.0", "url": "http://" } }
+    // Parse the JSON response and check if the version is newer
+    // If it is, set has_new_version_ to true and store the new version and URL
+    
+    cJSON *root = cJSON_Parse(response.c_str());
+    if (root == NULL) {
+        ESP_LOGE(TAG, "Failed to parse JSON response");
+        return;
+    }
+    cJSON *firmware = cJSON_GetObjectItem(root, "firmware");
+    if (firmware == NULL) {
+        ESP_LOGE(TAG, "Failed to get firmware object");
+        cJSON_Delete(root);
+        return;
+    }
+    cJSON *version = cJSON_GetObjectItem(firmware, "version");
+    if (version == NULL) {
+        ESP_LOGE(TAG, "Failed to get version object");
+        cJSON_Delete(root);
+        return;
+    }
+    cJSON *url = cJSON_GetObjectItem(firmware, "url");
+    if (url == NULL) {
+        ESP_LOGE(TAG, "Failed to get url object");
+        cJSON_Delete(root);
+        return;
+    }
+
+    firmware_version_ = version->valuestring;
+    firmware_url_ = url->valuestring;
+    cJSON_Delete(root);
+
+    // Check if the version is newer, for example, 0.1.0 is newer than 0.0.1
+    has_new_version_ = IsNewVersionAvailable(current_version, firmware_version_);
+    if (has_new_version_) {
+        ESP_LOGI(TAG, "New version available: %s", firmware_version_.c_str());
+    } else {
+        ESP_LOGI(TAG, "Current is the latest version");
+    }
+}
+
+void FirmwareUpgrade::MarkCurrentVersionValid() {
+    auto partition = esp_ota_get_running_partition();
+    if (strcmp(partition->label, "factory") == 0) {
+        ESP_LOGI(TAG, "Running from factory partition, skipping");
+        return;
+    }
+
+    ESP_LOGI(TAG, "Running partition: %s", partition->label);
+    esp_ota_img_states_t state;
+    if (esp_ota_get_state_partition(partition, &state) != ESP_OK) {
+        ESP_LOGE(TAG, "Failed to get state of partition");
+        return;
+    }
+
+    if (state == ESP_OTA_IMG_PENDING_VERIFY) {
+        ESP_LOGI(TAG, "Marking firmware as valid");
+        esp_ota_mark_app_valid_cancel_rollback();
+    }
+}
+
+void FirmwareUpgrade::Upgrade(const std::string& firmware_url) {
+    ESP_LOGI(TAG, "Upgrading firmware from %s", firmware_url.c_str());
+    esp_ota_handle_t update_handle = 0;
+    auto update_partition = esp_ota_get_next_update_partition(NULL);
+    if (update_partition == NULL) {
+        ESP_LOGE(TAG, "Failed to get update partition");
+        return;
+    }
+
+    ESP_LOGI(TAG, "Writing to partition %s at offset 0x%lx", update_partition->label, update_partition->address);
+    bool image_header_checked = false;
+    std::string image_header;
+
+    if (!http_.Open("GET", firmware_url)) {
+        ESP_LOGE(TAG, "Failed to open HTTP connection");
+        return;
+    }
+
+    size_t content_length = http_.GetBodyLength();
+    if (content_length == 0) {
+        ESP_LOGE(TAG, "Failed to get content length");
+        http_.Close();
+        return;
+    }
+
+    char buffer[4096];
+    size_t total_read = 0, recent_read = 0;
+    auto last_calc_time = esp_timer_get_time();
+    while (true) {
+        int ret = http_.Read(buffer, sizeof(buffer));
+        if (ret < 0) {
+            ESP_LOGE(TAG, "Failed to read HTTP data: %s", esp_err_to_name(ret));
+            http_.Close();
+            return;
+        }
+
+        // Calculate speed and progress every second
+        recent_read += ret;
+        total_read += ret;
+        if (esp_timer_get_time() - last_calc_time >= 1000000 || ret == 0) {
+            size_t progress = total_read * 100 / content_length;
+            ESP_LOGI(TAG, "Progress: %zu%% (%zu/%zu), Speed: %zuB/s", progress, total_read, content_length, recent_read);
+            if (upgrade_callback_) {
+                upgrade_callback_(progress, recent_read);
+            }
+            last_calc_time = esp_timer_get_time();
+            recent_read = 0;
+        }
+
+        if (ret == 0) {
+            break;
+        }
+
+
+        if (!image_header_checked) {
+            image_header.append(buffer, ret);
+            if (image_header.size() >= sizeof(esp_image_header_t) + sizeof(esp_image_segment_header_t) + sizeof(esp_app_desc_t)) {
+                esp_app_desc_t new_app_info;
+                memcpy(&new_app_info, image_header.data() + sizeof(esp_image_header_t) + sizeof(esp_image_segment_header_t), sizeof(esp_app_desc_t));
+                ESP_LOGI(TAG, "New firmware version: %s", new_app_info.version);
+
+                auto current_version = esp_app_get_description()->version;
+                if (memcmp(new_app_info.version, current_version, sizeof(new_app_info.version)) == 0) {
+                    ESP_LOGE(TAG, "Firmware version is the same, skipping upgrade");
+                    http_.Close();
+                    return;
+                }
+
+                if (esp_ota_begin(update_partition, OTA_WITH_SEQUENTIAL_WRITES, &update_handle)) {
+                    esp_ota_abort(update_handle);
+                    http_.Close();
+                    ESP_LOGE(TAG, "Failed to begin OTA");
+                    return;
+                }
+
+                image_header_checked = true;
+            }
+        }
+        auto err = esp_ota_write(update_handle, buffer, ret);
+        if (err != ESP_OK) {
+            ESP_LOGE(TAG, "Failed to write OTA data: %s", esp_err_to_name(err));
+            esp_ota_abort(update_handle);
+            http_.Close();
+            return;
+        }
+    }
+    http_.Close();
+
+    esp_err_t err = esp_ota_end(update_handle);
+    if (err != ESP_OK) {
+        if (err == ESP_ERR_OTA_VALIDATE_FAILED) {
+            ESP_LOGE(TAG, "Image validation failed, image is corrupted");
+        } else {
+            ESP_LOGE(TAG, "Failed to end OTA: %s", esp_err_to_name(err));
+        }
+        return;
+    }
+
+    err = esp_ota_set_boot_partition(update_partition);
+    if (err != ESP_OK) {
+        ESP_LOGE(TAG, "Failed to set boot partition: %s", esp_err_to_name(err));
+        return;
+    }
+
+    ESP_LOGI(TAG, "Firmware upgrade successful, rebooting in 3 seconds...");
+    vTaskDelay(pdMS_TO_TICKS(3000));
+    esp_restart();
+}
+
+void FirmwareUpgrade::StartUpgrade(std::function<void(int progress, size_t speed)> callback) {
+    upgrade_callback_ = callback;
+    Upgrade(firmware_url_);
+}
+
+std::vector<int> FirmwareUpgrade::ParseVersion(const std::string& version) {
+    std::vector<int> versionNumbers;
+    std::stringstream ss(version);
+    std::string segment;
+    
+    while (std::getline(ss, segment, '.')) {
+        versionNumbers.push_back(std::stoi(segment));
+    }
+    
+    return versionNumbers;
+}
+
+bool FirmwareUpgrade::IsNewVersionAvailable(const std::string& currentVersion, const std::string& newVersion) {
+    std::vector<int> current = ParseVersion(currentVersion);
+    std::vector<int> newer = ParseVersion(newVersion);
+    
+    for (size_t i = 0; i < std::min(current.size(), newer.size()); ++i) {
+        if (newer[i] > current[i]) {
+            return true;
+        } else if (newer[i] < current[i]) {
+            return false;
+        }
+    }
+    
+    return newer.size() > current.size();
+}
+
+void FirmwareUpgrade::SetBoardJson(const std::string& board_json) {
+    board_json_ = board_json;
+}
+
+std::string FirmwareUpgrade::GetPostData() {
+    /* 
+        {
+            "flash_size": 4194304,
+            "psram_size": 0,
+            "minimum_free_heap_size": 123456,
+            "mac_address": "00:00:00:00:00:00",
+            "chip_model_name": "esp32s3",
+            "chip_info": {
+                "model": 1,
+                "cores": 2,
+                "revision": 0,
+                "features": 0
+            },
+            "application": {
+                "name": "my-app",
+                "version": "1.0.0",
+                "compile_time": "2021-01-01T00:00:00Z"
+                "idf_version": "4.2-dev"
+                "elf_sha256": ""
+            },
+            "partition_table": [
+                "app": {
+                    "label": "app",
+                    "type": 1,
+                    "subtype": 2,
+                    "address": 0x10000,
+                    "size": 0x100000
+                }
+            ],
+            "ota": {
+                "label": "ota_0"
+            }
+        }
+    */
+    std::string json = "{";
+    json += "\"flash_size\":" + std::to_string(SystemInfo::GetFlashSize()) + ",";
+    json += "\"minimum_free_heap_size\":" + std::to_string(SystemInfo::GetMinimumFreeHeapSize()) + ",";
+    json += "\"mac_address\":\"" + SystemInfo::GetMacAddress() + "\",";
+    json += "\"chip_model_name\":\"" + SystemInfo::GetChipModelName() + "\",";
+    json += "\"chip_info\":{";
+
+    esp_chip_info_t chip_info;
+    esp_chip_info(&chip_info);
+    json += "\"model\":" + std::to_string(chip_info.model) + ",";
+    json += "\"cores\":" + std::to_string(chip_info.cores) + ",";
+    json += "\"revision\":" + std::to_string(chip_info.revision) + ",";
+    json += "\"features\":" + std::to_string(chip_info.features);
+    json += "},";
+
+    json += "\"application\":{";
+    auto app_desc = esp_app_get_description();
+    json += "\"name\":\"" + std::string(app_desc->project_name) + "\",";
+    json += "\"version\":\"" + std::string(app_desc->version) + "\",";
+    json += "\"compile_time\":\"" + std::string(app_desc->date) + "T" + std::string(app_desc->time) + "Z\",";
+    json += "\"idf_version\":\"" + std::string(app_desc->idf_ver) + "\",";
+
+    char sha256_str[65];
+    for (int i = 0; i < 32; i++) {
+        snprintf(sha256_str + i * 2, sizeof(sha256_str) - i * 2, "%02x", app_desc->app_elf_sha256[i]);
+    }
+    json += "\"elf_sha256\":\"" + std::string(sha256_str) + "\"";
+    json += "},";
+
+    json += "\"partition_table\": [";
+    esp_partition_iterator_t it = esp_partition_find(ESP_PARTITION_TYPE_ANY, ESP_PARTITION_SUBTYPE_ANY, NULL);
+    while (it) {
+        const esp_partition_t *partition = esp_partition_get(it);
+        json += "{";
+        json += "\"label\":\"" + std::string(partition->label) + "\",";
+        json += "\"type\":" + std::to_string(partition->type) + ",";
+        json += "\"subtype\":" + std::to_string(partition->subtype) + ",";
+        json += "\"address\":" + std::to_string(partition->address) + ",";
+        json += "\"size\":" + std::to_string(partition->size);
+        json += "},";
+        it = esp_partition_next(it);
+    }
+    json.pop_back(); // Remove the last comma
+    json += "],";
+
+    json += "\"ota\":{";
+    auto ota_partition = esp_ota_get_running_partition();
+    json += "\"label\":\"" + std::string(ota_partition->label) + "\"";
+    json += "},";
+
+    json += "\"board\":" + board_json_;
+
+    // Close the JSON object
+    json += "}";
+    return json;
+}
--- a/main/FirmwareUpgrade.h
+++ b/main/FirmwareUpgrade.h
@@ -0,0 +1,39 @@
+#ifndef _FIRMWARE_UPGRADE_H
+#define _FIRMWARE_UPGRADE_H
+
+#include <functional>
+#include <string>
+#include <map>
+
+#include <Http.h>
+
+class FirmwareUpgrade {
+public:
+    FirmwareUpgrade(Http& http);
+    ~FirmwareUpgrade();
+
+    void SetBoardJson(const std::string& board_json);
+    void SetCheckVersionUrl(std::string check_version_url);
+    void SetHeader(const std::string& key, const std::string& value);
+    void CheckVersion();
+    bool HasNewVersion() { return has_new_version_; }
+    void StartUpgrade(std::function<void(int progress, size_t speed)> callback);
+    void MarkCurrentVersionValid();
+
+private:
+    Http& http_;
+    std::string check_version_url_;
+    bool has_new_version_ = false;
+    std::string firmware_version_;
+    std::string firmware_url_;
+    std::string board_json_;
+    std::map<std::string, std::string> headers_;
+
+    void Upgrade(const std::string& firmware_url);
+    std::function<void(int progress, size_t speed)> upgrade_callback_;
+    std::vector<int> ParseVersion(const std::string& version);
+    bool IsNewVersionAvailable(const std::string& currentVersion, const std::string& newVersion);
+    std::string GetPostData();
+};
+
+#endif // _FIRMWARE_UPGRADE_H
--- a/main/Kconfig.projbuild
+++ b/main/Kconfig.projbuild
@@ -1,14 +1,20 @@
 menu "Xiaozhi Assistant"

+config OTA_VERSION_URL
+    string "OTA Version URL"
+    default "https://api.tenclass.net/xiaozhi/ota/"
+    help
+        The application will access this URL to check for updates.
+
 config WEBSOCKET_URL
    string "Websocket URL"
-    default "wss://"
+    default "wss://api.tenclass.net/xiaozhi/v1/"
    help
        Communication with the server through websocket after wake up.

 config WEBSOCKET_ACCESS_TOKEN
    string "Websocket Access Token"
-    default ""
+    default "test-token"
    help
        Access token for websocket communication.

@@ -24,48 +30,206 @@ config AUDIO_OUTPUT_SAMPLE_RATE
    help
        Audio output sample rate.

-config AUDIO_DEVICE_I2S_GPIO_BCLK
-    int "I2S GPIO BCLK"
-    default 5
+choice AUDIO_CODEC
+    prompt "Audio Codec"
+    default AUDIO_CODEC_NONE
    help
-        GPIO number of the I2S BCLK.
+        Audio codec.
+    config AUDIO_CODEC_ES8311_ES7210
+        bool "Box: ES8311 + ES7210"
+    config AUDIO_CODEC_NONE
+        bool "None"
+endchoice

-config AUDIO_DEVICE_I2S_GPIO_WS
-    int "I2S GPIO WS"
-    default 4
-    help
-        GPIO number of the I2S WS.
+menu "Box Audio Codec I2C and PA Control"
+    depends on AUDIO_CODEC_ES8311_ES7210
+    
+    config AUDIO_CODEC_I2C_SDA_PIN
+        int "Audio Codec I2C SDA Pin"
+        default 39
+        help
+            Audio codec I2C SDA pin.

-config AUDIO_DEVICE_I2S_GPIO_DOUT
-    int "I2S GPIO DOUT"
-    default 6
-    help
-        GPIO number of the I2S DOUT.
+    config AUDIO_CODEC_I2C_SCL_PIN
+        int "Audio Codec I2C SCL Pin"
+        default 38
+        help
+            Audio codec I2C SCL pin.
+    
+    config AUDIO_CODEC_PA_PIN
+        int "Audio Codec PA Pin"
+        default 40
+        help
+            Audio codec PA pin.
+    
+    config AUDIO_CODEC_INPUT_REFERENCE
+        bool "Audio Codec Input Reference"
+        default y
+        help
+            Audio codec input reference.
+endmenu

-config AUDIO_DEVICE_I2S_GPIO_DIN
-    int "I2S GPIO DIN"
-    default 3
+choice AUDIO_I2S_METHOD
+    prompt "Audio I2S Method"
+    default AUDIO_I2S_METHOD_SIMPLEX if AUDIO_CODEC_NONE
+    default AUDIO_I2S_METHOD_DUPLEX if AUDIO_CODEC_ES8311_ES7210
    help
-        GPIO number of the I2S DIN.
-    
-config AUDIO_DEVICE_I2S_SIMPLEX
-    bool "I2S Simplex"
-    default y
-    help
-        Enable I2S Simplex mode.
-    
-config AUDIO_DEVICE_I2S_MIC_GPIO_BCLK
-    int "I2S MIC GPIO BCLK"
-    default 11
-    depends on AUDIO_DEVICE_I2S_SIMPLEX
-    help
-        GPIO number of the I2S MIC BCLK.
-    
-config AUDIO_DEVICE_I2S_MIC_GPIO_WS
-    int "I2S MIC GPIO WS"
-    default 10
-    depends on AUDIO_DEVICE_I2S_SIMPLEX
-    help
-        GPIO number of the I2S MIC WS.
+        Audio I2S method.
+    config AUDIO_I2S_METHOD_SIMPLEX
+        bool "Simplex"
+        help
+            Use I2S 0 as the audio input and I2S 1 as the audio output.
+    config AUDIO_I2S_METHOD_DUPLEX
+        bool "Duplex"
+        help
+            Use I2S 0 as the audio input and audio output.
+endchoice
+
+menu "Audio I2S Simplex"
+    depends on AUDIO_I2S_METHOD_SIMPLEX
+
+    config AUDIO_DEVICE_I2S_MIC_GPIO_WS
+        int "I2S MIC GPIO WS"
+        default 4
+        help
+            GPIO number of the I2S MIC WS.
+
+    config AUDIO_DEVICE_I2S_MIC_GPIO_SCK
+        int "I2S MIC GPIO BCLK"
+        default 5
+        help
+            GPIO number of the I2S MIC SCK.
+
+    config AUDIO_DEVICE_I2S_MIC_GPIO_DIN
+        int "I2S MIC GPIO DIN"
+        default 6
+        help
+            GPIO number of the I2S MIC DIN.
+
+    config AUDIO_DEVICE_I2S_SPK_GPIO_DOUT
+        int "I2S SPK GPIO DOUT"
+        default 7
+        help
+            GPIO number of the I2S SPK DOUT.
+        
+    config AUDIO_DEVICE_I2S_SPK_GPIO_BCLK
+        int "I2S SPK GPIO BCLK"
+        default 15
+        help
+            GPIO number of the I2S SPK BCLK.
+        
+    config AUDIO_DEVICE_I2S_SPK_GPIO_LRCK
+        int "I2S SPK GPIO WS"
+        default 16
+        help
+            GPIO number of the I2S SPK LRCK.
+
+endmenu
+
+menu "Audio I2S Duplex"
+    depends on AUDIO_I2S_METHOD_DUPLEX
+
+    config AUDIO_DEVICE_I2S_GPIO_MCLK
+        int "I2S GPIO MCLK"
+        default -1
+        help
+            GPIO number of the I2S WS.
+
+    config AUDIO_DEVICE_I2S_GPIO_LRCK
+        int "I2S GPIO LRCK"
+        default 4
+        help
+            GPIO number of the I2S LRCK.
+
+    config AUDIO_DEVICE_I2S_GPIO_BCLK
+        int "I2S GPIO BCLK / SCLK"
+        default 5
+        help
+            GPIO number of the I2S BCLK.
+
+    config AUDIO_DEVICE_I2S_GPIO_DIN
+        int "I2S GPIO DIN"
+        default 6
+        help
+            GPIO number of the I2S DIN.
+
+    config AUDIO_DEVICE_I2S_GPIO_DOUT
+        int "I2S GPIO DOUT"
+        default 7
+        help
+            GPIO number of the I2S DOUT.
+
+endmenu
+
+config BOOT_BUTTON_GPIO
+    int "Boot Button GPIO"
+    default 0
+    help
+        GPIO number of the boot button.
+
+config VOLUME_UP_BUTTON_GPIO
+    int "Volume Up Button GPIO"
+    default 40
+    help
+        GPIO number of the volume up button.
+
+config VOLUME_DOWN_BUTTON_GPIO
+    int "Volume Down Button GPIO"
+    default 39
+    help
+        GPIO number of the volume down button.
+
+config USE_AFE_SR
+    bool "Use Espressif AFE SR"
+    default y
+    help
+        Use AFE SR for wake word detection.
+
+config USE_ML307
+    bool "Use ML307"
+    default n
+    help
+        Use ML307 as the modem.
+
+config ML307_RX_PIN
+    int "ML307 RX Pin"
+    default 11
+    depends on USE_ML307
+    help
+        GPIO number of the ML307 RX.
+
+config ML307_TX_PIN
+    int "ML307 TX Pin"
+    default 12
+    depends on USE_ML307
+    help
+        GPIO number of the ML307 TX.
+
+config USE_DISPLAY
+    bool "Use Display"
+    default n
+    help
+        Use Display.
+
+config DISPLAY_HEIGHT
+    int "Display Height"
+    default 32
+    depends on USE_DISPLAY
+    help
+        Display height in pixels.
+
+config DISPLAY_SDA_PIN
+    int "Display SDA Pin"
+    default 41
+    depends on USE_DISPLAY
+    help
+        GPIO number of the Display SDA.
+
+config DISPLAY_SCL_PIN
+    int "Display SCL Pin"
+    default 42
+    depends on USE_DISPLAY
+    help
+        GPIO number of the Display SCL.

 endmenu
--- a/main/SystemInfo.cc
+++ b/main/SystemInfo.cc
@@ -0,0 +1,127 @@
+#include "SystemInfo.h"
+#include <freertos/task.h>
+#include <esp_log.h>
+#include <esp_flash.h>
+#include <esp_mac.h>
+#include <esp_system.h>
+#include <esp_partition.h>
+#include <esp_app_desc.h>
+#include <esp_ota_ops.h>
+
+
+#define TAG "SystemInfo"
+
+size_t SystemInfo::GetFlashSize() {
+    uint32_t flash_size;
+    if (esp_flash_get_size(NULL, &flash_size) != ESP_OK) {
+        ESP_LOGE(TAG, "Failed to get flash size");
+        return 0;
+    }
+    return (size_t)flash_size;
+}
+
+size_t SystemInfo::GetMinimumFreeHeapSize() {
+    return esp_get_minimum_free_heap_size();
+}
+
+size_t SystemInfo::GetFreeHeapSize() {
+    return esp_get_free_heap_size();
+}
+
+std::string SystemInfo::GetMacAddress() {
+    uint8_t mac[6];
+    esp_read_mac(mac, ESP_MAC_WIFI_STA);
+    char mac_str[18];
+    snprintf(mac_str, sizeof(mac_str), "%02x:%02x:%02x:%02x:%02x:%02x", mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+    return std::string(mac_str);
+}
+
+std::string SystemInfo::GetChipModelName() {
+    return std::string(CONFIG_IDF_TARGET);
+}
+
+esp_err_t SystemInfo::PrintRealTimeStats(TickType_t xTicksToWait) {
+    #define ARRAY_SIZE_OFFSET 5
+    TaskStatus_t *start_array = NULL, *end_array = NULL;
+    UBaseType_t start_array_size, end_array_size;
+    configRUN_TIME_COUNTER_TYPE start_run_time, end_run_time;
+    esp_err_t ret;
+    uint32_t total_elapsed_time;
+
+    //Allocate array to store current task states
+    start_array_size = uxTaskGetNumberOfTasks() + ARRAY_SIZE_OFFSET;
+    start_array = (TaskStatus_t*)malloc(sizeof(TaskStatus_t) * start_array_size);
+    if (start_array == NULL) {
+        ret = ESP_ERR_NO_MEM;
+        goto exit;
+    }
+    //Get current task states
+    start_array_size = uxTaskGetSystemState(start_array, start_array_size, &start_run_time);
+    if (start_array_size == 0) {
+        ret = ESP_ERR_INVALID_SIZE;
+        goto exit;
+    }
+
+    vTaskDelay(xTicksToWait);
+
+    //Allocate array to store tasks states post delay
+    end_array_size = uxTaskGetNumberOfTasks() + ARRAY_SIZE_OFFSET;
+    end_array = (TaskStatus_t*)malloc(sizeof(TaskStatus_t) * end_array_size);
+    if (end_array == NULL) {
+        ret = ESP_ERR_NO_MEM;
+        goto exit;
+    }
+    //Get post delay task states
+    end_array_size = uxTaskGetSystemState(end_array, end_array_size, &end_run_time);
+    if (end_array_size == 0) {
+        ret = ESP_ERR_INVALID_SIZE;
+        goto exit;
+    }
+
+    //Calculate total_elapsed_time in units of run time stats clock period.
+    total_elapsed_time = (end_run_time - start_run_time);
+    if (total_elapsed_time == 0) {
+        ret = ESP_ERR_INVALID_STATE;
+        goto exit;
+    }
+
+    printf("| Task | Run Time | Percentage\n");
+    //Match each task in start_array to those in the end_array
+    for (int i = 0; i < start_array_size; i++) {
+        int k = -1;
+        for (int j = 0; j < end_array_size; j++) {
+            if (start_array[i].xHandle == end_array[j].xHandle) {
+                k = j;
+                //Mark that task have been matched by overwriting their handles
+                start_array[i].xHandle = NULL;
+                end_array[j].xHandle = NULL;
+                break;
+            }
+        }
+        //Check if matching task found
+        if (k >= 0) {
+            uint32_t task_elapsed_time = end_array[k].ulRunTimeCounter - start_array[i].ulRunTimeCounter;
+            uint32_t percentage_time = (task_elapsed_time * 100UL) / (total_elapsed_time * CONFIG_FREERTOS_NUMBER_OF_CORES);
+            printf("| %-16s | %8lu | %4lu%%\n", start_array[i].pcTaskName, task_elapsed_time, percentage_time);
+        }
+    }
+
+    //Print unmatched tasks
+    for (int i = 0; i < start_array_size; i++) {
+        if (start_array[i].xHandle != NULL) {
+            printf("| %s | Deleted\n", start_array[i].pcTaskName);
+        }
+    }
+    for (int i = 0; i < end_array_size; i++) {
+        if (end_array[i].xHandle != NULL) {
+            printf("| %s | Created\n", end_array[i].pcTaskName);
+        }
+    }
+    ret = ESP_OK;
+
+exit:    //Common return path
+    free(start_array);
+    free(end_array);
+    return ret;
+}
+
--- a/main/SystemInfo.h
+++ b/main/SystemInfo.h
@@ -0,0 +1,19 @@
+#ifndef _SYSTEM_INFO_H_
+#define _SYSTEM_INFO_H_
+
+#include <string>
+
+#include <esp_err.h>
+#include <freertos/FreeRTOS.h>
+
+class SystemInfo {
+public:
+    static size_t GetFlashSize();
+    static size_t GetMinimumFreeHeapSize();
+    static size_t GetFreeHeapSize();
+    static std::string GetMacAddress();
+    static std::string GetChipModelName();
+    static esp_err_t PrintRealTimeStats(TickType_t xTicksToWait);
+};
+
+#endif // _SYSTEM_INFO_H_
--- a/main/SystemReset.cc
+++ b/main/SystemReset.cc
@@ -1,10 +1,10 @@
 #include "SystemReset.h"
-#include "esp_log.h"
-#include "nvs_flash.h"
-#include "driver/gpio.h"
-#include "esp_partition.h"
-#include "esp_system.h"
-#include "freertos/FreeRTOS.h"
+#include <esp_log.h>
+#include <nvs_flash.h>
+#include <driver/gpio.h>
+#include <esp_partition.h>
+#include <esp_system.h>
+#include <freertos/FreeRTOS.h>


 #define TAG "SystemReset"
--- a/main/WakeWordDetect.cc
+++ b/main/WakeWordDetect.cc
@@ -0,0 +1,209 @@
+#include <esp_log.h>
+#include <model_path.h>
+
+#include "WakeWordDetect.h"
+#include "Application.h"
+
+#define DETECTION_RUNNING_EVENT 1
+#define WAKE_WORD_ENCODED_EVENT 2
+
+static const char* TAG = "WakeWordDetect";
+
+WakeWordDetect::WakeWordDetect()
+    : afe_detection_data_(nullptr),
+      wake_word_pcm_(),
+      wake_word_opus_() {
+
+    event_group_ = xEventGroupCreate();
+}
+
+WakeWordDetect::~WakeWordDetect() {
+    if (afe_detection_data_ != nullptr) {
+        esp_afe_sr_v1.destroy(afe_detection_data_);
+    }
+
+    if (wake_word_encode_task_stack_ != nullptr) {
+        free(wake_word_encode_task_stack_);
+    }
+
+    vEventGroupDelete(event_group_);
+}
+
+void WakeWordDetect::Initialize(int channels, bool reference) {
+    channels_ = channels;
+    reference_ = reference;
+    int ref_num = reference_ ? 1 : 0;
+
+    srmodel_list_t *models = esp_srmodel_init("model");
+    for (int i = 0; i < models->num; i++) {
+        ESP_LOGI(TAG, "Model %d: %s", i, models->model_name[i]);
+        if (strstr(models->model_name[i], ESP_WN_PREFIX) != NULL) {
+            wakenet_model_ = models->model_name[i];
+        }
+    }
+
+    afe_config_t afe_config = {
+        .aec_init = reference_,
+        .se_init = true,
+        .vad_init = true,
+        .wakenet_init = true,
+        .voice_communication_init = false,
+        .voice_communication_agc_init = false,
+        .voice_communication_agc_gain = 10,
+        .vad_mode = VAD_MODE_3,
+        .wakenet_model_name = wakenet_model_,
+        .wakenet_model_name_2 = NULL,
+        .wakenet_mode = DET_MODE_90,
+        .afe_mode = SR_MODE_HIGH_PERF,
+        .afe_perferred_core = 1,
+        .afe_perferred_priority = 1,
+        .afe_ringbuf_size = 50,
+        .memory_alloc_mode = AFE_MEMORY_ALLOC_MORE_PSRAM,
+        .afe_linear_gain = 1.0,
+        .agc_mode = AFE_MN_PEAK_AGC_MODE_2,
+        .pcm_config = {
+            .total_ch_num = channels_,
+            .mic_num = channels_ - ref_num,
+            .ref_num = ref_num,
+            .sample_rate = 16000
+        },
+        .debug_init = false,
+        .debug_hook = {{ AFE_DEBUG_HOOK_MASE_TASK_IN, NULL }, { AFE_DEBUG_HOOK_FETCH_TASK_IN, NULL }},
+        .afe_ns_mode = NS_MODE_SSP,
+        .afe_ns_model_name = NULL,
+        .fixed_first_channel = true,
+    };
+
+    afe_detection_data_ = esp_afe_sr_v1.create_from_config(&afe_config);
+
+    xTaskCreate([](void* arg) {
+        auto this_ = (WakeWordDetect*)arg;
+        this_->AudioDetectionTask();
+        vTaskDelete(NULL);
+    }, "audio_detection", 4096 * 2, this, 1, NULL);
+}
+
+void WakeWordDetect::OnWakeWordDetected(std::function<void()> callback) {
+    wake_word_detected_callback_ = callback;
+}
+
+void WakeWordDetect::OnVadStateChange(std::function<void(bool speaking)> callback) {
+    vad_state_change_callback_ = callback;
+}
+
+void WakeWordDetect::StartDetection() {
+    xEventGroupSetBits(event_group_, DETECTION_RUNNING_EVENT);
+}
+
+void WakeWordDetect::StopDetection() {
+    xEventGroupClearBits(event_group_, DETECTION_RUNNING_EVENT);
+}
+
+bool WakeWordDetect::IsDetectionRunning() {
+    return xEventGroupGetBits(event_group_) & DETECTION_RUNNING_EVENT;
+}
+
+void WakeWordDetect::Feed(std::vector<int16_t>& data) {
+    input_buffer_.insert(input_buffer_.end(), data.begin(), data.end());
+
+    auto chunk_size = esp_afe_sr_v1.get_feed_chunksize(afe_detection_data_) * channels_;
+    while (input_buffer_.size() >= chunk_size) {
+        esp_afe_sr_v1.feed(afe_detection_data_, input_buffer_.data());
+        input_buffer_.erase(input_buffer_.begin(), input_buffer_.begin() + chunk_size);
+    }
+}
+
+void WakeWordDetect::AudioDetectionTask() {
+    auto chunk_size = esp_afe_sr_v1.get_fetch_chunksize(afe_detection_data_);
+    ESP_LOGI(TAG, "Audio detection task started, chunk size: %d", chunk_size);
+
+    while (true) {
+        xEventGroupWaitBits(event_group_, DETECTION_RUNNING_EVENT, pdFALSE, pdTRUE, portMAX_DELAY);
+
+        auto res = esp_afe_sr_v1.fetch(afe_detection_data_);
+        if (res == nullptr || res->ret_value == ESP_FAIL) {
+            if (res != nullptr) {
+                ESP_LOGI(TAG, "Error code: %d", res->ret_value);
+            }
+            continue;;
+        }
+
+        // Store the wake word data for voice recognition, like who is speaking
+        StoreWakeWordData((uint16_t*)res->data, res->data_size / sizeof(uint16_t));
+
+        // VAD state change
+        if (vad_state_change_callback_) {
+            if (res->vad_state == AFE_VAD_SPEECH && !is_speaking_) {
+                is_speaking_ = true;
+                vad_state_change_callback_(true);
+            } else if (res->vad_state == AFE_VAD_SILENCE && is_speaking_) {
+                is_speaking_ = false;
+                vad_state_change_callback_(false);
+            }
+        }
+
+        if (res->wakeup_state == WAKENET_DETECTED) {
+            ESP_LOGI(TAG, "Wake word detected");
+            StopDetection();
+
+            if (wake_word_detected_callback_) {
+                wake_word_detected_callback_();
+            }
+        }
+    }
+}
+
+void WakeWordDetect::StoreWakeWordData(uint16_t* data, size_t samples) {
+    // store audio data to wake_word_pcm_
+    std::vector<int16_t> pcm(data, data + samples);
+    wake_word_pcm_.emplace_back(std::move(pcm));
+    // keep about 2 seconds of data, detect duration is 32ms (sample_rate == 16000, chunksize == 512)
+    while (wake_word_pcm_.size() > 2000 / 32) {
+        wake_word_pcm_.pop_front();
+    }
+}
+
+void WakeWordDetect::EncodeWakeWordData() {
+    if (wake_word_encode_task_stack_ == nullptr) {
+        wake_word_encode_task_stack_ = (StackType_t*)malloc(4096 * 8);
+    }
+    wake_word_encode_task_ = xTaskCreateStatic([](void* arg) {
+        auto this_ = (WakeWordDetect*)arg;
+        auto start_time = esp_timer_get_time();
+        // encode detect packets
+        OpusEncoder* encoder = new OpusEncoder();
+        encoder->Configure(16000, 1, 60);
+        encoder->SetComplexity(0);
+        this_->wake_word_opus_.resize(4096 * 4);
+        size_t offset = 0;
+
+        for (auto& pcm: this_->wake_word_pcm_) {
+            encoder->Encode(pcm, [this_, &offset](const uint8_t* opus, size_t opus_size) {
+                size_t protocol_size = sizeof(BinaryProtocol) + opus_size;
+                if (offset + protocol_size < this_->wake_word_opus_.size()) {
+                    auto protocol = (BinaryProtocol*)(&this_->wake_word_opus_[offset]);
+                    protocol->version = htons(PROTOCOL_VERSION);
+                    protocol->type = htons(0);
+                    protocol->reserved = 0;
+                    protocol->timestamp = 0;
+                    protocol->payload_size = htonl(opus_size);
+                    memcpy(protocol->payload, opus, opus_size);
+                    offset += protocol_size;
+                }
+            });
+        }
+        this_->wake_word_pcm_.clear();
+        this_->wake_word_opus_.resize(offset);
+
+        auto end_time = esp_timer_get_time();
+        ESP_LOGI(TAG, "Encode wake word opus: %zu bytes in %lld ms", this_->wake_word_opus_.size(), (end_time - start_time) / 1000);
+        xEventGroupSetBits(this_->event_group_, WAKE_WORD_ENCODED_EVENT);
+        delete encoder;
+        vTaskDelete(NULL);
+    }, "encode_detect_packets", 4096 * 8, this, 1, wake_word_encode_task_stack_, &wake_word_encode_task_buffer_);
+}
+
+const std::string&& WakeWordDetect::GetWakeWordStream() {
+    xEventGroupWaitBits(event_group_, WAKE_WORD_ENCODED_EVENT, pdTRUE, pdTRUE, portMAX_DELAY);
+    return std::move(wake_word_opus_);
+}
--- a/main/WakeWordDetect.h
+++ b/main/WakeWordDetect.h
@@ -0,0 +1,53 @@
+#ifndef WAKE_WORD_DETECT_H
+#define WAKE_WORD_DETECT_H
+
+#include <esp_afe_sr_models.h>
+#include <esp_nsn_models.h>
+
+#include <freertos/FreeRTOS.h>
+#include <freertos/task.h>
+#include <freertos/event_groups.h>
+
+#include <list>
+#include <string>
+#include <vector>
+#include <functional>
+
+
+class WakeWordDetect {
+public:
+    WakeWordDetect();
+    ~WakeWordDetect();
+
+    void Initialize(int channels, bool reference);
+    void Feed(std::vector<int16_t>& data);
+    void OnWakeWordDetected(std::function<void()> callback);
+    void OnVadStateChange(std::function<void(bool speaking)> callback);
+    void StartDetection();
+    void StopDetection();
+    bool IsDetectionRunning();
+    void EncodeWakeWordData();
+    const std::string&& GetWakeWordStream();
+
+private:
+    esp_afe_sr_data_t* afe_detection_data_ = nullptr;
+    char* wakenet_model_ = NULL;
+    std::vector<int16_t> input_buffer_;
+    EventGroupHandle_t event_group_;
+    std::function<void()> wake_word_detected_callback_;
+    std::function<void(bool speaking)> vad_state_change_callback_;
+    bool is_speaking_ = false;
+    int channels_;
+    bool reference_;
+
+    TaskHandle_t wake_word_encode_task_ = nullptr;
+    StaticTask_t wake_word_encode_task_buffer_;
+    StackType_t* wake_word_encode_task_stack_ = nullptr;
+    std::list<std::vector<int16_t>> wake_word_pcm_;
+    std::string wake_word_opus_;
+
+    void StoreWakeWordData(uint16_t* data, size_t size);
+    void AudioDetectionTask();
+};
+
+#endif
--- a/main/idf_component.yml
+++ b/main/idf_component.yml
@@ -1,24 +1,15 @@
 ## IDF Component Manager Manifest File
 dependencies:
-  78/esp-builtin-led: "^1.0.0"
-  78/esp-wifi-connect: "^1.0.0"
-  78/esp-ota: "^1.0.0"
-  78/esp-websocket: "^1.0.0"
-  78/esp-opus-encoder: "^1.0.1"
+  78/esp-builtin-led: "^1.0.2"
+  78/esp-wifi-connect: "^1.2.0"
+  78/esp-opus-encoder: "^1.0.2"
+  78/esp-ml307: "^1.2.1"
+  espressif/esp_codec_dev: "^1.3.1"
  espressif/esp-sr: "^1.9.0"
+  espressif/button: "^3.3.1"
+  lvgl/lvgl: "^8.4.0"
+  esp_lvgl_port: "^1.4.0"
  ## Required IDF version
  idf:
    version: ">=5.3"
-  # # Put list of dependencies here
-  # # For components maintained by Espressif:
-  # component: "~1.0.0"
-  # # For 3rd party components:
-  # username/component: ">=1.0.0,<2.0.0"
-  # username2/component2:
-  #   version: "~1.0.0"
-  #   # For transient dependencies `public` flag can be set.
-  #   # `public` flag doesn't have an effect dependencies of the `main` component.
-  #   # All dependencies of `main` are public by default.
-  #   public: true
-description: "An AI voice assistant for ESP32"
-url: "https://github.com/78/xiaozhi-esp32"
+
--- a/main/main.cc
+++ b/main/main.cc
@@ -1,22 +1,31 @@
-#include <cstdio>
+#include <esp_log.h>
+#include <esp_err.h>
+#include <nvs.h>
+#include <nvs_flash.h>
+#include <driver/gpio.h>
+#include <esp_event.h>

-#include "esp_log.h"
-#include "esp_err.h"
-#include "nvs.h"
-#include "nvs_flash.h"
-#include "driver/gpio.h"
-
-#include "WifiConfigurationAp.h"
 #include "Application.h"
 #include "SystemInfo.h"
 #include "SystemReset.h"
-#include "BuiltinLed.h"

 #define TAG "main"
-#define STATS_TICKS         pdMS_TO_TICKS(1000)

 extern "C" void app_main(void)
 {
+#ifdef CONFIG_AUDIO_CODEC_ES8311_ES7210
+    // Make GPIO15 HIGH to enable the 4G module
+    gpio_config_t ml307_enable_config = {
+        .pin_bit_mask = (1ULL << 15),
+        .mode = GPIO_MODE_OUTPUT,
+        .pull_up_en = GPIO_PULLUP_DISABLE,
+        .pull_down_en = GPIO_PULLDOWN_DISABLE,
+        .intr_type = GPIO_INTR_DISABLE,
+    };
+    gpio_config(&ml307_enable_config);
+    gpio_set_level(GPIO_NUM_15, 1);
+#endif
+
    // Check if the reset button is pressed
    SystemReset system_reset;
    system_reset.CheckButtons();
@@ -32,29 +41,15 @@ extern "C" void app_main(void)
    }
    ESP_ERROR_CHECK(ret);

-    // Get the WiFi configuration
-    nvs_handle_t nvs_handle;
-    ret = nvs_open("wifi", NVS_READONLY, &nvs_handle);
-
-    // If the WiFi configuration is not found, launch the WiFi configuration AP
-    if (ret != ESP_OK) {
-        auto& builtin_led = BuiltinLed::GetInstance();
-        builtin_led.SetBlue();
-        builtin_led.Blink(1000, 500);
-
-        WifiConfigurationAp::GetInstance().Start("Xiaozhi");
-        return;
-    }
-    nvs_close(nvs_handle);
-    
    // Otherwise, launch the application
    Application::GetInstance().Start();

    // Dump CPU usage every 10 second
    while (true) {
        vTaskDelay(10000 / portTICK_PERIOD_MS);
-        // SystemInfo::PrintRealTimeStats(STATS_TICKS);
-        int free_sram = heap_caps_get_minimum_free_size(MALLOC_CAP_INTERNAL);
-        ESP_LOGI(TAG, "Free heap size: %u minimal internal: %u", SystemInfo::GetFreeHeapSize(), free_sram);
+        // SystemInfo::PrintRealTimeStats(pdMS_TO_TICKS(1000));
+        int free_sram = heap_caps_get_free_size(MALLOC_CAP_INTERNAL);
+        int min_free_sram = heap_caps_get_minimum_free_size(MALLOC_CAP_INTERNAL);
+        ESP_LOGI(TAG, "Free internal: %u minimal internal: %u", free_sram, min_free_sram);
    }
 }
--- a/pack.py
+++ b/pack.py
@@ -1,68 +0,0 @@
-#! /usr/bin/env python3
-
-import csv
-import os
-
-# 例如：1000, 0x1000, 1M
-def read_value(text):
-    text = text.strip()
-    if text.endswith('K'):
-        return int(text[:-1]) * 1024
-    elif text.endswith('M'):
-        return int(text[:-1]) * 1024 * 1024
-    else:
-        if text.startswith('0x'):
-            return int(text, 16)
-        else:
-            return int(text)
-
-
-def write_bin(image_data, offset, file_path, max_size=None):
-    # Read file_path and write to image_data
-    with open(file_path, 'rb') as f:
-        data = f.read()
-        if max_size is not None:
-            assert len(data) <= max_size, f"Data from {file_path} is too large"
-        image_data[offset:offset+len(data)] = data
-        print(f"Write {os.path.basename(file_path)} to 0x{offset:08X} with size 0x{len(data):08X}")
-
-
-'''
-根据 partitions.csv 文件，把 bin 文件打包成一个 4MB 的 image 文件，方便烧录
-'''
-def pack_firmware_image():
-    # Create a 4MB image filled with 0xFF
-    image_size = 4 * 1024 * 1024
-    image_data = bytearray([0xFF] * image_size)
-
-    build_dir = os.path.join(os.path.dirname(__file__), 'build')
-    write_bin(image_data, 0, os.path.join(build_dir, 'bootloader', 'bootloader.bin'))
-    write_bin(image_data, 0x8000, os.path.join(build_dir, 'partition_table', 'partition-table.bin'))
-
-    # 读取 partitions.csv 文件
-    with open('partitions.csv', 'r') as f:
-        reader = csv.reader(f)
-        for row in reader:
-            if row[0] == 'model':   
-                file_path = os.path.join(build_dir, 'srmodels', 'srmodels.bin')
-            elif row[0] == 'factory':
-                file_path = os.path.join(build_dir, 'xiaozhi.bin')
-            else:
-                continue
-
-            offset = read_value(row[3])
-            size = read_value(row[4])
-            write_bin(image_data, offset, file_path, size)
-
-    # 写入 image 文件
-    output_path = os.path.join(build_dir, 'xiaozhi.img')
-    with open(output_path, 'wb') as f:
-        f.write(image_data)
-    print(f"Image file {output_path} created with size 0x{len(image_data):08X}")
-
-    # Compress image with zip without directory
-    os.system(f"zip -j {output_path}.zip {output_path}")
-
-
-if __name__ == '__main__':
-    pack_firmware_image()
--- a/partitions.csv
+++ b/partitions.csv
@@ -3,7 +3,7 @@
 nvs,      data, nvs,     0x9000,    0x4000,
 otadata,  data, ota,     0xd000,    0x2000,
 phy_init, data, phy,     0xf000,    0x1000,
-model,    data, spiffs,  0x100000,  1M,
-factory,  app,  factory, 0x200000,  2M,
-ota_0,    app,  ota_0,   0x400000,  2M,
-ota_1,    app,  ota_1,   0x600000,  2M,
+model,    data, spiffs,  0x10000,   0xF0000,
+factory,  app,  factory, 0x200000,  4M,
+ota_0,    app,  ota_0,   0x600000,  4M,
+ota_1,    app,  ota_1,   0xA00000,  4M,
--- a/partitions_4M.csv
+++ b/partitions_4M.csv
@@ -0,0 +1,7 @@
+# ESP-IDF Partition Table
+# Name,   Type, SubType, Offset,  Size, Flags
+nvs,      data, nvs,     0x9000,    0x4000,
+otadata,  data, ota,     0xd000,    0x2000,
+phy_init, data, phy,     0xf000,    0x1000,
+model,    data, spiffs,  0x10000,   0xF0000,
+factory,  app,  factory, 0x100000,  3M,
--- a/publish.py
+++ b/publish.py
@@ -1,51 +0,0 @@
-#! /usr/bin/env python3
-from dotenv import load_dotenv
-load_dotenv()
-
-import os
-import oss2
-import json
-
-def get_version():
-    with open('CMakeLists.txt', 'r') as f:
-        for line in f:
-            if line.startswith('set(PROJECT_VER'):
-                return line.split('"')[1]
-    return '0.0.0'
-
-def upload_bin_to_oss(bin_path, oss_key):
-    auth = oss2.Auth(os.environ['OSS_ACCESS_KEY_ID'], os.environ['OSS_ACCESS_KEY_SECRET'])
-    bucket = oss2.Bucket(auth, os.environ['OSS_ENDPOINT'], os.environ['OSS_BUCKET_NAME'])
-    bucket.put_object(oss_key, open(bin_path, 'rb'))
-
-
-if __name__ == '__main__':
-    # 获取版本号
-    version = get_version()
-    print(f'version: {version}')
-
-    # 上传 bin 文件到 OSS
-    upload_bin_to_oss('build/xiaozhi.bin', f'firmwares/xiaozhi-{version}.bin')
-
-    # File URL
-    file_url = os.path.join(os.environ['OSS_BUCKET_URL'], f'firmwares/xiaozhi-{version}.bin')
-    print(f'Uploaded bin to OSS: {file_url}')
-
-    firmware_json = {
-        "version": version,
-        "url": file_url
-    }
-    with open(f"build/firmware.json", "w") as f:
-        json.dump(firmware_json, f, indent=4)
-    
-    # copy firmware.json to server
-    firmware_config_path = os.environ['FIRMWARE_CONFIG_PATH']
-    ret = os.system(f'scp build/firmware.json {firmware_config_path}')
-    if ret != 0:
-        print(f'Failed to copy firmware.json to server')
-        exit(1)
-    print(f'Copied firmware.json to server: {firmware_config_path}')
-
-
-
-    
--- a/sdkconfig.box
+++ b/sdkconfig.box
--- a/sdkconfig.defaults
+++ b/sdkconfig.defaults
@@ -3,22 +3,11 @@ CONFIG_BOOTLOADER_LOG_LEVEL_NONE=y
 CONFIG_BOOTLOADER_SKIP_VALIDATE_ALWAYS=y
 CONFIG_BOOTLOADER_APP_ROLLBACK_ENABLE=y

-CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_240=y
-
-CONFIG_SPIRAM=y
-CONFIG_SPIRAM_MODE_OCT=y
-CONFIG_SPIRAM_SPEED_80M=y
-CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=4096
-CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y
-CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=32768
-CONFIG_SPIRAM_MEMTEST=n
-
 CONFIG_HTTPD_MAX_REQ_HDR_LEN=2048
 CONFIG_HTTPD_MAX_URI_LEN=2048

 CONFIG_PARTITION_TABLE_CUSTOM=y
 CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
-CONFIG_PARTITION_TABLE_FILENAME="partitions.csv"
 CONFIG_PARTITION_TABLE_OFFSET=0x8000

 CONFIG_USE_WAKENET=y
--- a/sdkconfig.defaults.esp32c3
+++ b/sdkconfig.defaults.esp32c3
@@ -0,0 +1,6 @@
+
+CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y
+
+CONFIG_PARTITION_TABLE_CUSTOM=y
+CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_4M.csv"
+CONFIG_PARTITION_TABLE_OFFSET=0x8000
--- a/sdkconfig.defaults.esp32s3
+++ b/sdkconfig.defaults.esp32s3
@@ -2,6 +2,17 @@
 CONFIG_ESPTOOLPY_FLASHSIZE_16MB=y
 CONFIG_ESPTOOLPY_FLASHMODE_QIO=y

+CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_240=y
+
+CONFIG_SPIRAM=y
+CONFIG_SPIRAM_MODE_OCT=y
+CONFIG_SPIRAM_SPEED_80M=y
+CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=4096
+CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y
+CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=32768
+CONFIG_SPIRAM_MEMTEST=n
+CONFIG_MBEDTLS_EXTERNAL_MEM_ALLOC=y
+
 CONFIG_ESP32S3_INSTRUCTION_CACHE_32KB=y
 CONFIG_ESP32S3_DATA_CACHE_64KB=y
-CONFIG_ESP32S3_DATA_CACHE_LINE_64B=y
+CONFIG_ESP32S3_DATA_CACHE_LINE_64B=y
--- a/versions.py
+++ b/versions.py
@@ -0,0 +1,158 @@
+#! /usr/bin/env python3
+from dotenv import load_dotenv
+load_dotenv()
+
+import os
+import struct
+import zipfile
+import oss2
+import json
+
+def get_chip_id_string(chip_id):
+    return {
+        0x0000: "esp32",
+        0x0002: "esp32s2",
+        0x0005: "esp32c3",
+        0x0009: "esp32s3",
+        0x000C: "esp32c2",
+        0x000D: "esp32c6",
+        0x0010: "esp32h2",
+        0x0011: "esp32c5",
+        0x0012: "esp32p4",
+        0x0017: "esp32c5",
+    }[chip_id]
+
+def get_flash_size(flash_size):
+    MB = 1024 * 1024
+    return {
+        0x00: 1 * MB,
+        0x01: 2 * MB,
+        0x02: 4 * MB,
+        0x03: 8 * MB,
+        0x04: 16 * MB,
+        0x05: 32 * MB,
+        0x06: 64 * MB,
+        0x07: 128 * MB,
+    }[flash_size]
+
+def get_app_desc(data):
+    magic = struct.unpack("<I", data[0x00:0x04])[0]
+    if magic != 0xabcd5432:
+        raise Exception("Invalid app desc magic")
+    version = data[0x10:0x30].decode("utf-8").strip('\0')
+    project_name = data[0x30:0x50].decode("utf-8").strip('\0')
+    time = data[0x50:0x60].decode("utf-8").strip('\0')
+    date = data[0x60:0x70].decode("utf-8").strip('\0')
+    idf_ver = data[0x70:0x90].decode("utf-8").strip('\0')
+    elf_sha256 = data[0x90:0xb0].hex()
+    return {
+        "name": project_name,
+        "version": version,
+        "compile_time": date + "T" + time,
+        "idf_version": idf_ver,
+        "elf_sha256": elf_sha256,
+    }
+
+def get_board_name(folder):
+    basename = os.path.basename(folder)
+    if basename.startswith("v0.2"):
+        return "simple"
+    if basename.startswith("v0.3") or basename.startswith("v0.4"):
+        if "ML307" in basename:
+            return "compact.4g"
+        else:
+            return "compact.wifi"
+    raise Exception(f"Unknown board name: {basename}")
+
+def read_binary(dir_path):
+    merged_bin_path = os.path.join(dir_path, "merged-binary.bin")
+    data = open(merged_bin_path, "rb").read()[0x200000:]
+    if data[0] != 0xE9:
+        print(dir_path, "is not a valid image")
+        return
+    # get flash size
+    flash_size = get_flash_size(data[0x3] >> 4)
+    chip_id = get_chip_id_string(data[0xC])
+    # get segments
+    segment_count = data[0x1]
+    segments = []
+    offset = 0x18
+    for i in range(segment_count):
+        segment_size = struct.unpack("<I", data[offset + 4:offset + 8])[0]
+        offset += 8
+        segment_data = data[offset:offset + segment_size]
+        offset += segment_size
+        segments.append(segment_data)
+    assert offset < len(data), "offset is out of bounds"
+    
+    # extract bin file
+    bin_path = os.path.join(dir_path, "xiaozhi.bin")
+    if not os.path.exists(bin_path):
+        print("extract bin file to", bin_path)
+        open(bin_path, "wb").write(data)
+
+    # The app desc is in the first segment
+    desc = get_app_desc(segments[0])
+    return {
+        "chip_id": chip_id,
+        "flash_size": flash_size,
+        "board": get_board_name(dir_path),
+        "application": desc,
+    }
+
+def extract_zip(zip_path, extract_path):
+    if not os.path.exists(extract_path):
+        os.makedirs(extract_path)
+    print(f"Extracting {zip_path} to {extract_path}")
+    with zipfile.ZipFile(zip_path, 'r') as zip_ref:
+        zip_ref.extractall(extract_path)
+
+def upload_dir_to_oss(source_dir, target_dir):
+    auth = oss2.Auth(os.environ['OSS_ACCESS_KEY_ID'], os.environ['OSS_ACCESS_KEY_SECRET'])
+    bucket = oss2.Bucket(auth, os.environ['OSS_ENDPOINT'], os.environ['OSS_BUCKET_NAME'])
+    for filename in os.listdir(source_dir):
+        oss_key = os.path.join(target_dir, filename)
+        print('uploading', oss_key)
+        bucket.put_object(oss_key, open(os.path.join(source_dir, filename), 'rb'))
+
+def main():
+    release_dir = "releases"
+    versions = []
+    # look for zip files startswith "v"
+    for name in os.listdir(release_dir):
+        if name.startswith("v") and name.endswith(".zip"):
+            tag = name[:-4]
+            folder = os.path.join(release_dir, tag)
+            if not os.path.exists(folder):
+                os.makedirs(folder)
+                extract_zip(os.path.join(release_dir, name), folder)
+                info = read_binary(folder)
+                target_dir = os.path.join("firmwares", tag)
+                info["tag"] = tag
+                info["url"] = os.path.join(os.environ['OSS_BUCKET_URL'], target_dir, "xiaozhi.bin")
+                open(os.path.join(folder, "info.json"), "w").write(json.dumps(info, indent=4))
+                # upload all file to oss
+                upload_dir_to_oss(folder, target_dir)
+            # read info.json
+            info = json.load(open(os.path.join(folder, "info.json")))
+            versions.append(info)
+
+    # sort versions by version
+    versions.sort(key=lambda x: x["tag"], reverse=True)
+    # write versions to file
+    versions_path = os.path.join(release_dir, "versions.json")
+    open(versions_path, "w").write(json.dumps(versions, indent=4))
+    print(f"Versions written to {versions_path}")
+
+    # copy versions.json to server
+    versions_config_path = os.environ['VERSIONS_CONFIG_PATH']
+    ret = os.system(f'scp {versions_path} {versions_config_path}')
+    if ret != 0:
+        print(f'Failed to copy versions.json to server')
+        exit(1)
+    print(f'Copied versions.json to server: {versions_config_path}')
+
+
+
+if __name__ == "__main__":
+    main()
Author	SHA1	Message	Date
Terrence	fe05a039a2	fix v0.4.x board name	2024-10-25 12:46:33 +08:00
Terrence	32a5977fcc	update to 0.4.1	2024-10-25 12:29:23 +08:00
Terrence	357cd95693	fix audio data overflow	2024-10-25 12:25:22 +08:00
Terrence	2833142870	update docs	2024-10-25 10:05:01 +08:00
Terrence	a2487f46c8	ES8311 + ES7210	2024-10-24 09:53:08 +08:00
Terrence	4c6da771ec	add versions.py for ota upgrade	2024-10-17 18:34:51 +08:00
Terrence	ad8033fb47	bump to 0.3.3	2024-10-15 04:02:41 +08:00
Terrence	caae7cb930	add volume up gpio button	2024-10-15 03:56:35 +08:00
Terrence	548b854777	bump to 0.3.2	2024-10-11 00:59:03 +08:00
Terrence	56560685b1	add tcp transport support	2024-10-10 21:41:20 +08:00
Terrence	073fd4046e	fix frame size calculation	2024-10-04 04:26:08 +08:00
Terrence	2be6217b1f	v0.3.1	2024-10-03 06:41:16 +08:00
Terrence	879f1cc21e	reconstruct application	2024-10-03 06:39:22 +08:00
Terrence	e59be04394	update to 4MB partition	2024-10-01 15:58:03 +08:00
Terrence	d26e8d25ff	support ML307, new version 0.3.0	2024-10-01 14:16:12 +08:00
Terrence	8e9be5abc7	add websocket protocol	2024-09-26 16:19:54 +08:00
Terrence	7fd72aa8e2	add more wake word packets	2024-09-26 16:19:06 +08:00
Terrence	0396b4a91c	fix bugs	2024-09-25 03:44:28 +08:00
Terrence	53b08843d4	add vad to detection and communication	2024-09-17 11:26:07 +08:00
Terrence	797f9c2515	start AP if WiFi station fails to connect	2024-09-15 14:03:11 +08:00