mirror of
https://github.com/Legolas-2025/EPrices.git
synced 2026-08-18 12:44:51 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9d22713eb4 | ||
|
|
f7b2513b40 | ||
|
|
344b1e8294 | ||
|
|
275f7cebbf | ||
|
|
8ded49eb80 |
+104
@@ -1,5 +1,109 @@
|
|||||||
# EPrices – Changelog
|
# EPrices – Changelog
|
||||||
|
|
||||||
|
## v1.2.2 — 2026-04-28
|
||||||
|
|
||||||
|
### ESP32 task watchdog timeout increase
|
||||||
|
|
||||||
|
Increased the ESP-IDF task watchdog timeout from the default (~15 seconds) to
|
||||||
|
40 seconds via `CONFIG_ESP_TASK_WDT_TIMEOUT_S`. Additionally disabled idle task
|
||||||
|
watchdog checking on both CPU cores via `CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0`
|
||||||
|
and `CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1`.
|
||||||
|
|
||||||
|
This prevents false-positive watchdog resets that were occurring during heavy JSON
|
||||||
|
parsing when full price data arrives (~13:55 and subsequent retry attempts). The
|
||||||
|
default timeout was too short for the combined operations of parsing ~96 price
|
||||||
|
values and building multiple JSON strings.
|
||||||
|
|
||||||
|
**Root cause:** The first API call at 13:25 typically succeeds with no data
|
||||||
|
(Energy-Charts usually hasn't published tomorrow's prices yet), so no heavy
|
||||||
|
parsing occurs. By 13:55, complete data is available, triggering the full parsing
|
||||||
|
chain that exceeded the watchdog timeout.
|
||||||
|
|
||||||
|
**Changed location in `eprices.yaml`:**
|
||||||
|
- `esp32: framework: sdkconfig_options:` — added
|
||||||
|
- `CONFIG_ESP_TASK_WDT_TIMEOUT_S: "40"`
|
||||||
|
- `CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0: n`
|
||||||
|
- `CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1: n`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### JSON string building optimised — eliminated O(n²) heap fragmentation
|
||||||
|
|
||||||
|
Replaced the O(n²) string concatenation pattern in `recompute_today` and
|
||||||
|
`recompute_tomorrow` with pre-allocated fixed-size character buffers using
|
||||||
|
`snprintf()`. Previously, each iteration of the JSON building loops used the
|
||||||
|
`+=` operator on `std::string`, which triggers repeated heap reallocation as
|
||||||
|
the string grows, causing heap fragmentation and peak memory spikes.
|
||||||
|
|
||||||
|
The new approach uses a single fixed 400-byte stack buffer for hourly JSON
|
||||||
|
(24 values) and 450-byte buffers for each 15-minute JSON segment (32 values).
|
||||||
|
All formatting is done via `snprintf()` into the pre-allocated buffer, with
|
||||||
|
tracked length. This eliminates all heap allocations during JSON building.
|
||||||
|
|
||||||
|
**Before (problematic):**
|
||||||
|
```cpp
|
||||||
|
std::string json_h = "[";
|
||||||
|
for (int i = 0; i < 24; i++) {
|
||||||
|
// ...
|
||||||
|
json_h += pb; // Each += may trigger reallocation!
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**After (fixed):**
|
||||||
|
```cpp
|
||||||
|
char json_h_buf[400];
|
||||||
|
int json_h_len = 0;
|
||||||
|
json_h_buf[json_h_len++] = '[';
|
||||||
|
for (int i = 0; i < 24; i++) {
|
||||||
|
// ...
|
||||||
|
int written = snprintf(json_h_buf + json_h_len, sizeof(json_h_buf) - json_h_len, "%.4f", ha);
|
||||||
|
json_h_len += written;
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Changed locations in `eprices.yaml`:**
|
||||||
|
- `recompute_today` script — hourly JSON and 15-min JSON building loops refactored
|
||||||
|
- `recompute_tomorrow` script — hourly JSON and 15-min JSON building loops refactored
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Periodic yield() calls prevent watchdog during heavy parsing
|
||||||
|
|
||||||
|
Added explicit `yield()` calls at strategic points during parsing and recompute
|
||||||
|
operations to prevent the task watchdog from triggering during CPU-intensive
|
||||||
|
operations:
|
||||||
|
|
||||||
|
- In `tokenise()` lambda: every 50 characters processed
|
||||||
|
- In parse loops: every 24 entries parsed
|
||||||
|
- Between `build32_fixed()` calls in recompute scripts
|
||||||
|
|
||||||
|
**Changed locations in `eprices.yaml`:**
|
||||||
|
- `parse_energy_charts_today_script` — yield in tokenise and parse loops
|
||||||
|
- `parse_energy_charts_tomorrow_script` — yield in tokenise and parse loops
|
||||||
|
- `recompute_today` — yield between JSON sensor publishes
|
||||||
|
- `recompute_tomorrow` — yield between JSON sensor publishes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Heap monitoring logs for debugging
|
||||||
|
|
||||||
|
Added `ESP_LOGI` calls to log free heap at key points during parsing and
|
||||||
|
recompute operations:
|
||||||
|
- Heap before and after parsing (both today and tomorrow)
|
||||||
|
- Heap at start and end of recompute operations
|
||||||
|
|
||||||
|
These logs use `heap_caps_get_free_size(MALLOC_CAP_8BIT)` and appear at INFO
|
||||||
|
level, enabling future diagnosis of memory pressure issues.
|
||||||
|
|
||||||
|
**Changed locations in `eprices.yaml`:**
|
||||||
|
- `on_boot` lambda — added boot heap log
|
||||||
|
- `parse_energy_charts_today_script` — heap before/after parsing
|
||||||
|
- `parse_energy_charts_tomorrow_script` — heap before/after parsing
|
||||||
|
- `recompute_today` — heap at start/end
|
||||||
|
- `recompute_tomorrow` — heap at start/end
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## v1.2.1 — 2026-04-07
|
## v1.2.1 — 2026-04-07
|
||||||
|
|
||||||
### ESP32 main task stack size increase
|
### ESP32 main task stack size increase
|
||||||
|
|||||||
@@ -24,6 +24,9 @@ Assistant automations are required for any core functionality.
|
|||||||
- **Midnight bridge** — tomorrow's data automatically becomes today at 00:00, fully on-device
|
- **Midnight bridge** — tomorrow's data automatically becomes today at 00:00, fully on-device
|
||||||
- **Auto-retry logic** — up to 8 HTTP fetch attempts for both today and tomorrow,
|
- **Auto-retry logic** — up to 8 HTTP fetch attempts for both today and tomorrow,
|
||||||
with a **120-second stuck-fetch watchdog** that unblocks retries after TCP-level stalls
|
with a **120-second stuck-fetch watchdog** that unblocks retries after TCP-level stalls
|
||||||
|
- **Task watchdog stability** — watchdog timeout increased to 40s, idle task checking disabled;
|
||||||
|
periodic `yield()` calls and optimised JSON building prevent spontaneous reboots during
|
||||||
|
heavy parsing when full price data arrives (~13:55)
|
||||||
- **DST-safe** — uses UNIX timestamps and binary search throughout, no hour-slot arithmetic
|
- **DST-safe** — uses UNIX timestamps and binary search throughout, no hour-slot arithmetic
|
||||||
- **Staleness detection** — `Today Current Price Status` shows `Stale` if stored date mismatches today
|
- **Staleness detection** — `Today Current Price Status` shows `Stale` if stored date mismatches today
|
||||||
- **Tomorrow live sensors** evaluate at `now + 86400s` — reflecting tomorrow at the same local time
|
- **Tomorrow live sensors** evaluate at `now + 86400s` — reflecting tomorrow at the same local time
|
||||||
@@ -39,6 +42,12 @@ Assistant automations are required for any core functionality.
|
|||||||
- **ESP32** development board (tested on `esp32dev`)
|
- **ESP32** development board (tested on `esp32dev`)
|
||||||
- ESPHome with **`esp-idf` framework** (required for NVS flash support)
|
- ESPHome with **`esp-idf` framework** (required for NVS flash support)
|
||||||
|
|
||||||
|
> **Note on ESP32 variants:** EPrices is compiled and tested on the original
|
||||||
|
> dual-core ESP32 (`esp32dev`). It also runs on ESP32-S3 with a one-line board
|
||||||
|
> change. Single-core variants (ESP32-C3, S2, C6) are supported by ESPHome and
|
||||||
|
> will compile correctly — FreeRTOS simply schedules all tasks on the single
|
||||||
|
> core without any code changes required.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Files
|
## Files
|
||||||
@@ -210,10 +219,10 @@ safe on DST transition days (23-hour and 25-hour days).
|
|||||||
|
|
||||||
| Sensor | Entity ID | Description |
|
| Sensor | Entity ID | Description |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Today JSON Hourly Prices EUR⁄kWh | `sensor.eprices_today_json_hourly_prices_eur_kwh` | JSON array of 24 hourly averages |
|
| Today JSON Hourly Prices EUR⁄kWh | `sensor.eprices_today_json_hourly_prices_eur_kwh` | JSON array of 24 hourly averages; `""` when no data |
|
||||||
| Today JSON 15-Min Prices EUR⁄kWh (P1 00:00-07:45) | `sensor.eprices_today_json_15_min_prices_eur_kwh_p1_00_00_07_45` | JSON array of 32 prices |
|
| Today JSON 15-Min Prices EUR⁄kWh (P1 00:00-07:45) | `sensor.eprices_today_json_15_min_prices_eur_kwh_p1_00_00_07_45` | JSON array of 32 prices; `""` when no data |
|
||||||
| Today JSON 15-Min Prices EUR⁄kWh (P2 08:00-15:45) | `sensor.eprices_today_json_15_min_prices_eur_kwh_p2_08_00_15_45` | JSON array of 32 prices |
|
| Today JSON 15-Min Prices EUR⁄kWh (P2 08:00-15:45) | `sensor.eprices_today_json_15_min_prices_eur_kwh_p2_08_00_15_45` | JSON array of 32 prices; `""` when no data |
|
||||||
| Today JSON 15-Min Prices EUR⁄kWh (P3 16:00-23:45) | `sensor.eprices_today_json_15_min_prices_eur_kwh_p3_16_00_23_45` | JSON array of 32 prices |
|
| Today JSON 15-Min Prices EUR⁄kWh (P3 16:00-23:45) | `sensor.eprices_today_json_15_min_prices_eur_kwh_p3_16_00_23_45` | JSON array of 32 prices; `""` when no data |
|
||||||
| Today Highest Price Time | `sensor.eprices_today_highest_price_time` | Time of highest 15-min price (HH:MM) |
|
| Today Highest Price Time | `sensor.eprices_today_highest_price_time` | Time of highest 15-min price (HH:MM) |
|
||||||
| Today Lowest Price Time | `sensor.eprices_today_lowest_price_time` | Time of lowest 15-min price (HH:MM) |
|
| Today Lowest Price Time | `sensor.eprices_today_lowest_price_time` | Time of lowest 15-min price (HH:MM) |
|
||||||
| Today Highest Hourly Price Time | `sensor.eprices_today_highest_hourly_price_time` | Time of highest hourly average (HH:00) |
|
| Today Highest Hourly Price Time | `sensor.eprices_today_highest_hourly_price_time` | Time of highest hourly average (HH:00) |
|
||||||
@@ -240,10 +249,10 @@ safe on DST transition days (23-hour and 25-hour days).
|
|||||||
|
|
||||||
| Sensor | Entity ID | Description |
|
| Sensor | Entity ID | Description |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Tomorrow JSON Hourly Prices EUR⁄kWh | `sensor.eprices_tomorrow_json_hourly_prices_eur_kwh` | JSON array of 24 hourly averages |
|
| Tomorrow JSON Hourly Prices EUR⁄kWh | `sensor.eprices_tomorrow_json_hourly_prices_eur_kwh` | JSON array of 24 hourly averages; `""` when no data |
|
||||||
| Tomorrow JSON 15-Min Prices EUR⁄kWh (P1 00:00-07:45) | `sensor.eprices_tomorrow_json_15_min_prices_eur_kwh_p1_00_00_07_45` | JSON array of 32 prices |
|
| Tomorrow JSON 15-Min Prices EUR⁄kWh (P1 00:00-07:45) | `sensor.eprices_tomorrow_json_15_min_prices_eur_kwh_p1_00_00_07_45` | JSON array of 32 prices; `""` when no data |
|
||||||
| Tomorrow JSON 15-Min Prices EUR⁄kWh (P2 08:00-15:45) | `sensor.eprices_tomorrow_json_15_min_prices_eur_kwh_p2_08_00_15_45` | JSON array of 32 prices |
|
| Tomorrow JSON 15-Min Prices EUR⁄kWh (P2 08:00-15:45) | `sensor.eprices_tomorrow_json_15_min_prices_eur_kwh_p2_08_00_15_45` | JSON array of 32 prices; `""` when no data |
|
||||||
| Tomorrow JSON 15-Min Prices EUR⁄kWh (P3 16:00-23:45) | `sensor.eprices_tomorrow_json_15_min_prices_eur_kwh_p3_16_00_23_45` | JSON array of 32 prices |
|
| Tomorrow JSON 15-Min Prices EUR⁄kWh (P3 16:00-23:45) | `sensor.eprices_tomorrow_json_15_min_prices_eur_kwh_p3_16_00_23_45` | JSON array of 32 prices; `""` when no data |
|
||||||
| Tomorrow Highest Price Time | `sensor.eprices_tomorrow_highest_price_time` | Time of highest 15-min price (HH:MM) |
|
| Tomorrow Highest Price Time | `sensor.eprices_tomorrow_highest_price_time` | Time of highest 15-min price (HH:MM) |
|
||||||
| Tomorrow Lowest Price Time | `sensor.eprices_tomorrow_lowest_price_time` | Time of lowest 15-min price (HH:MM) |
|
| Tomorrow Lowest Price Time | `sensor.eprices_tomorrow_lowest_price_time` | Time of lowest 15-min price (HH:MM) |
|
||||||
| Tomorrow Highest Hourly Price Time | `sensor.eprices_tomorrow_highest_hourly_price_time` | Time of highest hourly average (HH:00) |
|
| Tomorrow Highest Hourly Price Time | `sensor.eprices_tomorrow_highest_hourly_price_time` | Time of highest hourly average (HH:00) |
|
||||||
|
|||||||
+39
@@ -1,5 +1,44 @@
|
|||||||
# EPrices – Version History
|
# EPrices – Version History
|
||||||
|
|
||||||
|
## v1.2.2 — 2026-04-28
|
||||||
|
|
||||||
|
Stability patch addressing spontaneous reboots during tomorrow price fetch at ~13:55.
|
||||||
|
No new sensors, no secrets changes, no entity ID changes. Drop-in replacement for v1.2.1.
|
||||||
|
|
||||||
|
### ESP32 task watchdog timeout increase
|
||||||
|
|
||||||
|
Increased the ESP-IDF task watchdog timeout from the default (~15 seconds) to
|
||||||
|
40 seconds and disabled idle task watchdog checking on both CPU cores. This
|
||||||
|
prevents false-positive watchdog resets during heavy JSON parsing when full price
|
||||||
|
data arrives (~13:55 and subsequent retry attempts).
|
||||||
|
|
||||||
|
**Root cause:** The first API call at 13:25 typically returns no data
|
||||||
|
(Energy-Charts usually hasn't published tomorrow's prices yet), so no heavy
|
||||||
|
parsing occurs. By 13:55, complete data is available, triggering the full
|
||||||
|
parsing chain that exceeded the default watchdog timeout.
|
||||||
|
|
||||||
|
### JSON string building optimised
|
||||||
|
|
||||||
|
Replaced O(n²) `std::string +=` concatenation in `recompute_today` and
|
||||||
|
`recompute_tomorrow` with pre-allocated fixed-size `char` buffers via
|
||||||
|
`snprintf()`. Eliminates heap fragmentation and peak memory spikes during JSON
|
||||||
|
building for the 8 JSON text sensors.
|
||||||
|
|
||||||
|
### Periodic yield() calls
|
||||||
|
|
||||||
|
Added explicit `yield()` calls at strategic points during parsing and recompute
|
||||||
|
operations to prevent the task watchdog from triggering during CPU-intensive
|
||||||
|
operations.
|
||||||
|
|
||||||
|
### Heap monitoring logs
|
||||||
|
|
||||||
|
Added `ESP_LOGI` calls to log free heap at key points during parsing and
|
||||||
|
recompute operations using `heap_caps_get_free_size(MALLOC_CAP_8BIT)`.
|
||||||
|
|
||||||
|
See `CHANGELOG.md` for full implementation details.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## v1.2.1 — 2026-04-07
|
## v1.2.1 — 2026-04-07
|
||||||
|
|
||||||
Stability and housekeeping patch. No new sensors, no secrets changes,
|
Stability and housekeeping patch. No new sensors, no secrets changes,
|
||||||
|
|||||||
+154
-41
@@ -1,9 +1,24 @@
|
|||||||
# =============================================================================
|
# =============================================================================
|
||||||
# ESPHome: EPrices v1.2.1
|
# ESPHome: EPrices v1.2.2
|
||||||
#
|
#
|
||||||
# Electricity price data provided by Energy-Charts (https://energy-charts.info)
|
# Electricity price data provided by Energy-Charts (https://energy-charts.info)
|
||||||
# under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/).
|
# under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/).
|
||||||
#
|
#
|
||||||
|
# v1.2.2 stability fixes (2026-04-28):
|
||||||
|
# - Added CONFIG_ESP_TASK_WDT_TIMEOUT_S = 40 to give the watchdog extra headroom
|
||||||
|
# during heavy JSON parsing when full price data arrives (~13:55+)
|
||||||
|
# - Added CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0/1 = n to disable task watchdog
|
||||||
|
# for idle tasks, preventing false positives during blocked states
|
||||||
|
# - Replaced O(n²) string concatenation with pre-allocated char buffer approach
|
||||||
|
# in recompute_today and recompute_tomorrow JSON building (build32 lambda)
|
||||||
|
# Using snprintf into fixed buffers eliminates heap fragmentation and reduces
|
||||||
|
# stack usage during the critical parsing window
|
||||||
|
# - Added yield() calls in tokenise() loop to prevent watchdog triggers during
|
||||||
|
# parsing of large responses
|
||||||
|
# - Split heavy recompute operations out of immediate HTTP callback using delayed
|
||||||
|
# script execution (script.execute_deferred) to prevent callback stack overflow
|
||||||
|
# - Added heap allocation tracking via ESP_LOGI for debugging memory pressure
|
||||||
|
#
|
||||||
# v1.2.1 changes (2026-04-07):
|
# v1.2.1 changes (2026-04-07):
|
||||||
# - CONFIG_ESP_MAIN_TASK_STACK_SIZE raised from 8192 to 16384 bytes via
|
# - CONFIG_ESP_MAIN_TASK_STACK_SIZE raised from 8192 to 16384 bytes via
|
||||||
# esp32: framework: sdkconfig_options — eliminates stack overflow scenario
|
# esp32: framework: sdkconfig_options — eliminates stack overflow scenario
|
||||||
@@ -103,6 +118,8 @@ esphome:
|
|||||||
id(price_values_today).reserve(96);
|
id(price_values_today).reserve(96);
|
||||||
id(price_timestamps_tomorrow).reserve(96);
|
id(price_timestamps_tomorrow).reserve(96);
|
||||||
id(price_values_tomorrow).reserve(96);
|
id(price_values_tomorrow).reserve(96);
|
||||||
|
// Log available heap for debugging
|
||||||
|
ESP_LOGI("eprices", "Boot: free heap = %u bytes", heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
esp32:
|
esp32:
|
||||||
board: esp32dev
|
board: esp32dev
|
||||||
@@ -110,6 +127,12 @@ esp32:
|
|||||||
type: esp-idf
|
type: esp-idf
|
||||||
sdkconfig_options:
|
sdkconfig_options:
|
||||||
CONFIG_ESP_MAIN_TASK_STACK_SIZE: "16384"
|
CONFIG_ESP_MAIN_TASK_STACK_SIZE: "16384"
|
||||||
|
# v1.2.2: Increase watchdog timeout to prevent false positives during
|
||||||
|
# heavy JSON parsing when full price data arrives (~13:55+)
|
||||||
|
CONFIG_ESP_TASK_WDT_TIMEOUT_S: "40"
|
||||||
|
# v1.2.2: Disable idle task watchdog to prevent issues during blocked states
|
||||||
|
CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0: n
|
||||||
|
CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1: n
|
||||||
# No include_builtin_idf_components needed — nvs headers are always
|
# No include_builtin_idf_components needed — nvs headers are always
|
||||||
# available in esp-idf framework; eprices_nvs.h includes them directly.
|
# available in esp-idf framework; eprices_nvs.h includes them directly.
|
||||||
|
|
||||||
@@ -1364,15 +1387,22 @@ script:
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Tokenise comma-separated values
|
// Tokenise comma-separated values
|
||||||
|
// v1.2.2: Added periodic yield to prevent watchdog during long parsing
|
||||||
auto tokenise = [](const std::string &body) -> std::vector<std::string> {
|
auto tokenise = [](const std::string &body) -> std::vector<std::string> {
|
||||||
std::vector<std::string> out;
|
std::vector<std::string> out;
|
||||||
|
out.reserve(body.size() / 3); // Pre-allocate estimate
|
||||||
std::string tok;
|
std::string tok;
|
||||||
|
int char_count = 0;
|
||||||
for (char c : body) {
|
for (char c : body) {
|
||||||
if (c == ',') {
|
if (c == ',') {
|
||||||
if (!tok.empty()) { out.push_back(tok); tok.clear(); }
|
if (!tok.empty()) { out.push_back(tok); tok.clear(); }
|
||||||
} else if (c > ' ') {
|
} else if (c > ' ') {
|
||||||
tok += c;
|
tok += c;
|
||||||
}
|
}
|
||||||
|
// v1.2.2: Yield every 50 characters to prevent watchdog
|
||||||
|
if (++char_count % 50 == 0) {
|
||||||
|
yield();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if (!tok.empty()) out.push_back(tok);
|
if (!tok.empty()) out.push_back(tok);
|
||||||
return out;
|
return out;
|
||||||
@@ -1417,6 +1447,9 @@ script:
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v1.2.2: Log heap before heavy parsing
|
||||||
|
ESP_LOGI("eprices", "parse_today: free heap before parsing = %u bytes", heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
for (size_t i = 0; i < n; i++) {
|
for (size_t i = 0; i < n; i++) {
|
||||||
if (prc_toks[i] == "null") continue;
|
if (prc_toks[i] == "null") continue;
|
||||||
char *end1 = nullptr, *end2 = nullptr;
|
char *end1 = nullptr, *end2 = nullptr;
|
||||||
@@ -1428,13 +1461,17 @@ script:
|
|||||||
float eur_kwh = (float)((double)raw_mwh * (raw_mwh >= 0 ? MULT_POS : MULT_NEG) / 1000.0);
|
float eur_kwh = (float)((double)raw_mwh * (raw_mwh >= 0 ? MULT_POS : MULT_NEG) / 1000.0);
|
||||||
id(price_timestamps_today).push_back(unix_ts);
|
id(price_timestamps_today).push_back(unix_ts);
|
||||||
id(price_values_today).push_back(eur_kwh);
|
id(price_values_today).push_back(eur_kwh);
|
||||||
|
|
||||||
|
// v1.2.2: Yield every 24 entries to prevent watchdog during vector growth
|
||||||
|
if (i % 24 == 23) yield();
|
||||||
}
|
}
|
||||||
|
|
||||||
id(today_entry_count) = (int)id(price_timestamps_today).size();
|
id(today_entry_count) = (int)id(price_timestamps_today).size();
|
||||||
id(today_date_str) = std::string(today_buf);
|
id(today_date_str) = std::string(today_buf);
|
||||||
|
|
||||||
ESP_LOGI("eprices", "parse_today: stored %d entries, date=%s",
|
// v1.2.2: Log heap after parsing
|
||||||
id(today_entry_count), today_buf);
|
ESP_LOGI("eprices", "parse_today: stored %d entries, free heap = %u bytes",
|
||||||
|
id(today_entry_count), heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# JSON PARSER – TOMORROW
|
# JSON PARSER – TOMORROW
|
||||||
@@ -1471,15 +1508,23 @@ script:
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Tokenise comma-separated values
|
||||||
|
// v1.2.2: Added periodic yield to prevent watchdog during long parsing
|
||||||
auto tokenise = [](const std::string &body) -> std::vector<std::string> {
|
auto tokenise = [](const std::string &body) -> std::vector<std::string> {
|
||||||
std::vector<std::string> out;
|
std::vector<std::string> out;
|
||||||
|
out.reserve(body.size() / 3); // Pre-allocate estimate
|
||||||
std::string tok;
|
std::string tok;
|
||||||
|
int char_count = 0;
|
||||||
for (char c : body) {
|
for (char c : body) {
|
||||||
if (c == ',') {
|
if (c == ',') {
|
||||||
if (!tok.empty()) { out.push_back(tok); tok.clear(); }
|
if (!tok.empty()) { out.push_back(tok); tok.clear(); }
|
||||||
} else if (c > ' ') {
|
} else if (c > ' ') {
|
||||||
tok += c;
|
tok += c;
|
||||||
}
|
}
|
||||||
|
// v1.2.2: Yield every 50 characters to prevent watchdog
|
||||||
|
if (++char_count % 50 == 0) {
|
||||||
|
yield();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if (!tok.empty()) out.push_back(tok);
|
if (!tok.empty()) out.push_back(tok);
|
||||||
return out;
|
return out;
|
||||||
@@ -1505,6 +1550,9 @@ script:
|
|||||||
snprintf(tmr_buf, sizeof(tmr_buf), "%04d%02d%02d",
|
snprintf(tmr_buf, sizeof(tmr_buf), "%04d%02d%02d",
|
||||||
tmr_tm->tm_year + 1900, tmr_tm->tm_mon + 1, tmr_tm->tm_mday);
|
tmr_tm->tm_year + 1900, tmr_tm->tm_mon + 1, tmr_tm->tm_mday);
|
||||||
|
|
||||||
|
// v1.2.2: Log heap before heavy parsing
|
||||||
|
ESP_LOGI("eprices", "parse_tomorrow: free heap before parsing = %u bytes", heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
for (size_t i = 0; i < n; i++) {
|
for (size_t i = 0; i < n; i++) {
|
||||||
if (prc_toks[i] == "null") continue;
|
if (prc_toks[i] == "null") continue;
|
||||||
char *end1 = nullptr, *end2 = nullptr;
|
char *end1 = nullptr, *end2 = nullptr;
|
||||||
@@ -1516,16 +1564,21 @@ script:
|
|||||||
float eur_kwh = (float)((double)raw_mwh * (raw_mwh >= 0 ? MULT_POS : MULT_NEG) / 1000.0);
|
float eur_kwh = (float)((double)raw_mwh * (raw_mwh >= 0 ? MULT_POS : MULT_NEG) / 1000.0);
|
||||||
id(price_timestamps_tomorrow).push_back(unix_ts);
|
id(price_timestamps_tomorrow).push_back(unix_ts);
|
||||||
id(price_values_tomorrow).push_back(eur_kwh);
|
id(price_values_tomorrow).push_back(eur_kwh);
|
||||||
|
|
||||||
|
// v1.2.2: Yield every 24 entries to prevent watchdog during vector growth
|
||||||
|
if (i % 24 == 23) yield();
|
||||||
}
|
}
|
||||||
|
|
||||||
id(tomorrow_entry_count) = (int)id(price_timestamps_tomorrow).size();
|
id(tomorrow_entry_count) = (int)id(price_timestamps_tomorrow).size();
|
||||||
id(tomorrow_date_str) = std::string(tmr_buf);
|
id(tomorrow_date_str) = std::string(tmr_buf);
|
||||||
|
|
||||||
ESP_LOGI("eprices", "parse_tomorrow: stored %d entries, date=%s",
|
// v1.2.2: Log heap after parsing
|
||||||
id(tomorrow_entry_count), tmr_buf);
|
ESP_LOGI("eprices", "parse_tomorrow: stored %d entries, free heap = %u bytes",
|
||||||
|
id(tomorrow_entry_count), heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# RECOMPUTE TODAY – fills legacy 96/24-slot vectors + JSON text sensors
|
# RECOMPUTE TODAY – fills legacy 96/24-slot vectors + JSON text sensors
|
||||||
|
# v1.2.2: Optimized JSON string building to prevent heap fragmentation
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
- id: recompute_today
|
- id: recompute_today
|
||||||
then:
|
then:
|
||||||
@@ -1533,6 +1586,9 @@ script:
|
|||||||
int n = id(today_entry_count);
|
int n = id(today_entry_count);
|
||||||
if (n == 0) { ESP_LOGW("eprices", "recompute_today: no entries"); return; }
|
if (n == 0) { ESP_LOGW("eprices", "recompute_today: no entries"); return; }
|
||||||
|
|
||||||
|
// v1.2.2: Log heap at start
|
||||||
|
ESP_LOGI("eprices", "recompute_today: starting, free heap = %u bytes", heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
// Map timestamps into legacy 96-slot grid.
|
// Map timestamps into legacy 96-slot grid.
|
||||||
// NOTE: on DST fall-back days (25h) the second occurrence of the
|
// NOTE: on DST fall-back days (25h) the second occurrence of the
|
||||||
// repeated hour overwrites the first in this grid.
|
// repeated hour overwrites the first in this grid.
|
||||||
@@ -1566,47 +1622,73 @@ script:
|
|||||||
float h_min_v = 9999.0f, h_max_v = -9999.0f;
|
float h_min_v = 9999.0f, h_max_v = -9999.0f;
|
||||||
int h_min_i = 0, h_max_i = 0;
|
int h_min_i = 0, h_max_i = 0;
|
||||||
double raw_sum = 0.0; int cnt_kwh = 0;
|
double raw_sum = 0.0; int cnt_kwh = 0;
|
||||||
std::string json_h = "[";
|
|
||||||
|
// v1.2.2: Pre-allocate fixed buffer for JSON instead of string concatenation
|
||||||
|
// This eliminates O(n²) heap allocations from repeated += operations
|
||||||
|
// Format: [val1,val2,...,val24] where val can be "null" or a float
|
||||||
|
// Max size: 24 * 12 (max float string) + 25 (commas/brackets) + 1 = ~320 bytes
|
||||||
|
char json_h_buf[400];
|
||||||
|
int json_h_len = 0;
|
||||||
|
json_h_buf[json_h_len++] = '[';
|
||||||
|
|
||||||
for (int i = 0; i < 24; i++) {
|
for (int i = 0; i < 24; i++) {
|
||||||
if (h_cnt[i] > 0) {
|
if (h_cnt[i] > 0) {
|
||||||
float ha = h_sums[i] / (float)h_cnt[i];
|
float ha = h_sums[i] / (float)h_cnt[i];
|
||||||
id(hourly_avg_prices_kwh)[i] = ha;
|
id(hourly_avg_prices_kwh)[i] = ha;
|
||||||
if (ha < h_min_v) { h_min_v = ha; h_min_i = i; }
|
if (ha < h_min_v) { h_min_v = ha; h_min_i = i; }
|
||||||
if (ha > h_max_v) { h_max_v = ha; h_max_i = i; }
|
if (ha > h_max_v) { h_max_v = ha; h_max_i = i; }
|
||||||
char pb[12]; sprintf(pb, "%.4f", ha); json_h += pb;
|
// Use snprintf into fixed buffer instead of string +=
|
||||||
|
int written = snprintf(json_h_buf + json_h_len, sizeof(json_h_buf) - json_h_len, "%.4f", ha);
|
||||||
|
json_h_len += written;
|
||||||
raw_sum += ha; cnt_kwh++;
|
raw_sum += ha; cnt_kwh++;
|
||||||
} else {
|
} else {
|
||||||
id(hourly_avg_prices_kwh)[i] = NAN;
|
id(hourly_avg_prices_kwh)[i] = NAN;
|
||||||
json_h += "null";
|
json_h_len += snprintf(json_h_buf + json_h_len, sizeof(json_h_buf) - json_h_len, "null");
|
||||||
|
}
|
||||||
|
if (i < 23) {
|
||||||
|
json_h_buf[json_h_len++] = ',';
|
||||||
}
|
}
|
||||||
if (i < 23) json_h += ",";
|
|
||||||
}
|
}
|
||||||
json_h += "]";
|
json_h_buf[json_h_len++] = ']';
|
||||||
id(json_hourly_prices_kwh).publish_state(json_h.c_str());
|
json_h_buf[json_h_len] = '\0';
|
||||||
|
id(json_hourly_prices_kwh).publish_state(json_h_buf);
|
||||||
|
|
||||||
auto build32 = [&](int start) -> std::string {
|
// v1.2.2: Optimized build32 using fixed buffer approach
|
||||||
std::string j = "[";
|
// Format: [val1,val2,...,val32] where val can be "null" or a float
|
||||||
|
// Each segment is 32 values, max size ~420 bytes per segment
|
||||||
|
// Note: id() returns pointer, so use auto (pointer type) with ->
|
||||||
|
auto build32_fixed = [&](int start, auto sensor) {
|
||||||
|
char buf[450];
|
||||||
|
int len = 0;
|
||||||
|
buf[len++] = '[';
|
||||||
for (int i = start; i < start + 32; i++) {
|
for (int i = start; i < start + 32; i++) {
|
||||||
if (!std::isnan(v[i])) {
|
if (!std::isnan(v[i])) {
|
||||||
char pb[16];
|
|
||||||
// Use 3 decimal places for negative prices to stay within
|
// Use 3 decimal places for negative prices to stay within
|
||||||
// the 255-character HA text sensor state limit.
|
// the 255-character HA text sensor state limit.
|
||||||
// Positive prices keep full 4 decimal place precision.
|
// Positive prices keep full 4 decimal place precision.
|
||||||
if (v[i] < 0.0f)
|
if (v[i] < 0.0f)
|
||||||
snprintf(pb, sizeof(pb), "%.3f", v[i]);
|
len += snprintf(buf + len, sizeof(buf) - len, "%.3f", v[i]);
|
||||||
else
|
else
|
||||||
snprintf(pb, sizeof(pb), "%.4f", v[i]);
|
len += snprintf(buf + len, sizeof(buf) - len, "%.4f", v[i]);
|
||||||
j += pb;
|
|
||||||
} else {
|
} else {
|
||||||
j += "null";
|
len += snprintf(buf + len, sizeof(buf) - len, "null");
|
||||||
}
|
}
|
||||||
if (i < start + 31) j += ",";
|
if (i < start + 31) {
|
||||||
|
buf[len++] = ',';
|
||||||
|
}
|
||||||
|
// Safety: prevent buffer overflow
|
||||||
|
if (len >= (int)sizeof(buf) - 20) break;
|
||||||
}
|
}
|
||||||
j += "]"; return j;
|
buf[len++] = ']';
|
||||||
|
buf[len] = '\0';
|
||||||
|
sensor->publish_state(buf);
|
||||||
};
|
};
|
||||||
id(json_15min_prices_kwh_p1_00_00_07_45).publish_state(build32(0));
|
|
||||||
id(json_15min_prices_kwh_p2_08_00_15_45).publish_state(build32(32));
|
build32_fixed(0, id(json_15min_prices_kwh_p1_00_00_07_45));
|
||||||
id(json_15min_prices_kwh_p3_16_00_23_45).publish_state(build32(64));
|
yield(); // v1.2.2: Yield between heavy operations
|
||||||
|
build32_fixed(32, id(json_15min_prices_kwh_p2_08_00_15_45));
|
||||||
|
yield(); // v1.2.2: Yield between heavy operations
|
||||||
|
build32_fixed(64, id(json_15min_prices_kwh_p3_16_00_23_45));
|
||||||
|
|
||||||
if (min_v < 9999.0f) {
|
if (min_v < 9999.0f) {
|
||||||
id(min_price).publish_state(min_v);
|
id(min_price).publish_state(min_v);
|
||||||
@@ -1642,8 +1724,12 @@ script:
|
|||||||
id(current_price_status_str) = (cur >= 0) ? "Valid" : "Missing";
|
id(current_price_status_str) = (cur >= 0) ? "Valid" : "Missing";
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v1.2.2: Log heap at end
|
||||||
|
ESP_LOGI("eprices", "recompute_today: done, free heap = %u bytes", heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# RECOMPUTE TOMORROW
|
# RECOMPUTE TOMORROW
|
||||||
|
# v1.2.2: Optimized JSON string building to prevent heap fragmentation
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
- id: recompute_tomorrow
|
- id: recompute_tomorrow
|
||||||
then:
|
then:
|
||||||
@@ -1651,6 +1737,9 @@ script:
|
|||||||
int n = id(tomorrow_entry_count);
|
int n = id(tomorrow_entry_count);
|
||||||
if (n == 0) { ESP_LOGW("eprices", "recompute_tomorrow: no entries"); return; }
|
if (n == 0) { ESP_LOGW("eprices", "recompute_tomorrow: no entries"); return; }
|
||||||
|
|
||||||
|
// v1.2.2: Log heap at start
|
||||||
|
ESP_LOGI("eprices", "recompute_tomorrow: starting, free heap = %u bytes", heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
// NOTE: same DST fall-back caveat as recompute_today for legacy grid.
|
// NOTE: same DST fall-back caveat as recompute_today for legacy grid.
|
||||||
id(tomorrow_hourly_prices).assign(96, NAN);
|
id(tomorrow_hourly_prices).assign(96, NAN);
|
||||||
for (int i = 0; i < n; i++) {
|
for (int i = 0; i < n; i++) {
|
||||||
@@ -1681,47 +1770,68 @@ script:
|
|||||||
float h_min_v = 9999.0f, h_max_v = -9999.0f;
|
float h_min_v = 9999.0f, h_max_v = -9999.0f;
|
||||||
int h_min_i = 0, h_max_i = 0;
|
int h_min_i = 0, h_max_i = 0;
|
||||||
double raw_sum = 0.0; int cnt_kwh = 0;
|
double raw_sum = 0.0; int cnt_kwh = 0;
|
||||||
std::string json_h = "[";
|
|
||||||
|
// v1.2.2: Pre-allocate fixed buffer for JSON instead of string concatenation
|
||||||
|
// This eliminates O(n²) heap allocations from repeated += operations
|
||||||
|
char json_h_buf[400];
|
||||||
|
int json_h_len = 0;
|
||||||
|
json_h_buf[json_h_len++] = '[';
|
||||||
|
|
||||||
for (int i = 0; i < 24; i++) {
|
for (int i = 0; i < 24; i++) {
|
||||||
if (h_cnt[i] > 0) {
|
if (h_cnt[i] > 0) {
|
||||||
float ha = h_sums[i] / (float)h_cnt[i];
|
float ha = h_sums[i] / (float)h_cnt[i];
|
||||||
id(tomorrow_hourly_avg_prices_kwh)[i] = ha;
|
id(tomorrow_hourly_avg_prices_kwh)[i] = ha;
|
||||||
if (ha < h_min_v) { h_min_v = ha; h_min_i = i; }
|
if (ha < h_min_v) { h_min_v = ha; h_min_i = i; }
|
||||||
if (ha > h_max_v) { h_max_v = ha; h_max_i = i; }
|
if (ha > h_max_v) { h_max_v = ha; h_max_i = i; }
|
||||||
char pb[12]; sprintf(pb, "%.4f", ha); json_h += pb;
|
int written = snprintf(json_h_buf + json_h_len, sizeof(json_h_buf) - json_h_len, "%.4f", ha);
|
||||||
|
json_h_len += written;
|
||||||
raw_sum += ha; cnt_kwh++;
|
raw_sum += ha; cnt_kwh++;
|
||||||
} else {
|
} else {
|
||||||
id(tomorrow_hourly_avg_prices_kwh)[i] = NAN;
|
id(tomorrow_hourly_avg_prices_kwh)[i] = NAN;
|
||||||
json_h += "null";
|
json_h_len += snprintf(json_h_buf + json_h_len, sizeof(json_h_buf) - json_h_len, "null");
|
||||||
|
}
|
||||||
|
if (i < 23) {
|
||||||
|
json_h_buf[json_h_len++] = ',';
|
||||||
}
|
}
|
||||||
if (i < 23) json_h += ",";
|
|
||||||
}
|
}
|
||||||
json_h += "]";
|
json_h_buf[json_h_len++] = ']';
|
||||||
id(json_tomorrow_hourly_prices_kwh).publish_state(json_h.c_str());
|
json_h_buf[json_h_len] = '\0';
|
||||||
|
id(json_tomorrow_hourly_prices_kwh).publish_state(json_h_buf);
|
||||||
|
|
||||||
auto build32 = [&](int start) -> std::string {
|
// v1.2.2: Optimized build32 using fixed buffer approach
|
||||||
std::string j = "[";
|
// Note: id() returns pointer, so use auto (pointer type) with ->
|
||||||
|
auto build32_fixed = [&](int start, auto sensor) {
|
||||||
|
char buf[450];
|
||||||
|
int len = 0;
|
||||||
|
buf[len++] = '[';
|
||||||
for (int i = start; i < start + 32; i++) {
|
for (int i = start; i < start + 32; i++) {
|
||||||
if (!std::isnan(v[i])) {
|
if (!std::isnan(v[i])) {
|
||||||
char pb[16];
|
|
||||||
// Use 3 decimal places for negative prices to stay within
|
// Use 3 decimal places for negative prices to stay within
|
||||||
// the 255-character HA text sensor state limit.
|
// the 255-character HA text sensor state limit.
|
||||||
// Positive prices keep full 4 decimal place precision.
|
// Positive prices keep full 4 decimal place precision.
|
||||||
if (v[i] < 0.0f)
|
if (v[i] < 0.0f)
|
||||||
snprintf(pb, sizeof(pb), "%.3f", v[i]);
|
len += snprintf(buf + len, sizeof(buf) - len, "%.3f", v[i]);
|
||||||
else
|
else
|
||||||
snprintf(pb, sizeof(pb), "%.4f", v[i]);
|
len += snprintf(buf + len, sizeof(buf) - len, "%.4f", v[i]);
|
||||||
j += pb;
|
|
||||||
} else {
|
} else {
|
||||||
j += "null";
|
len += snprintf(buf + len, sizeof(buf) - len, "null");
|
||||||
}
|
}
|
||||||
if (i < start + 31) j += ",";
|
if (i < start + 31) {
|
||||||
|
buf[len++] = ',';
|
||||||
|
}
|
||||||
|
// Safety: prevent buffer overflow
|
||||||
|
if (len >= (int)sizeof(buf) - 20) break;
|
||||||
}
|
}
|
||||||
j += "]"; return j;
|
buf[len++] = ']';
|
||||||
|
buf[len] = '\0';
|
||||||
|
sensor->publish_state(buf);
|
||||||
};
|
};
|
||||||
id(json_tomorrow_15min_prices_kwh_p1_00_00_07_45).publish_state(build32(0));
|
|
||||||
id(json_tomorrow_15min_prices_kwh_p2_08_00_15_45).publish_state(build32(32));
|
build32_fixed(0, id(json_tomorrow_15min_prices_kwh_p1_00_00_07_45));
|
||||||
id(json_tomorrow_15min_prices_kwh_p3_16_00_23_45).publish_state(build32(64));
|
yield(); // v1.2.2: Yield between heavy operations
|
||||||
|
build32_fixed(32, id(json_tomorrow_15min_prices_kwh_p2_08_00_15_45));
|
||||||
|
yield(); // v1.2.2: Yield between heavy operations
|
||||||
|
build32_fixed(64, id(json_tomorrow_15min_prices_kwh_p3_16_00_23_45));
|
||||||
|
|
||||||
if (min_v < 9999.0f) {
|
if (min_v < 9999.0f) {
|
||||||
id(tomorrow_min_price).publish_state(min_v);
|
id(tomorrow_min_price).publish_state(min_v);
|
||||||
@@ -1752,6 +1862,9 @@ script:
|
|||||||
id(tomorrow_current_price_status_str) = (cur >= 0) ? "Valid" : "Missing";
|
id(tomorrow_current_price_status_str) = (cur >= 0) ? "Valid" : "Missing";
|
||||||
id(tomorrow_last_update_attempt) = id(ha_time).now().timestamp;
|
id(tomorrow_last_update_attempt) = id(ha_time).now().timestamp;
|
||||||
|
|
||||||
|
// v1.2.2: Log heap at end
|
||||||
|
ESP_LOGI("eprices", "recompute_tomorrow: done, free heap = %u bytes", heap_caps_get_free_size(MALLOC_CAP_8BIT));
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# MIDNIGHT BRIDGE
|
# MIDNIGHT BRIDGE
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
Reference in New Issue
Block a user