Enhance README with stuck-fetch watchdog details (v1.2)

Added explanation for stuck-fetch watchdog mechanism and updated status message details.

CHANGELOG.md

@@ -1,5 +1,70 @@
 # EPrices – Changelog
 
+## v1.2 — 2026-04-06
+
+### HTTP fetch stuck-flag watchdog
+
+Fixed a reliability issue where an HTTP fetch that stalled at the TCP level
+(connection accepted by the server but data delivery suspended) could hold
+`is_updating_today` or `is_updating_tomorrow` locked at `true` indefinitely.
+While the flag was stuck, all subsequent auto-retry triggers and manual button
+presses were silently blocked.
+
+**Root cause:** The application-level 25 s HTTP timeout does not fire when the
+TCP connection is established but the server stops sending data mid-transfer.
+The socket never goes idle from the OS perspective, so the timeout never
+triggers, and the fetch hangs until the TCP stack eventually resets the
+connection — which can take several minutes.
+
+**Fix:** Added a 120-second watchdog to both the today and tomorrow worker
+loops (`seconds: /10`). If `is_updating_*` has been `true` for more than 120
+seconds, the worker force-clears the flag and sets the status message to
+`"Fetch timeout – will retry"`. The next scheduled retry or manual press then
+proceeds normally.
+
+**New globals:**
+- `is_updating_today_since` (`time_t`) — timestamp when `is_updating_today` was last set to `true`
+- `is_updating_tomorrow_since` (`time_t`) — timestamp when `is_updating_tomorrow` was last set to `true`
+
+Both timestamps are set immediately alongside `is_updating_* = true` inside
+`smart_price_update` and `smart_tomorrow_price_update`.
+
+**Changed locations in `eprices.yaml`:**
+- `globals:` — added `is_updating_today_since` and `is_updating_tomorrow_since`
+- `smart_price_update` script — added `id(is_updating_today_since) = ...` alongside flag set
+- `smart_tomorrow_price_update` script — added `id(is_updating_tomorrow_since) = ...` alongside flag set
+- Today worker (`seconds: /10`) — watchdog block added before existing worker logic
+- Tomorrow worker (`seconds: /10`) — watchdog block added before existing worker logic
+
+---
+
+### Tomorrow auto-retry attempt counter fix
+
+Fixed a bug where `tomorrow_retry_count` was not incremented in the 13:55 and
+14:55–19:55 hourly retry triggers. Those triggers called
+`smart_tomorrow_price_update` directly without incrementing the counter first,
+causing the `Tomorrow API Fetch Attempts` diagnostic sensor to undercount
+attempts after the first one at 13:25.
+
+**Changed locations in `eprices.yaml`:**
+- 13:55 trigger — added `id(tomorrow_retry_count)++` before `.execute()`
+- 14:55–19:55 hourly trigger — added `id(tomorrow_retry_count)++` before `.execute()`
+
+---
+
+### Status message improvements
+
+Two status message strings that contained a hardcoded time reference have been
+corrected. Both were misleading after a device reboot in the afternoon or a
+manual `clear_tomorrow_prices` call outside the normal fetch window.
+
+| Location | Old value | New value |
+|---|---|---|
+| `tomorrow_update_status_message` global `initial_value` | `"Waiting for 13:20"` | `"No data yet"` |
+| `clear_tomorrow_prices` script end | `"Cleared – awaiting next 13:20 window"` | `"Cleared – awaiting fetch window"` |
+
+---
+
 ## v1.1 — 2026-04-06
 
 ### Negative price provider fee support

README.md

@@ -22,7 +22,8 @@ Assistant automations are required for any core functionality.
   for use in HA automations and dashboards
 - **NVS persistence** — prices survive device reboots without re-fetching
 - **Midnight bridge** — tomorrow's data automatically becomes today at 00:00, fully on-device
-- **Auto-retry logic** — up to 8 HTTP fetch attempts for both today and tomorrow
+- **Auto-retry logic** — up to 8 HTTP fetch attempts for both today and tomorrow,
+  with a **120-second stuck-fetch watchdog** that unblocks retries after TCP-level stalls
 - **DST-safe** — uses UNIX timestamps and binary search throughout, no hour-slot arithmetic
 - **Staleness detection** — `Today Current Price Status` shows `Stale` if stored date mismatches today
 - **Tomorrow live sensors** evaluate at `now + 86400s` — reflecting tomorrow at the same local time
@@ -133,6 +134,18 @@ For the full list see the [Energy-Charts API documentation](https://api.energy-c
 | 13:55, 14:55 … 19:55 | Retry tomorrow fetch if previous attempt failed (max 8 total) |
 | Manual button press | Force immediate fetch for today or tomorrow |
 
+### Stuck-fetch watchdog
+
+If an HTTP request stalls at the TCP level (connection accepted but data
+delivery suspended), the 25 s application timeout may not fire. In that case
+the `is_updating_*` flag can stay `true` for several minutes, blocking all
+subsequent retries and manual button presses.
+
+The worker loop checks every 10 seconds: if `is_updating_today` or
+`is_updating_tomorrow` has been `true` for more than **120 seconds**, the flag
+is force-cleared and the status message is set to `"Fetch timeout – will retry"`.
+The next scheduled trigger or manual press then proceeds normally.
+
 ### Price calculation
 
 Raw prices from the API are in **€/MWh**. EPrices converts them to **€/kWh**
@@ -259,7 +272,7 @@ safe on DST transition days (23-hour and 25-hour days).
 | Today Price Update Status | `sensor.eprices_today_price_update_status` | `SUCCESS` / `FAILED/WAITING` |
 | Tomorrow Price Update Status | `sensor.eprices_tomorrow_price_update_status` | `SUCCESS` / `FAILED/WAITING` |
 | Today Price Update Status Message | `sensor.eprices_today_price_update_status_message` | Detailed status string |
-| Tomorrow Price Update Status Message | `sensor.eprices_tomorrow_price_update_status_message` | Detailed status string |
+| Tomorrow Price Update Status Message | `sensor.eprices_tomorrow_price_update_status_message` | Detailed status string; `"Fetch timeout – will retry"` after watchdog trigger |
 
 ### Buttons
 

VERSION.md

@@ -1,5 +1,40 @@
 # EPrices – Version History
 
+## v1.2 — 2026-04-06
+
+### HTTP fetch stuck-flag watchdog
+
+Fixed a production-observed reliability issue where a TCP-level stall during
+an HTTP fetch could lock `is_updating_today` or `is_updating_tomorrow` at
+`true` for several minutes, silently blocking all auto-retry triggers and
+manual button presses for the duration.
+
+A 120-second watchdog was added to both worker loops. If either `is_updating_*`
+flag has been held for more than 120 seconds, the worker force-clears it and
+sets the status message to `"Fetch timeout – will retry"`, allowing the next
+scheduled retry or manual press to proceed immediately.
+
+New globals: `is_updating_today_since`, `is_updating_tomorrow_since`.
+
+### Tomorrow auto-retry attempt counter fix
+
+`tomorrow_retry_count` was not being incremented in the 13:55 and 14:55–19:55
+hourly retry triggers, causing the `Tomorrow API Fetch Attempts` diagnostic
+sensor to undercount after the first scheduled attempt at 13:25.
+
+### Status message improvements
+
+- `tomorrow_update_status_message` initial value changed from
+  `"Waiting for 13:20"` to `"No data yet"` — removes misleading time
+  reference shown after afternoon reboots
+- `clear_tomorrow_prices` end message changed from
+  `"Cleared – awaiting next 13:20 window"` to
+  `"Cleared – awaiting fetch window"`
+
+See `CHANGELOG.md` for full implementation details.
+
+---
+
 ## v1.1 — 2026-04-06
 
 ### Negative price provider fee

eprices.yaml

@@ -1,10 +1,25 @@
 # =============================================================================
-# ESPHome: EPrices v1.1
+# ESPHome: EPrices v1.2
 #
 # Electricity price data provided by Energy-Charts (https://energy-charts.info)
 # under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/).
 #
-# v1.1 changes:
+# v1.2 changes (2026-04-06):
+# - HTTP fetch stuck-flag watchdog (120 s) added to today and tomorrow workers:
+#   if is_updating_today / is_updating_tomorrow stays true for >120 s (TCP
+#   stall not caught by app-level timeout), the flag is force-cleared and
+#   status message set to "Fetch timeout – will retry"
+# - New globals: is_updating_today_since, is_updating_tomorrow_since
+#   set alongside is_updating_* = true in smart_price_update and
+#   smart_tomorrow_price_update
+# - tomorrow_retry_count++ added before .execute() in 13:55 and 14:55–19:55
+#   hourly retry triggers (counter was not incrementing after attempt #1)
+# - tomorrow_update_status_message initial_value: "Waiting for 13:20"
+#   → "No data yet" (removes misleading time reference after afternoon reboot)
+# - clear_tomorrow_prices end message: "Cleared – awaiting next 13:20 window"
+#   → "Cleared – awaiting fetch window"
+#
+# v1.1 changes (2026-04-06):
 # - Separate provider fee for negative spot prices (eprices_neg_prov_fee):
 #   positive prices: (raw / 1000) × (1 + prov_fee)     × (1 + vat_rate)
 #   negative prices: (raw / 1000) × (1 - neg_prov_fee) × (1 + vat_rate)
@@ -42,6 +57,7 @@
 #   eprices_country_bzn           e.g. "SI"
 #   eprices_prov_fee              e.g. "0.12"  (provider fee, decimal multiplier)
 #   eprices_vat_rate              e.g. "0.22"  (VAT rate, decimal multiplier)
+#   eprices_neg_prov_fee          e.g. "0.30"  (neg price provider fee, decimal multiplier)
 #
 # See README.md for full installation instructions and sensor reference.
 # See CHANGELOG.md for complete sensor and entity ID listing.
@@ -185,6 +201,15 @@ time:
       - seconds: /10
         then:
           - lambda: |-
+              if (id(is_updating_today) && id(is_updating_today_since) > 0) {
+                time_t now_ts = (time_t)id(ha_time).now().timestamp;
+                if (now_ts - id(is_updating_today_since) > 120) {
+                  ESP_LOGW("eprices", "Worker: is_updating_today stuck >120s, force-clearing");
+                  id(is_updating_today) = false;
+                  id(is_updating_today_since) = 0;
+                  id(update_status_message) = "Fetch timeout – will retry";
+                }
+              }
               if (id(need_today_update) && !id(is_updating_today)) {
                 ESP_LOGI("eprices", "Worker: starting Today update");
                 id(need_today_update) = false;
@@ -194,6 +219,15 @@ time:
       - seconds: /10
         then:
           - lambda: |-
+              if (id(is_updating_tomorrow) && id(is_updating_tomorrow_since) > 0) {
+                time_t now_ts = (time_t)id(ha_time).now().timestamp;
+                if (now_ts - id(is_updating_tomorrow_since) > 120) {
+                  ESP_LOGW("eprices", "Worker: is_updating_tomorrow stuck >120s, force-clearing");
+                  id(is_updating_tomorrow) = false;
+                  id(is_updating_tomorrow_since) = 0;
+                  id(tomorrow_update_status_message) = "Fetch timeout – will retry";
+                }
+              }
               auto t = id(ha_time).now();
               bool in_window = (t.hour > 13 && t.hour < 23) ||
                                (t.hour == 13 && t.minute >= 20) ||
@@ -322,7 +356,8 @@ time:
                 return;
               }
 
-              ESP_LOGW("eprices", "Auto tomorrow: attempt #%d at 13:55", id(tomorrow_retry_count) + 1);
+              id(tomorrow_retry_count)++;
+              ESP_LOGW("eprices", "Auto tomorrow: attempt #%d at 13:55", id(tomorrow_retry_count));
               id(smart_tomorrow_price_update).execute();
 
       # Attempts #3.. at 14:55, 15:55, 16:55, 17:55, 18:55, 19:55 (capped to 8)
@@ -343,7 +378,8 @@ time:
                 return;
               }
 
-              ESP_LOGW("eprices", "Auto tomorrow: attempt #%d at %02d:55", id(tomorrow_retry_count) + 1, t.hour);
+              id(tomorrow_retry_count)++;
+              ESP_LOGW("eprices", "Auto tomorrow: attempt #%d at %02d:55", id(tomorrow_retry_count), t.hour);
               id(smart_tomorrow_price_update).execute();
 
       - minutes: /1
@@ -462,6 +498,9 @@ globals:
   - id: is_updating_today
     type: bool
     initial_value: 'false'
+  - id: is_updating_today_since
+    type: time_t
+    initial_value: '0'
 
   - id: tomorrow_last_successful_update
     type: time_t
@@ -478,7 +517,7 @@ globals:
     initial_value: 'false'
   - id: tomorrow_update_status_message
     type: std::string
-    initial_value: '"Waiting for 13:20"'
+    initial_value: '"No data yet"'
   - id: tomorrow_retry_count
     type: int
     initial_value: '0'
@@ -494,6 +533,9 @@ globals:
   - id: is_updating_tomorrow
     type: bool
     initial_value: 'false'
+  - id: is_updating_tomorrow_since
+    type: time_t
+    initial_value: '0'
 
   # --- Country code for Energy-Charts API (wired from secret) ---
   - id: country_bzn
@@ -1769,6 +1811,7 @@ script:
             return;
           }
           id(is_updating_today) = true;
+          id(is_updating_today_since) = (time_t)id(ha_time).now().timestamp;
           id(last_update_attempt) = id(ha_time).now().timestamp;
           id(retry_count)++;
           { char buf[8]; snprintf(buf, sizeof(buf), "%d", id(retry_count)); id(today_price_update_attempts).publish_state(buf); }
@@ -1872,6 +1915,7 @@ script:
             return;
           }
           id(is_updating_tomorrow) = true;
+          id(is_updating_tomorrow_since) = (time_t)id(ha_time).now().timestamp;
           id(tomorrow_last_update_attempt) = id(ha_time).now().timestamp;
           id(tomorrow_retry_count)++;
           { char buf[8]; snprintf(buf, sizeof(buf), "%d", id(tomorrow_retry_count)); id(tomorrow_price_update_attempts).publish_state(buf); }
@@ -2101,7 +2145,7 @@ script:
           id(tomorrow_min_hourly_price_time_str) = "--:--";
           id(tomorrow_max_hourly_price_time_str) = "--:--";
           id(tomorrow_last_update_success) = false;
-          id(tomorrow_update_status_message) = "Cleared – awaiting next 13:20 window";
+          id(tomorrow_update_status_message) = "Cleared – awaiting fetch window";
           id(tomorrow_current_price_status_str) = "Cleared";
           id(tomorrow_price_update_status).update();
           id(tomorrow_price_status_message).update();