This updates all dependencies to their latest versions - except two
packages:
- eslint: Some plugins we use here aren't compatible yet to ESLint v10.
- node-ical: The new version has revealed an issue in our calendar
logic. I would prefer to address this in a separate PR.
After updating the dependencies, eslint-plugin-package-json rules
complained about missing fields in the package.json. I added them in the
second commit.
## Loading `config.js`
### Previously
Loaded on server-side in `app.js` and in the browser by including
`config.js` in `index.html`. The web server has an endpoint `/config`
providing the content of server loaded `config.js`.
### Now
Loaded only on server-side in `app.js`. The browser loads the content
using the web server endpoint `/config`. So the server has control what
to provide to the clients.
Loading the `config.js` was moved to `Utils.js` so that
`check_config.js` can use the same functions.
## Using environment variables in `config.js`
### Previously
Environment variables were not allowed in `config.js`. The workaround
was to create a `config.js.template` with curly braced bash variables
allowed. While starting the app the `config.js.template` was converted
via `envsub` into a `config.js`.
### Now
Curly braced bash variables are allowed in `config.js`. Because only the
server loads `config.js` he can substitute the variables while loading.
## Secrets in MagicMirror²
To be honest, this is a mess.
### Previously
All content defined in the `config` directory was reachable from the
browser. Everyone with access to the site could see all stuff defined in
the configuration e.g. using the url http://ip:8080/config. This
included api keys and other secrets.
So sharing a MagicMirror² url to others or running MagicMirror² without
authentication as public website was not possible.
### Now
With this PR we add (beta) functionality to protect sensitive data. This
is only possible for modules running with a `node_helper`. For modules
running in the browser only (e.g. default `weather` module), there is no
way to hide data (per construction). This does not mean, that every
module with `node_helper` is safe, e.g. the default `calendar` module is
not safe because it uses the calendar url's as sort of id and sends them
to the client.
For adding more security you have to set `hideConfigSecrets: true` in
`config.js`. With this:
- `config/config.env` is not deliverd to the browser
- the contents of environment variables beginning with `SECRET_` are not
published to the clients
This is a first step to protect sensitive data and you can at least
protect some secrets.
This migrates the Newsfeed module to use the centralized HTTPFetcher
class (introduced in #4016), following the same pattern as the Calendar
module.
This continues the refactoring effort to centralize HTTP error handling
across all modules.
## Changes
**NewsfeedFetcher:**
- Refactored from function constructor to ES6 class (like the calendar
module in #3959)
- Replaced manual fetch() + timer handling with HTTPFetcher composition
- Uses structured error objects with translation keys
- Inherits smart retry strategies (401/403, 429, 5xx backoff)
- Inherits timeout handling (30s) and AbortController
**node_helper.js:**
- Updated error handler to use `errorInfo.translationKey`
- Simplified property access (`fetcher.url`, `fetcher.items`)
**Cleanup:**
- Removed `js/module_functions.js` (`scheduleTimer` no longer needed)
- Removed `#module_functions` import from package.json
## Related
Part of the HTTPFetcher migration effort started in #4016.
Next candidate: Weather module (client-side → server-side migration).
This is another change to cleanup structure, already mentioned in
https://github.com/MagicMirrorOrg/MagicMirror/pull/4019#issuecomment-3792953018
After separating default and 3rd-party modules this PR moves the
`custom.css` from the mm-owned directory `css` into user owned directory
`config`.
It has a built-in function which moves the `css/custom.css` to the new
location `config/custom.css` (if the target not exists).
Let me know if there's a majority in favor of this change.
Since the project's inception, I've missed a clear separation between
default and third-party modules.
This increases complexity within the project (exclude `modules`, but not
`modules/default`), but the mixed use is particularly problematic in
Docker setups.
Therefore, with this pull request, I'm moving the default modules to a
different directory.
~~I've chosen `default/modules`, but I'm not bothered about it;
`defaultmodules` or something similar would work just as well.~~
Changed to `defaultmodules`.
Let me know if there's a majority in favor of this change.
- remove param `--enable-features=UseOzonePlatform` in start electron
tests (as we did already in `package.json`)
- update node versions in github workflows, remove `22.21.1`, add `25.x`
- fix formatting in tests
- update dependencies including electron to v40
This is still a draft PR because most calendar electron tests are not
running which is caused by the electron update from `v39.3.0` to
`v40.0.0`. Maybe @KristjanESPERANTO has an idea ...
---------
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
## Summary
PR [#3976](https://github.com/MagicMirrorOrg/MagicMirror/pull/3976)
introduced smart HTTP error handling for the Calendar module. This PR
extracts that HTTP logic into a central `HTTPFetcher` class.
Calendar is the first module to use it. Follow-up PRs would migrate
Newsfeed and maybe even Weather.
**Before this change:**
- ❌ Each module had to implemented its own `fetch()` calls
- ❌ No centralized retry logic or backoff strategies
- ❌ No timeout handling for hanging requests
- ❌ Error detection relied on fragile string parsing
**What this PR adds:**
- ✅ Unified HTTPFetcher class with intelligent retry strategies
- ✅ Modern AbortController with configurable timeout (default 30s)
- ✅ Proper undici Agent for self-signed certificates
- ✅ Structured error objects with translation keys
- ✅ Calendar module migrated as first consumer
- ✅ Comprehensive unit tests with msw (Mock Service Worker)
## Architecture
**Before - Decentralized HTTP handling:**
```
Calendar Module Newsfeed Module Weather Module
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ fetch() own │ │ fetch() own │ │ fetch() own │
│ retry logic │ │ basic error │ │ no retry │
│ error parse │ │ handling │ │ client-side │
└─────────────┘ └─────────────┘ └─────────────┘
│ │ │
└───────────────────────┴───────────────────────┘
▼
External APIs
```
**After - Centralized with HTTPFetcher:**
```
┌─────────────────────────────────────────────────────┐
│ HTTPFetcher │
│ • Unified retry strategies (401/403, 429, 5xx) │
│ • AbortController timeout (30s) │
│ • Structured errors with translation keys │
│ • undici Agent for self-signed certs │
└────────────┬──────────────┬──────────────┬──────────┘
│ │ │
┌───────▼───────┐ ┌────▼─────┐ ┌──────▼──────┐
│ Calendar │ │ Newsfeed │ │ Weather │
│ ✅ This PR │ │ future │ │ future │
└───────────────┘ └──────────┘ └─────────────┘
│ │ │
└──────────────┴──────────────┘
▼
External APIs
```
## Complexity Considerations
**Does HTTPFetcher add complexity?**
Even if it may look more complex, it actually **reduces overall
complexity**:
- **Calendar already has this logic** (PR #3976) - we're extracting, not
adding
- **Alternative is worse:** Each module implementing own logic = 3× the
code
- **Better testability:** 443 lines of tests once vs. duplicating tests
for each module
- **Standards-based:** Retry-After is RFC 7231, not custom logic
## Future Benefits
**Weather migration (future PR):**
Moving Weather from client-side to server-side will enable:
- **Same robust error handling** - Weather gets 429 rate-limiting, 5xx
backoff for free
- **Simpler architecture** - No proxy layer needed
Moving the weather modules from client-side to server-side will be a big
undertaking, but I think it's a good strategy. Even if we only move the
calendar and newsfeed to the new HTTP fetcher and leave the weather as
it is, this PR still makes sense, I think.
## Breaking Changes
**None**
----
I am eager to hear your opinion on this 🙂
This PR makes three small changes to the ESLint setup:
1. Migrate from
[eslint-plugin-vitest](https://www.npmjs.com/package/eslint-plugin-vitest)
to it's successor
[@vitest/eslint-plugin](https://www.npmjs.com/package/@vitest/eslint-plugin).
2. Change the eslint config so that only the test files are checked with
the vitest rules. Previously, it was just unnecessary and inefficient to
check all js files with them.
3. We had defined some of the test rules as warnings, but that is not
really ideal. I changed them to errors.
Adapts calendar module to node-ical changes and fixes a bug with moved
full-day recurring events in eastern timezones.
## Changes
### 1. Update node-ical to 0.23.1
- Includes upstream fixes for UNTIL UTC validation errors from CalDAV
servers (reported by @rejas in PR #4010)
- Changes to `getDateKey()` behavior for VALUE=DATE events (now uses
local date components)
- Fixes issue with malformed DURATION values (reported by MagicMirror
user here: https://github.com/jens-maus/node-ical/issues/381)
### 2. Remove dead code
- Removed ineffective UNTIL modification code (rule.options is read-only
in rrule-temporal)
- The code attempted to extend UNTIL for all-day events but had no
effect
### 3. Fix recurrence lookup for full-day events
node-ical changed the behavior of `getDateKey()` - it now uses local
date components for VALUE=DATE events instead of UTC. This broke
recurrence override lookups for full-day events in eastern timezones.
**Why it broke:**
- **before node-ical update:** Both node-ical and MagicMirror used UTC →
keys matched ✅
- **after node-ical update:** node-ical uses local date (RFC 5545
conform), MagicMirror still used UTC → **mismatch** ❌
**Example:**
- Full-day recurring event on October 12 in Europe/Berlin (UTC+2)
- node-ical 0.23.1 stores override with key: `"2024-10-12"` (local date)
- MagicMirror looked for key: `"2024-10-11"` (from UTC: Oct 11 22:00)
- **Result:** Moved event not found, appears on wrong date
**Solution:** Adapt to node-ical's new behavior by using local date
components for full-day events, UTC for timed events.
**Note:** This is different from previous timezone fixes - those
addressed event generation, this fixes the lookup of recurrence
overrides.
## Background
node-ical 0.23.0 switched from `rrule` to `rrule-temporal`, introducing
breaking changes. Version 0.23.1 fixed the UNTIL validation issue and
formalized the `getDateKey()` behavior for DATE vs DATE-TIME values,
following RFC 5545 specification that DATE values represent local
calendar dates without timezone context.
Updating `node-ical` and adapt logic to new behaviour.
## Problem
node-ical 0.23.0 switched from `rrule.js` to `rrule-temporal`, changing
how recurring event dates are returned. Our code assumed the old
behavior where dates needed manual timezone conversion.
## Solution
Updated `getMomentsFromRecurringEvent()` in `calendarfetcherutils.js`:
- Removed `tzid = null` clearing (no longer needed)
- Simplified timed events: `moment.tz(date, eventTimezone)` instead of
`moment.tz(date, "UTC").tz(eventTimezone, true)`
- Kept UTC component extraction for full-day events to prevent date
shifts
This PR changes the default `start` and `start:dev` scripts to use
Wayland instead of X11. I think after three years, it's time to take
this step.
### Background
Since Raspberry Pi OS Bookworm (2023), Wayland is the default display
server. As most MagicMirror installations run on Raspberry Pi, this
change aligns with what new users already have installed.
### Benefits
Especially for new users (which install the OS with Wayland) it's easier
- they can simply run `npm start` without needing to understand display
server differences or manually switch scripts.
And for projects in general it's better to rely on modern defaults than
on legacy.
### Breaking Changes
**None** - X11 support is maintained. Users who really use and need X11
can use `node --run start:x11`.
The Electron 38+ auto-detection of ozone-platform does not work reliably
in all environments (e.g., Docker containers) because it depends on
environment variables like XDG_SESSION_TYPE which may not be set.
Since `start:wayland` is explicitly called for Wayland sessions, it
makes sense to explicitly specify the platform flag for maximum
reliability.
The `--enable-features=UseOzonePlatform` flag remains removed as it is
no longer needed since Electron 38.
Fixes Docker compatibility issue reported in
MagicMirrorOrg/MagicMirror#3974
### 1. Replace `XMLHttpRequest` with the modern `fetch` API for loading
translation files
#### Changes
- **translator.js**: Use `fetch` with `async/await` instead of XHR
callbacks
- **loader.js**: Align URL handling and add error handling (follow-up to
fetch migration)
- **Tests**: Update infrastructure for `fetch` compatibility
#### Benefits
- Modern standard API
- Cleaner, more readable code
- Better error handling and fallback mechanisms
### 2. Migrate e2e tests to Playwright
This wasn't originally planned for this PR, but is related. While
investigating suspicious log entries which surfaced after the fetch
migration I kept running into JSDOM’s limitations. That pushed me to
migrate the E2E suite to Playwright instead.
#### Changes
- switch e2e harness to Playwright (`tests/e2e/helpers/global-setup.js`)
- rewrite specs to use Playwright locators + shared `expectTextContent`
- install Chromium via `npx playwright install --with-deps` in CI
#### Benefits
- much closer to real browser behaviour
- and no more fighting JSDOM’s quirks
`node-libgpiod` uses deprecated NAN which is incompatible with Electron
v39+. `serialport` uses N-API ensuring compatibility with current and
future Electron versions.
`node-libgpiod` is only used by 1 of ~1300 3rd-party-modules
(MMM-PresenceScreenControl), while serialport is used by at least 4
modules (MMM-Serial-Notification, MMM-RadarPresence, MMM-LKY-TIC and
MMM-Gestures), making it a better test candidate.
Also updates Electron to v39.
Fixes#3933
This is a big change, but I think it's a good move, as `vitest` is much
more modern than `jest`.
I'm excited about the UI watch feature (run `npm run test:ui`), for
example - it's really helpful and saves time when debugging tests. I had
to adjust a few tests because they had time related issues, but
basically we are now testing the same things - even a bit better and
less flaky (I hope).
What do you think?
seems we dont need the parse5 pin as long as jsdom is fixed to v27.0.0.
not sure if there is anything else we can do to the deps?
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: veeck <gitkraken@veeck.de>
## Description
This PR adds a new `server:watch` script that runs MagicMirror² in
server-only mode with automatic restart and browser reload capabilities.
Particularly helpful for:
- **Developers** who need to see changes immediately without manual
restarts.
- **Users setting up their mirror** who make many changes to `config.js`
or `custom.css` and need quick feedback.
### What it does
When you run `npm run server:watch`, the watcher monitors files you
specify in `config.watchTargets`. Whenever a monitored file changes:
1. The server automatically restarts
2. Waits for the port to become available
3. Sends a reload notification to all connected browsers via Socket.io
4. Browsers automatically refresh to show the changes
This creates a seamless development experience where you can edit code,
save, and see the results within seconds.
### Implementation highlights
**Zero dependencies:** Uses only Node.js built-ins (`fs.watch`,
`child_process.spawn`, `net`, `http`) - no nodemon or external watchers
needed.
**Smart file watching:** Monitors parent directories instead of files
directly to handle atomic writes from modern editors (VSCode, etc.) that
create temporary files during save operations.
**Port management:** Waits for the old server instance to fully release
the port before starting a new one, preventing "port already in use"
errors.
### Configuration
Users explicitly define which files to monitor in their `config.js`:
```js
let config = {
watchTargets: [
"config/config.js",
"css/custom.css",
"modules/MMM-MyModule/MMM-MyModule.js",
"modules/MMM-MyModule/node_helper.js"
],
// ... rest of config
};
```
This explicit approach keeps the implementation simple (~260 lines)
while giving users full control over what triggers restarts. If
`watchTargets` is empty or undefined, the watcher starts but monitors
nothing, logging a clear warning message.
---
**Note:** This PR description has been updated to reflect the final
implementation. During the review process, we refined the approach
multiple times based on feedback.
---------
Co-authored-by: Jboucly <contact@jboucly.fr>
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
Normally, I wouldn't update the dependencies again so soon, but
`node-ical` underwent some major changes (see
https://github.com/jens-maus/node-ical/pull/404) with the last release,
and I'd like to use it here as early as possible to see if there are any
problems with it.
This fixes security issue
[CVE-2023-42282](https://github.com/advisories/GHSA-78xj-cgh5-2h22),
which is not very likely to be exploitable in MagicMirror² setups, but
still should be fixed.
The [express-ipfilter](https://www.npmjs.com/package/express-ipfilter)
package depends on the obviously unmaintained
[ip](https://github.com/indutny/node-ip) package, which has known
security vulnerabilities. Since no fix is available, this commit
replaces both dependencies with a custom middleware using the better
maintained [ipaddr.js](https://www.npmjs.com/package/ipaddr.js) library.
Changes:
- Add new `js/ip_access_control.js` with lightweight middleware
- Remove `express-ipfilter` dependency, add `ipaddr.js`
- Update `js/server.js` to use new middleware
- In addition, I have formulated the descriptions of the corresponding
tests a little more clearly.
- removes the external unmaintained `module-alias` dependency ->
reducing complexity and risk
- introduces a small internal alias mechanism for `logger` and
`node_helper`
- preserves backward compatibility for existing 3rd‑party modules
- should simplify a future ESM migration of MagicMirror
I'm confident that it shouldn't cause any problems, but we could also
consider including it in the release after next. What do you think?
This PR is inspired by PR #2934 - so thanks to @thesebas! 🙇😃
This prevents `prettier` from failing when `lint-staged` passes
unknown/binary files, making the pre-commit hook more robust.
In concrete terms this could happen, when we, for example, add a new PNG
file. Since we rarely do this, it has not been noticed so far.
This is recommended when using asterisk:
https://github.com/lint-staged/lint-staged#automatically-fix-code-style-with-prettier-for-any-format-prettier-supports
## before
```bash
$ npx lint-staged <-- after staging a new PNG file
✔ Backed up original state in git stash (c3247d4b)
✔ Hiding unstaged changes to partially staged files...
⚠ Running tasks for staged files...
❯ package.json — 2 files
❯ * — 2 files
✖ prettier --write [FAILED]
↓ *.js — no files
↓ *.css — no files
↓ Skipped because of errors from tasks.
↓ Skipped because of errors from tasks.
✔ Reverting to original state because of errors...
✔ Cleaning up temporary files...
✖ prettier --write:
[error] No parser could be inferred for file "~/MagicMirror/test.png".
...
```
## after
```bash
$ npx lint-staged <-- after staging a new PNG file
✔ Backed up original state in git stash (0c3fe285)
✔ Running tasks for staged files...
✔ Applying modifications from tasks...
✔ Cleaning up temporary files...
```
electron uses node v22.18 in its [current
releases](https://releases.electronjs.org/), so we should go hand in
hand and use that as the minimal node version
nothing fancy in these though
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: veeck <gitkraken@veeck.de>
- Remove `sinon` dependency in favor of Jest native mocking
- Unify test helper functions across translation test suites
- Rename `setupDOMEnvironment` to `createTranslationTestEnvironment` for
consistency
- Simplify DOM setup by removing unnecessary Promise/async patterns
- Avoid potential port conflicts by using port 3001 for translator unit
tests
- Improve test reliability and maintainability
Fixes#3253
Adds a configuration option to overwrite the default `User-Agent` header
that is send at least by the calendar and news module. Allows other
modules to use the individual user agent as well.
The configuration accepts either a string or a function:
```
var config =
{
...
userAgent: 'Mozilla/5.0 (My User Agent)',
...
}
```
or
```
var config =
{
...
userAgent: () => 'Mozilla/5.0 (My User Agent)',
...
}
```
---------
Co-authored-by: Veeck <github@veeck.de>
Co-authored-by: veeck <gitkraken@veeck.de>
Co-authored-by: Karsten Hassel <hassel@gmx.de>
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
Just some normal maintainance after the holidays
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: veeck <gitkraken@veeck.de>
