Commit Graph

620 Commits

Author SHA1 Message Date
Kristjan ESPERANTO
2e97e29ab5 fix(http_fetcher): use undici.fetch when dispatcher is present (#4097)
### What's the problem?

The `selfSignedCert` option passes an undici `Agent` as `dispatcher` to
`fetch()`. But Node's built-in `fetch()` and undici@8's `Agent` use
different internal handler APIs - passing them together throws:

```
invalid onRequestStart method
```

### What's the fix?

When `selfSignedCert` is enabled (i.e. a `dispatcher` is set), use
undici's own `fetch()` instead of the global one. For all other
requests, keep using `globalThis.fetch`.

```js
const fetchFn = requestOptions.dispatcher ? undiciFetch : globalThis.fetch;
```

### Why not just always use undici's fetch?

That would fix the crash - but it would break some tests. MSW (Mock
Service Worker), which is used in our test suite to intercept HTTP
requests, only hooks into `globalThis.fetch`. Undici's fetch bypasses
those interceptors entirely, so tests would start making real network
requests instead of getting the mocked responses. We could rewrite all
tests to use undici-compatible mocking instead - but that would be a
massive change for no real benefit.

----

Fixes #4093
2026-04-08 18:42:30 +02:00
Kristjan ESPERANTO
d8c29d5ec3 fix: prevent crash when config is undefined in socket handler (#4096)
If a module uses this.io.of() to register a custom socket.io namespace,
connections on that namespace trigger the onAny handler in setSocketIO
before config is set, causing a TypeError.

Fixes #4089
2026-04-07 21:15:25 +02:00
Karsten Hassel
9b97add1ae fix cors function for alpine linux (#4091)
I provide docker images with alpine linux and tested the new cors
approach.

It didn't work because after calling

```js
  const dispatcher = new Agent({ connect: { lookup: (_h, _o, cb) => cb(null, address, family) } });
```

the dispatcher variable was undefined.

This PR solves this and I tested this under debian too.

The mix of internal fetch and newer undici did not work and alpine needs
additionally the `process.nextTick`.

---------

Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
2026-04-05 21:42:32 +02:00
Kristjan ESPERANTO
96c18ec8b0 fix(cors): prevent SSRF via DNS rebinding (#4090)
PR #4084 blocked SSRF by checking the IP before `fetch()` — but
`fetch()` resolves DNS again on its own. With DNS rebinding (TTL=0,
alternating IPs) an attacker can slip a private IP through between check
and connection.

Fix: resolve DNS once, validate, pin the validated IP for the
connection.

No second DNS query → no rebinding window. `isPrivateTarget()` is gone,
code is shorter than before.

Not a likely attack for a typical MagicMirror setup, but it doesn't add
complexity so there's no reason not to close the gap.
2026-04-04 20:40:14 +02:00
Karsten Hassel
abf6989faa add option to disable or restrict cors endpoint (#4087) 2026-04-04 11:55:13 +02:00
Kristjan ESPERANTO
03f268934a fix: prevent SSRF via /cors endpoint by blocking private/reserved IPs (#4084)
Resolve target hostname before proxying and reject any address that is
not globally routable (loopback, RFC 1918, link-local, etc.) using
ipaddr.js and dns.lookup().
2026-04-03 17:57:26 +02:00
Kristjan ESPERANTO
dce2df63af refactor: replace implicit global config with explicit global.config (#4085)
In PR #4072 the GitHub bot complained about a missing variable
declaration for `config` in `app.js` and suggested adding `let config`.
Applying that suggestion broke the app because `server_functions.js` was
accessing `config` as an implicit global variable - the `let`
declaration made it unreachable from there.

So instead of the `let` declaration, I replaced all bare `config`
references with explicit `global.config`. This makes the dependency on
the global variable visible without changing runtime behavior,
consistent with how other globals like `global.root_path` and
`global.version` are already handled throughout the codebase.

Related to #4073
2026-04-03 17:56:11 +02:00
Kristjan ESPERANTO
8e1630e8bf refactor: enable ESLint rule "no-unused-vars" and handle related issues (#4080)
In PR #4072 GitHub Bot complained about an unused var. Instead of just
removing that one, I checked why ESLint hadn't complained about it: We
had disabled the rule for it.

So I enabled rule and resolved the issues that ESLint then detected.

Related to #4073
2026-04-02 08:56:27 +02:00
Kristjan ESPERANTO
3ea3f0a605 chore: upgrade ESLint to v10 and fix newly surfaced issues (#4057)
`eslint-plugin-import-x` was the last thing blocking the ESLint v10
upgrade - it just got v10 support. So here we go.

The upgrade itself is tiny. The rest of the diff is cleanup from issues
ESLint v10 now catches: a few `let` declarations with initial values
that were immediately overwritten anyway (`no-useless-assignment`), and
`Translator` listed in `/* global */` in `main.js` and `module.js`.

Working through those `no-useless-assignment` warnings also surfaced a
dead default in `openmeteo`: `maxEntries: 5` in the constructor, which
was never actually doing anything - `openmeteo` never reads
`this.config.maxEntries` anywhere. And `weather.js` already sets that
default for all providers, so it was just a redundant duplicate. Removed
that too.

No runtime behavior changes.
2026-03-12 11:58:26 +01:00
Kristjan ESPERANTO
21d1e7472a refactor: simplify internal require() calls (#4056)
Remove unnecessary `__dirname` template-literal prefix from relative
`require()` paths. Node.js resolves relative require() paths correctly
without it.

While this may look like nitpicking: `require("./server")` is
transparent to static analysis tools - IDEs can resolve the path and
support go-to-definition. Template literals with `__dirname` are opaque
to them. It also removes another usage of `__dirname`, which has no
native equivalent in ESM and would need to be replaced there anyway
(when we switch to ESM anytime in the future).

The import reordering is a side effect: `import-x/order` treats
template-literal `require()` calls differently from plain strings, so
the previous order was no longer valid.
2026-03-12 10:35:42 +01:00
Kristjan ESPERANTO
cb61aebb5a chore: update ESLint and plugins, simplify config, apply new rules (#4052)
This PR updates ESLint and the ESLint plugins to their latest versions
and takes advantage of the new versions to simplify the config.

The main cleanup: removed all explicit `plugins: {}` registrations from
`eslint.config.mjs`. When passing direct config objects like
`js.configs.recommended`, the plugin registration is already included –
we were just doing it twice.

Two lint warnings are also fixed:
- A wrong import style for `eslint-plugin-package-json` (named vs.
default)
- `playwright/no-duplicate-hooks` is disabled for e2e tests – the rule
doesn't handle plain `beforeAll()`/`afterAll()` (Vitest style) correctly
and produces false positives. I've created an issue for that:
https://github.com/mskelton/eslint-plugin-playwright/issues/443.

Built-in Node.js imports were manually updated to use the `node:` prefix
(e.g. `require("fs")` → `require("node:fs")`). Minor formatting fixes
were applied automatically by `eslint --fix`.
2026-03-07 08:34:28 -07:00
Kristjan ESPERANTO
e7503a457b refactor: further logger clean-up (#4050)
After #4049 here are two small follow-up improvements to `js/logger.js`.

**1. Simpler bind syntax** —
`Function.prototype.bind.call(console.debug, console)` is an archaic
pattern. The equivalent `console.debug.bind(console)` works fine in all
supported engines (Node.js ≥ 22, modern browsers) and is much easier to
read. Also: `console.timeStamp` exists in all supported environments, so
the conditional fallback to an empty function is no longer needed.

**2. Simpler `setLogLevel`** — instead of iterating over all keys in the
logger object and permanently overwriting them, the method now loops
over the five log-level keys explicitly and rebinds from `console[key]`.
This makes the filtered set obvious at a glance and ensures utility
methods like `group`, `time`, and `timeStamp` are never accidentally
silenced — they're structural helpers, not log levels.
2026-03-06 18:56:16 +01:00
Kristjan ESPERANTO
3eb3745dd3 Fix Node.js v25 logging prefix and modernize logger (#4049)
On Node.js v25, the log prefix in the terminal stopped working - instead
of seeing something like:

```
[2026-03-05 23:00:00.000] [LOG]   [app] Starting MagicMirror: v2.35.0
```

the output was:

```
[2026-03-05 23:00:00.000] :pre() Starting MagicMirror: v2.35.0
```

Reported in #4048.

## Why did it break?

The logger used the `console-stamp` package to format log output. One
part of that formatting used `styleText("grey", ...)` to color the
caller prefix gray. Node.js v25 dropped `"grey"` as a valid color name
(only `"gray"` with an "a" is accepted now). This caused `styleText` to
throw an error internally - and `console-stamp` silently swallowed that
error and fell back to returning its raw `:pre()` format string as the
prefix. Not ideal.

## What's in this PR?

**1. The actual fix** - `"grey"` → `"gray"`.

**2. Cleaner stack trace approach** - the previous code set
`Error.prepareStackTrace` *after* creating the `Error`, which is fragile
and was starting to behave differently across Node versions. Replaced
with straightforward string parsing of `new Error().stack`.

**3. Removed the `console-stamp` dependency** - all formatting is now
done with plain Node.js built-ins (`node:util` `styleText`). Same visual
result, no external dependency.

**4. Simplified the module wrapper** - the logger was wrapped in a UMD
pattern, which is meant for environments like AMD/RequireJS. MagicMirror
only runs in two places: Node.js and the browser. Replaced with a simple
check (`typeof module !== "undefined"`), which is much easier to follow.
2026-03-06 13:10:59 +01:00
Kristjan ESPERANTO
729f7f0fd1 [core] refactor: enable ESLint rule require-await and handle detected issues (#4038)
Enable the `require-await` ESLint rule. Async functions without `await`
are just regular functions with extra overhead — marking them `async`
adds implicit Promise wrapping, can hide missing `return` statements,
and misleads readers into expecting asynchronous behavior where there is
none.

While fixing the violations, I removed unnecessary `async` keywords from
source files and from various test callbacks that never used `await`.
2026-02-25 10:55:56 +01:00
Kristjan ESPERANTO
8ce0cda7bf [weather] refactor: migrate to server-side providers with centralized HTTPFetcher (#4032)
This migrates the Weather module from client-side fetching to use the
server-side centralized HTTPFetcher (introduced in #4016), following the
same pattern as the Calendar and Newsfeed modules.

## Motivation

This brings consistent error handling and better maintainability and
completes the refactoring effort to centralize HTTP error handling
across all default modules.

Migrating to server-side providers with HTTPFetcher brings:
- **Centralized error handling**: Inherits smart retry strategies
(401/403, 429, 5xx backoff) and timeout handling (30s)
- **Consistency**: Same architecture as Calendar and Newsfeed modules
- **Security**: Possibility to hide API keys/secrets from client-side
- **Performance**: Reduced API calls in multi-client setups - one server
fetch instead of one per client
- **Enabling possible future features**: e.g. server-side caching, rate
limit monitoring, and data sharing with third-party modules

## Changes

- All 10 weather providers now use HTTPFetcher for server-side fetching
- Consistent error handling like Calendar and Newsfeed modules

## Breaking Changes

None. Existing configurations continue to work.

## Testing

To ensure proper functionality, I obtained API keys and credentials for
all providers that require them. I configured all 10 providers in a
carousel setup and tested each one individually. Screenshots for each
provider are attached below demonstrating their working state.

I even requested developer access from the Tempest/WeatherFlow team to
properly test this provider.

**Comprehensive test coverage**: A major advantage of the server-side
architecture is the ability to thoroughly test providers with unit tests
using real API response snapshots. Don't be alarmed by the many lines
added in this PR - they are primarily test files and real-data mocks
that ensure provider reliability.

## Review Notes

I know this is an enormous change - I've been working on this for quite
some time. Unfortunately, breaking it into smaller incremental PRs
wasn't feasible due to the interdependencies between providers and the
shared architecture.

Given the scope, it's nearly impossible to manually review every change.
To ensure quality, I've used both CodeRabbit and GitHub Copilot to
review the code multiple times in my fork, and both provided extensive
and valuable feedback. Most importantly, my test setup with all 10
providers working successfully is very encouraging.

## Related

Part of the HTTPFetcher migration #4016.

## Screenshots

<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-06-54"
src="https://github.com/user-attachments/assets/2139f4d2-2a9b-4e49-8d0a-e4436983ed6e"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-02"
src="https://github.com/user-attachments/assets/880f7ce2-4e44-42d5-bfe4-5ce475cca7c2"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-07"
src="https://github.com/user-attachments/assets/abd89933-fe03-40ab-8a7c-41ae1ff99255"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-12"
src="https://github.com/user-attachments/assets/22225852-f0a9-4d33-87ab-0733ba30fad3"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-17"
src="https://github.com/user-attachments/assets/7a7192a5-f237-4060-85d7-6f50b9bef5af"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-22"
src="https://github.com/user-attachments/assets/df84d9f1-e531-4995-8da8-d6f2601b6a08"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-27"
src="https://github.com/user-attachments/assets/4cf391ac-db43-4b52-95f4-f5eadc5ea34d"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-32"
src="https://github.com/user-attachments/assets/8dd8e688-d47f-4815-87f6-7f2630f15d58"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-37"
src="https://github.com/user-attachments/assets/ee84a8bc-6b35-405a-b311-88658d9268dd"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-42"
src="https://github.com/user-attachments/assets/f941f341-453f-4d4d-a8d9-6b9158eb2681"
/>

Provider "Weather API" added later:

<img width="1910" height="1080" alt="Ekrankopio de 2026-02-15 19-39-06"
src="https://github.com/user-attachments/assets/3f0c8ba3-105c-4f90-8b2e-3a1be543d3d2"
/>
2026-02-23 10:27:29 +01:00
Karsten Hassel
1dc3032171 allow environment variables in cors urls (#4033)
and centralize and optimize replace regex.

Another follow up to #4029 

With this PR you can use secrets in urls in browser modules if you use
the cors proxy.
2026-02-08 16:18:56 +01:00
Karsten Hassel
172ca18178 fix cors proxy getting binary data (e.g. png, webp) (#4030)
fixes #3266

---------

Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
2026-02-08 12:02:50 +01:00
Karsten Hassel
b9481d27fa fix: correct secret redaction and optimize loadConfig (#4031)
- fix copy/paste typo in redacted replacement
- create redacted content only if hideConfigSecrets is true

follow up for #4029
2026-02-08 00:26:40 +01:00
Karsten Hassel
9dd964e004 change loading config.js, allow variables in config.js and try to protect sensitive data (#4029)
## Loading `config.js`

### Previously

Loaded on server-side in `app.js` and in the browser by including
`config.js` in `index.html`. The web server has an endpoint `/config`
providing the content of server loaded `config.js`.

### Now

Loaded only on server-side in `app.js`. The browser loads the content
using the web server endpoint `/config`. So the server has control what
to provide to the clients.

Loading the `config.js` was moved to `Utils.js` so that
`check_config.js` can use the same functions.

## Using environment variables in `config.js`

### Previously

Environment variables were not allowed in `config.js`. The workaround
was to create a `config.js.template` with curly braced bash variables
allowed. While starting the app the `config.js.template` was converted
via `envsub` into a `config.js`.

### Now

Curly braced bash variables are allowed in `config.js`. Because only the
server loads `config.js` he can substitute the variables while loading.

## Secrets in MagicMirror²

To be honest, this is a mess.

### Previously

All content defined in the `config` directory was reachable from the
browser. Everyone with access to the site could see all stuff defined in
the configuration e.g. using the url http://ip:8080/config. This
included api keys and other secrets.

So sharing a MagicMirror² url to others or running MagicMirror² without
authentication as public website was not possible.

### Now

With this PR we add (beta) functionality to protect sensitive data. This
is only possible for modules running with a `node_helper`. For modules
running in the browser only (e.g. default `weather` module), there is no
way to hide data (per construction). This does not mean, that every
module with `node_helper` is safe, e.g. the default `calendar` module is
not safe because it uses the calendar url's as sort of id and sends them
to the client.

For adding more security you have to set `hideConfigSecrets: true` in
`config.js`. With this:
- `config/config.env` is not deliverd to the browser
- the contents of environment variables beginning with `SECRET_` are not
published to the clients

This is a first step to protect sensitive data and you can at least
protect some secrets.
2026-02-06 00:21:35 +01:00
Karsten Hassel
f6d559e3dc remove kioskmode (#4027)
Marked as deprecated since 2016.
2026-02-06 00:09:59 +01:00
Kristjan ESPERANTO
5c1cc476f3 [newsfeed] refactor: migrate to centralized HTTPFetcher (#4023)
This migrates the Newsfeed module to use the centralized HTTPFetcher
class (introduced in #4016), following the same pattern as the Calendar
module.

This continues the refactoring effort to centralize HTTP error handling
across all modules.

## Changes

**NewsfeedFetcher:**
- Refactored from function constructor to ES6 class (like the calendar
module in #3959)
- Replaced manual fetch() + timer handling with HTTPFetcher composition
- Uses structured error objects with translation keys
- Inherits smart retry strategies (401/403, 429, 5xx backoff)
- Inherits timeout handling (30s) and AbortController

**node_helper.js:**
- Updated error handler to use `errorInfo.translationKey`
- Simplified property access (`fetcher.url`, `fetcher.items`)

**Cleanup:**
- Removed `js/module_functions.js` (`scheduleTimer` no longer needed)
- Removed `#module_functions` import from package.json

## Related

Part of the HTTPFetcher migration effort started in #4016.
Next candidate: Weather module (client-side → server-side migration).
2026-01-29 19:41:59 +01:00
Karsten Hassel
6324ec2116 move custom.css from css to config (#4020)
This is another change to cleanup structure, already mentioned in
https://github.com/MagicMirrorOrg/MagicMirror/pull/4019#issuecomment-3792953018

After separating default and 3rd-party modules this PR moves the
`custom.css` from the mm-owned directory `css` into user owned directory
`config`.

It has a built-in function which moves the `css/custom.css` to the new
location `config/custom.css` (if the target not exists).

Let me know if there's a majority in favor of this change.
2026-01-28 10:50:25 +01:00
Karsten Hassel
d44db6ea10 move default modules from /modules/default to /defaultmodules (#4019)
Since the project's inception, I've missed a clear separation between
default and third-party modules.

This increases complexity within the project (exclude `modules`, but not
`modules/default`), but the mixed use is particularly problematic in
Docker setups.

Therefore, with this pull request, I'm moving the default modules to a
different directory.

~~I've chosen `default/modules`, but I'm not bothered about it;
`defaultmodules` or something similar would work just as well.~~

Changed to `defaultmodules`.

Let me know if there's a majority in favor of this change.
2026-01-27 08:37:52 +01:00
Kristjan ESPERANTO
34913bfb9f [core] refactor: extract and centralize HTTP fetcher (#4016)
## Summary

PR [#3976](https://github.com/MagicMirrorOrg/MagicMirror/pull/3976)
introduced smart HTTP error handling for the Calendar module. This PR
extracts that HTTP logic into a central `HTTPFetcher` class.

Calendar is the first module to use it. Follow-up PRs would migrate
Newsfeed and maybe even Weather.

**Before this change:**

-  Each module had to implemented its own `fetch()` calls
-  No centralized retry logic or backoff strategies
-  No timeout handling for hanging requests
-  Error detection relied on fragile string parsing

**What this PR adds:**

-  Unified HTTPFetcher class with intelligent retry strategies
-  Modern AbortController with configurable timeout (default 30s)
-  Proper undici Agent for self-signed certificates
-  Structured error objects with translation keys
-  Calendar module migrated as first consumer
-  Comprehensive unit tests with msw (Mock Service Worker)

## Architecture

**Before - Decentralized HTTP handling:**

```
Calendar Module          Newsfeed Module         Weather Module
┌─────────────┐         ┌─────────────┐         ┌─────────────┐
│ fetch() own │         │ fetch() own │         │ fetch() own │
│ retry logic │         │ basic error │         │ no retry    │
│ error parse │         │   handling  │         │ client-side │
└─────────────┘         └─────────────┘         └─────────────┘
      │                       │                       │
      └───────────────────────┴───────────────────────┘
                              ▼
                        External APIs
```

**After - Centralized with HTTPFetcher:**

```
┌─────────────────────────────────────────────────────┐
│                  HTTPFetcher                        │
│  • Unified retry strategies (401/403, 429, 5xx)     │
│  • AbortController timeout (30s)                    │
│  • Structured errors with translation keys          │
│  • undici Agent for self-signed certs               │
└────────────┬──────────────┬──────────────┬──────────┘
             │              │              │
     ┌───────▼───────┐ ┌────▼─────┐ ┌──────▼──────┐
     │   Calendar    │ │ Newsfeed │ │   Weather   │
     │    This PR  │ │  future  │ │   future    │
     └───────────────┘ └──────────┘ └─────────────┘
             │              │              │
             └──────────────┴──────────────┘
                          ▼
                   External APIs
```
## Complexity Considerations

**Does HTTPFetcher add complexity?**

Even if it may look more complex, it actually **reduces overall
complexity**:

- **Calendar already has this logic** (PR #3976) - we're extracting, not
adding
- **Alternative is worse:** Each module implementing own logic = 3× the
code
- **Better testability:** 443 lines of tests once vs. duplicating tests
for each module
- **Standards-based:** Retry-After is RFC 7231, not custom logic

## Future Benefits

**Weather migration (future PR):**

Moving Weather from client-side to server-side will enable:
- **Same robust error handling** - Weather gets 429 rate-limiting, 5xx
backoff for free
- **Simpler architecture** - No proxy layer needed

Moving the weather modules from client-side to server-side will be a big
undertaking, but I think it's a good strategy. Even if we only move the
calendar and newsfeed to the new HTTP fetcher and leave the weather as
it is, this PR still makes sense, I think.

## Breaking Changes

**None**

----

I am eager to hear your opinion on this 🙂
2026-01-22 19:24:37 +01:00
Karsten Hassel
82e39a2476 fix systeminformation not displaying electron version (#4012)
Bug was introduced with #4002

Because the sysinfo process runs as own subprocess the
`${process.versions.electron}` variable is always `undefined`.
2026-01-11 23:17:01 +01:00
Kristjan ESPERANTO
9731ea28eb refactor: unify favicon for index.html and Electron (#4006)
In #3407 we already talked about unifying them.

- Create SVG favicon (better then png)
- Replace base64 placeholder in index.html with SVG favicon
- Update electron.js to use SVG favicon instead of mm2.png
- Add favicon.svg to server static routes
- Remove mm2.png
2026-01-05 10:51:43 +01:00
Karsten Hassel
241921b79c [core] run systeminformation in subprocess so the info is always displayed (#4002)
If an error occurs during startup, we request system information from
the user. The problem is that this information is displayed too late,
for example, if the configuration check fails.

My initial idea was to use `await
Utils.logSystemInformation(global.version);`, but this increased the
startup time.

Therefore, the function is now called in a subprocess. This approach
provides the information in all cases and does not increase the startup
time.
2026-01-03 01:14:48 +01:00
Kristjan ESPERANTO
9d3b07db12 [core] fix: allow browser globals in config files (#3992)
The config checker previously only allowed Node.js globals, but since
the config file runs also in the browser context, users should be able
to access browser APIs like `document` or `window` when needed.

This was incorrectly flagged as an error by the `no-undef` ESLint rule.
The fix adds browser globals to the allowed globals in the linter
config.

Fixes #3990.
2025-12-21 12:44:03 +01:00
Karsten Hassel
4186cbf0b2 [core] auto create release notes with every push on develop (#3985)
and remove CHANGELOG.md logic.

This is my attempt to create a draft release instead of editing a
changelog, see discussion on discord.

Logic:
- new github workflow `.github/workflows/release-notes.yaml`
- runs with every push on `develop` (so after PR's are merged)
- collects the commits on `develop` which are newer than the latest tag
- searches the commit messages for keywords defined in an array and
group the messages into categories (this is a first shot, we will update
this ...)
- creates markdown content
- looks for an untagged and unpublished draft release with name
`unreleased`, if it exists, it will be deleted
- creates an untagged and unpublished draft release with name
`unreleased` with markdown content created above

Example created on my fork (this caused having `MagicMirrorOrg` in the
PR-Links):

<img width="952" height="1804" alt="grafik"
src="https://github.com/user-attachments/assets/38687bed-f5da-4dcb-93eb-242c317769df"
/>

Please review this PR, it is a draft release at the moment because I got
problems in my fork where I tested this: The created draft release is
not visible at the moment (they are visible via api). AFAIS this is a
queue problem on GitHub, maybe I flooded their queue while testing ...
So I will test this tomorrow again before removing `draft` here.
2025-12-10 11:56:31 -06:00
Kristjan ESPERANTO
3c4d69ea84 [calendar] refactor: migrate CalendarFetcher to ES6 class and improve error handling (#3959)
1. Convert CalendarFetcher from legacy constructor function pattern to
ES6 class (which simplifies future migration from CommonJS to ES
modules).
2. Implement targeted HTTP error handling with smart retry strategies
for common calendar feed issues:
   - 401/403: Extended retry delay (5× interval, min 30 min)
   - 429: Retry-After header parsing with 15 min fallback
   - 5xx: Exponential backoff (2^count, max 3 retries)
   - 4xx: Extended retry (2× interval, min 15 min)
   - Add serverErrorCount tracking for exponential backoff
- Error messages now include specific HTTP status codes and calculated
retry delays for better debugging and user feedback

Previously, CalendarFetcher did not respond appropriately to HTTP
errors, continuing to hammer endpoints without backoff, potentially
overloading servers and triggering rate limits. This refactoring
implements respectful retry strategies that adapt to server responses
and reduce unnecessary load.

Maybe we could later centralize the HTTP error handling and use it for
weather and newsfeed as well.

The PR was inspired by having worked on the calendar fetcher for
MMM-CalendarExt2, where there was already better error handling.
2025-11-14 20:14:23 +01:00
Jordan Welch
53df20f313 [weatherprovider] update subclass language use override (#3914) 2025-11-13 22:08:47 +01:00
Kristjan ESPERANTO
f29f424a62 [core] refactor: replace XMLHttpRequest with fetch and migrate e2e tests to Playwright (#3950)
### 1. Replace `XMLHttpRequest` with the modern `fetch` API for loading
translation files

#### Changes
- **translator.js**: Use `fetch` with `async/await` instead of XHR
callbacks
- **loader.js**: Align URL handling and add error handling (follow-up to
fetch migration)
- **Tests**: Update infrastructure for `fetch` compatibility

#### Benefits
- Modern standard API
- Cleaner, more readable code
- Better error handling and fallback mechanisms

### 2. Migrate e2e tests to Playwright

This wasn't originally planned for this PR, but is related. While
investigating suspicious log entries which surfaced after the fetch
migration I kept running into JSDOM’s limitations. That pushed me to
migrate the E2E suite to Playwright instead.

#### Changes
- switch e2e harness to Playwright (`tests/e2e/helpers/global-setup.js`)
- rewrite specs to use Playwright locators + shared `expectTextContent`
- install Chromium via `npx playwright install --with-deps` in CI

#### Benefits
- much closer to real browser behaviour
- and no more fighting JSDOM’s quirks
2025-11-08 21:59:05 +01:00
sam detweiler
c1aaea5913 [weather] add error handling to weather fetch functions, including cors (#3791)
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
fixes #3687
2025-11-08 14:21:31 +01:00
Kristjan ESPERANTO
d7348ed765 [tests] suppress debug logs in CI environment + improve calendar symbol test stability (#3941)
## CI Log Suppression

**Two-level approach for clean test output:**

1. **Suppress debug/info logs**: Call `logger.setLogLevel("ERROR")` in
CI to hide verbose logging
2. **Suppress intentional error logs**: Set `mmTestMode` flag and check
it before logging errors that are part of test assertions (e.g., testing
error handling in `git_helper.js` and `server_functions.js`)

This keeps CI output clean and makes genuine failures immediately
visible, while preserving full logging for local development.

**Before:** 1348 log lines with verbose debug/info output  
**After:** 168 log clean lines with only test results

## Calendar Symbol Test Stability

Convert the calendar symbol test from external URL (`calendarlabs.com`)
to existing local mock file (`12_events.ics`). This eliminates CI
timeouts caused by external dependencies and improves test reliability.

The test still validates the same symbol array feature but now runs
faster and deterministically without network dependencies.
2025-11-03 23:49:21 +01:00
Kristjan ESPERANTO
462abf7027 [tests] migrate from jest to vitest (#3940)
This is a big change, but I think it's a good move, as `vitest` is much
more modern than `jest`.

I'm excited about the UI watch feature (run `npm run test:ui`), for
example - it's really helpful and saves time when debugging tests. I had
to adjust a few tests because they had time related issues, but
basically we are now testing the same things - even a bit better and
less flaky (I hope).

What do you think?
2025-11-03 19:47:01 +01:00
Jboucly
961b3c96d6 feat(core): add server:watch script with automatic restart on file changes (#3920)
## Description

This PR adds a new `server:watch` script that runs MagicMirror² in
server-only mode with automatic restart and browser reload capabilities.

Particularly helpful for:
- **Developers** who need to see changes immediately without manual
restarts.
- **Users setting up their mirror** who make many changes to `config.js`
or `custom.css` and need quick feedback.

### What it does

When you run `npm run server:watch`, the watcher monitors files you
specify in `config.watchTargets`. Whenever a monitored file changes:

1. The server automatically restarts
2. Waits for the port to become available
3. Sends a reload notification to all connected browsers via Socket.io
4. Browsers automatically refresh to show the changes

This creates a seamless development experience where you can edit code,
save, and see the results within seconds.

### Implementation highlights

**Zero dependencies:** Uses only Node.js built-ins (`fs.watch`,
`child_process.spawn`, `net`, `http`) - no nodemon or external watchers
needed.

**Smart file watching:** Monitors parent directories instead of files
directly to handle atomic writes from modern editors (VSCode, etc.) that
create temporary files during save operations.

**Port management:** Waits for the old server instance to fully release
the port before starting a new one, preventing "port already in use"
errors.

### Configuration

Users explicitly define which files to monitor in their `config.js`:

```js
let config = {
  watchTargets: [
    "config/config.js",
    "css/custom.css",
    "modules/MMM-MyModule/MMM-MyModule.js",
    "modules/MMM-MyModule/node_helper.js"
  ],
  // ... rest of config
};
```

This explicit approach keeps the implementation simple (~260 lines)
while giving users full control over what triggers restarts. If
`watchTargets` is empty or undefined, the watcher starts but monitors
nothing, logging a clear warning message.

---

**Note:** This PR description has been updated to reflect the final
implementation. During the review process, we refined the approach
multiple times based on feedback.

---------

Co-authored-by: Jboucly <contact@jboucly.fr>
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
2025-10-28 19:14:51 +01:00
Kristjan ESPERANTO
9ad5618843 [check_config] refactor: improve error handling (#3927)
- Combine file existence and permission checks with better error
messages
- Replace thrown exceptions with clean error output (no stack traces)
- Support custom module positions by changing strict validation to
warnings
- Add missing process.exit(1) after validation errors

Users now see clear, actionable error messages without stack traces, and
custom region configurations work correctly.

## example before

```bash
$ npm run start

> magicmirror@2.34.0-develop start
> node --run start:x11

[2025-10-22 17:56:06.303] [LOG]   Starting MagicMirror: v2.34.0-develop 
[2025-10-22 17:56:06.304] [LOG]   Loading config ... 
[2025-10-22 17:56:06.304] [LOG]   config template file not exists, no envsubst 
[2025-10-22 17:56:06.356] [ERROR] File not found: /home/kristjan/MagicMirror/config/config.js
No config file present! 
[2025-10-22 17:56:06.356] [ERROR] [checkconfig] Error: Error: ENOENT: no such file or directory, access '/home/kristjan/MagicMirror/config/config.js'
No permission to access config file!
    at checkConfigFile (/home/kristjan/MagicMirror/js/check_config.js:43:9)
    at Object.<anonymous> (/home/kristjan/MagicMirror/js/check_config.js:138:2)
    at Module._compile (node:internal/modules/cjs/loader:1714:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1848:10)
    at Module.load (node:internal/modules/cjs/loader:1448:32)
    at Module._load (node:internal/modules/cjs/loader:1270:12)
    at c._load (node:electron/js2c/node_init:2:17993)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:244:24)
    at Module.require (node:internal/modules/cjs/loader:1470:12)
    at require (node:internal/modules/helpers:147:16)
    at loadConfig (/home/kristjan/MagicMirror/js/app.js:126:3)
    at App.start (/home/kristjan/MagicMirror/js/app.js:291:18)
    at Object.<anonymous> (/home/kristjan/MagicMirror/js/electron.js:228:7)
    at Module._compile (node:internal/modules/cjs/loader:1714:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1848:10) 
```

## example after

```bash
$ npm run start

> magicmirror@2.34.0-develop start
> node --run start:x11

[2025-10-22 21:33:27.930] [LOG]   Starting MagicMirror: v2.34.0-develop 
[2025-10-22 21:33:27.931] [LOG]   Loading config ... 
[2025-10-22 21:33:27.931] [LOG]   config template file not exists, no envsubst 
[2025-10-22 21:33:27.985] [ERROR] [check_config] File not found: /home/kristjan/MagicMirror/config/config.js 
```
2025-10-23 22:48:16 +02:00
Karsten Hassel
bc0d36503a logger: add calling filename as prefix on server side (#3926) 2025-10-22 22:50:31 +02:00
Veeck
a1c1e9560c [logger] Add prefixes to most Log messages (#3923)
Co-authored-by: veeck <gitkraken@veeck.de>
2025-10-21 20:32:48 +02:00
Kristjan ESPERANTO
37d1a3ae8f refactor: replace express-ipfilter with lightweight custom middleware (#3917)
This fixes security issue
[CVE-2023-42282](https://github.com/advisories/GHSA-78xj-cgh5-2h22),
which is not very likely to be exploitable in MagicMirror² setups, but
still should be fixed.

The [express-ipfilter](https://www.npmjs.com/package/express-ipfilter)
package depends on the obviously unmaintained
[ip](https://github.com/indutny/node-ip) package, which has known
security vulnerabilities. Since no fix is available, this commit
replaces both dependencies with a custom middleware using the better
maintained [ipaddr.js](https://www.npmjs.com/package/ipaddr.js) library.

Changes:
- Add new `js/ip_access_control.js` with lightweight middleware
- Remove `express-ipfilter` dependency, add `ipaddr.js`
- Update `js/server.js` to use new middleware
- In addition, I have formulated the descriptions of the corresponding
tests a little more clearly.
2025-10-18 19:56:55 +02:00
Karsten Hassel
d39e686f7a remove eslint warnings, add npm publish process to Collaboration.md (#3913) 2025-10-14 22:44:37 +02:00
Kristjan ESPERANTO
5f1f5bd291 feat: add ESlint rule no-sparse-arrays for config check (#3911)
Adding a rule to the config checker config so that unexpected commas in
the middle of arrays (reported in issue #3910) are better detected.

Two commas in a row inside the modules array create an empty entry
(`undefined`). JavaScript accepts that syntax, but MagicMirror would
later try to load that “module” and fail.

Alternatively, we could filter out undefined entries, but with this PR,
the user receives a clear message indicating where the error lies, can
easily fix it, and thus has a cleaner configuration.

## Before

```
[2025-10-10 19:33:30.874] [INFO]  Checking config file /home/kristjan/MagicMirror/config/config.js ... 
[2025-10-10 19:33:30.944] [INFO]  Your configuration file doesn't contain syntax errors :) 
[2025-10-10 19:33:30.945] [INFO]  Checking modules structure configuration ... 
[2025-10-10 19:33:31.027] [ERROR] This module configuration contains errors:
undefinedmust be object
```

## After

```
[2025-10-10 19:41:20.030] [INFO]  Checking config file /home/kristjan/MagicMirror/config/config.js ... 
[2025-10-10 19:41:20.107] [ERROR] Your configuration file contains syntax errors :(
Line 91 column 1: Unexpected comma in middle of array.
```
2025-10-13 23:40:23 +02:00
Kristjan ESPERANTO
787cc6bd1f refactor: replace module-alias dependency with internal alias resolver (#3893)
- removes the external unmaintained `module-alias` dependency ->
reducing complexity and risk
- introduces a small internal alias mechanism for `logger` and
`node_helper`
- preserves backward compatibility for existing 3rd‑party modules
- should simplify a future ESM migration of MagicMirror

I'm confident that it shouldn't cause any problems, but we could also
consider including it in the release after next. What do you think?

This PR is inspired by PR #2934 - so thanks to @thesebas! 🙇 😃
2025-09-30 20:12:58 +02:00
Karsten Hassel
1f2d1b92b5 update jsdoc and other deps (#3896)
other cosmetic code changes because of new `eslint-plugin-jsdoc` version
v60
2025-09-23 06:27:29 +02:00
Kristjan ESPERANTO
fb2aa438d8 feat: add clear log for occupied port at startup (#3890)
Having repeatedly seen that users are unaware of the meaning of the
EADDRINUSE error message (see, for example, this [forum
thread](https://forum.magicmirror.builders/topic/19871/update-package-list/5)),
I thought we should intercept this message and provide clearer output.
This may help users identify the cause of the problem more quickly
themselves.

## before

```
[2025-09-13 09:54:32.903] [LOG]   Starting MagicMirror: v2.33.0-develop 
...
[2025-09-13 09:54:33.533] [LOG]   Starting server on port 8080 ...  
[2025-09-13 09:54:33.537] [WARN]  You're using a full whitelist configuration to allow for all IPs 
[2025-09-13 09:54:33.568] [ERROR] Whoops! There was an uncaught exception... 
[2025-09-13 09:54:33.574] [ERROR] Error: listen EADDRINUSE: address already in use 0.0.0.0:8080
    at Server.setupListenHandle [as _listen2] (node:net:1940:16)
    at listenInCluster (node:net:1997:12)
    at node:net:2206:7
    at process.processTicksAndRejections (node:internal/process/task_queues:90:21) {
  code: 'EADDRINUSE',
  errno: -98,
  syscall: 'listen',
  address: '0.0.0.0',
  port: 8080
} 
[2025-09-13 09:54:33.574] [ERROR] MagicMirror² will not quit, but it might be a good idea to check why this happened. Maybe no internet connection? 
[2025-09-13 09:54:33.574] [ERROR] If you think this really is an issue, please open an issue on GitHub: https://github.com/MagicMirrorOrg/MagicMirror/issues 
[2025-09-13 09:54:35.235] [INFO]  
####  System Information  ####
...
```

## after

```
[2025-09-13 09:53:20.151] [LOG]   Starting MagicMirror: v2.33.0-develop 
...
[2025-09-13 09:53:20.928] [LOG]   Starting server on port 8080 ...  
[2025-09-13 09:53:20.931] [WARN]  You're using a full whitelist configuration to allow for all IPs 
[2025-09-13 09:53:20.970] [ERROR] 
────────────────────────────────────────────────────────────────
 PORT IN USE: 0.0.0.0:8080

 Another process (most likely another MagicMirror instance)
 is already using this port.

 Stop the other process (free the port) or use a different port.
──────────────────────────────────────────────────────────────── 
[2025-09-13 09:53:22.471] [INFO]  
####  System Information  ####
...
```
2025-09-13 13:01:55 +02:00
Karsten Hassel
2481bc621f revert changes breaking node --run config:check (#3885) 2025-09-10 07:55:05 +02:00
Karsten Hassel
b1865d8115 refactor: use global.root_path instead relative paths (#3883) 2025-09-09 08:09:45 +02:00
sam detweiler
eb719429d4 fix for #3380, socket.io timeout closure (#3862)
socket.io times out and closes the client side socket without any
callback
sendSocntNotification() from the server side data is lost as the socket
is closed. but the client doesn't know

increase the timeout 

fixes #3380
2025-08-28 18:02:21 +02:00
Marcel
76da0aa55e Make User-Agent configurable (#3255)
Fixes #3253 

Adds a configuration option to overwrite the default `User-Agent` header
that is send at least by the calendar and news module. Allows other
modules to use the individual user agent as well.

The configuration accepts either a string or a function:
```
var config =
	{
		...
		userAgent: 'Mozilla/5.0 (My User Agent)',
		...
	}
```
or
```
var config =
	{
		...
		userAgent: () => 'Mozilla/5.0 (My User Agent)',
		...
	}
```

---------

Co-authored-by: Veeck <github@veeck.de>
Co-authored-by: veeck <gitkraken@veeck.de>
Co-authored-by: Karsten Hassel <hassel@gmx.de>
Co-authored-by: Kristjan ESPERANTO <35647502+KristjanESPERANTO@users.noreply.github.com>
2025-08-27 13:50:37 +02:00
Kristjan ESPERANTO
931fe55022 refactor: optimize system information logging (#3843)
Additionally to #3839 did some rework on the system logging.

- feat: include MagicMirror version (like Sam suggested in #3839)
- refactor: use more variables to get the string array less complex
- refactor: get `installedNodeVersion` from si.versions (with that it
was possible to drop the import of `execSync`)
- fix: `used node` was always the same as the installed one. Since
Electron comes with its own node version, this can differ. This is now
shown correctly (again?) with the use of `process.version`.
- a bit formatting

I think these changes make the code easier to understand and therefore
easier to maintain. Except for showing the MM version there is no big
difference for the user.

## before

```bash
#####  System Information  #####
- SYSTEM:    manufacturer: Notebook; model: N650DU; virtual: false; timeZone: Europe/Berlin
- OS:        platform: linux; distro: Debian GNU/Linux; release: 12; arch: x64; kernel: 5.10.0-20-amd64
- VERSIONS:  electron: 36.3.2; used node: 22.15.0; installed node: 22.15.0; npm: 10.9.0; pm2: 6.0.6
- ENV:       XDG_SESSION_TYPE: wayland; MM_CONFIG_FILE: config/config_MMM-PublicTransportHafas.js;
             WAYLAND_DISPLAY:  wayland-0; DISPLAY: :0; ELECTRON_ENABLE_GPU: undefined
- RAM:       total: 15925.45 MB; free: 2716.90 MB; used: 13209.04 MB
- UPTIME:    259 minutes 
```

## after

```bash
####  System Information  ####
- SYSTEM:   manufacturer: Notebook; model: N650DU; virtual: false; MM: 2.33.0-develop
- OS:       platform: linux; distro: Debian GNU/Linux; release: 12; arch: x64; kernel: 5.10.0-20-amd64
- VERSIONS: electron: 36.3.2; used node: 22.15.1; installed node: 22.15.0; npm: 10.9.0; pm2: 6.0.6
- ENV:      XDG_SESSION_TYPE: wayland; MM_CONFIG_FILE: config/config_MMM-PublicTransportHafas.js
            WAYLAND_DISPLAY:  wayland-0; DISPLAY: :0; ELECTRON_ENABLE_GPU: undefined
- RAM:      total: 15925.45 MB; free: 2814.49 MB; used: 13110.96 MB
- OTHERS:   uptime: 260 minutes; timeZone: Europe/Berlin 
```
2025-07-12 08:24:09 +02:00