Running many TYPO3 installations means the same recurring question: which extension version fits which TYPO3 version, and which of our projects would a bump break?
This service answers it by scraping the Satis Composer repository for extension metadata, combining it with project requirements and rendering the result as a matrix you can read at a glance.
| Fact | Detail |
|---|---|
| Question answered | Extension × TYPO3 version compatibility |
| Input | Satis repository metadata, semantic versions |
| Stack | Koa, cheerio, handlebars, node-sass |
| Started | June 2018 |
What's in it
Built to decide an upgrade order across projects, not to archive data.
Satis metadata plus real version constraints rather than a hand-kept table.
In detail
What is actually inside it
It answers "which extension version fits which TYPO3 version, and what would a bump cost us" — the matrix is a decision aid, not a report.
Satis exposes a web UI, not an API, so cheerio parses what the repository actually serves. Pragmatic, and the caching layer keeps it from hammering internal infrastructure.
Built to compare many installations across projects, which is the opposite of what an extension maintainer needs — it optimises for the fleet owner.
| Detail | Value |
|---|---|
| Answers | extension × TYPO3 compatibility |
| Stack | Koa, cheerio, semver, node-cache, hbs + list.js |
| Started | June 2018 |
Technical notes
The decisions behind it
Why Satis at all?
Why a web service instead of a CLI?
Reading the fleet
One HTML fetch, then seven sequential raw-file probes per installation
Everything on the matrix page is produced from a single HTML fetch plus a fan-out of raw-file fetches. There is no API anywhere in the chain.
The service issues one authenticated GET https://$SATIS_HOST/ and hands the body to cheerio. Every #package-list .card is then classified by one selector: if the card header contains a link mentioning BASE_DISTRIBUTION_NAME (default rm-dist) it is a distribution — one of our installations — otherwise it is an extension. The package name is read straight out of the card's id attribute, with a .replace('"', '') because Satis renders a stray quote there.
Extensions contribute two fields: the distributions they are Required by (the <dd> after dt:contains(Required by)), and their current version, defined as the first release link whose text passes semver-regex. That definition matters: dev-master, dev-develop and any non-semver tag are skipped, and correctness depends entirely on Satis listing releases newest-first.
For a distribution the payload is gitUrl (dt:contains(Releases) → a:contains(dev-master)). The Bitbucket project key is not parsed as a URL: it is sliced out of that SSH string by byte offset between ${BITBUCKET_PORT}/ and /${BASE_DISTRIBUTION_NAME}.git, with a hard-coded + 5 that silently assumes a four-digit port like 7999. That key is then used to build raw-file URLs, which go over HTTPS — the SSH port in the git URL is only ever parsed, never connected to.
Each distribution triggers seven Bitbucket fetches, executed sequentially with await (the distributions themselves are fanned out in parallel through Promise.all): composer.lock and composer.json at refs/heads/master; two Bamboo spec scripts, build/03-write-install-files.sh and deploy/backup-typo3.util.sh, detected by existence alone; .rm-frontendrc.js, whose existence means "new frontend integration"; and two Bitbucket browse pages, src/Patches and patches, where #browse-table tbody tr length > 1 is taken as "patches exist". All of it is cached per URL in node-cache; the assembled result is cached under the single key matrixHomepage for HOMEPAGE_CACHE_TTL seconds (600 by default, same as FETCH_CACHE_TTL).
The last derived value is the colour. semver-diff(usedVersion, satisCurrentVersion) becomes the class on each cell — diff--patch, diff--minor, diff--major, or literally diff--null when the two versions are equal, because that is what semver-diff returns for identical input. That one class is the whole colour language of the matrix: on row or column hover it paints the cell yellow, orange, red or green (the UI legend calls the green state "current").
Sharp edges
Failure modes and dead scaffolding found in the source
Does the master/develop switch actually read the develop branch?
'master', so develop is never fetched. The switch is fully scaffolded — the develop label, span.diff--develop rules and body.branchToggled selectors all exist in the SCSS — but no script ever adds branchToggled to <body>; public/javascript/main.js only binds hover handlers. The app's own "How it's done" copy, which promises data "for both develop and master branches", describes intent rather than shipped behaviour.Why can clearing one repository's cache leave stale cells on screen?
GET /cache/clear/:repository deletes a hand-written key list, and that list drifted from the fetch code. A January 2021 commit moved two Bamboo fetches from build-distribution/… to build/… and from distribution-deployment/util/… to deploy/…, but the clear route still deletes the old URLs — so 03-write-install-files.sh and backup-typo3.util.sh keep their cached values. The two patch-folder listing URLs are absent from the clear list entirely. Only GET /cache/clear (flushAll) is guaranteed to drop everything.What happens when a distribution repo has no composer.lock?
fetchUrl throws on a 404, the caller catches any error whose message contains 404 and substitutes a [false] sentinel — but that sentinel is then assigned to distribution.extensions.master.rm, and the very next line calls false.forEach(...), which throws a TypeError inside the request. The rejection propagates through Promise.all and there is no wrapping try/catch in the route, so the matrix returns a 500 instead of the intended X cell. Only the pure existence probes (the Bamboo scripts, .rm-frontendrc.js) handle absence gracefully, because they discard the body and resolve [true]. A 500, a 401 from bad BITBUCKET_PASSWORD, or a network timeout fails the same way: only the exact string 404 is swallowed.Are misses cached?
fetchUrl calls cache.set only after a body has been read successfully; a 404 throws before it gets there. Since several fields are existence probes that are supposed to 404 on older distributions, every render after the cache expires re-requests all of them. A cold render therefore costs one Satis fetch, seven sequential Bitbucket requests for the slowest distribution, and one Packagist fetch for the current TYPO3 core version.What is list.js doing in the stack?
package.json — and named in this project's own stack table — but the string appears nowhere in server.js, the Handlebars template or public/javascript/main.js: no new List(), no <script> tag for it. All highlighting is hand-written jQuery: main.js copies the diff--* class from the version <span> up to its <td>, then toggles active on the hovered row or column. list.js — together with the unused body.distributionToggled SCSS — is leftover scaffolding.The README as a code review
What the TODO list says about the shape of the service
The README's TODO list is really a description of the code's shape. A single server.js (~18 KB, last changed in January 2021) does the scraping, the composer.lock evaluation and the condensation into view data, and it passes results between stages as positional tuples.
filterComposerLockToRmPackages returns a seven-element array, consumed by the caller as composerLock[0] through composerLock[6]; most other extract* functions return a one-element array ([hasPatches], [newFrontendIntegration]) purely so they can hang off .then(). The two halves are coupled only by index, and that coupling — not the HTTP code — is what the TODO "more general units (satis info extraction, composer.lock evaluation, data condensation)" is asking to break.
There is no test directory. server.js and public/javascript/main.js are the only application scripts, and the only proof the chain works is the rendered matrix.
Split satis info extraction, composer.lock evaluation and data condensation — currently three responsibilities in one 18 KB file, wired by array indices.
Bootstrap 4, jQuery 3.2, Font Awesome 5 and Typekit all load from CDNs (with SRI hashes). The README's "no CDNs" item is about an internal tool that currently phones four external hosts to render.
extension, version, typo3Version and isDivider are registered inline in server.js, and version builds HTML by string concatenation into new hbs.SafeString(...).
21 commits since June 2018; the SCSS refactor and const-by-default are done, while the unused list.js dependency and the CDN loading are not. Dependabot bumps are the only later changes.
Links
Where to find this one