Exports of a pattern library are hundreds of files, and a plain diff is useless: everything looks changed. node-diffraction compares two folders and tries to be smart about it — hashing files, beautifying the markup, clustering the changes and rendering them so a reviewer can see what actually moved.
It was built for exactly that: comparing pattern_exports dumps between two dates before a release.
| Fact | Detail |
|---|---|
| Input | Two folders of pattern-library exports |
| Technique | Hashing, beautify, clustering, word-level diffs |
| Stack | Node (diff, difflib, set-clustering, rainbow-code) |
| Started | November 2018 |
What's in it
Changes are grouped so the signal is visible before the details.
HTML-aware beautification before diffing — otherwise whitespace swamps the result.
In detail
What is actually inside it
Grouping similar files first is what turns "412 files changed" into "three components changed, and here is how".
diffFromBasePath = 'diffs/20181128-pattern_exports' / '…20181203-pattern_exports': the tool is pointed at two dated export folders, which is exactly how a release review worked.
HTML is beautified before comparison — without that, whitespace and attribute order swamp the signal and nobody reads the diff.
| Detail | Value |
|---|---|
| Built for | diffing pattern-library exports |
| Technique | hashing → beautify → clustering → word-level diff |
| Started | November 2018 |
Technical notes
The decisions behind it
Why not use a normal diff?
What are "pattern exports"?
*-pattern_exports folders. Comparing two of those was the pre-release check that nothing regressed visually or structurally.Inside the run
Seven steps, and the constants that decide what you see
diff.js is one file, and npm start is the whole entry point: node diff.js. Everything it knows about the world is two constants above the first function — in the last commit diffFromBasePath = 'diffs/20181128-pattern_exports' and diffToBasePath = 'diffs/20181203-pattern_exports'. The first commit compared 20181119 against 20181128; the second commit's real work was the report, and its configuration change was bumping those two strings to the next pair of exports.
Why a plain recursive diff was not an option: an export contains each pattern's rendered markup, including the markup of whatever it embeds, so one change to a shared element turns up verbatim in every export that includes it — the checked-in patch.html carries a single change record whose file list holds 32 different *.markup-only.html paths. A normal diff reports that as 32 separate changes, and the interesting question ("what actually changed, and how often?") never surfaces.
- Glob, not walk.
glob-fs, created with{ gitignore: false }, reads**/*.markup-only.htmlfrom both folders. Only themarkup-onlyvariant of a pattern is compared; the rest of the export never enters the run. - Classify by path membership. Files present on both sides become
unchangedFilePaths; everything else lands inaddedFilePaths/removedFilePaths. - Read everything up front. The union of both folders is read into one
Mapof path → string. No streaming, no caching — an export has to fit in memory. - Diff with five lines of context.
jsdiff.structuredPatch(from, to, a, b, '', '', { context: 5 }). The two empty strings suppress the file headers, because paths are rendered separately. - Hash the hunk.
sha256(hunk.lines.join())is the change identity: jsdiff's prefixed lines (' ','-','+') joined by an argument-lessjoin(), which joins with commas. Byte-identical hunks in different files collapse into one record with N file paths. Because the hash covers the context lines too, the same edit nested at a different depth counts as a different change — re-grouping those is what the next step is for. - Cluster at 0.6. Similarity is
new difflib.SequenceMatcher(null, a.hunk.lines.join(''), b.hunk.lines.join('')).ratio()— the concatenated hunk bodies scored by a 2012 port of Python'sdifflib.similarGroups(0.6)keeps groups whose members are at least 60 % alike. - Render two kinds of page.
diffractor/src/pages/index.htmlgets the group table (group id, similar changes, affected files, root-change breadcrumb, link);diffractor/src/pages/patches/patch-N.htmlgets the hunks, passed throughrainbow.colorSync(lines.join('\n'), 'html')into<pre class="change__diff">.
One thing worth knowing before reading the output: js-beautify (indent 2, indent_inner_html) runs on the generated dashboard markup, not on the files being compared. The markup handed to the patcher is exactly what the export wrote, so the export's own formatting sets the noise floor for the entire report.
What a run actually produced
Numbers from the two report snapshots committed in the repo
diffractor/src/pages/index.html (9.7 kB): 06-organisms/04-cards/_09-cards/corporate (2 changes, 3 files), 08-templates/01-artikel/03-pressemitteilung (1, 2), 05-molecules/quote/_quote (3, 11), 04-atoms/text/_blockquote (1, 1).patch.html snapshot (71.7 kB) lists the same 32 *.markup-only.html paths, spanning 08-templates-01-artikel-* and 09-pages-*. That repetition is the thing the tool exists to collapse. The snapshot's first title says 23 files while its own list holds 32 — it is a design mock, not regenerated output.diff.js is 231 lines and there is nothing else: no config file, no CLI flags, no test script. Pointing the tool at another pair of exports means editing two string constants and re-running it.Two projects, one report
What diffractor/ does that diff.js does not
The report is a static site, and that site is not home-grown: diffractor/ is the ZURB Foundation for Sites template — its package.json still says foundation-zurb-template, "Official ZURB Template for Foundation for Sites", MIT, author ZURB. It brings gulp 4, Panini for Handlebars, webpack, Sass and a BrowserSync server on port 8000, with its own dependency tree and its own 345 kB yarn.lock sitting next to the runner's 68 kB package-lock.json.
diff.js never builds anything. It writes finished HTML into diffractor/src/pages/ and diffractor/src/pages/patches/, and Panini's src/pages/**/*.{html,hbs,handlebars} glob wraps those files in src/layouts/default.html on the next gulp run. The layout is titled "Changes in component library export", sets the page background to #F7F7F7, and pairs the Rainbow theme monokai.css with a rainbow.min.js in src/assets/lib/js/ — even though the hunks were already highlighted server-side by rainbow.colorSync; the emitted <code> has none of the data-language or lang-* hooks the client script looks for.
The dashboard look is assembled from Foundation building blocks (dashboard-table, stats-list, accordion, breadcrumbs), with the report-specific additions in app.scss (.change__diff { margin-bottom: 0 }, the absolutely positioned .logo, breadcrumbs inside table cells, zebra rows). One Foundation default had to be switched off for the diffs to be readable at all: _settings.scss sets $code-color: inherit, $code-background: transparent and $code-border: 0, so the highlighted lines are not framed by the grey code box every line would otherwise sit in.
Two leftovers from the first commit are still visible in the tree. The report rows used to be written to src/partials/inference/change-table-rows.html, and each patch page ended in <script src="js/rainbow.min.js"> — a relative path that could only resolve for pages at the site root. The second commit dropped both, moved diffractor/src/pages/index.html and tmp/ into .gitignore, and left the two snapshots index.html and patch.html committed alongside the generated files. The Dockerfile (node:carbon, .nvmrc at lts/carbon, CMD [ "npm", "start" ]) containerises only the runner; the gulp build stays on the machine.
Rough edges
One TODO file, and what else was still guesswork
addedFilePaths and removedFilePaths are filled while classifying the files and then never read again: only the intersection is diffed and rendered, so a pattern that appears or disappears between two exports is invisible in the report. That also hides a matching bug in the classification itself — the unchanged check is toFilePaths.indexOf(...) > 0, so a match in the first array position is counted as removed. It costs nothing today, but only because removed files are never displayed.
getFilePathBreadcrumb takes the path out of the filename and applies -(\d+) → /$1, (\d+-\w+)- → $1/ and -_ → /_, which is what turns 06-organisms-04-cards-_09-cards-corporate… into the four-step breadcrumb in the report. TODO.md opens with this function and three worked examples of what it should produce instead — level names upper-cased, pattern names kept whole: 08-templates-03-corporate-01-cw-start-page → 08-TEMPLATES / 03-CORPORATE / 01-CW / START-PAGE, 05-molecules-modal-modal → 05-MOLECULES / MODAL-MODAL. It was the last piece of the pipeline that was still guesswork.
package.json has exactly one script (start) and one devDependency (eslint, never invoked); the config it would use declares ecmaVersion: 5 with airbnb-base/legacy, which cannot parse the async and arrow code in diff.js. The same file assigns filePathSegments without a declaration (an implicit global), and the de-duplication inside getUniqueFilepaths is a no-op because the .filter() result is never assigned — so an "Affected files" figure counts file slots, not distinct files. set-clustering is pulled from github:deestan/set-clustering#master rather than from the published npm releases (latest 1.1.0), so a fresh install depends on what master happens to be that day.
similarGroups returns connected components, not an average distance: if A≈B and B≈C both clear 60 %, A and C end up in the same group however little they share. Lowering the threshold does not simply merge more changes, it merges them by chaining. The cost is a full N×N table with the ratio function called once per pair and no memoisation — fine for a few hundred hunks, quadratic from there. TODO.md's second item is the same problem from the other side: an insert plus a delete that reorders code produces two nearly identical root-change entries that the threshold never combines.
Links
Where to find this one