- Hyperclay
- Behind the project
- Secret Sauce
Secret Sauce
Technical architecture and design blueprint for the Hyperclay hosted platform, from the DOM-as-database save pipeline to routing, storage, and sync.
This is a technical blueprint for the Hyperclay hosted platform. Every architectural decision, every code pattern, and every trade-off is documented here. A developer could read this and rebuild the system from scratch.
The stack is intentionally boring: Node.js, Express, PostgreSQL with Sequelize, Edge.js templates, PM2 for process management, Stripe for payments, Postmark for email, and Server-Sent Events for real-time sync. The interesting part isn’t the technology: it’s how the pieces fit together.
The core innovation: HTML files as a database#
Traditional web applications shuttle data between a frontend, an API, and a database. Hyperclay collapses all three into one artifact: the malleable document. The DOM is the data model. In ClayJS, saving means cloning the root element, synchronizing [persist] form state, running snapshot hooks (onbeforesnapshot and clay.onSnapshot), stripping regions marked clay="editor-ui" or no-snapshot along with browser extension debris, running save hooks (onbeforesave and clay.addDocumentTransform), converting mode attributes (editmode:contenteditable, viewmode:disabled) to their inert saved form, stripping no-save content and runtime root attributes (editmode, pageowner, canedit, draftmode, unsaved, savestatus, savetoken, documentetag), then writing <!DOCTYPE html> plus the prepared root to disk via POST /_/save. The next page load serves the modified file, so changes are permanent.
The save pipeline#
When a user saves, the client sends the full HTML string via POST /_/save. On the server, saveHTML runs through these steps:
-
Format: Formatting is opt-in. If the
<html>element carriesformathtml="true", the raw HTML is passed through js-beautify with force-expanded multiline attributes and 2-space indentation (SVG, script, and style tags are left unformatted). Otherwise the bytes are stored exactly as sent. -
Admit: The current file is read from disk. A stale
If-Matchanswers412. Then the write is admitted: a save that declares a lower block editor format than the document’s floor (X-Clay-Blockeditor-Formatagainst<meta name="clay-blockeditor-format">), or that replaces a block editor document whose live session it does not descend from, answers409. Nothing is backed up or written for a refused save. -
Deduplicate: The formatted HTML is compared to the current file. If the content hasn’t changed, the save is skipped entirely, with no backup, no broadcast, and no wasted I/O.
-
First-save safety net: If this document has zero backups in the database (for example, it was created before the backup system existed), the current content is backed up before the new content overwrites it.
-
Backup: The formatted HTML is passed to
BackupService.createBackup, which stores either a full snapshot or a unified diff inside a PostgreSQL transaction. -
Write: The formatted HTML is written to
/sites/{username}/{folderpath}/{sitename}.htmlviadx('sites', ownerUsername, sitePath).createFileOverwrite(). -
Tailwind compile: If the HTML contains any Tailwind
<link>, the Tailwind JIT compiler extracts classes and writes the compiled CSS to/public-assets/tailwindcss/{username}/{folderpath}/{sitename}.css. -
Broadcast: The new content and a SHA-256 checksum are broadcast over SSE to any connected live-sync clients and the owner’s disk-sync engine.
// The actual save handler (simplified)
const formattedHtml = formatHtml(html);
const currentHtml = await dx('sites', ownerUsername, sitePath, node.name).getContents();
if (currentHtml === formattedHtml) return next(); // Skip if unchanged
await BackupService.createBackup(node.id, formattedHtml, userId);
await dx('sites', ownerUsername, sitePath).createFileOverwrite(node.name, formattedHtml);
if (hasAnyTailwindLink(formattedHtml)) {
const css = await compileTailwind(formattedHtml);
const baseName = node.name.replace(/\.(html|htmlclay)$/, '');
await dx('public-assets/tailwindcss', ownerUsername, sitePath).createFileOverwrite(`${baseName}.css`, css);
}
liveSync.broadcastNodeSaved(ownerUsername, buildNodeSavedPayload(node, {
content: formattedHtml
}));
The three-dimensional state machine router#
Every request to Hyperclay is classified along three axes: who is making the request, what they’re accessing, and what action they’re performing. These three dimensions form a routing key like dev:regular_site:edit, and the routing table maps keys to handler pipelines.
User types#
| Type | Description |
|---|---|
no_auth | Not logged in |
app_user | Logged-in user on a multi-tenant app (account_type = 'app_user') |
dev | Developer account (account_type = 'dev'), can create documents and has billing |
admin | Superadmin (isSuperadmin = true), falls through to dev routes if no admin-specific route exists |
Resource types#
| Type | Description |
|---|---|
main_app | The hyperclay.com application itself (dashboard, auth pages, billing) |
regular_site | A standard user-created document |
multi_tenant | A document with enableSignups = true |
instance | A copy of a multi-tenant app owned by an app_user |
folder | Dashboard folder navigation |
upload | User-uploaded files |
The eight-step state detection pipeline#
Before any route handler executes, eight middleware functions build a complete picture of the request:
state__init -> Initialize req.state with safe defaults
state__meta -> Parse domain, method, path, body, query, subdomain
state__user -> Load Person from auth_token cookie, detect user type
state__resource -> Detect resource type, load Node from database, check ownership
state__shareAccess -> If valid share token, elevate no_auth -> app_user with ownership
state__action -> Map URL path to action name (e.g. /edit -> 'edit', /login GET -> 'login-page')
state__key -> Build final key: '{userType}:{resourceType}:{action}'
state__authcookies -> Set ownership and resource cookies for the client
After detection, req.state contains everything needed for routing:
{
meta: { fullDomain, rootDomain, method, path, subdomain, isCustomDomain, body, query },
user: { person, isLoggedIn, isOwner, username, hasSubscription, siteLimit, canCreateSites },
resource: { node, siteName, customDomain, sourceNode, instanceOwner, currentFolderId },
userType: 'dev',
resourceType: 'regular_site',
action: 'edit',
key: 'dev:regular_site:edit'
}
Action detection#
The state__action middleware maps URL paths to action names. The rules:
- Root path (
/):view - First segment is the action name:
/editmaps toedit,/savemaps tosave,/dashboardmaps todashboard - GET on auth pages gets a
-pagesuffix:GET /loginmaps tologin-page,POST /loginmaps tologin. This lets the routing table serve the form on GET and process the submission on POST with the same URL. - Compound routes have special mappings:
/edit/uploads/...maps toedit-upload,/save/uploads/...maps tosave-upload,/live-sync/streammaps tolive-sync-stream. Upload saves viaPOST /_/savewith aDocument-URLheader containing/edit/uploads/...are also detected assave-upload. - Versioning routes remap:
/version/123maps toview-version,/restore/123maps torestore-version,/versionsmaps tobackups - Unknown paths on documents fall through to
view. If the resource is a document and the action isn’t in the known actions list (edit,save,login,signup, etc.), it’s treated as aview. This enables client-side routing: any path on a document renders the root HTML, and the client code can readwindow.location.pathname.
The remaining path segments are stored in req.state.actionParams, available to handlers. For /version/123, the action is view-version and actionParams is ['123'].
The expansion syntax#
The + operator lets one route definition cover multiple combinations. This:
'dev+app_user:regular_site+instance:save': [requireOwnership, saveHTML, respond]
Expands at startup into four separate routes:
dev:regular_site:save
dev:instance:save
app_user:regular_site:save
app_user:instance:save
Wildcard matching#
When the exact key has no match, the router tries fallbacks in order:
- Exact match:
dev:regular_site:edit - Admin fallback to dev:
dev:regular_site:edit(if user is admin) - Wildcard user:
*:regular_site:edit - Wildcard resource:
dev:*:edit - Full wildcard:
*:*:edit
If nothing matches, the request gets a 403.
The state factory#
Route handlers use a state() factory for inline condition checks:
// Require ownership - returns 400 if false
'dev:regular_site:edit': [
requireOwnership,
serveCodeEditor
]
// Require condition with custom message
state(s => s.user.person?.hasActiveSubscription).require('Active subscription required')
// Redirect if condition fails
state(s => false).redirect('/_/dashboard', 'You already have an account.')
Key routes#
| Key | What it does |
|---|---|
*:main_app:view | Serve the homepage |
dev:main_app:dashboard | Serve the dashboard (folder browser, document list) |
dev:main_app:new | Create a new document |
*:regular_site+multi_tenant+instance:view | Serve a document’s HTML |
dev:regular_site+multi_tenant+instance:edit | Serve the code editor (ownership required) |
dev+app_user:regular_site+multi_tenant+instance:save | Save HTML (ownership required) |
*:multi_tenant:signup | Sign up for a multi-tenant app, create instance |
dev+app_user:*:live-sync-stream | SSE stream for real-time sync |
*:regular_site+multi_tenant+instance:data | Data extraction API |
The response layer#
The routing table maps keys to handler pipelines, and each pipeline produces output through the respond module, which handles the browser-vs-AJAX split automatically.
| Function | What it does |
|---|---|
respond.success(res, { msg, html }) | Sends JSON with msgType: 'success'. Used by save, create, and update operations. |
respond.error(res, error) | Browser requests get an Edge.js error page. AJAX requests get JSON. Sequelize errors are auto-classified: SequelizeUniqueConstraintError maps to 409, SequelizeValidationError maps to 400. |
respond.redirect(res, url, params) | Builds a URL with query parameters (msg, msgType, return) and issues a 302. |
respond.html(res, html) | Sends raw HTML, prepending <!DOCTYPE html> if missing. |
respond.nodes({ res, person, folderPath, msg }) | Reloads the person via getPersonWithNodes, renders the components/nodes Edge.js partial, and returns JSON with the rendered HTML fragment. |
respond.nodes is how the legacy dashboard stays in sync: after creating, deleting, or moving a document, the client gets back the updated node list as a pre-rendered HTML string, with no second request needed.
The routing table also uses serve('home') (renders a named Edge.js template for browser requests) and asyncHandler(fn) (wraps async handlers with try/catch, routing uncaught errors to respond.error).
The request type is detected via isBrowserRequest(), which checks the Accept header. This means the same route can serve both a full HTML page (direct navigation) and a JSON response (AJAX call from the client).
The db middleware factory#
Route handlers often need to check database conditions before proceeding, such as whether an email already exists or a document name conflicts. The db object provides a Proxy-based API for expressing these checks declaratively inside the routing table:
// Ensure no Person exists with this email (for signup)
db.person({ email: req => req.body.email }).exists(false)
// Ensure the node exists (404 if not)
db.node({ name: req => req.body.name }).exists(true)
// Update the current user's record
db.person().update({ username: req.body.username })
// Delete matching records
db.customDomain({ id: req => req.body.domainId }).destroy()
Each call returns a standard Express middleware function. The query parameter accepts static values or functions that receive req: values are resolved at execution time, not at startup.
.exists(false) returns 409 if a record is found. .exists(true) returns 404 if not found. .update(data) finds the record and updates it. .destroy() deletes matching records. .create(data) inserts a new record and stores it on req.createdRecord for later middleware.
Special handling for Person and Node: calling db.person().update(...) with no query defaults to req.state.user.person, and db.node().update(...) defaults to req.state.resource.node. These records were already loaded by the state pipeline, so no redundant lookup is needed.
The Proxy intercepts any property access, so db.customDomain(...), db.message(...), db.siteBackups(...), and others all work for any registered Sequelize model.
The hybrid storage system#
Hyperclay uses PostgreSQL for metadata and the filesystem for content.
Database models#
| Model | Purpose |
|---|---|
Person | User accounts: email, username, password (bcrypt), stripeCustomerId, account_type (dev/app_user), isSuperadmin |
Node | Unified entity for documents, folders, and uploads: name, type, parentId, path, enableSignups, shareToken, sourceNodeId |
PersonNode | Ownership junction table (many-to-many between Person and Node) |
SiteBackups | Version history: html/diffContent, isDiff, diffFromId, snapshotId, contentHash, backupNumber |
CustomDomain | Domain mappings: domain, status (pending/verifying/active/failed), sslStatus, saasCustomDomainsId |
LoggedInToken | Auth tokens: token (secure random), personId, expires (30 days from creation) |
EmailConfirmToken | Email verification tokens |
Message | Contact form submissions |
Event | Activity tracking (logins, saves, signups) |
ApiKey | Sync API keys: keyHash (SHA-256), keyPrefix (hcsk_), expiresAt, isActive |
The Node model is the unifying abstraction. Documents, folders, and uploads are all Nodes with a type field ('site', 'folder', or 'upload'). Folders form a tree via parentId. Instances link to their source via sourceNodeId. This lets the workspace and dashboard load documents, folders, and uploads in a single query via the ownerPersonId-keyed OwnedNodes association.
Filesystem layout#
/sites/{username}/{folderpath}/{sitename}.html # Live document files
/sites-versions/{username}/{folderpath}/{base}/{timestamp}.html # Timestamped backup files (e.g. on instance creation)
/sites-deleted/{sitename}.html # Soft-deleted documents
/uploads/{username}/{folderpath}/{filename} # User-uploaded files
/public-assets/tailwindcss/{username}/{folderpath}/{name}.css # Compiled Tailwind CSS per document
The dx.js file utility#
All filesystem operations go through dx, a chainable async API that absorbs errors instead of throwing:
// Read - returns null if file doesn't exist
const html = await dx('sites', 'mysite.html').getContents();
// Write - creates directories automatically
await dx('sites').createFileOverwrite('mysite.html', html);
// Check existence
const exists = await dx('sites', 'mysite.html').exists();
// Copy with automatic directory creation
await dx('sites-deleted', 'mysite.html').copyFileFrom('sites', 'mysite.html');
// JSON operations
await dx('config.json').appendJSON({ username: 'alice' });
const value = await dx('config.json').getKey('username');
// Chainable - copy then rename
await dx('backups', 'site.html')
.copyFileFrom('sites', 'site.html')
.renameTo('site-backup.html');
// Null safety - returns null instead of crashing
await dx('users', undefined).append('data'); // returns null
The chainable mechanism uses a Proxy that wraps each async operation, passing the resolved path from one step to the next. Native array methods like .length, .reverse(), and [0] work on the resolved values.
The diff-based backup system#
Every save creates a backup in version history. Every 20th backup is a full HTML snapshot stored in the SiteBackups table. The 19 saves between snapshots are stored as unified diffs (via the diff npm package) against the previous backup.
How it works#
// Inside BackupService.createBackup, within a transaction:
const backupNumber = lastBackupNumber + 1;
const isSnapshot = backupNumber % 20 === 0 || backupNumber === 1;
if (isSnapshot) {
// Store full HTML
await SiteBackups.create({
nodeId, html, contentHash, isDiff: false, backupNumber
});
} else {
// Get previous backup and create diff
const previousHtml = await lastBackup.getFullHtml();
const diff = createPatch(String(lastBackup.id), previousHtml, html);
await SiteBackups.create({
nodeId, diffContent: diff, diffFromId: lastBackup.id,
contentHash, isDiff: true, snapshotId, backupNumber
});
}
Reconstruction#
To reconstruct any backup’s full HTML, getFullHtml() does:
- If the backup is a snapshot (
isDiff: false), returnhtmldirectly - Otherwise, load the nearest snapshot (tracked by
snapshotId) - Load all diffs between the snapshot and this backup, ordered by ID
- Validate chain continuity: each diff’s
diffFromIdmust equal the previous backup’s ID - Apply diffs sequentially using
applyPatch - Verify the final SHA-256 hash matches
contentHash
Concurrency safety#
The entire createBackup call runs inside a PostgreSQL transaction that takes a row-level lock on the Node:
await Node.findByPk(nodeId, {
transaction,
lock: transaction.LOCK.UPDATE
});
This serializes concurrent saves for the same document, preventing two backups from both reading the same “last backup” and creating conflicting diffs.
Multi-tenant architecture#
Any document becomes a multi-tenant sign-up app by setting enableSignups = true on its Node record. The state machine detects this and classifies the resource as multi_tenant instead of regular_site.
Instance creation flow#
When a user signs up on a multi-tenant app:
- Create account: A
Personrecord withaccount_type: 'app_user'and a bcrypt-hashed password - Generate name: The instance gets named
{appname}.html. If taken, it becomes{appname}-1.html,{appname}-2.html, and so on. - Create Node: A new
Nodewithtype: 'site',ownerPersonIdset to the newapp_user, andsourceNodeIdpointing to the source app - Copy HTML: The source document’s HTML file is copied to
/sites/{username}/{instancename} - Create backup: An initial timestamped backup is created
- Send confirmation: An email with a confirmation link is sent
const instanceName = await generateInstanceName(baseName, person);
const instanceNode = await Node.create({
name: instanceName,
type: 'site',
sourceNodeId: sourceNode.id,
parentId: 'root',
ownerPersonId: person.id
});
const sourceHtml = await dx('sites', sourceOwner.username, sourcePath, sourceNode.name).getContents();
await dx('sites', person.username).createFileOverwrite(instanceNode.name, sourceHtml || '');
Instance URLs#
The subdomain is the owner’s username, and the instance filename is a path segment:
- Hyperclay subdomain:
https://{username}.hyperclay.com/{instance.name}(for example,https://alice.hyperclay.com/myapp.html) - Custom domain:
{username}.myapp.com/{instance.name}(requires wildcard domain*.myapp.com)
Each instance evolves independently from the source. Editing the source app doesn’t change existing instances.
Wildcard domain requirements#
If a multi-tenant app uses custom domains, enabling signups requires a wildcard domain (*.myapp.com) so that instance subdomains can be routed. Without custom domains, instances use Hyperclay subdomains by default.
Authentication and access control#
Token-based auth#
Login creates a LoggedInToken record with a cryptographically secure random ID and a 30-day expiry:
// LoggedInToken model
{
token: idSecure(), // Secure random string
personId: person.id,
expires: Date.now() + (30 * 24 * 60 * 60 * 1000) // 30 days
}
The token is stored as an httpOnly cookie. For Hyperclay subdomains, it’s scoped to .hyperclay.com (the leading dot covers all subdomains). For custom domains, SameSite: none and secure: true enable cross-domain cookie sharing.
On every request, state__user looks up the token, loads the Person with their Nodes (via eager loading), calculates document limits, and sets the user type.
Eager loading with getPersonWithNodes#
The getPersonWithNodes function is used everywhere a fully-populated user is needed: during login, on every authenticated request, and after account creation. It loads the Person with all their Nodes in a single query via Sequelize eager loading:
const person = await Person.findOne({
where: { id: personId },
include: [{
model: Node,
as: 'OwnedNodes',
required: false,
include: [{
model: Node,
as: 'SourceNode',
required: false
}]
}],
order: [sequelize.literal('LOWER("OwnedNodes"."name") ASC')]
});
This returns a Person with person.Nodes (aliased to OwnedNodes), an array of all documents, folders, and uploads they own, each with its SourceNode if it’s an instance. The dashboard, document limit calculation, ownership checks, and folder navigation all read from this pre-loaded array instead of making additional queries. The LOWER(name) ASC ordering means the dashboard lists documents alphabetically, case-insensitively.
Domain-aware cookie management#
The cookieManager handles the complexity of setting cookies that work across both Hyperclay subdomains and custom domains:
// For Hyperclay domains - scope to apex so all subdomains share auth
setCookieOnApexAndAllSubdomains(res, 'auth_token', token, {
maxAge: 30 * 24 * 60 * 60 * 1000,
httpOnly: true
});
// For custom domains - SameSite=none enables cross-domain sharing
res.cookie('auth_token', token, {
maxAge, httpOnly: true, secure: true,
sameSite: 'none',
domain: req.state.meta.rootDomain
});
The same split applies to ownership cookies. When a user visits a document, cookieManager.setOwnershipCookie sets currentResource (the document name) and isAdminOfCurrentResource (whether they own it) on the appropriate domain. These cookies are readable by client JavaScript: ClayJS reads isAdminOfCurrentResource at boot to set clay.canEdit and clay.isOwner. The document still opens in view, so clay.isEditMode stays false until the owner switches to editing. ClayJS writes editmode="true|false", pageowner="true|false" and, while the person may edit, canedit="true" onto <html> to control editing UI visibility ([show-when:editmode="true"]) and mode attributes.
On logout, clearAuthCookies clears all auth and ownership cookies across every possible domain variation to prevent stale sessions.
Two account types#
| Type | How they’re created | What they can do |
|---|---|---|
dev | Sign up via Stripe checkout (or free signup when FREE_SIGNUP=on) and set password | Create documents, manage uploads, and on paid plans edit code, manage billing, and attach custom domains |
app_user | Sign up on a multi-tenant app | Edit their own instance, manage their uploads |
Share token system#
Any document can use share links. In the token-based share flow, a random string is stored on the Node. When a visitor arrives with ?token=... in the URL, the state__shareAccess middleware:
- Validates the token against the Node’s
shareTokenfield (must haveshareEnabled: true) - Sets
req.state.hasShareAccess = trueandreq.state.user.isOwner = true - Elevates
no_authusers toapp_user(devs keep theirdevtype) - Stores the token in a document-specific cookie (
share_{sitename}) for future visits
This lets edit share link recipients save changes without creating an account. Free accounts can create both view and edit share links.
Password security#
- Bcrypt hashing with salt rounds of 10
- 8-character minimum
- Checked against a top-100k common passwords list loaded at server startup
- Timing-safe comparison on failed lookups (dummy bcrypt compare to prevent enumeration)
Real-time collaboration via SSE#
The livesync-hyperclay library handles real-time live sync between browser editors and the disk sync engine over the ClayJS wire protocol.
SSE stream#
Clients connect via GET. The document-url query parameter names the document because EventSource cannot set headers:
GET /_/sync?document-url=<full page URL>&lane=live
GET /_/sync?document-url=<full page URL>&lane=saved
The live lane carries pre-strip snapshots to editors. The saved lane carries durable documents to viewers after an accepted save.
The server sets SSE headers, registers the client, sends a : connected comment, and starts 30-second keep-alive pings. On disconnect, the client is unregistered.
const lane = req.query.lane === 'saved' ? 'saved' : 'live';
res.setHeader('Content-Type', 'text/event-stream');
res.setHeader('Cache-Control', 'no-cache');
res.setHeader('Connection', 'keep-alive');
res.setHeader('X-Accel-Buffering', 'no');
liveSync.subscribe(file, res, { lane });
const keepAlive = setInterval(() => {
res.write(': ping\n\n');
}, 30000);
Save broadcast#
When a save happens (from the code editor, the visual editor, or the Sync API), two broadcasts fire:
- Browser broadcast:
liveSync.broadcast(file, { html, sender }, { lane: 'saved' })sends the durable document to viewers on thesavedlane. ThesenderID lets clients ignore their own saves. - Sync engine broadcast:
liveSync.broadcastNodeSaved(username, buildNodeSavedPayload(node, { content }))notifies the owner’s Hyperclay Local sync client.buildNodeSavedPayloadbuilds the per-type wire shape (for a document, it carriescontentplus an auto-computed SHA-256checksum).
Editor snapshots travel separately through POST /_/sync and are broadcast on the live lane without writing to disk.
Access control#
The two lanes use different authorization. The live lane is gated like a write and requires ownership. The saved lane is gated like a read, so public documents are open while private documents require owner, share-token, or admin access.
app.get('/_/sync', gateLiveSyncStream, handleLiveSyncStream);
Edit share guests get live-lane access because state__shareAccess elevates them and sets isOwner = true.
The Sync API#
The Sync API powers Hyperclay Local, the desktop app that syncs a local folder of documents with hyperclay.com, as well as external agents and scripts. It uses a separate authentication system from the browser-based state machine: API keys instead of cookies.
API key authentication#
Keys use a hcsk_ prefix followed by 32 random bytes (hex-encoded). The raw key is shown once during generation and stored as a SHA-256 hash. Keys expire after 1 year. Each request validates the X-API-Key header by hashing the provided key and looking up the hash in the ApiKey table.
const keyHash = crypto.createHash('sha256').update(apiKeyHeader).digest('hex');
const apiKey = await ApiKey.findValidKey(keyHash);
The authenticateApiKey middleware runs before all sync endpoints, separate from the state machine pipeline. On success, it sets req.state.user.person to the key’s owner, so downstream handlers can use the same person reference.
Endpoints#
| Endpoint | Method | Purpose |
|---|---|---|
/_/sync/status | GET | Server time for clock synchronization |
/_/sync/stream | GET | SSE stream for real-time file change notifications |
/_/sync/nodes | GET | List all nodes (documents, uploads, and folders) |
/_/sync/nodes | POST | Create a node |
/_/sync/nodes/:id/content | GET | Download node content |
/_/sync/nodes/:id/content | PUT | Write node content |
/_/sync/nodes/:id/rename | PATCH | Rename a node |
/_/sync/nodes/:id/move | PATCH | Move a node to a different parent |
/_/sync/nodes/:id | DELETE | Delete a node |
Sync upload flow#
A node write (POST or PUT /_/sync/nodes...) does more than write a file:
- Parse path: Splits
folder/subfolder/sitenameinto folder components and document name - Create folders: If the path includes folders, creates
Noderecords for each level that doesn’t exist yet, with parent-child relationships - Check name collisions: If the document name exists globally and is owned by someone else, returns 409. If owned by the same user in a different folder, reuses the existing
Node. - Backup: If the file already exists on disk, creates a backup via
BackupService.createBackupbefore overwriting - Write: Writes to
/sites/{username}/{folderpath}/{sitename}.html, scoped by owner username and folder path on disk - Tailwind: Compiles Tailwind CSS if the HTML references a Tailwind stylesheet
- Broadcast: Sends the update to both the live-sync system (for connected browsers) and the sync engine (for other devices running Hyperclay Local)
SSE stream per user#
Unlike browser live-sync (which subscribes to individual files), the sync stream subscribes to an entire user’s file changes via liveSync.subscribeUser. When any of the user’s files is saved, whether from the browser code editor, another device’s Hyperclay Local, or in-page editing, the stream pushes the update.
Dual broadcast#
When a sync upload includes a snapshotHtml field and senderId, two additional broadcasts fire:
liveSync.broadcast(siteName, { html, sender })pushes to browser editors viewing the fileliveSync.broadcastToUser(username, siteName, { html, sender })pushes to other Hyperclay Local instances
This enables three-way sync across the local filesystem, the hosted platform, and the browser editor. The sender ID prevents echo loops, as each client ignores broadcasts from itself.
Tailwind CSS on-save compilation#
The tailwind-hyperclay library powers Tailwind on save with four functions used across the codebase:
| Function | Purpose |
|---|---|
hasTailwindLink(html, siteName) | Checks if HTML contains <link href="/tailwindcss/{siteName}.css"> |
hasAnyTailwindLink(html) | Checks if HTML contains any Tailwind link (for copies and renames) |
compileTailwind(html) | Extracts classes from HTML, runs Tailwind v4 JIT, and returns a CSS string |
replaceTailwindLink(html, oldName, newName) | Updates the link href when a document is renamed or copied |
Compilation hooks into every operation that changes HTML content:
- Save (
saveHTML): compile ifhasAnyTailwindLink - Sync upload (
uploadSyncFile): compile ifhasAnyTailwindLink - Backup restore (
restoreBackup): compile ifhasAnyTailwindLink - Document copy (
copySiteComplete): detect withhasAnyTailwindLink, replace link withreplaceTailwindLink, then compile
Output is written to /public-assets/tailwindcss/{username}/{folderpath}/{sitename}.css. The Express server serves this path with a fallback: if the file doesn’t exist yet (first load before any save), it returns empty CSS instead of a 404.
Domain routing#
Hyperclay routes both addresses and subdomains on hyperclay.com and custom domains.
Domain classification#
The classifyDomain function in state__meta determines the request type:
| Domain | Classification |
|---|---|
hyperclay.com or www.hyperclay.com | main_app |
*.hyperclay.com | hyperclay_subdomain |
| Dev tunnel domain | dev_tunnel (treated like main_app) |
| Everything else | custom_domain |
Subdomain resolution#
For Hyperclay subdomains, state__resource resolves the subdomain to the owner’s username, then the document by filename in the path:
- Look up a
Personwhoseusernamematches the subdomain - Resolve the
.htmlor.htmlclayfilename from the path and find the owner’sNodeby name and folder path - If that
NodehassourceNodeIdset, load its source node and owner, settingResourceType.INSTANCE - Otherwise, if the document has
enableSignups = true, setResourceType.MULTI_TENANT; else setResourceType.REGULAR_SITE - If no document matches, fall back to resolving an uploaded asset at that path
Custom domain resolution#
For custom domains:
- Look up the exact hostname in the
CustomDomaintable (status must beactive) - If not found and there’s a subdomain, try the root domain (for instance subdomains on custom domains)
- Load the associated
Node - If the
NodehasenableSignupsand there’s a subdomain, look for an instance (ResourceType.INSTANCE) - Otherwise serve the document directly
Custom domain lifecycle#
Custom domains are managed through the SaasCustomDomains.com API:
- User adds a domain:
POSTto the SaasCustomDomains API creates aCustomDomainrecord with statuspending - SaasCustomDomains provides DNS instructions (CNAME or TXT records)
- User updates DNS
- Webhook fires on DNS verification: status moves to
active - SSL is provisioned automatically
- Webhook fires on SSL issuance:
sslStatusmoves toissued
Wildcard domains (*.myapp.com) use the DNS-01 challenge type for SSL. Per-user rate limiting caps API calls at 10 per minute.
The Data Extraction API#
Any document’s HTML can be queried as structured JSON through page data endpoints by appending ?data={...} to its URL, or from inside the page with the ClayJS data plugin (clay.extractData and clay.readData). The extraction rules use a CSS-selector-based syntax.
Rule types#
String rules extract text or attributes:
".title" // text content of first .title element
".logo@src" // src attribute of first .logo element
"@data-user-id" // attribute from root element
".tag[]" // array of text from all .tag elements
Array rules iterate over elements:
// [selector, shape] - returns array of shaped objects
[".product", { name: ".name", price: ".price" }]
// Result: [{ name: "Widget A", price: "$99" }, { name: "Widget B", price: "$149" }]
Object rules compose nested structures:
{
user: { name: ".user-name", role: ".user-role" },
metrics: { revenue: ".revenue", orders: ".orders" },
products: [".product", { name: ".name", price: ".price" }],
tags: ".tag[]"
}
DOM property access#
The @ prefix also supports DOM properties: @textContent, @innerHTML, @outerHTML, @value, @checked, and others.
Relaxed JSON parser#
Since rules are passed as URL query parameters, the extraction endpoint includes a custom tokenizer that handles unquoted keys and CSS selectors (including pseudo-selectors with colons and attribute selectors with brackets) without requiring strict JSON escaping.
Caching#
Results are cached in memory for 5 minutes per document and query combination. When the map exceeds 100 entries, expired entries are swept; past 200 entries, the oldest entry is evicted.
File uploads#
Storage#
Uploads are stored directly on the filesystem at /uploads/{username}/{folder_path}/{filename}. The Node model tracks them in the database for folder organization, with parentId linking to a folder Node.
When a file is uploaded from inside a document using the ClayJS upload plugin (clay.upload posting to /_/upload), it is stored in that document’s own folder at uploads/assets-document-<document id>/ (created on first upload) and linked as /_/uploads/assets-document-<document id>/<file>. The folder is named after the document’s id, so renaming the document never moves its pictures, and once the account’s uploads are protected the file answers only to people who can open the document. The top-level uploads folder is locked and cannot be renamed, moved, or deleted.
Processing#
Upload handling uses Formidable for multipart parsing. Files pass through validation:
- Filename sanitization via
sanitize-filename - Extension-based type detection via
mime-typesandshared/file-kinds.js, plus byte sniffing on free accounts to verify image contents - Refused extensions:
.shtmland.xhtare blocked on all upload lanes, and.html,.htm,.xhtml,.htmlclay,.js,.mjs, and.xmlare blocked on the in-document upload lane so HTML files go through the save lane - Tier-based upload and storage caps: Free accounts are limited to images only (PNG, JPG, GIF, WebP, AVIF, or SVG) at 10 MB per file and 25 MB total; Personal allows 20 MB per file and 500 MB total; Maker allows 100 MB per file and 2 GB total. This is separate from the Express body parser limits (5 MB JSON, 16 MB text for whole-document saves, 2 MB URL-encoded forms), with special parsers raising the JSON cap to 150 MB on
/sync/nodesand/upload, lowering it to 1 MB on/collection, and setting 10 MB on/live-sync.
Editable files#
Clicking an upload in Files opens /dashboard/site/<id>, which renders images in place, PDFs in the browser’s viewer, text files as plain text, and other file types as a card with Download and Open in new tab. Text-based uploads (txt, md, markdown, json, html, htm, htmlclay, jsx, css, js, mjs, csv, tsv, xml, yml, yaml, log) plus svg can be edited in that page’s Code tab with Save (⌘S) or at /_/edit/uploads/{username}/{path}/{filename}, which routes saves to save-upload. Editing an upload’s code requires the paid canEditCode capability.
Folder management#
Folders are Node records with type: 'folder'. They support:
- 5 levels of nesting (enforced by a
beforeValidatehook) - Breadcrumb navigation via the
pathfield (stores the ancestor path) - Move operations between folders (with path recalculation for all descendants)
Payment integration#
Stripe checkout flow#
- Unauthenticated user submits their email on the pricing page
- Server creates a Stripe checkout session with the email
- User completes payment on Stripe’s hosted page
- Stripe webhook (
/stripe-webhook) fires with the raw body (parsed before Express JSON middleware) - Server verifies the webhook signature
- On
checkout.session.completed, the server creates aPersonwithhasActiveSubscription: trueand anEmailConfirmToken - User follows the confirmation link, sets a username and password, and the account is ready
Pricing tiers#
Plans and billing offer two paid tiers, billed monthly or annually (saving 33%), plus a Free tier ($0 forever, offered when FREE_SIGNUP is enabled; off by default in production):
| Tier | Monthly | Annual | Included highlights |
|---|---|---|---|
| Free | $0 | $0 | 10 documents, image-only uploads (10 MB per file, 25 MB total), view and edit share links |
| Personal | $6/month | $48/year | Unlimited documents, unlimited custom domains, 1 to 3 multi-tenant apps, 3 to 10 subdomains, 20 MB / 500 MB uploads, 1,000 monthly AI credits |
| Maker | $15/month | $120/year | Everything in Personal plus teams (4 people included: owner and 3 editors; $3/month or $2/month on annual per extra editor; viewers free), 5 to 25 multi-tenant apps, 6 to 25 subdomains, 100 MB / 2 GB uploads, 2,000 monthly AI credits |
Document limits and feature accrual#
Limits are tier-based, driven by TIER_CONFIG rather than a single formula. Free accounts can create 10 documents. Paid tiers (Personal and Maker) are effectively unlimited: the config sets the cap at 2000, which calculateSiteLimit reports as isUnlimited. Superadmins are unlimited.
// From TIER_CONFIG
const TIER_CONFIG = {
free: { sites: { limit: 10 } },
personal: { sites: { limit: 2000 } }, // treated as unlimited
maker: { sites: { limit: 2000 } }, // treated as unlimited
};
// calculateSiteLimit reports config.limit >= 2000 as isUnlimited
Custom domains are unlimited (domains.maximum: null) on paid tiers. The base + monthsActive * perMonth accrual (capped at max) applies to subdomain and multi-tenant app allowances:
- Personal: 3 subdomains (+1/month up to 10) and 1 multi-tenant app (+1/month up to 3)
- Maker: 6 subdomains (+2/month up to 25) and 5 multi-tenant apps (+2/month up to 25)
Billing portal and lapsed subscriptions#
Subscription management (Manage billing) is a redirect via /_/manage-billing to Stripe’s hosted billing portal. Subscriptions have a 30-day full refund policy and auto-cancel after 6 months of inactivity. When a subscription lapses, isLapsedFor in server-lib/capabilities.js drops the account to free capabilities: existing documents stay accessible for viewing, content editing, uploads, and copying, while creating new documents, opening the code editor, adding domains or subdomains, and enabling sign-ups require an active paid plan.
The code editor#
The /_/edit route and the dashboard Code tab (/_/dashboard/site/:nodeId/code) serve the code editor built on CodeMirror 6. The editor loads the document’s current HTML, provides syntax highlighting, and integrates with the live-sync system.
Access control#
Only dev users who own the document and hold the paid canEditCode capability can access the code editor. Share guests are elevated to app_user, which is explicitly forbidden from the code editor (app_user:*:edit returns 403). This is intentional: edit share access grants save permission for in-page editing, not raw code editing.
'dev:regular_site+multi_tenant+instance:edit': [requireOwnership, serveCodeEditor],
'app_user:regular_site+multi_tenant+instance:edit': [forbidden],
Upload editing#
Text-based and SVG uploads can be edited in the dashboard Code tab or at /_/edit/uploads/{username}/{path}. Ownership and canEditCode are verified before saving.
Security model#
Declarative access control#
Security is structural. The routing table is the access control policy:
// Only devs who own the document can edit code
'dev:regular_site:edit': [requireOwnership, serveCodeEditor]
// Nobody without auth can edit anything
'no_auth:*:edit': [redirect('/_/login?...')]
// App users can save (their instances), but not edit code
'app_user:regular_site+instance:save': [requireOwnership, requireEmailConfirmForAppUser, saveHTML, respond]
'app_user:regular_site+instance:edit': [forbidden]
Every request that doesn’t match a routing key gets a 403. There is no default-allow behavior.
Rate limiting#
The createRateLimiter factory produces configurable middleware:
- Login: 20 attempts per 15 minutes, keyed on
sha256(IP + email), resets on success - API: 100 requests per 15 minutes, keyed on IP only
- Strict (sensitive operations): 5 attempts per hour
Rate limiters use in-memory Map instances with periodic cleanup.
Password security#
- Bcrypt with 10 salt rounds
- 8-character minimum enforced server-side
- Checked against a top-100k common passwords list (loaded from
PwnedPasswordsTop100k.jsonat startup) - Timing-safe comparison: on failed lookup, a dummy bcrypt compare runs to prevent email enumeration via response timing
Cookie security#
| Cookie | Flags |
|---|---|
auth_token | httpOnly, secure, SameSite=lax, domain=.hyperclay.com, 30-day expiry |
auth_token (custom domain) | httpOnly, secure, SameSite=none, scoped to root domain |
isAdminOfCurrentResource | secure (readable by ClayJS for UI state) |
share_{sitename} | httpOnly, secure (production only), SameSite=lax, 1-year expiry |
Additional protections#
- Sequelize parameterized queries prevent SQL injection
- Stripe webhook signature verification (raw body, not parsed JSON)
- Node name validation blocks reserved words and special characters
- File upload sanitization via
sanitize-filenameand byte sniffing for image-only free accounts - Express body parser limits (5 MB JSON, 16 MB text for whole-document saves, 2 MB URL-encoded), with tier-based per-file upload caps (Free 10 MB, Personal 20 MB, Maker 100 MB) and account storage caps (Free 25 MB, Personal 500 MB, Maker 2 GB) layered on top
Deployment#
Production#
// ecosystem.config.js
{
name: 'hyperclay',
script: './hey.js',
instances: 1,
autorestart: true,
max_memory_restart: '1G'
}
Production runs a single PM2 instance without cluster mode: the filesystem-based architecture means a single process avoids file locking complexity. The 1 GB memory limit triggers an automatic restart if the process leaks.
Development#
Dev mode runs four PM2 processes:
| Process | What it does |
|---|---|
hyperclay | Express server with file watching on hey.js and server-lib/ |
tailwind-dev | PostCSS watcher compiling the admin UI’s Tailwind CSS |
backup-dev | Periodic local backup script |
malleabledocs-dev | Cloudflare tunnel for external access during development |
Database#
PostgreSQL with Sequelize ORM. Connection pool: min 2, max 10, 30-second acquire timeout, 10-second idle timeout. There is no sequelize.sync() or model auto-sync at startup. The schema is managed by an explicit numbered migration runner (node migrate.js --all), run on both dev startup (the predev script) and deploy (the deploy script); the numbered migrations live in migrations/.
Graceful shutdown#
The server listens for SIGTERM and closes the HTTP server cleanly, letting in-flight requests complete before exiting:
process.on('SIGTERM', () => {
server.close(() => process.exit(0));
});
The philosophy#
Hyperclay works because it embraces constraints. A single HTML file per document limits complexity. DOM-as-database eliminates impedance mismatch. Direct manipulation means what you see is what gets saved.
The trade-offs are real:
- No complex queries: You can’t JOIN across documents or SELECT with SQL from the HTML. The Data Extraction API gives you CSS-selector-based reads, not relational queries.
- Entire HTML in memory: Every save reads and writes the full file (up to 8 MB on hyperclay.com). Documents with hundreds of thousands of DOM nodes will hit performance limits.
- Whole-document conflict handling: Saves send the complete HTML file rather than field-level patches. With conditional saves and receipts (
If-Matchand HTTP 412) and live sync (?plugins=sync), open tabs perform a save-gated three-way DOM merge and record colliding edits inclay.conflictsrather than silently overwriting, while un-stamped saves fall back to last-write-wins.
What it gains:
- Radical simplicity: The entire backend is one Express application with one routing table. There is no separate content API layer, no ORM for document content, and no frontend build step.
- Instant persistence: Save and it’s live. There is no deploy step, no cache invalidation, and no eventual consistency.
- True ownership: Users own their HTML files. They can download, modify, and re-upload them. The platform adds features on top; it doesn’t lock content away.