Widget Builder
Describe a small app, a model in your browser builds it for the room
Screenshots
About this app
Describe what the room needs ("a lunch poll: pizza, sushi or salad", "a kanban board for our sprint") and a language model running in your own browser builds it as a small app. You try it out in a private preview first; publishing puts it into the room, where everyone uses it together, and adds it to the room as a widget of its own.
Most requests fit one of the built-in templates (poll, checklist, board game for two such as tic-tac-toe, kanban board, random picker, scoreboard, countdown, drawing): then the model only fills in the template's settings and the tested code stays as it is. Anything else gets new code. Before you see a new app, the builder uses it once in a hidden sandbox (types, clicks, draws) and sends what is broken back to the model. Published apps can be changed later, through their settings or code without the model, or by describing the change.
The model runs on your GPU through WebGPU (WebLLM). Nothing is sent to an AI service; the only network traffic is the one-time download of the model (from Hugging Face) and its GPU code (WebLLM's, from GitHub). It needs a browser with WebGPU on a real GPU (current Chrome or Edge), and a small model makes mistakes: the click-through test catches many, not all.
Published apps are room state, which Matrix never encrypts, even in an encrypted room: the homeserver can read an app's code. What people do in an app (votes, items) are timeline events, encrypted in an encrypted room. Apps run in a sandboxed frame without network access or access to the room; they see only their own events.
Add to a room
Paste this into any room you may configure — Element and most clients understand /addwidget:
/addwidget https://marcade.store/apps/de.datanauten.widget-builder/0.3.1/Needs WebGPU on a real GPU; the default model is a one-time 1.6 GB download.
Rooms should pin the versioned URL above. https://marcade.store/apps/de.datanauten.widget-builder/latest/ always points at the newest release.
Name and icon in Element
/addwidget can carry only the URL: Element lists the app as “Custom” without an icon. Add the Marcade Store once per room and it gives every Marcade app pasted in there its name and icon, and tells you when an update is out.
/addwidget https://marcade.store/apps/de.minagawa.marcade-store/0.1.4/Permissions
Capabilities this widget may request. The Marcade Player asks for your consent before granting any of them.
- Requires running inside a Matrix client
io.element.requires_client - Send room state events of type “de.datanauten.widget_builder.app”
org.matrix.msc2762.send.state_event:de.datanauten.widget_builder.app - Read room state events of type “de.datanauten.widget_builder.app”
org.matrix.msc2762.receive.state_event:de.datanauten.widget_builder.app - Send room events of type “de.datanauten.widget_builder.event”
org.matrix.msc2762.send.event:de.datanauten.widget_builder.event - Read room events of type “de.datanauten.widget_builder.event”
org.matrix.msc2762.receive.event:de.datanauten.widget_builder.event - Send room state events of type “im.vector.modular.widgets”
org.matrix.msc2762.send.state_event:im.vector.modular.widgets - Read room state events of type “im.vector.modular.widgets”
org.matrix.msc2762.receive.state_event:im.vector.modular.widgets
What's new in 0.3.1
Apps look finished out of the box: every app gets built-in styles in the room client's colours (light and dark), with cards, lists, badges, progress bars and avatars, and you can give each app its own colour. The templates are rebuilt on them, and a new one plays board games for two, such as tic-tac-toe or five in a row: two people take turns, everyone else watches. App frames fit their app. Removing an app or replacing an unpublished draft asks first. The click-through test also clicks elements that are not buttons, so a game board that shares nothing is caught. The experimental LFM2 model is gone, and with it most of the download: 39 MB becomes 13 MB. Apps built with earlier versions keep their settings form. Each template's list settings take the number of entries their hint names.
Information
- Version
- 0.3.1
- Section
- Community — Built by Marcade CI from a pinned commit; the capability list has been audited.
- License
- AGPL-3.0-or-later
- Author
- Yan Minagawa (datanauten)
- Issue tracker
- https://github.com/yncyrydybyl/widget-builder/issues
- State event types
de.datanauten.widget_builder.app,im.vector.modular.widgets- Encrypted rooms
- Not declared
- Widget ID
de.datanauten.widget-builder