Skip to content

Let first run show what OpenBot does, and what is left to set up - #915

Open
jerelvelarde wants to merge 52 commits into
CopilotKit:mainfrom
jerelvelarde:jerel/openbot-onboarding
Open

jerelvelarde wants to merge 52 commits into
CopilotKit:mainfrom
jerelvelarde:jerel/openbot-onboarding

Conversation

@jerelvelarde

@jerelvelarde jerelvelarde commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Let first run show what OpenBot does, and what is left to set up

Depends on #913. Its own change is the commits after that one: compare.

The problem

First run is an introduction that changes scene every four seconds, then asks for a name. A new person watches it at a pace they don't control and leaves without connecting an app or learning whether anything works. The server stores a step for each person, and its docblock says why that does nothing: "NOTHING RESUMES FROM IT YET". A reload starts again.

An admin finishing first run can't see what the deployment still lacks. The server checks the model key once at boot because, in its own words, a key without access "fails at the first message, in front of whoever sent it", but that answer goes to a log.

The approach

Steps, not a loop. First run is Welcome, a browser and files of its own, the team of Bots, asking before it acts, Running it yourself (only for somebody trying OpenBot alone), Check your setup, Connect your apps, Get notified, and the name. The checklist comes before the apps, so an empty app list is already explained, and Skip intro lands on it. On a desktop the step comes first: one line, "Step 5 of 9", opens the list over the page, where a step already reached can be picked, and the heading, which is also the step's name in that list, starts right under it. Back sits left of the primary on every step, and a screen reader hears ", done" or ", not yet available" with each name in the list. On a phone the steps are dashes with the buttons pinned below. Every step's buttons fit at 1280×800 and 1280×700.

Pictures look like pictures. Each introductory step has one, read by its label, playing only while on screen, muted, with nothing inside to focus or press. Get notified shows a browser window, since a notification arrives only while OpenBot is open in a tab. When there is no Bot computer, the browser-and-files step says what one gives and who adds it rather than promising one.

A person's place survives. The step lives in the address and is saved on every move, as a number of its own rather than a position, since the self-hosting step joins only for some people. A reload, browser Back and a consent's return keep it. A link to another page, such as an admin's Add apps, opens at the end, rather than skipping the steps between.

The steps do the work. Connect your apps signs in from the step and comes back whether the consent connected, failed or was cancelled; a brokered app reads Connected only when the vendor says so. Get notified asks the browser. The asking step carries the Ask before making changes switch, the same preference as Settings → Approvals. The self-hosting step offers only Back and Continue.

Check your setup asks the server. GET /api/setup answers row by row for Intelligence, the model key, the Bot computer, more apps, and Slack and Teams. Ready means it works, not that a variable is set: the model from the provider's answer for the key in use, the computer from each running one taking its network policy. An admin, or somebody running OpenBot alone, is told the page or variable that closes a gap; everybody else gets one neutral line, "Your admin sets this up.", with no variable name in the response, because anybody opens onboarding. It says "Checking your setup…" while it asks. Since its fixes end in a restart, the checklist reopens in Settings → Setup for those who can act on it.

The name step stays last. Get started reads Saving…, then All set while the orb finishes.

The cost: first run is eight or nine steps where it was two or three, and reading the checklist can start calls to Intelligence, the provider and the computers.

Before After
First run before: the looping introduction First run after: Welcome, beside the list of steps
Nothing showed what the deployment lacked, outside a server log Settings → Setup after: the checklist an admin reopens, with Check again

What is not covered

  • Get notified covers notifications while OpenBot is open in a tab. There is no web push.
  • Any signed-in person can press Check again in onboarding, which asks every running computer.
  • More apps, and Slack and Teams, read Ready when configured, not proven.
  • Until the checklist answers, the browser-and-files step reads as if a Bot computer exists.
  • If reading which model key is in use fails at boot, the boot check that logs a refused key does not run.
  • In the desktop app, Get notified says notifications aren't available there yet, and All set points to Tasks and each Bot's page. It knows the desktop app by Tauri's isTauri flag, not yet checked in a desktop build. A Bot's own page still offers notifications there.
  • Google may refuse a sign-in started from Connect your apps inside the desktop app's webview.

Verification

typecheck, lint, format:check and the assertion-shape guard are clean. The full app suite is 2316 pass, 0 fail, against 2206 on #913 and 2129 on upstream main. CI's startup job passes on macOS and Windows. In Chromium, WebKit and Firefox, at 1280 and 390 wide, every changed screen loads with no page error and no sideways scroll.

  • app/tests/onboarding-frame.test.tsx: two panes on a desktop, one on a phone; one line above the heading opens the list; full-strength focus rings; a picture holds still off screen. Happy-dom can't measure layout, so a browser check put every step's buttons inside the window at both sizes.
  • app/tests/onboarding-route.test.tsx: the order of steps, each named by its heading; reload, Back, a skip and a consent's return keep the place; a carried link moves on; step 2 without a computer; the approval switch saves.
  • app/tests/onboarding-steps.test.tsx: Connect starts the real connection and comes back; an empty list says why. onboarding-pictures.test.tsx, onboarding-name-step.test.tsx: inert pictures; All set.
  • app/tests/setup-checklist.test.tsx and settings-setup.test.tsx: an admin sees the variable and the page, a member a neutral line and no Settings page.
  • server/tests/setup-checklist.test.ts: Ready means it works; a member's response names no variable; old or failed answers are asked again.
  • server/tests/plugin-oauth-callback.test.ts and three more, extended: the consent's return.
  • app/tests/onboarding-name-step.test.tsx also: Continue on Get notified goes to the name and never submits it. In all three browsers the same button had become the name form's submit mid-click.

Merge notes

This and #917 both add imports to server/src/app.ts and server/src/index.ts; a trial merge conflicts on those import lines alone. #918 carries both, with both sets kept.

The last two places still drawing boring-avatars' gradient circles now draw the
orb in the colour the Bot's id picks, so a Bot looks the same in the sidebar,
the conversation header and its profile. Nothing imports boring-avatars any
more, so it leaves the app's dependencies.
…ll when asked

A Bot's orb is a block, so a group conversation's 12px faces no longer sit on
the header's text baseline and drop half out of their circles. It resizes its
canvas when the screen's pixel density changes, so dragging a window to another
screen or zooming no longer leaves it soft, and it follows the reduced-motion
setting live rather than only when it first draws. The changelog and the
stylesheet now describe what this step draws, not what later steps will.
The skins import the orb's state type from agent-orb now that ai-core is gone,
OpenBot's mark keeps its agent-orb class for them, and their tests check the
orb's colour rather than boring-avatars' SVG. The Appearance test's preferences
carry the new skin field.
AgentOrb read "100%" as 100px, so with no Bot yet the header drew a 100px orb
in a 44 or 52px slot. With a Bot, the classes that used to rescale
boring-avatars' SVG squeezed the avatar's circle to 44px below 900px wide while
the orb inside stayed 52px, clipped and off centre. The frame now picks 44 or
52px from the window's width and draws the face at that size, the selectors
that matched only the old SVG are gone here and from OpenDots' Bot rows, and
AgentOrb takes its size only in px and refuses anything else.
Every orb on screen redrew on every animation frame, uncapped, so a sidebar of
conversations kept a few dozen canvases repainting at the screen's full rate
while the app sat idle: a small orb's colour still drifted, so no two frames
were ever the same. A small orb's colour now holds still, an orb skips any
frame that would draw what its canvas already shows in the same theme and at
the same density, and a small orb draws at most about thirty frames a second.
A face at rest with the pointer still now draws only to blink.
…violet

A Bot's face was one of nine colours and nothing else, so a team of four had
about an even chance of two identical faces. In the Bots OpenBot ships with,
three were the same sky blue, and release notes wore violet, OpenBot's own
mark. A Bot is now drawn from its seed: one of the eight colours that aren't
violet, with its colour masses turned, its hue nudged and its eyes spaced by
the same hash, so two Bots in one colour still look apart. BotOrb takes the
seed rather than a colour and draws OpenBot's violet when it has none. The
OpenDots avatar test checked one colour in nine, so it passed or failed on the
hash; it now checks that the seed reaches the orb.
… Bots apart

At 28px an orb reads as the average of its colours, and several averages sat
side by side: coral beside ember, sky beside ocean, berry beside violet. The
two Bots every deployment ships, general-assistant and knowledge, drew as
coral and ember and looked alike, and sun read bronze at 72px. The eight
colours keep their names but are re-tuned to read as red, orange, yellow,
green, cyan, blue, pink and grey, with lightness keeping neighbours apart.
orbSwatch gives the colour an orb reads as when small, and a test holds every
pair, violet included, at least 0.08 apart in OKLab. The colour now takes the
hash's top byte, so the shipped Bots draw yellow, red, cyan and grey; a test
pins that. Every Bot's face changes once, before this merges.
The outline every element falls back on, and OpenMuse's header and tab focus
rings, drew the violet ring at half strength with no border beside it: about
2.4:1 on the light background and 2.8:1 in the dark, under WCAG's 3:1 for a
focus indicator. They now draw it at full strength. The visual-system test
blends each ring at the strength its class gives it over the background and a
card, in both themes, and fails under 3:1.
…hem all

Every orb blinked about every four seconds and watched the pointer forever;
only the system's reduced motion stopped it, so a list of Bots never sat
still (WCAG 2.2.2). An orb under 40px now never follows the pointer and is
still at rest: it moves only while it listens, works or lands, and holds
where it was when it comes back to rest. A Bot faces choice, kept in this
browser like Appearance, can make every orb move only while busy, the same
way, or never; reduced motion still stills every face. The small-orb tests
now check that a resting face draws nothing for ten seconds with the pointer
circling it, and new tests cover each choice and reduced motion.
…y what they do

Appearance was a select whose description read the same whichever theme was
chosen. It is now a radio group drawn as a segmented control, so all three
choices are in view, and its description says what the chosen one does. Bot
faces sits beside it the same way, with Animated, Only while working and
Still, and says when the device's reduced motion is stilling faces anyway.
Arrow keys move through the choices and choose, and Tab leaves the group.
type-nav was 18px and type-body 17px: one pixel apart, which no one sees as
a step. type-nav is now 20/24, so every step of the scale is at least 2px
from the next, and a test holds that. Nothing on this branch uses type-nav
yet; keeping both names leaves the branches built on this one working.
…ead of throwing

AgentOrb threw when its size wasn't in px, so a style slip or a cast past
the type took down the whole route around it. The type still takes only px,
but at run time a size it can't read now draws the mark at its default 192px
and warns once for that size. The test renders "100%" twice and checks the
default size and the single warning.
…tops no other

A touch that turns into a scroll ends in pointercancel, not pointerup, and
nothing listened for it, so every orb kept looking at where the finger had
been. The shared pointer now lets go on a cancel. Every orb also draws in one
shared frame, and a callback that threw ended that frame and every frame
after it, stopping every orb on the page; each one is now reported and
skipped. A test for each.
Onboarding's roster drew each Bot's face, announced by name, beside a
heading with the same name, so a screen reader said every Bot twice.
AbstractAvatar takes decorative for a face beside a heading that already
names its Bot, and the roster passes it. The test checks that a decorative
face has no role or name and that the roster uses it.
The small-orb test played ten seconds of frames, then changed the orb
to thinking at the real clock, a few milliseconds after mounting. The
thinking motion had long settled by the frames that followed, so the
test passed only when a blink, timed by how many orbs the process had
drawn before, fell in that window: it failed in the full suite and on
reruns. become now changes the state at the time the frames reached.
…n as on Retina

The orb draws under a transform scaled by the screen's density, and
shadowBlur is the one length a canvas transform never scales. So on a
1x screen the shadow under the eye-lights and the check was twice as
soft against them as on a Retina one. The draw now takes the density
and scales the blur by it.
GET /api/setup says, row by row, whether Intelligence, the model, the Bot
computer, more apps and Slack and Teams work here, from their own answers rather
than from a variable being set. POST /api/setup/check asks them again. An admin,
or somebody running OpenBot alone, is told the page or the variable that closes a
gap; everybody else is told only that it works or that their admin sets it up,
and their response carries no variable name.
Onboarding is the only page somebody who has not finished it may open, so a
consent started on its Connect your apps step returns to that step, worked or
not, for OAuth and brokered apps alike. A consent cancelled at the vendor now
returns to the screen it started on too, instead of the Connections list. The
returnTo name is still narrowed to a closed set before it is sealed or used.
A step-by-step onboarding frame that works on a desktop screen too. One DOM,
laid out by CSS: from lg up, the steps as a list (the ones reached can be
picked) with the step's heading, sentence and Continue/Back on the left, and its
picture filling the right; below lg, a dash for each step, a round Back and the
buttons pinned to the bottom. Focus moves to each new heading. A demo's clock
runs only while it is on screen in a shown tab, and holds still with reduced
motion.
Welcome types a request under OpenBot's listening orb, a computer of its own
shows the Bot's desktop with its pointer on an errand, the team shows three
Bots' orbs taking turns at work, and an approval card waits for Allow once
before it sends. Somebody running OpenBot alone also sees it on a laptop, with
the agents they have, and for their organization. Each is a picture, read by
its label, with nothing inside to click.
…the box

The step promised that a Bot asks before it sends, buys or deletes anything.
Ask before making changes starts off, so out of the box a Bot asks only where
a rule says to, and paying or changing account security is handed to the
person. The step now says that, and where to turn asking on.
With AGENT_COMPUTER_URL set and no computer running, the usual state and
always true on a first run, the row told an admin Bots can't browse the web
and to set the variables and restart, for something already set. Not checked
now says it is set up and none is running to check, and opens Computers.
Intelligence, model and computer answers were asked at boot and then served
for as long as the server ran. An Intelligence slower to start than OpenBot
read as a red required gap until somebody pressed Check again, and a key
refused days after boot still read Ready. A read now asks again, in the
background and under the shared 10-second limit, when an answer failed or is
five minutes old, so the next read has it. Reads are answered from the last
answers that are all in, so none waits on a slow fleet.
The changelog said the intro was replaced by another product's steps, and two
comments cited it, which reads as a skin being added. They now say what
OpenBot does. The changelog also said a cancelled connect returns to where it
started from anywhere; only Connect your apps changed, so it says that.
At 1280 by 800 the self-hosting step and the name step ran past the bottom
of the window, cutting Continue and Back in half; at 700 tall, four steps
did. The left pane stacked the nine-step list, the step's words and its
buttons, and the page was left to scroll.

On a desktop the frame is now one screen tall and the step list's row is
the only one that can shrink, so the step's words and its buttons always
keep their full size. When the screen is too short for all of it, the list
scrolls within itself, fades at the edge it scrolls under, and keeps the
step on screen in view. Its rows and the space under it are a little
tighter, so every step still fits whole at 1280 by 800. Only a window too
short for the words and buttons themselves scrolls the page.

The picture pane now stretches to the screen instead of being set to its
height: a set height spanning every row gave that height to the empty last
row before the list got any. It drops the phone's margin on a desktop, which
would stop it short of the bottom on the name step.
… place

Check your setup came after Connect your apps, so an empty app list on a
deployment with no app broker was never explained. The checklist now comes
first, Skip intro lands on it, and Back after a skip, a pick from the list
or a consent's return follows the path actually taken.

An empty app list now says why: nobody has added apps, and for an admin,
that more apps aren't set up and the key that turns them on.

An admin's Add apps, and a checklist link, jumped straight to the name
step and skipped the steps between. Both now carry the page to the end and
move on to the next step, and every step after says what opens once the
person gets started. Skip intro is gone from the last intro step, where it
did only what Continue does.
A member saw amber and red Needs setup pills, a Required tag and a count
of what was required, all about the deployment's infrastructure, which
only an admin can change. A member's gap is now the server's one line,
"Your admin sets this up.", with no pill, and the summary says only that
their admin sets up what isn't ready. What works still reads Ready, and an
admin keeps every pill, count and fix.

While the answer was on its way the checklist drew nothing at all. It now
says "Checking your setup…" until the rows arrive.
The checklist's fixes say "set this, then restart OpenBot", but it lived
only inside onboarding, and after that restart nothing could open it again.

Settings → Setup draws the same checklist, with Check again, for an admin
and for the one person running OpenBot on their own machine. A fix's page
opens straight from there. A member is sent to General and is never
offered the page in the sidebar. Onboarding's Check your setup step tells
the people who get the page where to find it.

The checklist moves from components/onboarding to components/setup, since
two screens draw it now.
On a desktop the nine-row step list sat above every step's heading, so the
step's words started 300 to 640 px down a laptop's screen, and four names
in the list didn't match the heading under them ("Your team of Bots" over
"Pick a Bot or make your own").

Where somebody is now takes one line beside the product's name, "Step 5
of 9", and pressing it opens the list of steps over the page, where a step
already reached can be picked. The heading starts right under that line on
every step, and opening the list moves nothing. The phone keeps its dashes.

A step's heading is now its name in the list: the wizard hands the title
to the step's intro, which draws it unless told otherwise. The name step's
heading becomes "What should we call you?", which also names it in the
list.
The name step drew its own Get started inside its form, so Back sat under
it there and beside Continue on every other step. And while the orb played
its finish, the button still read a disabled "Saving…".

The name step now hands its form to the frame, whose primary submits it,
so Back sits to the left of Get started as everywhere else and Enter in
the field still submits. On the way out it reads Saving… while the name
and the completion are saved, then All set, green with a check, while the
orb finishes. A press during the finish does nothing, and a failure is
said under the field as before.

The step moves out of Settings' profile form into onboarding's own
name-step.tsx; the form and the step share one save. The frame no longer
needs a step that draws no primary, so that option is gone.
The self-hosting step showed Talk to an engineer, Back, Continue and Skip
intro at once. Skip intro is already gone from the last intro step, where
it did what Continue does. Talk to an engineer is now a link at the end of
the step's words ("Taking it to your team?"), still opening in a new tab,
so the step's actions are Back and Continue like every other.
…e clear

A completed step's tick was hidden from assistive technology, so "Welcome
to OpenBot, button" never said it was done, and a step ahead was a plain
name with nothing saying it couldn't be opened yet. Each row now says it
in words a screen reader reads with the name: ", done" for a step passed,
", not yet available" for one ahead.

The focus rings on the step list's rows, the phone's Back and the setup
checklist's links were the ring colour at half strength, about 2.4:1
against the page. They are full strength now, clearing 3:1.
Step 2 promised every Bot "a browser and files of its own", while four
steps later the checklist could say the Bot computer needs setup.

Onboarding now reads the setup checklist from the start, and when its Bot
computer row needs setup, step 2 says what a Bot computer gives once it is
set up, that this OpenBot doesn't have one yet, and who adds one: an admin
is pointed at Check your setup, a member told their admin sets it up.
With a computer, or before the checklist answers, it reads as before.
… browser's

Get notified showed a phone's home screen while its footnote says the
notifications show only while OpenBot is open in a browser tab. Its picture
is now a browser window with the notification at its corner, named by the
site it comes from, in the same inert frame as every other picture.

The Welcome picture's composer, with its filled Send, and the approval
picture's Decline and Allow once, drawn with the app's own button styles,
looked like controls. The frame now mutes every picture's colours a
little, the drawn Send is muted, and the approval's answers are drawn as
labels in muted tones with no hover state. Nothing inside a picture can
be focused or pressed, as before.
… step

The approval step told the person to turn on Ask before making changes in
Settings without offering it. The step now has the switch itself, under
its words and outside the picture: the same preference and mutation as
Settings → Approvals, taking effect when switched. A change that fails is
said under it, and the step's sentence now sends only a Bot's own rules
to Settings.
The steps run Check your setup before Connect your apps, the approval step
offers its switch, a desktop says where somebody is in one line that opens
the list, and an admin reopens the checklist in Settings → Setup.
Opened at 1280 wide, the list cut "Each Bot has its own browser and files"
short. It is a little wider now, so every step's name, which is also its
heading, reads whole.
…he team step

The look branch checked that onboarding's roster passed decorative to
AbstractAvatar. The steps replaced that roster with the team demo, which
names each Bot in text beside a bare orb that already hides itself, so
the check now makes sure no AbstractAvatar comes back there to name a
Bot twice.
…nishes

With notifications blocked or the prompt dismissed, Continue on Get
notified went straight into the app and marked onboarding done. React
redrew the same button as the name step's submit before the browser
acted on the click, so that one click also submitted the prefilled name.
Each step now draws its own buttons, so a click only acts on its step.
…ren't there yet

OpenBot Desktop's macOS webview has no Notification object, so the step
always said "This browser can't show notifications" and pointed at a
browser's site settings. In the desktop app it now says notifications
aren't available there yet and where to look instead, asks nothing, and
offers Continue. A browser sees the step as before.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant