The criterion: Not every question carries equal weight. A vendor can ace the demo and still fail you on the three things it can't fake — where your data lives, where the math runs, and whether compliance is built in. Weight those, and the gap shows itself.
Everything runs in your browser. Nothing you enter is sent anywhere — and the questions are yours to use on anyone, FactoryOS included.
The eight questionsWeightYour current AIThe one you're evaluating
1Where does the AI actually run — and can it see only what each seat is allowed, or everything? carries weightStrong answer: Runs on hardware you control, and each person sees only what their role allows — least privilege, enforced, not promised.
Weight
Your current AI
The one you're evaluating
2What does it cost in year three, not month one — and are you renting or owning? Strong answer: A three-year, all-in number for a machine you own and depreciate — not a subscription that compounds with headcount.
Weight
Your current AI
The one you're evaluating
3Where does the arithmetic run — inside the model, or on an engine that shows its work? carries weightStrong answer: Math runs on a deterministic engine that computes exactly and shows the work with the source — not inside the model.
Weight
Your current AI
The one you're evaluating
4Does it read your documents, or map them? Strong answer: It builds a map of your documents (a knowledge graph) and follows cross-references — not just skims the top.
Weight
Your current AI
The one you're evaluating
5Where does the compliance live — in the paperwork, or in the architecture? carries weightStrong answer: Compliance is enforced by the architecture (sealed mode, unbypassable log) — not requested by a policy.
Weight
Your current AI
The one you're evaluating
6Where do the gates go before the AI acts — and who decides, you or the vendor? Strong answer: You set where a human signs off before the AI acts. The dial is yours, not the vendor's.
Weight
Your current AI
The one you're evaluating
7If you walked away tomorrow, what would you actually keep? Strong answer: You keep your data, your corrections, your workflows, your access rules — not stranded inside a wrapper.
Weight
Your current AI
The one you're evaluating
8Where are the limits — and is the vendor honest enough to hand you the limits sheet? Strong answer: It tells you what it can't do and hands you a limits sheet. Honesty over a polished demo.
Weight
Your current AI
The one you're evaluating
Your current AI——
The one you're evaluating——
The three that carry the most weight: the defaults load on the questions a glossy demo can't fake —
Q1 (where the data lives, and who can see it), Q3 (where the math runs), and
Q5 (whether compliance is enforced by the architecture, not a policy). A vendor can score well on the rest
and still fail you on these. Re-weight them higher and watch the gap.
The point was never to sell you a vendor — it was to make you the kind of buyer who can grade one. These eight are yours to use on anyone. Grade FactoryOS on all eight the same way; we built it to answer them a particular way, and whether it does is yours to check.
Bring the eight questions to your next vendor call — ours included. If a column sits at policy-only on where the data lives, where the math runs, or how compliance is enforced, that's the one to chase before you sign.