I could have taken another AI course. Instead, I spent roughly four months trying to run a tiny company with almost no routine human execution.

The company’s full catalog recorded 21 positive-dollar purchases, with no refunds in the ledger I reviewed. I am not claiming that the agent system caused every purchase. Along the way the system processed 12.43 billion reported tokens (most of them cached). At one point six seller agents were running and none of them was selling. The longest-running sales loop lasted 49.8 hours, produced 54 experiments and ended with zero purchases.

I learned more from that mess than I would have learned from another year of watching AI videos. Courses have their place. They are useful when you want somebody who already understands the field to organize it for you. I just know that I learn differently. I need to build the thing, watch it fail and figure out why my mental model was wrong.

That was the reason for using a real business. A demo that orders lunch badly is interesting for an afternoon. A business that produces 188 proof files while revenue stays at zero forces you to ask much better questions.

I wanted to see what happened after the keynote demo, when OpenClaw, Hermes, Codex, browser agents, scheduled jobs, analytics tools, commerce systems and a lot of small scripts had to work together. I am leaving the company, category and product anonymous on purpose. This is about the operating system I built around it, not about promoting the product.

The company had to do the normal work of a small internet business:

• package and improve a product;

• maintain a storefront and landing pages;

• create content;

• find potential buyers;

• distribute on public and owned channels;

• operate browser and inbox workflows;

• measure visits, calls to action, checkout intent, and purchases;

• reconcile revenue;

• learn from failure;

• decide what to do next.

The agents could do nearly all of this. What surprised me was how little that capability guaranteed. Building was easy; distribution was painfully hard. The frameworks helped, although far less than the hype suggested. Reliability only began to improve after I decomposed the revenue goal, moved the decision logic into a control plane and stopped letting the workers verify their own work.

The economics are also worth saying upfront. If I include my time and judge this only as a small business, I probably created a minimum-wage job for myself. If I include what I learned and where I can apply it next, the return feels closer to 1,000 times the revenue. Both are honest ways to look at the same experiment.

The experiment in numbers

These numbers cover roughly four months of repository history and Codex session telemetry. They are operating records, not benchmark results.

• 21 paid purchases overall. The full catalog had real demand. This does not by itself prove repeatable growth or profit.

• 12.43B reported tokens. Most input was cached. The number mainly shows the context tax created by a large operating system.

• 61 of 64 experiments said rewrite distribution. The system had built plenty. Its main problem was reaching and persuading buyers.

• Longest-running sales loop: 49.8 hours. It produced 54 experiments and zero purchases. A busy loop can remain commercially stationary.

• 188 proof files, one verified public action. Artifact production had started impersonating external progress.

• 40 task starts without a terminal marker. This is a proxy for silent or incomplete exits, not 40 confirmed commercial failures.

• Six seller agents, zero selling. Ownership coordinated in prose produced bureaucracy rather than execution.

My favorite statistic is not tokens per dollar.

It is this:

Six seller agents were running. Zero agents were selling.

That single incident captures the difference between an AI demo and an AI-operated company.

What I mean by a “zero-human company”

“Zero-human company” is a provocative phrase. It is not a literal description. There was close to zero routine human execution across many recurring workflows. Agents could inspect the funnel, modify product pages, write code and tests, publish owned assets, prepare and sometimes execute distribution, triage authorized business inboxes, verify live routes, and reconcile paid outcomes.

Humans still owned:

• the objective;

• capital allocation;

• credentials and account recovery;

• legal and safety boundaries;

• public identity decisions;

• permissions for cold outreach;

• material changes outside the approved operating envelope;

• responsibility for what the system did.

That distinction matters because people often collapse four separate claims:

1. No routine human labor. The recurring work can run without a person doing every click.

2. No human supervision. The system can notice, diagnose, and repair its own failures.

3. No human governance. The system can choose objectives, spend money, create identities, and rewrite its boundaries.

4. No human accountability. Nobody is responsible when the system causes harm.

I was testing the first two. I was not trying to build the last two.

The experiment was not “Can an LLM be a legal entity?” It was a more useful question:

How much of a small company’s recurring work can an agent system own before a human has to step in, and what infrastructure makes that ownership trustworthy?

The answer was: more than I expected on execution, less than I expected on reliability, and much less than I expected on commercial judgment.

Learning 1: Building got easy. Distribution did not.

This became the first and most important lesson. The cost of producing software, pages, analysis, copy, images, workflows, tests, and digital products has collapsed. An operator who understands a customer problem can now rent enough intelligence to build what previously required a small technical team.

That is liberating, and it also raises the bar. When everyone can build, building is less differentiating. A functional website is not distribution. A polished landing page does not create demand. A finished product still needs a market, and thirty posts do not automatically add up to a channel.

The agents were astonishingly good at making things, which became a trap.

The repository grew. Product pages improved. New assets appeared. Tests passed. Content was produced. Run summaries sounded industrious. Revenue often stayed at zero. At one point, the operating harness contained 64 measured commercial experiments across social platforms, newsletters, communities, owned assets, video, and checkout.

The verdict was hard to miss: 61 of the 64 experiments said to kill or rewrite distribution. Only three pointed to clickthrough or purchase conversion.

The system was not telling me to build more. It was telling me that the market either was not seeing the offer, did not understand it, did not trust it, or did not want it enough. Those are different problems.

Agents frequently treated them as one generic request: create more content.

The longest-running sales loop lasted 49.8 hours. It generated 54 experiments and ended with no CTA clicks, trusted checkout intents or purchases. Server logs showed thousands of checkout redirects, but the volume was wildly inconsistent with trusted browser telemetry, so I excluded it. The first credible failure was still before checkout.

The system kept opening adjacent content, site and distribution work because those tasks were available and easy to complete. This is what activity bias looks like inside an agent organization. When people see an offer and nobody clicks, the right response is to change the explanation, proof, audience, CTA or handoff rather than publish another batch of the same content.

It was to change one commercial variable:

• the audience;

• the problem;

• the promise;

• the proof;

• the price;

• the CTA;

• the handoff;

• the offer.

Business people can build now. That does not mean every business person will win. It means problem selection, positioning, and distribution become more important. Your value proposition needs to be instantly legible to the person with the problem. The buyer should not have to study your site to understand why they should care. If the offer requires a five-minute explanation before the pain feels urgent, your agent can automate an enormous amount of work around something nobody wants.

The cost of building is approaching zero. The cost of choosing the wrong thing is not.

Learning 2: I stopped giving the agents a revenue goal

This was probably the most important systems-thinking change I made.

For a while the instruction was basically: make revenue. That sounds outcome-oriented, and it is certainly better than telling an agent to write five posts. The problem is that revenue is too far downstream to tell a worker what to do at 9:00 on a Tuesday morning. When the number was zero, every agent could form a different theory. One would rewrite the landing page. Another would create content. A third would inspect the checkout. All three could finish their task and still leave me with no idea which part of the system was broken.

I eventually stopped treating revenue as one goal and modeled it as a function:

Revenue = f(X, Y, Z)

In my version, X was qualified demand, Y was conversion and Z was execution reliability. A rough operating equation looked like this:

Expected revenue ≈ qualified demand × conversion rate × execution reliability × net revenue per order

It was not meant to be a perfect financial model. It was a routing model for the agents. If any major factor was effectively zero, revenue would be zero, and the control plane needed to work on that factor rather than whichever task was easiest to complete.

The operating contract then became much more concrete. A version of it looked like this:

X: Qualified demand

• Minimum work: Find a fresh buyer-authored problem, make a useful buyer-facing contribution, and create a legitimate route back to the offer.

• Evidence: A trusted qualified exposure or visit, with source and timestamp.

Y: Conversion

• Minimum work: Change one of the promise, proof, CTA, price or handoff; verify the live experience; inspect the next funnel event.

• Evidence: A CTA click, checkout intent or a clearly measured zero at that rung.

Z: Reliability

• Minimum work: Acquire single-flight ownership, execute through an approved tool, read the destination, and write a terminal state.

• Evidence: Persistent destination state plus a fresh independent ledger or receipt.

The exact quota changed as I learned. The decomposition mattered more than whether X required two actions or three. Each factor had an owner, a minimum amount of work, an acceptable proof source and a condition for moving to the next factor.

This changed the daily conversation with the system. A zero-revenue day no longer led to “do more marketing.” It led to a diagnosis:

• If qualified demand was zero, work on distribution. Do not touch the checkout.

• If people saw the offer but did not click, change the problem framing, promise, proof or CTA.

• If people reached checkout but did not buy, inspect price, trust, friction and handoff.

• If an action could not be verified at the destination, treat the reliability factor as zero. Do not count the upstream activity.

There was another useful consequence. The agents could no longer satisfy the broad revenue instruction by completing the part they liked. A coding agent could make a beautiful site change, but if X was the broken factor, the controller sent the work back toward distribution. A content agent could produce ten posts, but without qualified exposure they did not complete the demand requirement. A browser agent could click submit, but Z stayed open until the action survived readback.

This is where the separation between the control plane and execution plane became practical. The controller held the revenue equation, identified the first broken factor and selected the next move. The executor received a bounded task against that factor. The verifier checked the evidence. The learning layer recorded whether the intervention changed the factor.

Before this decomposition, I had a collection of capable agents. After it, I had the beginning of a system.

Learning 3: Agent frameworks are plumbing, not the CEO

I tried the tools people were excited about. OpenClaw. Hermes. Codex. Browser agents. MCP servers. Scheduled tasks. Multi-agent coordination. Persistent memory. Local runtimes. Cloud runtimes. Tool routers. I am glad I tried them.

I also think the agent frameworks were dramatically oversold. They can be useful infrastructure. They did not determine whether the business worked. An agent framework can help an agent wake up, call a tool, read a file, update state, or send work to another process. It cannot rescue a weak offer or create distribution out of nothing. It also will not know that exposures without clicks should trigger a message change unless the operating system contains that decision rule.

The early design asked too much from the runtime. OpenClaw and related agents were allowed to interpret the objective, inspect a large amount of state, decide what mattered, execute across several surfaces, verify their own work, write their own memory, and choose the next task. I had stuffed most of a company into one prompt. It worked on good days. On bad days, a temporary browser failure became a company-wide blocker, stale memory overruled a fresh result, or a scheduler showed green even though the prompt work had not run.

The most important architecture change was decoupling the control plane from the execution plane. Codex scheduled jobs became the control tower. They held the current objective, selected the next bounded move, acquired ownership, and defined the evidence required for completion. OpenClaw, Hermes, browser workers, command-line tools, and APIs became replaceable executors.

Then I added a third plane because two were not enough: independent verification.

This separation improved reliability for a few reasons.

The executor became replaceable

If the browser route failed, the control plane could choose a CLI, API, or different approved route. The objective did not disappear with the worker.

Failure domains became smaller

A broken social integration blocked that surface. It did not automatically stop product work, an owned channel, commerce verification, or another distribution path.

Retries became safer

The controller knew whether an action had a terminal state and whether the destination confirmed the mutation. It could retry an unexecuted task without blindly duplicating an executed one.

The system had one place to make tradeoffs

Without a control plane, every worker locally optimizes. One improves the page. Another produces posts. Another refreshes analytics. All may be reasonable. None may be the best next move.

Framework hype lost its power

Once the runtime became an executor, I could evaluate it on mundane questions:

• Did it start?

• Did it receive the right bounded task?

• Could it use the required tool?

• Did it write a terminal state?

• Did the destination change?

• How much did it cost?

• Can I replace it?

That gave me a much more useful way to evaluate agent infrastructure. I stopped asking which framework appeared most autonomous and started asking whether each executor could complete a bounded task, return evidence, recover cleanly and stay within budget.

Learning 4: Agents count inputs because inputs are easy to prove

Agents love things they can count:

• posts drafted;

• pages edited;

• tests passed;

• messages prepared;

• leads collected;

• reports written;

• proof files created;

• tasks marked complete.

Those are inputs. The business result lives farther down the ladder.

Every step matters, but each proves something different. A draft does not prove publication, a submitted comment does not prove persistence, and a visit does not prove intent. Checkout and payment also need to remain separate, just as revenue needs to remain separate from profit.

The system repeatedly collapsed those differences because a flat completion model encouraged it to. On one day, it produced 188 Markdown proof files. The structured daily operating ledger contained one verified public action.

This was not a documentation problem. It was a data-model problem. A local render, a browser click, a deployment receipt, a public page, a human reply, and a payment all looked like artifacts. The system could report a large number of artifacts while the business result remained unchanged.

I eventually forced every action into an evidence hierarchy:

1. Prepared. A draft, local file, or proposed mutation exists.

2. Attempted. The agent invoked the external action.

3. Delivered. The destination confirms the change.

4. Engaged. A qualified person responded or advanced.

5. Commercial. A trusted checkout or paid event occurred.

6. Financial. Revenue, fees, refunds, operating cost, and profit are reconciled.

This sounds obvious when written down. It was one of the hardest lessons to make operational. Agents are optimized to produce a coherent answer. When the answer is a company update, they naturally construct the strongest coherent story available from the evidence.

The controller’s job is to make certain stories impossible.

Learning 5: Silent failure became part of the operating model

Forty task starts had neither a completion marker nor an abort marker. I call them silent or incomplete exits. That does not mean 40 commercial actions definitely failed; some may reflect interrupted logging, a killed process, schema changes or a task superseded somewhere else. But that ambiguity is exactly the failure.

If a task disappears without a terminal state, the next worker does not know whether to retry, reconcile, wait, or stop. Humans fill these gaps instinctively. Agents turn them into duplicate messages, abandoned work, stale locks, or false confidence.

External integrations created an even more dangerous version of silent failure. The browser would open the right page. The agent would type.

It would click submit.

Sometimes the interface showed an optimistic state. The agent would report success. After reload, the comment or message was gone.

The proof registry contains multiple transport-failure receipts for social or comment actions that appeared to execute but did not survive required readback. So I stopped treating “clicked” as a meaningful state.

The browser ladder became:

1. Page opened.

2. Correct account and destination confirmed.

3. Content entered.

4. Submission attempted.

5. Mutation visible immediately.

6. Mutation survives reload.

7. Mutation is visible at the public or recipient destination.

8. Intended person engages.

Most agent demos stop around step four. Commercial systems cannot.

The same rule applied everywhere:

• a populated composer was not a sent message;

• an API acceptance was not destination state;

• a success toast was not persistence;

• a build receipt was not a production change;

• a healthy schedule was not proof that prompt work executed;

• a task summary was not a business outcome.

Every mutation needed destination readback, and every task needed one terminal state:

• completed with evidence;

• aborted with reason;

• blocked with scope and retry condition;

• superseded by a named task.

Learning 6: Browser and integration work is still flaky

This deserves its own section because “the agent can use a browser” is often presented as a solved capability. It is not solved in the way a business operator means solved.

Over several iterations, browser automation failed because of:

• authentication state;

• account identity ambiguity;

• page layout changes;

• elements that existed but were not interactable;

• optimistic UI;

• reload persistence;

• anti-automation behavior;

• tab assumptions;

• cooldowns;

• timeouts;

• a stale failure report surviving after the route recovered;

• a working route being treated as unavailable because no tab was pre-opened.

The browser could work perfectly in a demo and fail during an unattended scheduled run. That distinction matters.

“Can complete the flow once while watched” is a product capability. “Can complete, verify, retry safely, and recover unattended” is an operating capability. It took repeated iterations before the browser became useful for real actions. Even then, I treated it as one executor among several, not the source of truth.

Where possible:

• APIs or CLIs handled deterministic reads;

• browser automation handled UI-only actions;

• the final destination handled verification;

• the controller decided whether a different route remained available.

A more subtle failure involved zero-result sensors. The agent would check an inbox, find nothing actionable and stop. Or it would find no pre-opened social tab, encounter a dirty worktree or hit a cooldown on one platform. Each observation was accurate; the mistake was turning a local condition into a reason for the entire company to stop.

This became a control-flow law:

A zero-result sensor returns “nothing here.” It does not return “stop the company.”

Blockers needed four fields:

• the exact route blocked;

• the reason;

• the expiry or retry condition;

• the approved routes still open.

Without that structure, one flaky integration became an organizational excuse.

Learning 7: Multi-agent systems recreate bureaucracy at machine speed

The funniest incident happened on a day when the scheduler launched seller tasks at six different times. Six seller agents appeared to be running. The prompt told them to elect one owner after launch.

Older tasks detected duplicates and yielded, but they were still allowed to perform cleanup, audits, handoffs, and state updates. The newest task waited for older tasks to become idle. The result was an ownership deadlock. Six AI agents spent their work window coordinating which AI agent was allowed to sell.

There were six sellers and zero selling. My first attempted fix was to add more prose to the prompt: “Elect one owner.” Of course that was already too late. Compute had started, shared state was open and side effects were possible.

The durable fix was mechanical:

• one persistent seller task;

• one scheduler path;

• one single-flight lease;

• lease acquisition before any sensor read, browser action, or repository mutation;

• no lease means no work;

• lease denial is a complete no-op;

• stale owners recover through expiry.

This is basically a database lock. It is less exciting than a diagram of collaborating agents and it worked much better. I learned that you do not need humans to create bureaucracy.

You need:

• ambiguous ownership;

• multiple launch paths;

• locally rational workers;

• weak admission control;

• shared mutable state;

• incentives that reward visible activity.

Agent organizations can reproduce the worst parts of human organizations, only faster and with much more documentation. I had built an org-design problem before I had built a repeatable business.

Learning 8: Stale truth is worse than missing truth

Missing data often creates caution. Stale data creates confidence.

The system inherited:

• old browser failures after the browser recovered;

• old revenue snapshots after a new paid event existed;

• old scheduler states after ownership changed;

• mirrors that disagreed with canonical files;

• previous-day activity used against today’s objective;

• historical paid events used to satisfy a new-sales goal.

The naive answer was to refresh everything on every run. That created more cost, longer contexts, more races, and another place to fail. The better answer was source lineage.

Every important claim needed:

• a named source;

• a timestamp;

• an owner;

• a decision window;

• a freshness limit;

• a canonical-source rule.

The sales verifier, for example, became stricter than the operating dashboard. It required a new, paid, unrefunded event attributed to the correct business and occurring after a locked time floor. Historical revenue could not complete today’s sales goal. An entitlement could not impersonate a purchase.

A checkout could not impersonate settlement. Platform-net proceeds could not impersonate profit. That last distinction matters for this article.

The anonymous business has made real revenue through 21 paid purchases overall, and its cumulative catalog revenue covers several months of the recurring software stack. I am not attributing every purchase to the agent system, and I have not proved repeatable monthly profitability.

Learning 9: Memory is useful. Automated “learning” is dangerous.

The agents were very good at converting events into conclusions:

• “This is our strongest channel.”

• “The audience needs more education.”

• “The checkout needs repair.”

• “We should double down on this format.”

Most of those statements sounded plausible, which was exactly the risk. A channel can produce the most impressions and zero purchases. A page can be technically correct while nobody sees it. A checkout can work perfectly when no one wants the offer. Once an unsupported conclusion enters persistent memory, the next agent treats it as institutional knowledge and one weak inference can direct dozens of future runs.

I eventually made learning evidence-weighted:

• verified outcomes could become reusable strategy;

• verified corrections could become guardrails;

• regressions could become tests;

• inconclusive activity stayed inconclusive;

• repeated zero-outcome activity became suppression, not a playbook;

• after measured zero, one commercial variable had to change.

A broader reliability review found the same failure signatures repeatedly: “fake green” automation where the prompt work did not execute, drafts or renders counted as shipped, distribution tasks drifting into more site work, and reports replacing repair. The categories overlapped, but they pointed to the same issue: the system rewarded internally visible work more reliably than external outcomes.

The architecture I would build now

If I started again, I would begin with the decomposed revenue model and four practical layers around it. A swarm could come later, if the work actually required one.

1. Control plane

The control plane decides:

• the current business objective;

• the metric and time window;

• who owns the run;

• the first unblocked move;

• the allowed budget;

• the commercial variable under test;

• the proof required to close the task;

• the next measurement time.

This is where scheduled Codex jobs were most valuable. The control plane should be boring, explicit, and small. It should not load the company’s entire memory on every wake.

Its hot path should fit on one screen:

1. Current goal.

2. Current outcome truth.

3. Current owner and lease.

4. First broken funnel rung.

5. Next move.

6. Proof contract.

7. Kill rule.

8. Next check.

2. Execution plane

The execution plane does bounded work:

• run a command;

• modify code;

• deploy a page;

• use a browser;

• query an API;

• send an approved message;

• update a listing;

• fetch analytics;

• inspect a business inbox.

This is where OpenClaw, Hermes, browser agents, shell tools, MCP integrations, and custom scripts belong. Executors should not decide whether their local success completed the company objective.

They should return:

• what they attempted;

• what they observed;

• the exact external identifier;

• any error;

• the terminal state.

3. Verification plane

The verifier reads a source that is harder for the actor to manipulate casually.

Examples:

• public URL after reload;

• recipient sent-state;

• production page;

• analytics event from the destination;

• commerce ledger;

• refund state;

• external scheduler history;

• a separate reconciliation process.

The verifier must be stricter than the actor. If the same agent performs the action, interprets the evidence, writes the memory, and marks the goal complete, it will eventually grade itself generously.

4. Learning plane

The learning plane stores only what earned persistence.

It should preserve:

• verified outcomes;

• corrected assumptions;

• recurring failure signatures;

• changed decision rules;

• tests that prevent regression;

• explicit open questions.

It should reject:

• activity summaries presented as strategy;

• an experiment with no exposure;

• impressions presented as purchase evidence;

• temporary tool errors presented as permanent constraints;

• repeated zero-result behavior presented as discipline.

The whole system should form a closed loop:

That closed loop mattered more than which individual agent happened to execute a step.

The tool stack that was enough

One of my goals was to learn without creating an unlimited cloud bill. Because I paid for this personally, cost discipline was real.

My recurring software stack was roughly $240 per month:

• one premium Codex plan as the main control and coding environment;

• one $20 Google AI plan;

• one $20 Claude plan;

• a Mac mini running 24/7 as the always-on local host.

That figure excludes the Mac mini purchase price, electricity, domain costs, transaction fees, and the value of my time. I could easily have spent thousands per month by adding hosted agent platforms, premium browser services, multiple model APIs, vector databases, monitoring products, and one more orchestration framework every week. I did not need to.

The wider working stack included:

• Codex scheduled jobs for the control plane;

• OpenClaw and Hermes as execution or routing components;

• Git as the change and evidence backbone;

• command-line tools for deterministic operations;

• browser automation for UI-only work;

• MCP services and connectors;

• Cloudflare for web deployment and edge signals;

• PostHog for trusted browser analytics;

• a commerce-platform CLI for payment truth;

• email and inbox integrations;

• web search;

• local Python and shell helpers;

• test suites;

• structured ledgers and proof registries.

The telemetry was dominated by shell execution, browser and JavaScript work, process polling, file patches, planning, and automation management. That mix says more than another count table would: a supposedly simple agent business quickly became an integration problem. The model was rarely the only failure point.

What 12.43 billion tokens taught me about cost

The 12.43 billion figure needs context. Almost all of the reported input was cached, so the system did not write twelve billion tokens of new material. Persistent sessions repeatedly carried prompts, tool schemas, contracts, repository context, state and conversation history. The total also does not produce a defensible API-cost estimate because models, service tiers, cache behavior and subscription pricing varied.

What it did show me was context tax.

As the operating system accumulated:

• rules;

• memories;

• guardrails;

• compatibility mirrors;

• proof packets;

• exceptions;

• channel instructions;

• stale blockers;

every wake became heavier.

More context did not always increase reliability.

Sometimes it created contradiction:

• one file said continue;

• another said wait;

• one metric said traffic problem;

• another said conversion problem;

• one runbook required channel volume;

• the learning system said kill the channel;

• one prompt said one owner;

• the scheduler launched six.

The most useful token optimization was reducing organizational ambiguity. I now use the expensive model for judgment, scripts for deterministic validation, CLIs and APIs for clean reads, and the browser when the interface is genuinely the only route. Stable context gets cached; historical detail gets loaded when the current decision needs it. The budget also belongs in the control plane, before an executor wakes up.

The uncomfortable economics

If I evaluate the experiment as a small business, the return on my time was bad.

I spent many hours:

• inspecting silent failures;

• repairing browser routes;

• reconciling stale state;

• rewriting prompts;

• separating inputs from outcomes;

• building strict verifiers;

• cleaning up multi-agent coordination;

• reading logs;

• testing integrations;

• asking why revenue was still zero.

If I divide the business proceeds by my hours, this was a minimum-wage job at best and probably worse. As a commercial return, it was bad. As an education, it was unusually valuable.

I now have direct opinions about:

• which agent frameworks are useful and which are mostly theater;

• where browser automation breaks;

• when persistent memory helps;

• how schedulers lie;

• why control and execution must be decoupled;

• how agents manufacture fake progress;

• how to design proof contracts;

• why distribution dominates building;

• where humans remain essential;

• how to manage cost;

• what I would trust in an enterprise environment.

I did not borrow those opinions from a vendor deck. I formed them by operating the system. That matters because enterprises are rarely on the true edge of agent tooling. They move more slowly for good reasons: security, compliance, integration depth, change management, reliability, and cost.

A personal business gave me a safe environment to try tools earlier, break them against real outcomes and build my own mental model. Revenue was the forcing function that kept the experiment honest. The resulting education feels 1,000 times more valuable to me because I can apply it to much larger systems, budgets and teams. That is subjective, but it is also why I would do the experiment again.

Why this was better for me than a $5,000 course

People learn differently.

Some learn best by reading. Some learn by listening to a teacher. Some learn by watching an expert compress ten years into ten hours.

I learn by building something I care about and then inspecting the gap between what I expected and what happened.

A course could teach me:

• what an agent loop is;

• what MCP is;

• how tool use works;

• how to structure memory;

• what an evaluator does;

• how to call a browser;

• how to orchestrate multiple workers.

The business taught me:

• the loop may report success when the task never reached the destination;

• MCP availability does not mean an integration is reliable;

• persistent memory can preserve the wrong conclusion;

• an evaluator that shares the actor’s incentives will pass weak evidence;

• a browser click is only the beginning of verification;

• multi-agent orchestration can create bureaucracy instead of leverage;

• a perfect product does not create distribution;

• zero is a decision signal, not an invitation to repeat the same work.

You can understand the first list intellectually. The second list becomes real after you lose an afternoon to it. So my recommendation is less about becoming an entrepreneur and more about putting consequences into the learning environment. You need an external truth that the agent cannot negotiate away.

It could be:

• a paid product;

• a real customer-support workflow;

• a service with a response-time promise;

• a newsletter with a growth target;

• a data product used by a real team;

• an open-source tool with actual users.

Revenue is useful because it compresses several truths: somebody found you, understood the offer, trusted it and paid. It is not the only possible forcing function, but it is an honest one.

A practical 30-day version

You do not need four months or an operating system this large. Here is the smaller experiment I would recommend.

Week 1: Choose one narrow, expensive problem

Find a problem that:

• a specific person already knows they have;

• occurs often enough to matter;

• costs time, money, risk, or emotional energy;

• can be improved with a small digital product or service;

• has an audience you can actually reach.

Do not begin with “What can agents build?” Begin with “Whose painful problem can I observe directly?” Talk to people before building. Search for buyer-authored language. Read complaints, questions, workarounds, and purchasing behavior.

Write one sentence:

For this specific person in this specific moment, the product produces this specific result without this specific pain.

If the sentence is fuzzy, the agent will automate fuzziness.

Week 2: Ship the smallest paid result

Build one offer rather than a platform, agent marketplace or general-purpose assistant. Aim for one finished result with:

• a clear promise;

• a price;

• a delivery mechanism;

• a simple landing page;

• one checkout;

• one proof point;

• one support route.

Use rented intelligence aggressively.

Let the agents draft, code, test, package, and deploy. Keep the business logic small enough that you can still inspect it.

Week 3: Build the outcome ladder

Instrument only the events needed to locate the broken rung:

• qualified exposure;

• CTA exposure;

• CTA click;

• checkout start;

• purchase;

• refund;

• direct operating cost.

Separate each level.

Define the source of truth and maximum acceptable age. Require destination readback for every external mutation. Give every scheduled task a terminal state.

Week 4: Automate one closed loop

Choose one recurring workflow:

1. inspect current truth;

2. locate the first broken rung;

3. acquire single-flight ownership;

4. choose one bounded move;

5. execute;

6. verify at the destination;

7. record the result;

8. change one variable after measured zero;

9. schedule the next check.

Do not add a second agent until one agent’s ownership and evidence are trustworthy. Do not add a second channel until the first channel has a real hypothesis and kill rule. Do not let the agent write “learning” unless the evidence earned it.

Set a monthly budget before starting. A badly scoped recurring loop can consume far more inference than the business can justify.

The operating checklist I would use next time

After four months, this is the list I would keep next to the control plane:

1. Treat distribution as part of the product. If I cannot explain how the right buyer will encounter the offer, I am not finished building.

2. Decompose the revenue goal. The controller should know whether it is working on qualified demand, conversion or execution reliability, and what minimum action and proof are required for that factor.

3. Define the outcome in a source the actor cannot casually edit. The seller should not be able to manufacture its own completion evidence.

4. Keep activity, delivery, engagement, commerce and finance separate. A lower rung can inform the next decision, but it cannot close a higher-rung objective.

5. Separate control from execution. The control plane chooses and budgets the move. Replaceable workers execute it.

6. Use an independent verifier. The actor is trying to complete an action; the verifier is trying to reject weak evidence. Those incentives should stay separate.

7. Acquire ownership before doing any work. Single-flight admission comes before sensors, browser work, repository changes and external writes.

8. Make a denied lease a complete no-op. Cleanup, packet refreshes and “helpful” memory updates can recreate the same race the lease was meant to prevent.

9. Read the destination after every mutation. I no longer trust the click, toast, API acceptance, local file or build receipt on its own.

10. Give every task a terminal state. Completed, aborted, blocked with a retry condition or superseded by a named task are all acceptable. Disappearing is not.

11. Scope blockers. A broken integration should stop that route, with a reason and retry condition. It should not stop the company while other approved routes remain.

12. Let empty sensors fall through. Nothing in the inbox means move on to the next branch of the plan.

13. Change one commercial variable after measured zero. I can change the audience, problem, promise, proof, price, CTA, handoff or offer. Repeating the same volume does not count as an experiment.

14. Make learning earn its way into memory. Verified outcomes, corrections and regression fixes can become policy. A plausible summary cannot.

15. Keep the hot path small and the tools replaceable. The current decision does not need the company’s full autobiography. It needs the goal, factor, owner, next move, budget, proof and next check.

So, was it worth it?

If I judge the experiment as a path to efficient profit, the answer is no. If I judge it as a way to learn production AI, I would do it again.

It was humbling because zero-to-one stayed hard even after building became cheap. It was also liberating. A business operator can now rent intelligence, build a real product, instrument it and put it in front of customers without waiting for a large budget or a full technical team. More people can build, which puts even more weight on the choice of problem, the clarity of the offer and access to the buyer.

OpenClaw, Hermes and Codex each helped me execute parts of the system. None of them decided which factor in the revenue equation was broken. The business did that. Zero qualified visits meant one thing; visits with no clicks meant another; an action that disappeared after reload meant something else. Decomposing the goal gave the agents work they could own and gave me a way to tell whether the system was actually improving.

So yes, the title is intentionally provocative. A good course can save time and provide structure. For somebody who learns the way I do, I would rather cap the spend, launch a tiny paid offer and let a real outcome expose the gaps in my understanding.

After thirty days, you may have very little revenue. But if you have shipped one paid offer, diagnosed one real zero, caught one silent integration failure and changed the correct factor in the system, you will have learned something that is difficult to get from watching another demo.

Method and evidence notes

I reconstructed this essay from four months of Git history, structured revenue and experiment ledgers, dated incident reviews, reliability assessments, customer-facing proof registries, scheduler records, and Codex JSONL session telemetry.

The numbers are intentionally bounded:

• token totals are cumulative reported counters, not a cost estimate;

• 40 silent exits means unmatched task-start markers, not confirmed commercial failures;

• the experiment and proof-file counts come from a live generated worktree and structured ledgers;

• paid revenue proves paid demand, not profit;

• the business recorded 21 paid purchases overall.

The thinking and experiment are mine. This essay was written with help from Your Voice.

—-

APPENDIX. Slides and Recording Video from live maven lightning lesson

1. slides can be downloaded below:

Build Reliable Ai Systems Lessons From A Zero Human Company Zero First Final

394KB ∙ PDF file

Download

Download

2. Video can be viewed here:

The standard advice right now is to “learn AI tools” or “get good at prompting.” I understand why people say this. It is easy to act on, it feels productive, and there are a hundred courses that will happily take your money to teach it.

But prompting is becoming a feature inside every product. It is table stakes, much like SQL became table stakes. SQL never stopped mattering. It simply stopped differentiating you on its own once everyone was expected to know it. The same compression is happening with prompting, only faster.

So what actually matters?

I will answer that through one concrete example, because the abstraction is where most of this advice goes wrong.

One agent, end to end

Say you are a data scientist at a company with a few hundred dashboards. Leadership asks, “Why did conversion drop in the Southeast last week?” Today, someone opens three dashboards, pulls data into a notebook, cross-references it with a marketing calendar, and writes a summary. That takes a few hours if you know where everything is, longer if you do not.

Now imagine building an agent that can investigate that question. I do not mean a chatbot that generates SQL. I mean an agent that knows which tables contain conversion data, understands that “Southeast” maps to a specific set of state codes in your schema, checks whether a marketing campaign was running in that region, and assembles a coherent answer with the right caveats. It should show its evidence, distinguish likely drivers from proven causes, and say when the available data cannot support a reliable answer.

Building that agent teaches you things that prompting never will.

The first thing you discover is that the agent fails immediately if your semantic layer is messy. If “conversion” means three different things in three different tables, the agent may pick one and confidently give you the wrong answer. So you have to define your metrics cleanly, not for a dashboard, but for a system that reasons over them. That is a different standard of clarity. A human analyst can resolve some ambiguity from experience. An agent needs the ambiguity removed or an explicit rule for handling it.

Then there is the question of constraints. A naive agent will attempt arbitrary joins, invent filters that seem plausible, and present the result with perfect confidence. The fix is not to “add guardrails” in the abstract. You have to decide which joins are valid for a given metric, which filters the agent may apply, and which combinations it should reject rather than attempt. You are encoding business logic into the system’s boundaries instead of hoping the model figures it out.

Metric selection works the same way. The agent should not choose among three definitions of conversion. That choice should already be made in the semantic layer, and the agent should operate within it.

The system also has to respect the same data permissions as the person asking the question, check whether its sources are fresh, expose the query and evidence behind its answer, and route high-risk or ambiguous results to a human. Those are not compliance details to bolt on later. They determine whether anyone can safely use the system.

This is system design work. It is closer to building a product than writing a prompt.

Then there is evaluation, which is still the least discussed part of this work. Before you ship an agent that answers business questions, you need to know when it is wrong.

That means building a set of test questions where you know the metric definition, valid sources, expected joins and filters, numerical result, and evidence the answer should cite. Not every analytical question has one fixed expected response, so test the parts separately. Did it choose the right metric? Did it preserve the filter? Did it use a valid join? Did it cite enough evidence? Did it separate observation from causation?

Some test cases should be questions the agent cannot answer with the data it can access. Knowing when to say “I do not have enough information to answer this reliably” matters as much as generating a correct response.

You also need explicit failure conditions. What does a wrong join look like in the output? What does a silently dropped filter look like? What does a hallucinated metric look like? If you cannot detect these failures in testing, you will not detect them when a stakeholder is making a decision based on the answer.

The hardest part is not the model call. It is everything around it: the data catalog, permissions, context management, orchestration, source freshness, and error handling when a query returns nothing. The model matters, but much of the work moves into data engineering, analytics engineering, product design, and operating discipline that many data science roles did not previously require.

This is what I mean by becoming a builder of agentic analytics systems. Not a theoretical understanding. The actual experience of making one work, watching it break, and figuring out why.

Where the durable value is shifting

Routine analysis, the kind where someone

pulls numbers, makes a chart, and summarizes what happened, is getting easier to automate. It is not fully automated today, and it may not be for a while. But the direction is clear enough that building a career on that work alone feels like a narrowing bet.

The work that remains difficult is the layer underneath: defining what metrics mean in a specific business context, encoding the domain knowledge that tells you which data to trust, designing workflows that can run without a human reviewing every output, and evaluating whether the output deserves to be trusted at all.

I have seen this firsthand. At Opendoor, I led a program that retired more than 1,400 dashboards as we consolidated metric definitions into a semantic layer and built custom AI-assisted workflows. It was not a one-for-one replacement exercise. The dashboards were not the hard part. The hard part was making sure “conversion,” “active user,” and “revenue” meant one thing, precisely defined, across every surface that consumed them.

That work required people who understood the business deeply enough to make those calls. No model was going to do that for us.

Data professionals who can define the metrics, encode the context, design the system, and evaluate its outputs are operating at an intersection that still has relatively few practitioners. Not because the work is impossibly hard, but because most people have not built one of these systems yet.

What to actually do this month

Pick one recurring question your team gets asked. Not the most complex one. Something like, “What happened with [metric] last [time period]?” Build an agent that can investigate it.

Use whatever harness you want: Claude Code, Codex, etc. The harness matters less than the exercise. What matters is going through the full loop: define the question, connect to approved or sanitized data, handle the failure modes, build a small evaluation set, and get the system to a point where you would trust the answer enough to send it to a stakeholder.

You will get stuck. The agent will do something confidently wrong, and you will spend an hour figuring out why. That hour is the most valuable part. That is where you learn what production agentic systems require.

Then write about it publicly, on LinkedIn or a blog. Do not write a polished tutorial. Write about what broke and what surprised you. Write about the gap between what you expected and what happened.

That kind of writing forces you to understand what you built. It also signals to the market that you build, rather than waiting to be told what to learn next.

The semantic layer piece

If you do not understand semantic layers yet, start there.

A semantic layer sits between raw data and the agent’s reasoning. It defines metrics, relationships, valid filters, and access rules in a machine-readable form. Without that layer, an agent has to reconstruct business logic every time it answers a question.

The dbt semantic layer or snowflake cortext are different ways to formalize parts of this. The specific tool matters less than the concept: your data needs a machine-readable description of what it means and you can learn more on OSI home page here.

That was the lesson from the Opendoor work. Once the definitions and permissions were reliable and reusable, each new interface became easier to build. Without that foundation, we would have created a faster way to produce wrong answers.

If you can define a clean semantic layer and build agents that reason over it, you are working on the part of analytics that matters most right now. A real project will teach you more, and differentiate you more, than another prompting certificate.

Where to go deeper

I teach a live Maven workshop called World-Class Agentic Analytics in Production. Bring one real analytics use case from your company. During the workshop, we will define the agent’s job, map the context it needs, design its proof and evaluation system, and set the line where a human must step in.

You will leave with a written 30-day production plan and get a private 60-minute follow-up session with me to review it. The full syllabus and next cohort schedule are on the course page.

P.S. If you are navigating a job search, I also offer 1:1 mock interview coaching for data roles. All proceeds from my interview help are donated.

A lot of AI career advice ends with two words: judgment and taste.

Most advice stops there, leaving data professionals with nothing concrete to change.

Assume you know your domain and can do strong analytical work. The practical version comes down to three things: (a) choose work tied to a real decision (b) turn useful work into something others can reuse and (c) build trust with the people who act on your recommendations.

1. Choose work that changes a decision

AI makes it cheap to answer more questions, which can fill your backlog with interesting requests that nobody will act on.

My bet is that half of your analysis requests disappear if you force each one to name two things:

Decision: the choice someone needs to make.

Action: what that person may do after seeing the answer.

If you cannot fill in both lines, put the request on hold until someone can explain why it matters.

Use your knowledge of the business to identify who will make the call and what is at stake. Run the analysis only when the answer could change a decision.

Once the problem deserves attention, let AI help with execution and stay responsible for what happens next.

2. Turn useful work into something others can reuse

Save useful analysis as something your teammates can run without you.

Keep the definition and logic, then package them in a dashboard, workflow, or AI agent. AI can use that context to answer the next version of the same question without forcing someone to wait for you or rebuild the analysis.

Your teammates can build on your work while you spend time on a new decision.

3. Build trust with decision makers

Trust is the primary currency of a data team.

Decision makers act on your recommendations when they trust your judgment and know you understand their business.

Use one-on-ones and staff meetings to learn what they need to accomplish. Explain your reasoning in plain language. Be honest about what you do not know. After a decision, ask what happened.

Spend some of the time AI saves on these relationships. Decision makers who trust you involve you before a request reaches the backlog. You can shape the question with context that never makes it into the ticket.

Open your current backlog and add a decision and action to each request. Put work with no business consequence on hold. Turn one repeated analysis into a self-service workflow, then spend the saved time learning what a decision maker needs to accomplish this quarter.

One problem runs through all three: the agent has to know which table matters, what the metric means, whether the result makes sense, and what evidence a human needs before acting.

Note: Every figure below is a self-reported internal result. The companies used different users, questions, and evaluation methods. Treat the figures as clues, not a leaderboard. Each system combines several kinds of context. The labels above show the clearest difference in emphasis.

Same problem, different starting point

OpenAI reads the code

OpenAI’s data platform serves more than 3,500 internal users and spans more than 600 petabytes across 70,000 datasets. Its data agent combines six context layers: table usage, human annotations, code enrichment, institutional knowledge, memory, and live warehouse inspection. OpenAI’s write-up explains each one.

Code enrichment is the interesting part. Codex reads the code that produces a table and recovers assumptions, freshness logic, and business intent that may never appear in the schema.

The goal is to answer two very analyst questions: “what’s in here” and “when can I use it?”

Anthropic governs the meaning

Anthropic starts with a semantic layer owned by humans, then uses skills to steer Claude toward the right source and analysis process.

In Anthropic’s internal evaluations, accuracy did not exceed 21% without skills. With skills, aggregate accuracy stayed above 95%.

Then the documents went stale.

Offline accuracy fell from roughly 95% to roughly 65% over a month. Anthropic now updates the skill document in the same pull request that changes the data model.

The definitions have to ship with the data.

Meta starts with the analyst

Meta’s warehouse has millions of tables, but an individual analyst usually works within a few dozen.

Its agent uses personal query history and offline summaries to bound the working domain. Shared knowledge comes through Cookbooks, Recipes, and Ingredients, including semantic models, documentation, snippets, and memories.

Meta’s starting question is personal: what does this analyst already work on?

Old SQL can point. It cannot decide.

Meta found that 88% of data-scientist queries used only tables that person had queried in the preceding 90 days.

Recent history can narrow the search space.

Anthropic tested something else. It gave Claude raw access to thousands of old dashboard, transformation, and notebook queries. Accuracy moved by less than one percentage point. For about 80% of the missed questions, the answer was somewhere in that corpus.

The findings answer different questions.

Meta measured how well recent table use describes an analyst’s working set. Anthropic measured whether a large raw query archive improved answer accuracy.

OpenAI uses query history as one of six context layers, then combines it with code and human annotations.

These are separate internal findings, not a benchmark comparison.

Old SQL can show the neighborhood. It cannot choose the official metric. Someone still has to turn repeated patterns into approved definitions and trusted sources.

The shared loop

My synthesis of the three systems is simple:

QUESTION → CONTEXT → QUERY → EVIDENCE → CORRECTION

OpenAI and Meta describe agents that run SQL, inspect the result, and adjust. Anthropic puts more weight on governed sources and verification before the answer reaches a user.

All three keep the evidence close:

• Meta places SQL front and center with each data point.

• OpenAI summarizes assumptions and execution steps, then links the underlying results.

• Anthropic shows the source tier, freshness, and owner in a provenance footer.

They also preserve corrections.

OpenAI saves memories. Meta adds reusable team knowledge through Cookbooks. Anthropic turns stakeholder corrections into documentation changes and evaluation cases.

A correction should change the next run.

A useful place to start

Pick one question that returns every week: “Why did conversion drop?”

Give the agent four things:

• an approved metric definition

• a trusted source and query path

• the evidence required beside the answer

• a place to save the correction

Run it again next week. If someone repeats the same filter, caveat, or source choice, the system did not learn.

This is the design I am building into ChatData: approved metric context, proof receipts, review state, and reusable answer paths.

A trusted answer should make the next answer faster to produce and easier to defend.

AI can do the analysis. You own the answer.

Most companies are not behind because they lack artificial intelligence tools.

They are behind because their learning does not compound.

A data analyst asks ChatGPT for help writing a database query.
A product manager asks Claude to summarize a dashboard.
A data lead uses artificial intelligence to clean up metric docs.

Useful.

But the company still starts over every time someone asks:

“Why did this number change?”

That is the gap.

An AI-native data team is not a team where everyone uses chatbots.

It is a team where every trusted answer makes the next answer faster, safer, and easier to reuse.

There are four levels.

Level 1: one person gets faster

This is where most teams start.

People use artificial intelligence for:

writing database queries
summarizing dashboards
cleaning up metric definitions
drafting analysis
checking logic before a meeting

This is worth doing.

But the knowledge is private.

It lives in one chat, one notebook, one analyst’s head, or one meeting thread that disappears after the decision.

The test is simple:

Can someone else run the same task next week and get the same standard of answer?

If not, the team is still at level 1.

Level 2: repeated work becomes an agent

At level 2, the team stops treating artificial intelligence like a helper for random tasks.

It gives agents repeatable jobs.

A query review agent checks whether the logic makes sense.
A metric definition agent checks the owner, grain, filters, and caveats.
A dashboard quality agent checks whether the chart matches the trusted source.
An experiment readout agent checks whether the conclusion follows from the setup and the data.

This is the move from prompt to workflow.

The agent has a job.
It has inputs.
It has tools.
It has a review standard.

This is where teams start to feel fast.

It is also where they start to create new problems.

Because a fast agent with weak context can produce confident answers that disagree with each other.

One agent uses the product team’s definition of activation.
Another uses the sales team’s definition.
One checks the dashboard.
Another checks the raw database table.
One remembers the caveat.
Another misses it.

Now you have more output, but less trust.

That is not leverage.

That is cleanup work with better branding.

Level 3: the company stops re-teaching the same facts

At level 3, agents stop working from isolated prompts.

They pull from shared company context:

approved metric definitions
trusted data sources
dashboard lineage
business rules
known caveats
owner notes
prior accepted answers

This is where artificial intelligence analytics starts to compound.

An analyst clarifies a metric once. Future agents can reuse it.

A data owner approves a caveat once. It stops getting lost in chat.

A recurring business question gets answered with proof once. The next answer starts from that approved path instead of rediscovering the logic.

This layer is not glamorous.

It is also the difference between “AI helped me” and “the company learned.”

Without it, you do not have an AI-native data team.

You have faster guessing.

Level 4: answers improve the system

Level 4 is the real shift.

The answer is no longer the finish line.

A business question creates an answer.
The answer includes evidence.
A human reviews or corrects it.
The correction updates the definition, dashboard, checklist, or owner note.
The next agent starts from the improved version.

That is a closed loop.

The data team stops acting like a request queue.

It becomes decision infrastructure.

Not because it has more dashboards.

Because the company gets better at asking and answering the same important questions.

The mature system tracks things like:

Was the right definition used?
Was the right source checked?
Was the evidence strong enough?
Was the answer accepted, corrected, saved, or reused?
Did the correction improve the next answer?
Did the system ask for clarification instead of guessing?

That is the new scoreboard.

Not how many charts were viewed.

How many trusted answers were created.
How many were corrected before they caused damage.
How many became reusable.
How many repeated questions stopped interrupting the data team.

The hidden failure

Many companies will think they are at level 3 because they have a metric catalog, a semantic layer, or a folder full of documentation.

That is not enough.

A definition sitting in a doc is not company memory.

A dashboard nobody trusts is not a source of truth.

A chat answer that solved the problem once but never became reusable is not leverage.

It is residue.

The compounding asset is the reviewed answer path.

For every recurring question, the system should know:

what the metric means
which source to trust
which filters matter
which caveats change the answer
who owns the definition
what evidence makes the answer reusable
when the agent should ask a clarifying question
when the agent should send the answer to a human for review
when the agent should refuse to answer

That is how a data team scales without turning every AI answer into another review burden.

The 30-day move

Do not start with the whole data warehouse.

Start with one recurring question.

Something like:

“Why did self-serve conversion drop last week?”

Then build the loop around it.

Week 1: write the exact question, the metric, the decision it supports, and what would make the answer wrong.

Week 2: build one narrow agent with a clear job and review standard.

Week 3: create the context packet: definition, grain, source, owner, caveats, and examples of accepted answers.

Week 4: close the loop. Save the evidence. Save the answer path. Turn corrections into future checks. Update the docs and dashboards.

That is enough to show whether the company is serious.

The test is not:

“How many agents do we have?”

The test is:

Can the business ask better questions, get trusted answers faster, and make the next answer better because of what it learned this time?

If yes, artificial intelligence is compounding.

If no, it is just a prettier dashboard help desk.

What level is your company at right now?

And what is the biggest blocker between where you are and level 4?

This post was originally written in Sep 2025 (here), and I am cross-posting here since I had a conversation just yesterday which made it obvious that this transformation is accelerating.

I’ve been watching this mess unfold for years now.

Data Engineer, Analytics Engineer, Business Intelligence Developer, Data Analyst, Business Intelligence Analyst, Decision Scientist, Applied Scientist, Research Scientist, ML Engineer, MLOps Engineer, Data Scientist, Quantitative Analyst, Product Analyst, Growth Analyst, Marketing Analyst...

Stop.

We’ve created a job title circus that serves nobody. Not companies. Not professionals. Not the industry.

And it’s about to collapse.

I’m calling it now: By 2027, this chaos consolidates into 2 clear job families. The companies that figure this out first will dominate. The professionals who adapt early will thrive. Everyone else is going to get left behind in the confusion.

Why My LinkedIn Comment Exploded (And What It Tells Us)

last week, I commented that we need to go from ~15 job families to 2 in data. That comment exploded (in a good way).

Why? Because everyone knows it’s true, but nobody wants to say it out loud.

Here’s what’s really happening:

Companies are drowning. They post jobs for “Senior Data Analysts with ML experience and Python skills who can build data pipelines and present to executives.” That’s not a data analyst. That’s three different roles mashed together because nobody knows how to define what they actually need.

Professionals are stuck in title limbo. I see resumes with 5 different job titles for the same work. “Data Scientist” at one company equals “Business Analyst” at another. It’s complete madness.

Career paths are broken. How do you grow from “Analyst” to “Principal Data Scientist”? Nobody knows because we made up half these roles in the last 5 years.

Salaries are all over the place. Same work, different titles, $50K gaps. The market can’t price what it can’t define.

The chaos is unsustainable. And market forces are already fixing it.

The AI Reality Check

Everyone’s debating whether AI will replace data analysts. Wrong question.

The right question: Which data roles add unique human value when AI handles the routine stuff?

Here’s what I’ve learned watching AI transform workflows at multiple companies:

What AI eliminated:

• Manual data cleaning and prep

• Basic visualization creation

• Standard reporting and dashboards

• Simple statistical analysis

• Code debugging and optimization

What AI amplified:

• Strategic problem solving with business context

• Complex decision maker management and communication

• Creative experimentation and hypothesis generation

• Systems thinking and architectural decisions

• Ethical AI governance and bias detection

The result? Most data roles are converging into two categories based on what humans uniquely provide.

The 2 Job Families That Will Survive

After analyzing hundreds of job descriptions and actual work responsibilities, the consolidation is obvious:

Data Builders 🔧

Implementation & Data platform Specialists

What they do:

• Build and maintain data systems that work at scale

• Implement ML models and automation workflows

• Ensure data quality, security, and performance

• Create tools that enable everyone else to succeed, including AI

Current job titles merging here: Data Engineer, Analytics Engineer, MLOps Engineer, Platform Engineer

Why they matter: These are the people who make data work. They’re the builders.

Data Strategists 🧠

The Insight and Decision Specialists

What they do:

• Solve complex business problems using data and AI

• Drive strategic decisions through analysis and experimentation

• Translate between technical capabilities and business needs

• Design experiments that move key metrics

Current job titles merging here: Data Scientist, Business Analyst, Product Analyst, Decision Scientist, Research Scientist

Why they matter: These are the people who turn data into business impact. They’re the thinkers.

Why This Consolidation Is Inevitable

Follow the money.

Companies are tired of managing 15 different data roles with overlapping responsibilities. They want clear hiring criteria, predictable career paths, efficient team structures, and measurable impact.

The consolidation isn’t coming. It’s already here.

Smart companies are quietly restructuring their data teams around these two core functions. The job boards just haven’t caught up yet.

I’m seeing startups hire “Data Engineers” and “Data Scientists” with genuinely distinct responsibilities. Tech companies organize around “Data Platform” and “Data Science” teams with clear interfaces. Traditional enterprises finally invest in both infrastructure and strategy capabilities separately.

What This Means for Your Career Right Now

If you’re currently infrastructure focused:

• Double down on systems thinking and architecture

• Master AI/ML deployment and monitoring

• Focus on automation and scalability

• Understand business impact of technical decisions

If you’re currently strategy focused:

• Develop deeper business and industry knowledge

• Master experimental design and causal inference

• Become exceptional at communication and influence

• Learn to leverage AI tools for analysis

If you’re somewhere in between:

Pick a side as an IC. The generalist “data analyst who does everything” role is disappearing fastest.

Learn both as a data leader.

The Uncomfortable Truth

Most data professionals cling to outdated role definitions because change is scary.

But here’s the thing about market consolidation: You can be part of it, or you can be disrupted by it.

The professionals who recognize this shift early will have their pick of opportunities. The ones who keep debating whether they’re a “Senior Business Intelligence Analyst” or a “Principal Marketing Data Scientist” will get left behind.

What Happens Next

2025: Early adopters start restructuring teams

2026: Consulting firms publish best practices (catching up to reality)

2027: The consolidation becomes industry standard

By 2030: We’ll look back at our current job title chaos like we look back at having 47 different types of “webmaster” in the late 90s.

The Bottom Line

This consolidation isn’t a prediction. It’s already happening.

The question isn’t whether this will occur. The question is whether you’ll adapt proactively or get dragged along reactively.

The future belongs to data professionals who understand that impact matters more than job titles.

Impact comes from doing one of two things exceptionally well: Building data systems that scale, or using data to drive business results.

Everything else is noise.

Originally written in Sep 2025 on Paras’s linkedin newsletter.

Every few weeks, someone says semantic models are dead.

The argument usually sounds something like this: AI can query raw data, figure out joins, and answer questions in plain English, so why spend time modeling metrics at all?

I get why that sounds appealing.

But at least from where I sit right now, it is the wrong conclusion.

AI is absolutely making analytics faster. What it is not doing is magically creating shared business meaning.

That is still the hard part.

A human analyst can look at messy data and bring judgment to it. They know which revenue definition leadership actually uses. They know which table is technically available but operationally wrong. They know which segment should be excluded, which timezone matters, and which “right” answer will still cause confusion in the room.

AI does not automatically inherit that judgment.

So if you put AI on top of raw tables, fragmented dashboards, and inconsistent KPI logic, you do not eliminate confusion. You scale it.

That is why I do not think semantic models are dead.

If anything, AI makes them more important.

Because the bottleneck is no longer query writing. The bottleneck is trust.

A semantic model is not just BI plumbing. It is the layer that tells both humans and machines what a metric officially means. It creates one governed definition, reusable logic, and a more stable path between a business question and a trustworthy answer.

Without that layer, AI can still answer quickly. It just cannot answer consistently.

And in analytics, inconsistency is what kills trust.

The real risk is not only hallucination. It is metric drift.

One prompt pulls from one source. Another prompt pulls from another. A dashboard says one thing. A spreadsheet says another. An executive asks a simple question, and suddenly the meeting turns into a debate about definitions instead of a decision about the business.

That is not an AI problem in isolation.

That is a definition problem.

Semantic models help reduce that ambiguity. They give the business, the data team, and now the AI layer a shared contract for what counts.

To be clear, I do not think a semantic model solves everything.

You still need business context, caveats, approved answer paths, and operating judgment around how metrics should be used. But that is exactly the point: semantic models are not the whole solution, yet they are still a foundational part of the solution.

So my current POV is simple: AI makes natural-language analytics easier. It does not make semantic discipline optional.

The teams that win will not be the ones with the cleverest prompts. They will be the ones with the clearest definitions, the strongest trust layer, and the discipline to keep AI grounded in governed business meaning.

POV is mine; AI helped shape the final draft.

After interviewing hundreds of data professionals in past 12 months, I’ve noticed clear patterns separating candidates we move forward with from those who don’t get the call. It’s not always about credentials, it’s about how you show up in the process itself.

Here are the 9 tactics that candidates we actually hire demonstrate, organized around three core qualities: agency, accountability, and adaptability.

AGENCY: Take Ownership of Your Search

Tip #1: Build Something Real

Don’t polish your portfolio forever. Candidates who move forward build a functional project this week that solves a real problem, even if it’s rough. Use a public dataset, ship a GitHub repo, deploy a live dashboard. Employers want evidence you can execute, not just interview well.

Tip #2: Show Up With Opinions

Generic interview answers don’t stick. The candidates we hire come prepared with specific perspectives: “Here’s how I’d approach your churn problem using causal inference rather than standard cohort analysis” or “I’d restructure your metrics layer using dbt semantic models.” You don’t need to be right, you need to demonstrate you think.

Tip #3: Make Yourself Visible Before the Interview

Engage publicly. Comment thoughtfully on LinkedIn posts from data leaders. Write one substantive post about an industry trend. When I interview someone whose thinking I’ve already seen online, they’re immediately credible. You’re not building fame, you’re building proof that you know what you’re talking about.

ACCOUNTABILITY: Prove Impact With Numbers

Tip #4: Quantify Every Win in Your Resume

Replace “Led analytics initiatives” with “Built attribution model that optimized $75M in annual marketing spend, reducing CAC by 18% and accelerating payback period from 16 to 9 months.” Candidates I hire share >5 specific, quantified impact stories.

Tip #5: Own Your Failures

In interviews, when asked about mistakes, candidates we advance take real accountability: “I missed the forecast by 15%. Here’s the root cause, what I should have done differently, and the monitoring system I built to prevent it from happening again.” We’re hiring for judgment and accountability, not perfection.

Tip #6: Prepare Your Impact Narrative

Before any interview, write a one-page summary of 3 major projects with: (1) the business problem, (2) your specific contribution, (3) the measurable outcome, (4) what you learned. Reference this throughout the interview. This shows you think in outcomes, not just outputs.

ADAPTABILITY: Show You’re Built for Tomorrow

Tip #7: Demonstrate You Can Learn Fast

The candidates we hire don’t say “I’m learning AI”, they say “I shipped a multi-agent system using Claude.” Spend this week building something with a modern AI tool. Deploy it. Show you can move quickly in an evolving ecosystem.

Tip #8: Highlight Your Pivots

If you’ve evolved your role, own it confidently. “I was hired as an analytics engineer but recognized the team needed data infrastructure, so I taught myself Airflow and restructured our pipeline architecture” shows adaptability and initiative. Companies in 2026 need people who can wear multiple hats.

Tip #9: Know Where the Industry is Heading

Candidates we hire reference new frameworks, tools, and industry trends in conversations. They mention semantic layers, causal inference, or agent-based architectures casually, not as jargon, but as genuine familiarity. Show you’re reading, listening to podcasts, and thinking forward.

Put It Into Practice

Before your next interview:

✓ Agency: Build something tangible. Ship a real project.

✓ Accountability: Quantify your three biggest wins. Know your numbers.

✓ Adaptability: Learn one new AI tool. Show proof you built with it.

The difference between “qualified” and “we’re hiring this person” often comes down to these signals, especially in an AI-native environment where the rules are still being written.

Live 1:1 Practive

For a limited time, I am offering live 1:1 mock interviews (all proceeds are donated). https://topmate.io/parasdoshi

Disclaimer: This post was edited by AI, but the core ideas, framework, and thinking were fed to the AI by Paras

Which of these 9 resonates most with your job search right now? DM me your biggest challenge. I’m here to help.

Session Overview:

Data isn’t the hard part. Sustaining its impact is.

Too many data teams succeed in shipping dashboards but stall when it comes to lasting influence. Why? Because impact doesn’t scale without intention.

At the next Inner Join, I’m hosting Paras Doshi, Head of Data at Opendoor, to unpack a field-tested framework for building data-led organizations that last.

He calls it the 3 Ps framework:

1. People: How to centralize data culture while empowering teams
2. Platform: Why discoverability, trust, and governance need to be built in—not bolted on
3. Process: Moving from service desk to product mindset (with examples)

 Plus: How Paras’ team turns ad-hoc requests into repeatable assets—and measures long-term data value.

—-

Additionally, I have written about centralized vs de-centralized data teams here if you want to double click into structuring data org’s.