What data science professionals need to do now to stay relevant

Prompting is becoming table stakes. The durable opportunity is building analytics systems that know what the data means, what they can do, and when to stop.

The data science role as most people learned it is being compressed, and the replacement is not what most career advice suggests.

The standard advice right now is to “learn AI tools” or “get good at prompting.” I understand why people say this. It is easy to act on, it feels productive, and there are a hundred courses that will happily take your money to teach it.

But prompting is becoming a feature inside every product. It is table stakes, much like SQL became table stakes. SQL never stopped mattering. It simply stopped differentiating you on its own once everyone was expected to know it. The same compression is happening with prompting, only faster.

So what actually matters?

I will answer that through one concrete example, because the abstraction is where most of this advice goes wrong.

One agent, end to end

Say you are a data scientist at a company with a few hundred dashboards. Leadership asks, “Why did conversion drop in the Southeast last week?” Today, someone opens three dashboards, pulls data into a notebook, cross-references it with a marketing calendar, and writes a summary. That takes a few hours if you know where everything is, longer if you do not.

Now imagine building an agent that can investigate that question. I do not mean a chatbot that generates SQL. I mean an agent that knows which tables contain conversion data, understands that “Southeast” maps to a specific set of state codes in your schema, checks whether a marketing campaign was running in that region, and assembles a coherent answer with the right caveats. It should show its evidence, distinguish likely drivers from proven causes, and say when the available data cannot support a reliable answer.

Building that agent teaches you things that prompting never will.

The first thing you discover is that the agent fails immediately if your semantic layer is messy. If “conversion” means three different things in three different tables, the agent may pick one and confidently give you the wrong answer. So you have to define your metrics cleanly, not for a dashboard, but for a system that reasons over them. That is a different standard of clarity. A human analyst can resolve some ambiguity from experience. An agent needs the ambiguity removed or an explicit rule for handling it.

Then there is the question of constraints. A naive agent will attempt arbitrary joins, invent filters that seem plausible, and present the result with perfect confidence. The fix is not to “add guardrails” in the abstract. You have to decide which joins are valid for a given metric, which filters the agent may apply, and which combinations it should reject rather than attempt. You are encoding business logic into the system’s boundaries instead of hoping the model figures it out.

Metric selection works the same way. The agent should not choose among three definitions of conversion. That choice should already be made in the semantic layer, and the agent should operate within it.

The system also has to respect the same data permissions as the person asking the question, check whether its sources are fresh, expose the query and evidence behind its answer, and route high-risk or ambiguous results to a human. Those are not compliance details to bolt on later. They determine whether anyone can safely use the system.

This is system design work. It is closer to building a product than writing a prompt.

Then there is evaluation, which is still the least discussed part of this work. Before you ship an agent that answers business questions, you need to know when it is wrong.

That means building a set of test questions where you know the metric definition, valid sources, expected joins and filters, numerical result, and evidence the answer should cite. Not every analytical question has one fixed expected response, so test the parts separately. Did it choose the right metric? Did it preserve the filter? Did it use a valid join? Did it cite enough evidence? Did it separate observation from causation?

Some test cases should be questions the agent cannot answer with the data it can access. Knowing when to say “I do not have enough information to answer this reliably” matters as much as generating a correct response.

You also need explicit failure conditions. What does a wrong join look like in the output? What does a silently dropped filter look like? What does a hallucinated metric look like? If you cannot detect these failures in testing, you will not detect them when a stakeholder is making a decision based on the answer.

The hardest part is not the model call. It is everything around it: the data catalog, permissions, context management, orchestration, source freshness, and error handling when a query returns nothing. The model matters, but much of the work moves into data engineering, analytics engineering, product design, and operating discipline that many data science roles did not previously require.

This is what I mean by becoming a builder of agentic analytics systems. Not a theoretical understanding. The actual experience of making one work, watching it break, and figuring out why.

Where the durable value is shifting

Routine analysis, the kind where someone

pulls numbers, makes a chart, and summarizes what happened, is getting easier to automate. It is not fully automated today, and it may not be for a while. But the direction is clear enough that building a career on that work alone feels like a narrowing bet.

The work that remains difficult is the layer underneath: defining what metrics mean in a specific business context, encoding the domain knowledge that tells you which data to trust, designing workflows that can run without a human reviewing every output, and evaluating whether the output deserves to be trusted at all.

I have seen this firsthand. At Opendoor, I led a program that retired more than 1,400 dashboards as we consolidated metric definitions into a semantic layer and built custom AI-assisted workflows. It was not a one-for-one replacement exercise. The dashboards were not the hard part. The hard part was making sure “conversion,” “active user,” and “revenue” meant one thing, precisely defined, across every surface that consumed them.

That work required people who understood the business deeply enough to make those calls. No model was going to do that for us.

Data professionals who can define the metrics, encode the context, design the system, and evaluate its outputs are operating at an intersection that still has relatively few practitioners. Not because the work is impossibly hard, but because most people have not built one of these systems yet.

What to actually do this month

Pick one recurring question your team gets asked. Not the most complex one. Something like, “What happened with [metric] last [time period]?” Build an agent that can investigate it.

Use whatever harness you want: Claude Code, Codex, etc. The harness matters less than the exercise. What matters is going through the full loop: define the question, connect to approved or sanitized data, handle the failure modes, build a small evaluation set, and get the system to a point where you would trust the answer enough to send it to a stakeholder.

You will get stuck. The agent will do something confidently wrong, and you will spend an hour figuring out why. That hour is the most valuable part. That is where you learn what production agentic systems require.

Then write about it publicly, on LinkedIn or a blog. Do not write a polished tutorial. Write about what broke and what surprised you. Write about the gap between what you expected and what happened.

That kind of writing forces you to understand what you built. It also signals to the market that you build, rather than waiting to be told what to learn next.

The semantic layer piece

If you do not understand semantic layers yet, start there.

A semantic layer sits between raw data and the agent’s reasoning. It defines metrics, relationships, valid filters, and access rules in a machine-readable form. Without that layer, an agent has to reconstruct business logic every time it answers a question.

The dbt semantic layer or snowflake cortext are different ways to formalize parts of this. The specific tool matters less than the concept: your data needs a machine-readable description of what it means and you can learn more on OSI home page here.

That was the lesson from the Opendoor work. Once the definitions and permissions were reliable and reusable, each new interface became easier to build. Without that foundation, we would have created a faster way to produce wrong answers.

If you can define a clean semantic layer and build agents that reason over it, you are working on the part of analytics that matters most right now. A real project will teach you more, and differentiate you more, than another prompting certificate.

Where to go deeper

I teach a live Maven workshop called World-Class Agentic Analytics in Production. Bring one real analytics use case from your company. During the workshop, we will define the agent’s job, map the context it needs, design its proof and evaluation system, and set the line where a human must step in.

You will leave with a written 30-day production plan and get a private 60-minute follow-up session with me to review it. The full syllabus and next cohort schedule are on the course page.

P.S. If you are navigating a job search, I also offer 1:1 mock interview coaching for data roles. All proceeds from my interview help are donated.