The 2026 product leader's toolset

Executive summary

• Categories, not brands. Eight categories, each with its job, its families, a test for choosing and its traps. The product names will go stale; the tests won’t.
• Think and build. Pay for one frontier model, learn it deeply, and judge it by whether it changes your decisions. Prototype app-first to validate. Use an agentic coding tool to read code, write SQL and build internal tools, not to ship to production.
• Know what’s true. Get read-only warehouse access and read every generated query. For a single study, your model and a folder of transcripts synthesize as well as a dedicated platform. Keep evals in a spreadsheet you own until a second person or model joins in.
• Keep state where agents can read it. The tracker’s new job is to be accurate and machine-readable, with the “why” beside the work. A tracker and docs are enough; a second source of truth ruins both.
• Hear what you didn’t ask about. A short weekly digest of customer voice, sales calls, competitors and your own metrics, pushed to you and linked to its sources.

You can set up six of the eight alone, in a month, with no budget approval; only warehouse access and call recording need someone else. That is what “full-stack” now means in a product title.

One note on how to read this, because it decides whether the piece is still useful next year.

The categories are the argument. The brand names are examples, and they will go stale. Every product I name is what I’d point to in September 2026; by the time you read this, several will have been bought, renamed or overtaken. That’s fine. What lasts is why the category exists and the test you run to choose within it, because those come from the shape of the work, not the shape of the market.

Each section follows the same pattern:

1. What the category is for — the job it does, whoever sells it.
2. The families — the two or three real approaches, which change far more slowly than products.
3. The test — what to do to choose. Run it against whatever exists when you read this.
4. The traps — what goes wrong, which hardly changes at all.

Named products are marked (as of Sept 2026). A year from now, ignore the names and run the tests.

1. The reasoning partner

What it’s for

One frontier model you use all day to think, draft, critique and analyze. Not for any one task, but for the everyday work of thinking a problem through with something that argues back.

This is the most valuable item on the list, and most people use a fraction of it, because they treat it as a writing tool rather than a thinking one.

The families

There are no real families here, just a few frontier labs. The choice that matters is one, deeply or several, shallowly.

Pick one.

As of Sept 2026 the frontier is Anthropic, OpenAI and Google, with a few credible others. The list has held for about two years; the order changes every few months.

The test

Take five real tasks from the past month. Not sample prompts, but real work: a positioning argument, a messy transcript, a pricing decision, a piece of writing you were proud of, a question about data. Run all five through each candidate on the same afternoon, and ask which answer you would send.

Weigh these:

• Judgment under ambiguity. Give it a decision with no clean answer. Does it take a position and defend it, or hand back a balanced list of considerations? You want the position; you can always ask for the list.
• Pushback. Assert something wrong, confidently. The one that corrects you is a partner. The one that agrees is a mirror, and a mirror will let you walk into a wall.
• Long context. Give it forty pages and ask a question that needs all of them. Some models summarize without saying so and lose the detail that mattered.
• Writing you’d sign. Read the output aloud. If it sounds like a press release, editing costs more than drafting saved.
• Ecosystem. Does the same model power the coding and prototyping tools you’ll pick next? Fewer context switches beat a few benchmark points.

The one-month test: has it changed a decision? Not made you faster: changed what you decided. If it hasn’t, you’re using it as a typist.

The traps

• Choosing by the news. Releases never stop, and this week’s “best” model is rarely the best for your work.
• Staying on the free tier. The gap between free and paid is bigger than the gap between vendors. Whatever you choose, pay for it.
• Benchmark shopping. Benchmarks measure tasks you don’t do. Your five real tasks tell you more than every leaderboard.
• Reassessing constantly. Reassess every six months, not at every launch. Switching costs are real, and mostly hidden.

2. The prototyping tool

What it’s for

It turns a description into a working, clickable thing without an engineer. This is the tool that replaces the spec, and the clearest sign of how the job has changed.

The families

First decide what the prototype is for, because one word covers three different jobs:

Most product people need only the first two.

Then choose between two families:

Design-first. Starts from your design system and produces something that looks like your product. Best for alignment; weak when the prototype needs real logic or data. (Sept 2026: Figma Make and peers.)

App-first. Produces a working app with state, forms and a database. Best for validation, because users can do real things in it. It looks generic unless you push it. (Sept 2026: Lovable, Bolt, v0, Replit.)

Choose the family by which failure hurts more: a prototype that looks off-brand, or one that can’t do anything.

The test

One real feature idea, two tools, one afternoon.

1. Time to first clickable version. Under an hour, or it isn’t a prototyping tool.
2. Iteration. Ask for three changes a user might want. Does it make them, or break what already worked?
3. A URL by end of day. Without one it’s a demo tool, not a prototyping tool.
4. Data. Can it hold a list, a login, a saved state? Validation needs this; alignment doesn’t.
5. Escape hatch. Can you get the code out, or edit it directly when the tool gets stuck? It will get stuck.
6. Cost per experiment. Credit-based pricing punishes iteration, and iteration is the whole point.

The traps

• Polishing. A prototype’s job is to be wrong in public, fast. A day spent on spacing defeats it.
• Showing it to executives as if it were real. Say plainly that it’s a prototype, or you’ll be held to a ship date you never agreed to.
• Over-thinking the choice. These tools are converging fast. Pick one, get fluent, move on.
• Missing the moment you’ve outgrown it. Once you’re comfortable with an agentic coding tool, it becomes your prototyping tool: more control, less hand-holding. Starting app-first and moving on within a few months is the normal path, not a mistake.

3. The agentic coding tool

What it’s for

Reading code, writing queries and building small tools, even if you never ship a line to production. Whether you can do this now divides the two versions of the job.

Be honest about what you’ll use it for. In order of how often:

1. Reading. “How does the invite flow work?” answered from the real code in two minutes, not a Slack thread.
2. Data. Writing the SQL, running the analysis, making the chart.
3. Internal tools. Things you’ll use yourself, not ship to customers.
4. Prototypes. Later, once you’re fluent.

You will almost never write production features. A tool built for senior engineers who do may be more than you need.

The families

Terminal agents. Describe a task in plain language; the agent reads the codebase, makes a plan, runs commands and edits files. The most autonomous, the least visual, and the easiest start for a non-engineer, because it’s a conversation, not a code editor. (Sept 2026: Claude Code, Codex CLI, Gemini CLI.)

IDE agents. The same power inside a code editor. Better if you want to watch and edit the code as it goes; more intimidating if you’ve never opened one. (Sept 2026: Cursor, Devin Desktop (formerly Windsurf), Copilot.)

Cloud agents. You assign a task; it works in the background and comes back with a result. Useful once you trust it. The wrong place to start.

The test

Clone one real repository: your company’s, or an open-source app in your field.

1. Ask a question only the code can answer. Is it right? Does it cite files? Would an engineer have been faster?
2. Ask for a small change. Does it explain before it acts? Can you stop it halfway?
3. Build something from scratch. A script that reads a CSV of tickets and groups them by theme. Does it run the first time?
4. Give it a bad instruction. Does it push back, or comply and break things?
5. Check the leash. Does it ask permission before running commands and editing files? For a non-engineer this is the safety feature.

The traps

• Pointing it at production. Not because it can’t, but because you don’t yet know what you don’t know about deploys, secrets and data. Internal tools and reading code are your lane for the first few months, and it’s a wide lane.
• Quitting in the first week. You will feel like you’re pretending to be an engineer. The feeling passes about when your first tool becomes something you use.
• Accepting output you can’t read. If you can’t roughly follow what it did, you can’t tell when it’s wrong. Slow down until you can.
• A different model from your reasoning partner. Then you spend the day re-explaining context.

4. Direct access to your data

What it’s for

Answering product questions yourself, today, without filing a ticket. Here is the test of whether you can:

• What share of active accounts touched this feature last month?
• Which step in the flow loses the most people?
• Did retention move after the launch?
• Who are the ten accounts most likely to churn, and what do they have in common?

If any of those requires someone else, you don’t have direct access. You have a dashboard.

The families

Three layers, and you need all three, because they answer different questions.

The test

You usually inherit the analytics tool rather than choose it, so the real test is of your access, not the vendor:

1. Plain-language question in, number out, with the query shown. If the query is hidden, you can’t check it, and sooner or later you’ll show leadership a wrong number.
2. Joins across sources. Usage plus revenue, usage plus support. Single-source tools give single-source answers.
3. Save and rerun. A monthly question belongs in a saved query, not a repeated conversation.
4. Push, not just pull. A metric that changes should tell you.

And one request matters more than any tool: ask for read-only warehouse access in your first month anywhere. It’s the most valuable thing a product person can ask for, it’s usually granted, and it ends the two-day wait for an analyst for good. Then let your coding agent write the SQL; it’s good at it, and it’s the fastest way to learn the schema.

The traps

• Trusting generated SQL. Natural-language-to-SQL is confidently wrong more often than people expect, especially on large schemas, joins and warehouse-specific functions. Always read the query. Always check it against a number you already know.
• Assuming the events exist. The bottleneck is rarely the tool; it’s whether what you care about was ever tracked. Check before you build an analysis on it.
• Asking for write access. You don’t need it, and it removes the safety net. Read-only means read-only.
• Mistaking a dashboard for access. A dashboard answers the questions someone predicted. Access answers the one you have now.

Human experts · BIRD93%

Best system · BIRD, 202682%

ChatGPT · BIRD, 202340%

Best system · Spider 2.0-Lite, 202676%

o1-preview · Spider 2.0 enterprise tasks, 202417%

Sources: BIRD (Li et al., 2023, and its leaderboard); Spider 2.0 (Lei et al.). Both have climbed fast: by 2026 purpose-built agents top the Spider 2.0 leaderboards, but even the best still miss about one enterprise task in four on Spider 2.0-Lite, and a general model without your schema’s context does far worse. Hence: always read the query, and check it against a number you already know.

The numbers

System	Benchmark	Correct
Human experts	BIRD	92.96%
Best system, Sep 2026	BIRD	82.39%
ChatGPT + chain of thought	BIRD (2023)	40.08%
Best system, Sep 2026	Spider 2.0-Lite	76.23%
o1-preview	Spider 2.0 (2024)	17.1%

5. Research synthesis

What it’s for

Turning a pile of unstructured text (transcripts, session notes, open-ended survey answers, tickets, call recordings) into an answer to one question: what are people telling us?

This is where the timing problem lived. A week to recruit, a week of interviews, a week of synthesis, and the decision was made before the deck was ready.

The families

This is the one category where the dedicated tools are losing to the general ones, and it’s worth saying plainly, because it saves money.

Dedicated research platforms. Transcript storage, tagging, highlight reels, a searchable repository across studies. (Sept 2026: Dovetail, Marvin, Condens, Notably.)

Your reasoning partner and a folder. Transcripts in a folder, one carefully written prompt, outputs you control. For a single study, this now synthesizes at least as well, at no extra cost.

A dedicated tool earns its price in three cases: a research team of more than two people who need to find last year’s findings; video, when stakeholders want clipped highlight reels; and compliance, meaning consent tracking, participant data and retention rules. If none apply, you don’t need one yet. Most product leaders at startups don’t.

The test

Take a study you’ve already synthesized by hand. You know the answer, which makes this the cleanest test on the list.

1. Patterns. Did it find the themes you found? Missing one is bad. Finding a real one you missed is the point.
2. Contradictions. Did it notice where participants wanted opposite things, or flatten them into a majority view? This is where most tools fail and where most of the value is.
3. Traceability. Every claim links to a quote, and the quote is real. Check three at random. If one is paraphrased into something the person didn’t say, the tool is no good for decisions.
4. Surprises. Does it flag the outlier who fits no theme, or bury them?
5. Speed. Under an hour from raw transcripts to something you’d share, or the timing problem isn’t solved.

The traps

• A lazy prompt. “Summarize these interviews” produces mush. “Find claims made by three or more participants, list where participants disagree, and quote each with speaker and line” produces something you can act on. The prompt is most of the product.
• Letting it replace listening. Read two transcripts in full before you look at any synthesis, so you know the people behind the themes. Otherwise you’ll believe a tidy summary of a messy reality.
• Synthesizing before you’ve asked good questions. Fast synthesis of bad interviews is fast garbage.
• Losing the disagreements. People and tools both drift toward consensus. The disagreement is usually the finding.

6. The eval harness

What it’s for

Measuring the quality of anything model-powered that you ship: a place where your test set, rubric and scores live, and a command that tells you whether the number moved.

A traditional feature has acceptance criteria, and QA checks them once. A model feature gives a hundred slightly different answers to the same input, some wrong in ways only domain knowledge can spot. “It worked when I tried it” measures nothing. The method behind this category has its own piece; this section is only about where it lives.

The families

The test

1. Can you add a case from a real failure in under a minute? A support ticket about a bad answer should become a test case that afternoon. If it takes longer, the set goes stale.
2. Can a non-engineer write the rubric? Defining “good” is product work. If expressing “must not invent a source” requires code, you’ll stop maintaining it.
3. Does it show the diff — which cases changed and how, not just the aggregate score?
4. Can a model grade, and can you audit the grader against your own judgments?
5. Can it connect to production later? Not on day one, but the path should exist.

The traps

• Test sets that are too easy. Inputs you wrote yourself are cleaner than reality, so the score means nothing. Use real inputs, weighted toward the hard ones.
• Rubrics that measure fluency. Models write well. Measure what they’re bad at: not inventing, not leaving things out, following constraints, admitting uncertainty.
• Over-fitting. Iterating until the number goes up, then shipping into failures you never saw. Hold back cases you never look at while you iterate.
• One number for everything. An 87% average hides fabrication at 99% and completeness at 61%. Report per criterion.
• Building it, then abandoning it. The most common failure. Tie it to a ritual, such as running it before every release without exception, or it’s dead in two months.
• The test set is the asset, not the tool. Switching platforms is easy; rebuilding two hundred real cases is not. Keep it in a format you own.

7. The system of record

What it’s for

The least glamorous category, and the one that decides whether half the others work.

The tracker’s old job was to hold state for people to find, with the PM keeping it accurate and passing it on. Its new job is a place where state lives so that agents can read it. Every tool that monitors, drafts or builds a roadmap depends on the tracker being accurate and machine-readable. If the truth lives in Slack threads and people’s heads, none of them work.

That changes the choice. You’re not picking the tool with the nicest board. You’re picking the one an agent can read from and, carefully, write to.

The families

Tracker — the work. (Sept 2026: Linear, Jira, and a long tail.)

Docs — the reasoning. (Sept 2026: Notion, Confluence.)

Roadmap tools — only if leadership insists, and then as a generated view of the tracker, never a second source of truth.

Fewer tools, on purpose. A tracker and docs are enough. Every extra system is one more place for the truth to drift.

The test

1. A real API, and an MCP server or equivalent. This is now the first question, not the last. If your coding agent can’t ask the tracker “what shipped this week?”, you’ll be copying and pasting forever.
2. Structure that means something. Status, owner, target date, linked bet, linked metric. A list of titles gives an agent nothing to work with.
3. Room for the bet, not just the ticket. The “why” and the success metric belong next to the work, or they drift into a separate doc nobody updates.
4. Stays accurate without you. Engineers update the tracker because it’s part of their day. Nobody updates a roadmap tool. Pick what the people doing the work already maintain.
5. Leadership can read it directly. If someone has to turn it into a slide, you’re still the relay, and nothing has changed.

The traps

• Fighting the company’s choice of tracker. You’ll lose. Fight for API access and the fields instead — those you can win.
• Letting agents write unsupervised. Start read-only. An agent updating tickets on its own is how a confidently wrong status reaches the CEO. Draft, review, then write.
• Undated decisions. “We chose X because of Y, on date Z” is the most useful thing you can leave for your future agents and your future self. Without the date and the reason, it’s trivia.
• Two sources of truth. The moment the roadmap tool and the tracker disagree, both become untrustworthy.

8. The watcher

What it’s for

Every other category answers a question you asked. This one raises questions you didn’t know to ask.

A product leader’s biggest misses are rarely wrong answers. They’re things nobody looked at until a customer left or a competitor launched. The fix is to have the news come to you: customer voice, sales calls, competitor moves and your own metrics, arriving as a digest.

The families

Four feeds, from four different sources:

• Customer voice — tickets, in-app feedback, reviews. (Sept 2026: Zendesk/Intercom AI summaries, Enterpret, Canny — or a weekly export and your own prompt.)
• Sales and success calls — the richest source in most companies, and the least read. (Sept 2026: Gong, Chorus, Fireflies.) Buy this one; recording and transcription is real infrastructure.
• Competitors — pricing pages, changelogs, app store releases and job postings, the underrated signal: they tell you what a competitor will build before it’s built. (Sept 2026: Crayon, Klue, or a script that diffs pages weekly.)
• Your own metrics — the ones you defined at launch and nobody checked. Mostly you build this yourself.

At a startup, a weekly script (export tickets and call summaries, run one good prompt, post to a channel) covers most of this in an afternoon. Build the rest; buy the call recording.

The test

1. Push, not pull. A dashboard you have to remember to open fails the category by definition.
2. Short enough to read every time. If it takes more than five minutes you’ll skip it in a busy week, then every week. Three items and a link.
3. Links to the source. Every claim one click from the ticket, the call or the page. Otherwise you’re trusting a summary you can’t check.
4. Tunable without a vendor call. “Pricing complaints and integration requests; ignore login issues.”
5. Accumulates. “Third week running that export comes up” is a finding. One mention isn’t.

The traps

• Letting the digest become the truth. It’s a sample, summarized by a model. Read three raw tickets a week, unsummarized, forever.
• Too many alerts. Every new alert must replace an old one, or within a month it’s noise. One digest, one channel, one rhythm.
• Watching without acting. A digest nobody acts on is a more expensive way of not knowing.

The whole set

1. Reasoning partner — one frontier model, paid, used daily
2. Prototyping tool — app-first for validation; show, don’t tell
3. Agentic coding tool — reads code, writes SQL, builds your internal tools
4. Data access — read-only warehouse, query shown, joins across sources
5. Research synthesis — your model and a folder, until you need a repository
6. Eval harness — a test set you own, a rubric you wrote, a score you rerun
7. System of record — the tracker agents can read, with the “why” beside the “what”
8. The watcher — pushed digests from customers, sales, competitors, your metrics

Five years ago that wasn’t possible. It is what “full-stack” now means in a product title.

Read this again in a year and ignore every product name. The categories will still be there; run the tests on whatever exists then.

— Chris

Product examples current as of September 2026.

Sources

Where the evidence comes from. Each row opens the document itself; the tag says which part of the argument it supports, and ↩︎ goes back to the sentence that uses it.

1. Towards Understanding Sycophancy in Language ModelsMrinank Sharma, Meg Tong, ... Ethan Perez (Anthropic)Model sycophancyPaper2023-10 (ICLR 2024)5 state-of-the-art assistants, 4 free-form tasks↗︎Models agree with users over truth
2. Sycophancy in GPT-4o: What happened and what we're doing about itOpenAIGPT-4o sycophancyArticle2025-04-29↗︎Production model update rolled back for flattery
3. Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs (BIRD)Jinyang Li, Binyuan Hui, et al.Text-to-SQL accuracyPaper2023-05 (NeurIPS 2023)12,751 question-SQL pairs, 95 databases, 37 domains↗︎ChatGPT 40% vs humans 93% accuracy
4. Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL WorkflowsFangyu Lei, Jixuan Chen, et al.Enterprise text-to-SQLPaper2024-11 (ICLR 2025)632 real enterprise tasks; o1-preview solves 17.1% (revised from 21.3%)↗︎o1-preview agent solved 17.1% of enterprise tasks
5. Introducing the Model Context ProtocolAnthropicMCP launchArticle2024-11-25↗︎Open standard connecting AI to tools/data
6. Spider 2.0 leaderboardSpider 2.0 team (XLANG Lab, HKU) · spider2-sql.github.ioEnterprise text-to-SQLNewsletterSep 2026Spider 2.0-Lite top 76.23%; Spider 2.0-Snow top 96.70 (Genloop, Mar 2026)↗︎Current best systems on Spider 2.0