Measures, Metrics, and Signals

More Data Won't Help You Decide.

You've been told the answer is better metrics. Or more of them. A new framework. A new dashboard. Another round of data. DORA, OKRs, whichever framework the industry is selling this quarter. None of it will help your leaders make decisions they can't already make.

Not because the metrics are wrong. Because no one on your team can tell you what action each number actually demands.

The dashboards are built. The numbers update weekly. The reports go out. And they sit there, reported but not used. Your leaders can't read what the data is telling them, so the data does nothing. Capacity calls get made on gut. Team problems surface too late. The executive layer asks a question and no one in the room can answer it from the numbers they have.

You don't need another framework. You need leaders who can turn data into decisions.

How Signal Mapping Works

Engineering leaders use data for three things: investigation, monitoring, and decision-making. Each requires different thinking, and most organizations get them tangled. That's why so much measurement effort produces so little decision-making power. The workshop teaches how to do each effectively.

Signal Mapping is the approach to the decision-making use case. It's the unique method you'll leave with, it's where most leaders are most stuck, and it's what separates a dashboard from a working decision system.

Most measurement efforts start with "what should we measure?" The answer is always a list: DORA, OKRs, deployment frequency, velocity, whatever's trending.

Signal Mapping starts with a different question: what decision are you trying to make, and how will you know it's time to act?

From that question, we work backward:

1. Name the decision. What call does your leader need to make? Capacity, intervention, scheduling, investment, team health? Be specific.
2. Form a hypothesis. An explicit, falsifiable claim about what's driving the situation. "We believe X is causing Y. If we can see X change, we'll know how to act."
3. Identify the signal. The measurable thing that would validate or falsify the hypothesis. Not what's easy to track. What would actually tell you something.
4. Pair it with a solution. If the hypothesis holds, what's the action? Every signal has a paired response: a conversation, an experiment, a change.

Your leaders leave with a working set of decisions, hypotheses, signals, and actions tied together. Not a dashboard. A decision-making system.

What This Looks Like in Practice

A QA department supporting over a thousand distributed locations was chronically behind. They spot-checked releases when they should have been testing comprehensively. An automation initiative was underway, but it was over a year from results. The executive assumption was that the team was understaffed and under-qualified.

When we applied Signal Mapping, the decision wasn't "how do we measure QA velocity?" It was "where do we actually invest to restore this team's capacity?"

That decision led to a hypothesis: interruptions are killing capacity, and tracking interruption count and duration will surface it. If the hypothesis held, the solution would follow: reduce interruptions, restore capacity.

When we tracked it, the hypothesis held. Interruptions were eating a full day of capacity per person per week. We piloted a single change, scheduled office hours for interruptions only, and virtually eliminated them. The team caught up on their backlog, started leaving work on time, and pioneered automation months ahead of the year-long plan.

The assumption was that the team couldn't deliver. The measurement thinking showed they couldn't be left alone long enough to.

Other places this thinking has held up:

• Improved engineering delivery speed by 20% without sacrificing quality
• Kept a product operation compliant under federal law
• Drove a product portfolio worth over $30 million
• Built one of the highest-performing teams in an organization of 500

What Your Leaders Will Take Away

After this workshop, your leaders will be able to:

• Connect measurement to decisions, not dashboards. For any decision they need to make, they'll know what would tell them when and how to act, and what's just noise.
• Form hypotheses they can test. Move from "something's wrong" to explicit, falsifiable claims about what's driving a situation. Claims that tell them what to measure and what to do when the data comes back.
• Build signals that trigger action. Not metrics that describe activity. Signals that prompt the right behavior at the right time, paired with a clear response.
• Spot when a measure is lying to them. Recognize when a number looks good but the underlying situation isn't, and vice versa.
• Use what they already have. Stop chasing more data. Most teams already have what they need. They haven't learned how to read it.
• Build their own measurement for their situation. Stop implementing the popular metrics of the day. Learn the principles that let them create measurement that actually fits their situation, not the industry's.

This thinking applies broadly across capacity calls, scheduling decisions, team health, interpersonal friction, and organizational performance. Your leaders leave with a transferable skill, not a fixed template.

How the Workshop Runs

This isn't a lecture. Your leaders work on their own current goals, current teams, and current problems throughout. Everything they build is applied to real work.

They get expert guidance while doing the work themselves. Practice under pressure, with someone in the room who has seen how this goes right and wrong across many organizations.

Before the workshop: I conduct interviews with the participants to understand their goals, their current measurement practices, their concerns, and the decisions they're struggling with. The workshop is shaped around what I find.

During the workshop: Your leadership team does the work. I facilitate, challenge, and pattern-match to what I've seen elsewhere.

After the workshop: Your leaders have measurement thinking they built themselves, can defend to their own teams, and are equipped to act on.

This is for Your Team if…

• Leaders are generating reports but can't confidently answer what decision the data should drive
• Dashboards keep multiplying but decisions aren't getting better
• "We need more data" has become the default answer to every hard question
• The last team problem that surprised the leadership team should have been visible in the data
• The connection between what the team is doing and what the business cares about feels assumed, not demonstrated

How This is Purchased

Engineering leadership workshops are typically sponsored by a VP of Engineering, CTO, or equivalent senior leader for their direct and extended leadership team. That sponsor is the buyer; the team is the audience. This workshop is built for that model.

If you're a senior leader looking to sharpen your own team, or a leader in a partnership role who sees the gap and wants to bring it to your engineering counterpart, this is structured to fit how you actually make purchasing decisions.

Options

Three ways to work together. The right fit depends on how deep the current measurement problem runs and how much reinforcement the team needs to make new thinking stick.

Option 1: The Workshop

The core engagement. Pre-workshop interviews, the session itself, and all participant materials.

Option 2: Workshop + Follow-Up

Everything in Option 1, plus a follow-up session 30 days after. We review what's actually working in practice, troubleshoot what isn't, and refine the system against real data the team has collected since.

Option 3: Workshop + Ongoing Coaching

Everything in Option 2, plus 1:1 coaching with the senior engineering leader on implementation, and a quarterly check-in to assess progress and adjust. For leaders who want the measurement thinking to become part of how they run the organization, not a one-time event.

Why This Works

The QA case isn't an outlier. Over years embedded inside engineering organizations, from Fortune 100 to startups, I've seen the same pattern repeat: teams buried under dashboards that report activity instead of driving decisions, leaders blamed for outcomes their measurement systems were never designed to support.

A new framework doesn't fix that. Prescribed metrics don't fix it. What fixes it is leaders who can make the move from decision to hypothesis to signal to action, and who can run that process themselves, on their own work, without waiting for a consultant to tell them what to measure.

That's what this workshop teaches. It's the same thinking I bring to every consulting engagement I run, the part that actually moves organizations, compressed so your leaders can own it for themselves.

When this approach is in place, leaders stop reporting activity and start reporting what they've learned. Capacity questions get sharper answers. Team health issues surface earlier. Executive conversations about whether the organization is improving have data behind them. Not because you adopted a new framework, but because your leaders learned to build the signals their decisions actually need.

Let's Talk

If you're not sure whether this fits your team, that's a conversation worth having. A short call is the fastest way to know.

ryan@ryanlatta.com