Defining customer value (strategic research)
How do customers actually derive value from your product — and does it match what you assume? As research lead, I designed and ran an 8-week mixed-methods study to answer this question, validating our pricing plan assumptions against real usage data, customer interviews, and in-product experiments.
THE STRATEGIC CHALLENGE
Assumptions vs. Reality
Without concrete data, customer success couldn't demonstrate ROI, product couldn't prioritise confidently, and marketing risked promoting features that didn't match real behavior. We needed to test our assumptions.
RESEARCH APPROACH
Core research questions
I designed a comprehensive research methodology that combined quantitative usage analytics with qualitative customer insights to definitively answer our core research question:
"How do Intruder's customers actually define and derive value from the platform, and does their usage match our existing assumptions?"
The research was structured as a sprint-based programme, with each pricing plan investigated over a two-week cycle to maintain stakeholder momentum and close data gaps incrementally.
What have we discovered?
Essential customers confirmed our core assumption: more than 70% of their scans were scheduled, meaning they treat Intruder as a procedural, "set-and-forget" tool. However, we couldn't confirm compliance-driven seasonality; no quarterly usage peaks were found, suggesting compliance is a motivation to buy, but not a driver of how they use the product.
Pro findings were the hardest to validate. Reporting data proved unreliable due to tracking gaps — a finding in itself that led to recommendations for analytics infrastructure improvements. Interestingly, Pro users remediated vulnerabilities faster than Premium users, partially confirming that speed matters more to this segment than advanced features.
Premium delivered the most strategically significant insight: Attack Surface Monitoring — our biggest feature bet and key differentiator for this tier — accounted for only ~5% of platform views. The assumed value driver simply wasn't driving value. Instead, Premium users added significantly more team members than other plans, revealing that collaboration, not features, is what actually justifies the higher price point.
Customer interviews added critical context that the data alone couldn't explain.
We also discovered that some customers use Intruder for "passive oversight" rather than active remediation — a value proposition we hadn't considered at all.
VALIDATING
Through In-Product Experiments
To validate these findings at scale, we ran a modal experiment across 146 user interactions, asking customers to choose what action mattered most to them.
The experiment strongly validated that compliance reporting — not remediation — is the primary action Pro customers associate with value, challenging our assumption that speed-to-fix was their key driver.
RESEARCH IMPACT
Through In-Product Experiments
Although Phase 2 being closed early due to a team restructure, the research delivered lasting strategic value across four areas:
Validated: essential customers buy procedurally. 70% of scans were scheduled, confirming "set-and-forget" purchasing behaviour. Premium customers value collaboration over features, adding 2x more team members than other plans.
Refuted: premium customers barely used ASM (~5% of platform views) — our biggest feature bet on this tier was wrong. Compliance reporting didn't show the expected quarterly usage patterns, suggesting compliance motivates the purchase but not ongoing usage.
Frameworks built: replicable research methodology including interview protocols, modal experiment templates, and quantitative validation approaches — all reusable for future research initiatives.
Gaps identified: unreliable integration event tracking, inconsistent report download measurement, and unclear behaviour metric definitions — a roadmap was delivered to engineering for future improvements.
This project reinforced what I believe is the most important skill in design leadership: designing work that delivers value incrementally. When Phase 2 was closed due to shifting team priorities, the insights didn't disappear — the validated hypotheses informed product strategy, the interview findings shaped how customer success approached renewals, and the replicable frameworks (interview protocols, experiment templates, validation approaches) gave the team tools to continue the work without me.
Most importantly, having the courage to prove our own assumptions wrong — like the Premium ASM bet — was far more valuable to the business than confirming what we hoped was true.