What should a startup measure from day one?
TL;DR. Track one number from your first week, as two counts over the same group. Of the real users who signed up in a given week, count how many did your product's core action within their first seven days (activation), and how many did it in days 7 to 13 after their own signup (the start of retention). Real means your team, friends and test accounts carry a flag and stay out. A product used monthly, like many business tools, uses the first 30 days and days 30 to 59, and counts accounts, not users.
Four things fake that number: insiders in the count, one blended figure, an undefined "came back", and percentages on too few users. Before you can read it, keep the raw rows.
What is the one number, in plain words?
- Real users. Team, friends, family and test accounts carry a flag and stay out, as far as you can tell them apart.
- Signup week. Grouped by the week they joined, so new sign-ups cannot hide older ones.
- The core action. What the product exists to do (a recipe cooked, a ride completed), not an app open or a finished welcome tour.
- By source. Recorded from day one, read once each group is big enough. A two-sided product gets a number per side.
Picture a recipe app with 60 sign-ups in its first week. The dashboard says 45 percent came back in week two. But 12 of the 60 are the team, friends and testers, and every one came back. Of the 48 real users, 16 cooked a recipe in their first week and 15 cooked one in days 7 to 13: 31 percent. The owner plans around 45; the product is at 31. The numbers are invented; the gap shrinks once sign-ups come from ads.
What did the advice agree on?
In early November 2026 I read a founder forum thread that asked this question: 44 comments, eight substantive. As counts, not evidence: 6 said activation over downloads or onboarding completion; 4 a source column from day one; 3 retention by signup week; 3 called onboarding completion noise; 2 counts beside percentages; none said take the team, friends and testers out.
Investor pages add the one the thread missed. In a guide of 5 October 2026, the venture firm CRV groups cohorts by signup week for high-frequency products and by month for business software, reports activation separately from retention when cohorts start at signup, and takes test and internal accounts out before any computation; its example is a founding team on staging data, retaining at 100 percent forever. A July 2026 piece from the same firm says retention cohorts are among the first things seed investors check, and that friends, former colleagues and family do not show value for the market. Stage 2 Capital made a leading indicator of retention, one observable behaviour per customer, its first metric in February 2026. My reading: a first data hire used to build this table by hand in month six; now a dashboard arrives in an afternoon and counts everyone.
How does the first number lie?
Insiders inside the count. They lift the number and tell you little about the market. Record when the flag changes, and keep billing test mode apart.
GuardrailA named owner keeps the insider list; every number leaves insiders out by default.
CheckCount the insiders in your newest closed signup week and recompute without them.
One blended number. Sequoia's data science team, in an undated article with examples from 2016 and 2017, notes that total retention is the weighted average of cohort retentions, so one large cohort can set the figure, and that early users often retain best until the product fits its market. CRV adds that averaging hides channel differences.
GuardrailOne row per signup week, cohort size on the row, source recorded.
CheckOnce a source group has a few dozen users, the owner splits the number by source.
"Came back" left undefined. CRV says logins and app opens do not count as retention; Sequoia's example is a completed ride, not an app open. Lenny's Newsletter (2022) defines activation as the earliest onboarding step that predicts long-term retention, a hypothesis to validate.
GuardrailOne written line beside the number: which event, which window, who owns it.
CheckA teammate can name the event.
A base too small. Khosla Ventures noted in 2015 that small cohorts vary more; CRV says cohorts under roughly 30 accounts give noise, a rule of thumb its page does not source. My arithmetic: a 95 percent interval for 15 of 48 is about 20 to 45 percent; for 150 of 300, about 44 to 56.
GuardrailThe count beside every percentage, release dates and crash rate on the dashboard, several usage cycles before calling a trend.
CheckIf moving two users changes the story, do not act yet.
What should you record from day one, even before you can read it?
Ask whoever builds the product for an event log that is only added to: one row per action with user id, timestamp, event name and source, plus an event id against double counting. Record welcome steps as events, not as a flag overwritten in place: when each step happened is what finds the core action later. When a user deletes the account, keep a row that says they existed and left, personal fields removed, so old cohorts still add up. The same rule, each with a named owner, covers your billing system, CRM and ad spend exports.
This is the keep-history condition from the guide. Keep one written definition per metric. If you ask a coding agent for a retention dashboard, I think you should expect it to fill the gap with app opens and everyone, so write yours first.
What changes by stage?
I think volume and stakes matter more than stage; stage decides who owns the work and what you may conclude.
- Before launch. The founder sets up the event log and the insider flag. Nothing to conclude yet.
- First paying customers. The founder or a product lead owns the definition and the weekly spreadsheet below. You may conclude that a channel or a cohort differs, not that the product works. A business product with a dozen accounts reads them one by one.
- Growing with a first data person. The data person owns the definition and the insider list, runs the query, splits by source, shows the interval, compares cohorts over time.
No data team? Export two files from your database or analytics tool: one row per user (signup time, source, insider yes or no) and one row per core action (user id, time). In a spreadsheet, derive two columns per user: did the core action within 7 days, and in days 7 to 13. Pivot by signup week, insiders filtered out, weeks under three weeks old left out. Built this way from the raw rows of the test set below, the pivot returned 16 of 48 and 15 of 48.
For your data team
For your data team
This part gets technical. The query was tested on 6 November 2026 in SQLite on a synthetic set: 60 users signed up in the week of Monday 26 October, 12 flagged is_internal (every one returns), 48 real (15 return), as-of date 16 November. A signup week is reported only once it is 21 days old at the as-of date, so a half-finished week cannot hide under a full week's label. Adapt the date functions for your warehouse; count accounts for a business product.
-- Week cohorts, real users only: share who did the core action in days 0 to 6 (activation)
-- and in days 7 to 13 after their own signup; a signup week appears only once every member's window has closed
with params as (select '2026-11-16 00:00:00' as as_of),
cohort as (
select user_id, date(signed_up_at, '-6 days', 'weekday 1') as cohort_week
from users, params
where is_internal = 0
and datetime(date(signed_up_at, '-6 days', 'weekday 1'), '+21 days') <= as_of
),
core as (
select e.user_id,
julianday(e.occurred_at) - julianday(u.signed_up_at) as days_after
from events e
join users u on u.user_id = e.user_id
where e.event_name = 'recipe_cooked'
),
week1 as (select distinct user_id from core where days_after >= 0 and days_after < 7),
week2 as (select distinct user_id from core where days_after >= 7 and days_after < 14)
select c.cohort_week,
count(*) as cohort_size,
count(w1.user_id) as activated,
round(100.0 * count(w1.user_id) / count(*), 1) as pct_activated,
count(w2.user_id) as active_week2,
round(100.0 * count(w2.user_id) / count(*), 1) as pct_week2
from cohort c
left join week1 w1 on w1.user_id = c.user_id
left join week2 w2 on w2.user_id = c.user_id
group by c.cohort_week;
It returned cohort 2026-10-26, size 48, activated 16 (33.3 percent), active in week 2 15 (31.3 percent); without the is_internal filter, 60, 28 and 27 (46.7 and 45.0 percent). With the as-of date set to 13 or 15 November it returned no rows. The flag usually comes from the team's email domain, a list of ids and test devices.
To choose the core action, treat each candidate as a hypothesis. On this set, 7 of the 16 who cooked in week one cooked again in week two (44 percent), against 8 of the other 32 (25 percent), a gap that is not significant (p about 0.2) and uses the same event as the thing it predicts. The decision waits for cohorts in the hundreds, compared inside one source, or a better channel looks like a better action.
The intervals are 95 percent Wilson intervals.
Check it in 10 minutes
- Write the core action and the period in one line, and name who owns it.
- Take the newest signup week at least three weeks old. Count its insiders, and name who owns the list.
- Recompute both counts for that week without them, and put counts beside percentages.
- No users yet? Do step 1, then ask for the event log and the insider flag.
See where your setup stands with the two minute self-check, then read the draft checks your coding agent can run.
Spotted a mistake or a newer source? Tell us and we check it against the source. How we work.
Check your setup in 2 minutesAll postsGet new posts by email