Essay · Operations
I Killed a Survey App Over 50 Cents. A Company in My Category Sold for $1.07 Billion.
In 2016 I co-founded a survey app in San Francisco called Chati.Chat. About a year later I shut it down, partly because I couldn't figure out how to pay people 20 to 50 cents a question. Four years after that, Lucid, a company that matched researchers with survey respondents, sold to Cint for $1.07 billion.
Lucid wasn't my idea, and nobody stole anything. Lucid was in my category, though, and so was Swagbucks. Swagbucks' parent company did more than $300 million in revenue in 2021.
This post covers that call, the opposite mistake, and the system I use now to avoid both.
The survey app I killed too early
Brands and researchers wanted answers. We used Facebook's ad targeting to find the right people, then asked them questions over Messenger and paid them per answer. It was bootstrapped, with a small team, and I paid developers out of my own pocket.
Three things killed it. I'll give you all three, because the clean one-reason version would be a lie.
1. The payout math. We paid through PayPal, and the obvious rails were brutal for pocket change. The standard rate was 2.9% plus 30 cents, and PayPal's newer Payouts API charged a flat 25 cents per US payment, so the fee could eat most or all of a 20 to 50 cent answer. The cheap option, Mass Pay at 2%, needed PayPal's permission, and PayPal could pull it any time. I couldn't crack the formula, and I overthought the economics until I talked myself out of it.
2. Facebook's rules. Facebook only opened Messenger to bots in April 2016, then rewrote the rules that August. From then on you could only message people within 24 hours of their last reply. Opt-in subscription messages were limited to news, productivity and personal trackers. On top of that, the platform policy only let you reward people for logging in, entering a promotion or checking in. Paying strangers per answer inside Messenger was never what those rules were built for. It was a great use of existing tech that the terms didn't make room for.
3. My focus. I had a growing Amazon business at the same time, and I wanted to put my hours there.
All three are real. The one that stings is the payout problem, because the market had already solved it before I started. Swagbucks had been running on points since 2008. You pile up small earnings and cash out once you pass a threshold. Today that's $5 to PayPal, or a $1 Amazon gift card. Survey Junkie cashes out at $5. Google Opinion Rewards pays Android users in Play credit and only sends PayPal cash once you reach $2. Nobody solved micro-payments. They batched them.
The fix for my "impossible" problem was points and gift cards, and they were sitting right there in the market while I was building. I didn't look sideways.
I'm about to kill [IDEA] because of [THE PROBLEM].
Before I do, look sideways:
1. Who in this category has already solved this problem or worked around it? Name real companies and say how they did it.
2. Is mine a kill problem or an adjust problem? Explain why.
3. If it's an adjust problem, what's the cheapest 2-week test of the workaround?
Only name companies you're confident exist. If you're not sure, say so. So I killed a whole business over one choke point. That's an adjust problem, and I treated it like a kill problem.
The opposite mistake: an agency I kept alive too long
Killing too early is loud. Running too long is quiet, and it costs more.
At an agency I ran, we got stuck in experiment loops. We'd try something, read the tea leaves, then try something else. The problem was that we tried each direction once and made big decisions off that single data point.
One attempt is an anecdote, not a test.
Both failures come from the same place. No gate was set before launch. With Chati.Chat I quit on a feeling about the math. With the agency I kept going on a feeling that the next experiment would be the one. Neither time had I decided upfront what "working" looked like, or how many reps a direction got before I judged it.
You can't score apples like oranges
Every business is shaped differently, so every business needs its own scorecard. Take two I've worked with.
A game publisher lives and dies on fandom. One trusted expert reviewer can move thousands of buyers, and the fans pay attention to what reviewers say. Reviews and fan sentiment are leading numbers.
A hospitality technology optimization business is B2B back-office tech. It squeezes small margin and operations gains out of hospitality businesses. That makes it pure numbers. Customer opinion barely matters as long as the tech delivers what it promised. What counts is whether the margin moved, whether the system stayed up, and whether the client renewed.
Score those two on the same field and you'll kill the wrong one.
A reorder rate is a great metric for a consumable and a useless one for a SaaS tool. Ask every opportunity the same questions so you can compare them, but let each niche set its own metrics, thresholds and timelines.
Sometimes the deciding gate isn't the P&L at all. It's a rule, a license or a platform policy. Chati.Chat is proof.
What AI speeds up, and what it can't
AI changed the front half of this. I can build a site, a shop, an app, ad creative and a launch in a fraction of the old time. Six months of "is there anything here?" turns into a couple of weeks of real signal. (Here's how I build a site with AI.)
There's plenty AI can't rush: long-term loyalty, reorders, a real organic following and trust from reviewers. A reorder only shows up after a full reorder cycle, and no tool shortens that.
So you get fast signal: can we buy attention, and do people convert? You also get slow signal: do they come back, and is a customer worth more than they cost? Don't kill on a slow question using fast data.
Gates, not formulas
A gate is a scheduled check-in with yourself, and your partners if you have them, to decide where to go next.
Here's one example clock, for a consumer product people reorder fast:
- Day 15: ad and creative signal, cost to acquire a customer
- Day 30: early retention
- Day 45: conversion and first reorders
- Day 60: reorder pace
- Day 90: lifetime value and the big verdict
That's one niche's clock. A venue, a SaaS product or a service line runs a different one. Set yours before day 0, and decide what day 0 even means (site live, first ad or first sale).
I'm about to launch [IDEA]. Before I start, help me write my kill criteria.
Business shape: [fandom / pure numbers / B2B tech / other]
Day 0 means: [site live / first ad / first sale]
Time and money I can risk: [AMOUNT]
1. Pick the 3 numbers that matter most for this shape, and say why.
2. Set gates for my niche (for example day 15, 30 and 90) and what each one checks.
3. For each gate, write what kills it, what revives it, and the cheapest next test.
4. Write my stop rule and my scale rule in one sentence each.
"I'm tired of this" is not a kill condition. Don't let me use it. Every gate has three possible verdicts: continue, adjust or kill. Every verdict has to name three things:
- What kills it. Something specific you can observe, like losing money on every order after a set number of orders, or a platform saying no in writing.
- What revives it. Evidence, not enthusiasm. A cohort that reorders, a real price from a real buyer, or a gatekeeper's yes.
- The cheapest next test. Run a free check before a paid one, and have a conversation before you spend money.
Run Chati.Chat back through that. Kill: the platform rules block delivery. Revive: a payout method where fees stop eating the reward. Cheapest next test: a points wallet with a gift-card threshold for a small beta group, two weeks, near-zero cost. I never ran that test. I just hit kill.
Adjust means fix the choke point
Take a hypothetical. You launch an e-commerce product and your acquisition cost comes in lower than expected, which is great, but returns are high.
Don't throw it out. Find the single step with the most friction and fix that first. That might mean better instructions, better packaging or a small product tweak.
That's the bottleneck idea: a system only moves as fast as its tightest point. It comes from Eli Goldratt's Theory of Constraints.
When you adjust, change one variable, whether that's the offer, the audience, the channel, the price or the product. Then give it enough reps to mean something. That's the agency lesson again.
My launch has a problem. Help me find the choke point before I kill it.
What's working: [e.g. acquisition cost is lower than expected]
What's not: [e.g. returns are high]
My funnel: [e.g. ad > page > checkout > delivery > reorder]
1. Name the single step with the most friction, and show your reasoning.
2. Give me 3 fixes for that step only, cheapest first.
3. Pick one variable to change (offer, audience, channel, price or product) and tell me how many reps it needs before I judge it.
Don't pitch me a new idea. Fix this one first. The captain picks the rule
There's no single right way to make the call. It depends on your time, budget, effort and reason for doing it. Three setups work:
- Pure numbers, ruthless. Set thresholds in advance and let the numbers decide.
- An automated KPI kill. For example, a subscription business shuts itself off if acquisition cost or lifetime value isn't better than average by the gate. (I wire tripwires like this in PostHog.)
- Gate questions. What did this launch prove? What did it generate? What value did it bring? What lessons did it give us? Answer them at the gate and decide on the spot.
AI can watch the dashboards. The captain still has to stay awake at the wheel.
My gut vs. the spreadsheet
My gut wins on executive, brand, customer-experience, design and creative calls. There's a line attributed to Henry Ford: "If I had asked people what they wanted, they would have said faster horses." Sometimes the captain sees past what customers can put into words.
The spreadsheet wins on the verdict itself. Gut decides what to try. The numbers decide whether it worked.
My gut has also lied to me, in two patterns I've learned to override.
The honeymoon. When I get excited about something, I take it as a sign I should do it. A lot of that early excitement is the honeymoon phase of a romance that won't last. It's a new toy. Being excited to do something is different from being excited to do it for a very long time, and only the second one survives the long, boring middle.
Quitting right before the diamonds. You've probably seen the meme: a miner tunneling through rock turns around and walks away, and he's inches from a wall of diamonds. My gut tells me to stop a bit too early. Gates are how I catch it, because "I'm tired of this" isn't one of the kill conditions.
Am I excited about [IDEA], or excited about a new toy?
Interview me. Ask one question at a time, 7 questions max, about:
- what I'd be doing on a boring Tuesday in year 3
- the parts of the work I'd have to do every single week
- the lifestyle this business drags behind it if it works
Then tell me straight: is this honeymoon excitement or a 5-year interest? Quote my own answers back as evidence. Before the kill switch comes the pick
Everything above is for after you launch. Picking the idea comes first, and I covered it in my 10-sign checklist video. Sign five is that competition is proof the dogs are eating the dog food. With Chati.Chat I saw Swagbucks as a threat when it was actually a playbook.
Sidebar: The Gate Sheet
Fill this in before day 0.
- Business shape: fandom / pure numbers / B2B tech / other: ______
- Day 0 is: ______ Gate dates: ______
- My 3 niche numbers: ______ Universal line (P&L, margin, cash at risk): ______
- Stop rule: ______ Scale rule: ______
- Who decides, and how (pure numbers / automated KPI / gate questions): ______
- At each gate: verdict ______ kills it ______ revives it ______ cheapest next test ______
- Premortem: "It's day 90 and this failed. Why?" ______
Run a premortem on [IDEA].
It's day 90 and this failed. Write the post-mortem.
- Give me the 5 most likely reasons, ranked by how likely they are.
- For each one, name the earliest signal I'd see and the day I'd see it.
- Flag any reason that's a rule, a license or a platform policy, not a P&L number.
- Tell me the cheapest test I could run this week to rule out the top reason.
Be blunt. I'd rather hear it now than on day 90. The short version
Set your gates before you launch, read your own niche's numbers, and make every verdict name what kills it, what revives it, and the cheapest next test. Before you kill something, look sideways. Someone in your category may have already paid for the answer.
If you're picking your next idea, start with the 10-sign checklist. If you're already running one, steal the prompts above and set your gates this week. I post what I'm building, and what I kill, on X @PalafoxPalmer.
The idea, in brief
Sources
- Cint completes Lucid acquisition (USD 1,070M, Dec 29, 2021): https://news.cision.com/cint-ab/r/cint-group-ab--publ--completes-the-acquisition-of-lucid-holdings--llc,c3479611
- Cint to acquire Lucid (deal terms): https://news.cision.com/cint-ab/r/cint-group-ab--publ--to-acquire-lucid--intends-to-raise-equity-and-has-secured-debt-financing,c3441847
- Prodege 2021 results (more than $300M revenue): https://www.prodege.com/resources/press-release/audience/prodeges-record-year-showcases-innovation-in-evolving-tech-landscape-and-charts-a-strong-course-for-2022/
- Swagbucks PayPal and gift-card thresholds: https://www.swagbucks.com/rewards/PayPal
- Survey Junkie $5 cash-out: https://www.surveyjunkie.com/
- Google Opinion Rewards $2 PayPal threshold: https://support.google.com/opinionrewards/answer/7378183
- Facebook, "Bots for Messenger" (Apr 12, 2016): https://developers.facebook.com/blog/post/2016/04/12/bots-for-messenger/
- Facebook, "Updated Messenger Platform Policies" (Aug 15, 2016): https://developers.facebook.com/blog/post/2016/08/15/updated-messenger-platform-policies/
- TechCrunch on the Aug 2016 Messenger rules: https://techcrunch.com/2016/08/16/i-bot/
- Facebook Platform Policy as of Dec 2016 (incentives, data use, Messenger): https://web.archive.org/web/20161216072616/https://developers.facebook.com/policy/
- Chati.Chat dates and role: https://linkedin.com/in/shipwithmax
- Theory of Constraints, introduced in Eliyahu Goldratt's The Goal (1984), Theory of Constraints Institute: https://www.tocinstitute.org/history-of-toc.html
- "Faster horses" quote, attributed to Henry Ford, with no evidence he said it (Quote Investigator): https://quoteinvestigator.com/2011/07/28/ford-faster-horse/
- PayPal standard rate, 2.9% + $0.30 (merchant fees, Sep 2016): https://web.archive.org/web/20160930120748/https://www.paypal.com/us/webapps/mpp/merchant-fees
- PayPal Mass Pay, 2% capped at $1 (Sep 2016): https://web.archive.org/web/20160930120820/https://www.paypal.com/us/webapps/mpp/mass-payments
- PayPal User Agreement (effective Oct 19, 2016): Payouts API flat $0.25 per US payment, Payouts and Mass Pay need PayPal's permission: https://web.archive.org/web/20170228230632/https://www.paypal.com/us/webapps/mpp/ua/useragreement-full
- Full notes: source-facebook-terms-2016.md, source-paypal-2016.md