Quibb lets you share what you're reading for work. Use Quibb to share news about your industry, discuss what matters, and see what colleagues are reading.
Our mission is to connect professionals over business news and informed commentary — targeting every industry, profession, and geography.
Some have criticized Adii's lean approach to validating PublicBeta. I think it's really smart. Thoughts?
I'm a fan of the "frontend" PM/UX work approach. But I think the key to this is making it work is executing quickly after validation. -- "Taking credit card details is the only way to really know whether someone is willing to pay you." <|-- so true.
I suppose this is the logical extension of the (entirely valid) "we act like it's an automated service, but we're actually staying up all night doing it by hand" model, but I don't like it.
If the point of the exercise was to determine whether people were *really* interested in the original product, then it seems that that lesson was learned and then something else happened. The original product -- the one that interested people enough to get them to pay -- wasn't delivered. From that point forward all of this seems to be a two part lesson:
1. Once you have someone's credit card on file and authorized for a recurring transaction, you're very likely to make money. Negative option billing is pretty bad ass that way.
2. Apparently people who are deep in the startup cult will pay for anything that might make them more startup-y, even if it's not the thing that they thought they were buying.
Seriously, if this were not an internet startup tarting other internet startup people, how sketchy would this sound?
"I have a great car that you can lease for $30/mo."
"Great, here's my credit card."
"Thank you. By the way, we don't actually have any cars, but don't worry -- we won't start charging you until we get you a car."
"But I wanted the car now."
"I know, and now we know that you *really* want a car, which is totally awesome! We'll be in touch when we actually get some!"
"Um...okay. I guess."
"Hi there! We're not going to be leasing cars any more, but I know you'll love our washing machines! And you can just opt out if you don't want a washing machine instead of a car!"
And I'd be more than a little concerned about what else the company's management might consider acceptable as long as their "intention is right." That they didn't charge the cards immediately means that this probably didn't constitute actionable fraud, but the founder/author doesn't even seem to recognize how close he was to that line -- and the fact that they went opt out, rather than opt in, for the "pivoted" service doesn't make me feel any better about their judgement.
True, but to push back a bit, I think there are other ways to message this behavior and make it seem ok.
"We have great cars that you can lease for $30/mo, but we only have a limited supply, so first come first serve."
"Great, here's my credit card."
"Thank you. Unfortunately, we are just out of stock, but we are actively working on getting some more now." <--- how do you know that when your favorite ecommerce company is out of stock they aren't just testing demand for a product? :) This is exactly what Bill Gross did to test selling cars on the internet.
I don't think that's "other ways to message the behavior," but rather different behavior.
While I don't like several aspects of this, the biggest one for me is the headline item: he made $4,000 from a product he didn't have -- and a product that he never actually built.
Some number of people didn't opt out -- opt *out* -- after being (eventually) told that they would not receive the product they thought they were buying, so the new product inherited some money from that bait and switch. Fine, but don't cast that as anything other than what it is.
I'm not a huge fan of the term "dark patterns," but I'm tempted to use it here. The business model of getting a recurring charge authorized through whatever means necessary and then making money from the people who don't opt out is not new, lean, or worthy of any applause.
Great example, Elizabeth Yin . Here's the case study she's referring to for the uninitiated: http://pandodaily.com/2012/05/10/bill-gross-the-best-advice-i-ever-received/ Later in his career, he launched CarsDirect.com in 1998 to validate the hypothesis that he wasn’t the only person who would be brave enough to buy a car online. After 24-hours of the site going live, he had sold four cars at $1,000 over invoice that he didn’t even own yet. He had to then go to local dealerships and negotiate purchases that would not lose him money.
Whitney McNamara Just to clarify one thing: We only changed gears and tweaked our roadmap (+ primary value proposition), because the vast majority of the people that had signed up, told us to do that. I didn't know that, that would happen to be honest and until I had those customer interviews, I was fully expecting to build & produce the V1 previously planned.
Furthermore, not one, single customer ultimately left with bad vibes. Everyone has said they'll consider being a customer in the future. Yes, they were disappointed when I told them that this was a validation test, but after explaining the reasoning behind this along with our future roadmap, they were fine.
I'm not saying that's perfect; I'm just saying that 99% of the criticism I've gotten has been from people that were never even customers (and probably not even heard of PublicBeta before).
Hey, Adii -
Great to have you in the discussion here, and I'm grateful that you don't seem to be taking my criticism personally -- I think we have rather different perspectives on this particular case, but disagreement and discussion can be very healthy.
Starting from there, it seems to me that if your customers requested the change in focus from the original product idea, that suggests a somewhat different and more complicated story than the one I take away from your post.
If people were willing to sign up for the v1 service that you presented, but then told you that they didn't actually want v1 but rather something else entirely, that raises a lot of questions for me. Why did those users register and (as far as they knew) pay for a service that they didn't actually want?
It sounds to me like the exercise may not have validated anything about the original product, but rather identified a group of people who had both money to spend and an interest in startups/entrepreneurship. Once you had identified that group, you figured out the product that they actually wanted (which was not the v1 that they had been offered). That's not a bad thing, but the fact that you yourself weren't feeling great about the means to that end seems like something worth continuing thought: what other mechanisms could you use to achieve this end, that don't involve the same degree of deception?
I also still feel strongly that taking credit cards up front for product A and then billing for product B on an opt out basis is treading close to an area that could get you in real trouble, and I wouldn't want to see startups pushing that boundary.
The initial value proposition had two main components: content & community. So when we started doing customer interviews, our customers told us that they signed up mostly for the community. Hence why we tweaked our roadmap to focus on that first and build out content over time.
How can this line of thinking be applauded? This is deception pure and simple. While bootstrappers and entrepreneurs on the sidelines are lusting after these kinds of posts because it shows you don't really have to do work before making any kind of money how would you feel if you were on the other end of the transaction and someone had your credit card details with nothing to show for it? I understand the MVP movement and agree you shouldn't spend 5 years making a business without testing its validity, but this is nothing more than 5 minutes of work with the facade that you have created a valuable resource.
* "It doesn’t feel good to deceive prospective customers (or anyone for that matter). I didn’t like this bit."
Um, so yeah, don't deceive. It just isn't cool.
* "Regardless of how ethical (big or small) the deception is..."
Regardless? Deception isn't ethical at all.
The #1 trait of a designer or product person is empathy. In no way is any of that shown with this technique. This is a gateway to getting sued.
This line of reasoning is a bit hand wavy to me, but I'll leave that for a moment.
I think this is perfectly ethical to do, if the intent is in fact to create a product that fills a need that people might have. If you can actually attract an audience for a product you're willing to build if there's a market for it, you owe it your early adopters to ensure that the product will have enough demand that you'll continue working on it.
Consider the alternative: Building a product, having some (middling) traction, and then realizing that you just have to shut the product down shortly thereafter. That's arguable more frustrating for the user b/c they've now updated all of their user data and might have grown reliant on your service.
This is somewhat beside the point, but isn't an extreme version of selling a product one doesn't have just Kickstarter? Do you consider that unethical as well? I'm sure that the people's lives who've been improved both by purchasing products on Kickstarter and the new upstart companies/bands/artists who can spend time fulfilling their creative endeavors consider it perfectly ethical and quite marvelous.
As I've said before, the key issue for me is that the company never actually delivered the product that their customers bought.
The post never addresses why they decided not to actually make the educational videos that their customers thought they were getting (my guess is that the company figured out that the time and cost of making those videos were prohibitive based on the user interest -- $2k in recurring revenue sounds nice, but it's not enough to come close to creating even semi-professional video).
If the exercise has stopped there, with the company telling customers that they couldn't deliver and that no charges would be placed on the cards, I wouldn't have much of an issue with this approach.
What really bothers me is that instead of taking this approach, the company created a completely unrelated product (a startup focused online forum is not in any way educational videos), and then started charging the credit card of any customer who didn't opt out of receiving this new product -- which, again, was not the product that they were promised when they authorized the charge.
I have a very hard time seeing how that ends up being different from the classic bait-and-switch approach to shady commerce.
PublicBeta has $100k in seed funding (from my personal savings). If the majority of customers wanted semi-professional video content (even at $2k a pop), I could go out now and produce that in a heartbeat.
Extreme version of Kickstarter? Not even close. With Kickstarter you get to see the final product or know that the person behind the campaign is actively getting money to pursue development. The approach in this article is completely opposite. It makes people think they are getting something right away when they actually got nothing. That's deception.
"Consider the alternative: Building a product, having some (middling) traction, and then realizing that you just have to shut the product down shortly thereafter."
With all the business books, research, and insights published over the past 20 years there aren't better ways to test a business's validity? Come on now. Lying to people is never ethical and it bothers me that someone working at a big VC firm would even think so.
What is the problem of saying "I plan on creating this product and if you are interested I would like your help?"
Even better, why not offer an early adopter price or pre-beta signup price to test and see if people want to join?
The current $30 pm is an early adopter price. ;)
A pre-beta signup won't validate the ultimate assumption: will people repeatedly buy and subscribe to the service. This would only validate that I can find X many people to initially take the plunge with this thing I want to build. That too is validation, but not the validation that exclusively leads to a longer-term, sustainable business.
That's very similar to getting investors to give me $500k. Yes, they're backing my ideas, but they're not really buying the product.
I had to find this comment thread again to post this.... Adii Pienaar Have you seen onlycoin.com yet? They're (immediately) charging $55 and it wont ship until the summer of 2014!!
I can't say too much in detail here, but I know of startup in Seattle that has more than $1m in bookings before they had engineers. And we're not talking 2-3 big deals here, we're talking several 100s of smaller deals with a validated user need. Needless to say, I was astonished and impressed... and even made me rethink some of my own approaches to the lean startup.
I love this - and I even mentioned this method in the class I teach last night (Marketing for Startups at a local college). I think it needs to be handled well, and I think the people who sign up but aren't charged yet should also be send a handwritten thank you from the team and a t-shirt. It takes a LOT of chutzpah to do this to someone.
I think you can't validate a business launching a landing page only, the only thing you can validate is the problem. You still need to validate the solution and the market.
In this particular case, if someone is willing to pay such an amount, for me it means that this problem is big one. So my next questions will be:
- How many people have this problem?
- How often these clients will use your product?
and of course many other questions:)
Overall, I think it's a smart and bold move, and I really appreciate sharing this experience. Thanks Adii Pienaar