Th Episerver acquisition is indeed a bad exit, and I think I will lose >$100k in stock I exercised (which is OK, I'll be fine). But I hope all readers will take from this saga a lesson in humility and the pitfalls of intellectual dishonesty and hubris. Just because your startup is skyrocketing isn't enough. Success is not guaranteed. Your company's leadership needs to be honest with itself, which Optimizely's leadership was not. They need to be humble and work hard, which Optimizely did not do.
[1] https://zachholman.com/posts/fuck-your-90-day-exercise-windo...
Is it not the case that these startups begin developer-facing, get market traction, are lavishly funded, and then discover that the self-service offering they've built simply can't satisfy the projections they've made to justify their valuation?
Which is to say: would Optimizely be doing much better if they hadn't pivoted into enterprise hell? Or would they be a much smaller company?
I see why customers would have a strong preference! But it's less clear to me what the right decision for the business is. But I'm just asking!
I completely agree with you on (at least in terms of Twilio's experience):
> would Optimizely be doing much better if they hadn't pivoted into enterprise hell? Or would they be a much smaller company?
We were definitely going to hit a ceiling if we didn't add enterprise features and work on getting all the various certifications that large companies will require to even start talking to you, and build a sales force that knew how to sell to larger companies. If we hadn't done all that, we'd be a much smaller company today, and likely a competitor would have done it instead and eaten our lunch.
One of the important distinctions between Optimizely and most developer-first platforms is that experimentation is a hard practice to pick up. Most companies have difficulties getting their programs off the ground and keeping them funded, let alone grow or scale them. Small digital businesses struggle more for several reasons: 1) they have few resources, so teams are understaffed and resources are pulled easily, 2) they have little money, so the percentage uplifts are rarely motivating, 3) they have little traffic, so it is harder to get a statistically significant measure in their experiments
Because of these issues, Optimizely always had really poor retention in the SMB space. Nonetheless, the SMB customers helped Optimizely build up a brand name, build up legions of practitioners, and get the skills and experience to go after the Enterprise market. When Optimizely started acquiring enterprise customers, retention improved substantially.
This isn't to say that there aren't lots of problems with Enterprise sales and that Optimizely didn't make tons of cultural mistakes in that pivot. But on the core financials, Enterprise kept Optimizely afloat. The problem wasn't the pivot to enterprise, but the trade-offs that were mismanaged along the way. The path to enterprise was inevitable and correct.
CROs and VP of Sales types don't like the idea because they can't control the messaging and manage the sales funnel. But if you have people who genuinely love your product and will champion it internally, ignore these fools. Just keep delivering a quality product with value and if there is a market, the revenue will follow.
GitHub, Twilio, and OpenDNS all built their businesses off people taking tools to work.
For what it's worth, I would guess that had Optimizely not pivoted to the enterprise they would indeed be smaller, but more importantly would have had a slower growth rate at least at that point in time. In an industry obsessed with high growth rates that's the kiss of death and I imagine the reason behind their pivot. But that's just my guess.
I suspect a lot of startups sell their investors on enterprise from the get-go. Self-serve can be seen as a foot in the door, a way to prospect potential enterprise opportunities. They watch self-serve sign-ups for a bigcorp.com domain and then hand it over to account sales and swing for the fences.
I think a big problem is differentiating your self-serve from your enterprise offering. You don't want bigcorp.com to feel happy enough with your $20/month foot in the door offering. And at the same time enterprises aren't stupid money fountains and they don't just sign $100k/year contracts unless they see major value. I think this creates a volatile business where a dozen or so enterprises make up the lions share of revenue for a startup and the thousands of self-serve customers are just kind of there like background noise.
Problems are often:
-- Beyond just VC, it becomes an issue ongoing to having full-time dedicated sales: quota, qualified leads, etc, it's a beast. Internal culture and priority shift to start/maintain/grow.
-- if not naturally pulled here and no obvious tier separatation, above can easily destroy the SMB side, instead of using as part of your moat + growth. The marketing+sales org will kill it, unless you split those as well or otherwise solve
-- VC money just means artificially faster deadlines and expectations for all of the above, so even harder to do 2 things, even if long term natural and better to habdle
I think the problem is to become "lavishly funded" means that you need to have a pitch that creates a narrative of how you will reach a lavish level of revenue that is believable.
For B2B SaaS, usually that means moving upmarket and raising prices.
I don't think we can say that Optimizely was necessary wrong in doing what they did with knowing what they knew at the time. There are many examples of B2B SaaS companies successfully starting with SMB, then going enterprise. First company that comes to mind is New Relic ($3.6 billion market cap)
But I distinctly remember at the second job, that incredibly rough transition to that vague enterprise pricing structure along with the software getting clunkier and clunkier, which led us to abandon them and never look back.
I think at one point they were trying to get us to go from paying $99 a month (!!!) to like $3,000 a month for virtually the same service??
I've been there.
When they sent me the details for the on-site they sent me a role two levels lower. When I asked about it I was told the Director of Engineering felt that was a suitable role.
I withdrew from the process at that point. I was pretty bummed as I was super high on Optimizely back then. But I knew enough about myself to know that wasn't a good way to start a job (assuming I got an offer).
But, seriously, your description does sound exactly like a company on the decline. I couldn't blame you for bailing. What do you think the first signs that things were terminal there?
For example, say you're running a tool that allows people to quickly experiment with multiple versions of their web site, measure some quantifiable success rate for each version, and perform some basic statistical analysis to guide future changes and improve conversion rates. A basic but useful version of this tool can be implemented in a few days by one competent developer and one competent statistician; I suspect quite a few people reading this discussion have done exactly that. Polishing the tool might take longer and improve its utility somewhat, but it's not as though it's using some secret technique that no normal business can implement for themselves in-house.
At the mostly-self-service end of the spectrum, it might still be worth customers spending a bit of money on the pre-existing tool you make to do that job for them, because you're really competing on immediacy and convenience as much as technical capabilities. At the enterprise level, your competition could instead be some in-house team or some freelancer or agency being brought in from outside just to develop a tool directly for your customer. If they're potentially doing that at a cost less than just the first year of annual fees you're demanding up-front, and according to the customer's exact requirements, both of which seem quite plausible in a case like Optimizely's based on information in other comments here, what exactly is your sales pitch?
My company is going through a similar phase of trying to move more up market, and I'm glad we've tried to keep the free/self-serve tiers explicitly because we think they build good mindshare.
And do you know if the self-sever market was drying up for Optimizely in 2015? I assume they wouldn't abandon it if it was growing at a decent rate.
Slowing down users meant higher prices to sustain VC promised growth. Higher prices meant enterprise sales. And it went down from there.
When Optimizely hiked their prices and lost the smallest plan, all these clients went elsewhere. I'm sure their contribution was noise to Optimizely's turnover, but the awareness of what was possible with A/B testing and the way they'd tell their friends about Optimizely and what they'd just done on their website - you couldn't buy that exposure.
Based on what information? So far, all that's been publicly confirmed is that the sale price was below $600M, which presumably leaves opportunity for your shares to be worth something.
Maybe my serendipitous reading explain your observations of what happened to the company.
Continuing self-service while expanding "up market" into the enterprise. If AWS can be self-service, pretty much any service can be.
Additionally as a sidenote, that market is quite crowded with a LOT of choices. This is certainly not a winner take all industry, there are literally 20+ choices that will be reliable. Typically most large enterprises will end up with a vendor that will give them a good deal on rates, which isn't Stripe.
I don't know how you can be so sure that Stripe will win, Adyen is already on the enterprise space, and they are a bigger company than Stripe.
> Continuing self-service while expanding "up market" into the enterprise.
Things are more complex than a sentence, the whole company needs to operate differently https://blog.luap.info/why-most-saas-companies-cant-be-succe...
Also, to clarify, I used to end my interviews with TWO questions: (1) if you had 10 years to live, what would you do? [wait for answer] (2) if you had 10 years to live, would you take this job?
The things I was assessing in these questions were candor, intellectual honesty, and passion. Sure, it would be great if people authentically were passionate about taking the job if they had ten years to live. That was a tiny minority of responses.
The only "wrong" answer to these question was when someone would answer YES to the second question after clearly answering something completely different to the first one. For example, if someone would say travel the world to #1 and yes to #2.
The reason why this measured candor was because if someone could tell me to my face during an interview they wouldn't take this job, then I knew they would tell me to my face when something was broken in the company after I hired them. I was looking for the exact opposite of what this thread implies I was looking for. I didn't want ass kissers. I wanted truth tellers.
1. The op is lying about his experience.
He states he answered NO to question 2, which is what you claim to be looking for. Unless his answer to question 1 was "Work for Optimizely" (which, I guess, someone might say, maybe) then I don't see how you get a contradiction. By your logic, he would have been a hire.
2. The recruiter lied to him about why he was rejected.
Maybe everyone got a short sit down with the CEO back then and he took that to mean he passed all the other interviews? Maybe something else didn't check out and that was an easy way to let him down?
Anyway, if you're A/B testing, wouldn't you hire people regardless of their answers and then assess their performance over a longer timeframe to determine the efficacy of the questions and answers?
That is a strange assumption.
My answers to #1 and #2 would have been contradictory to you, but with 10 years to live I would see working at Optimizely for 3-5, then doing what I wanted to do for 7-5 years a great tradeoff.
Would I have said that in answer to #1? No. I'd have said "Travel and more time with friends and family", while knowing full-well I had to work, or I would starve.
Would I have said it in answer to #2? Perhaps, but unlikely.
What was the success rate with these two questions?
But there's a silver lining: we created and open-sourced Alephbet[0] - a simple A/B testing platform together with a couple of backend options with AWS Lambda/redis[1] and couldn't be happier :)
They've priced out self-service and smaller users, and repositioned their sales model for larger companies with immature internal capabilities. They lock you in with that annual pricing, and include enough margin to throw a massive amount of support resources at you to ensure you get everything fully off the ground and deeply embedded into your internal workflows.
As an early self-service user, it was really irritating when I tried to bring them into a new company I started at and realized they made that change. But after working for a major marketing agency for a while, I've realized that it makes sense for them (even if it sucks for my purposes). In the world of large scale brand marketing companies (such as CPG companies), even a rudimentary informational/branding/brochure-ware website tends to be a $500k+ abomination, involving a super complex IAT[1] consisting of 3-6 external agencies and internal teams. In that world, the single greatest cost for anything is the man-hours required for account management, since even the tiniest of thing involves so much coordination (both logistically and politically). Optimizely's absurd looking price bakes in the cost of providing that level of account management support as well as initial implementation/usage technical support. Without those, it's entirely likely that the brand could purchase Optimizely and it'll sit unused because the agency scopes don't account for it and no one is willing to eat the unscoped hours required to implement/support/use it.
[1] https://isl.co/2018/10/agile-iat-four-principles-for-better-...
Ultimately, i believe they got squeezed between smaller companies using free or cheaper offerings and larger companies probably building it themselves.
Full Story has gone this route. They used to have a plan that would work for smaller companies, now they have nothing between their free tier (1000 sessions a month) and their lowest level paid tier which is "5 figures annually" (they won't even announce any pricing on their website, though not sure if they ever did).
HUGE thank you to YC and the entire HN community for all their support over the years. It was almost ten years ago that we launched here on HN. [1]
Congratulations for the acquisition. We've enjoyed competing with Optimizely over last the last 10 years, and certainly learned a lot in the process. Hopefully, that will continue even after the acquisition.
The way we look at things, experimentation as a market is certainly in an early phase. The more complex the world becomes, the more necessary experimentation becomes to understand what customers really want.
As an aside, it is true though that for experimentation to work, a company needs to be ready for it. Their culture needs to support being proven wrong and having patience to build momentum of wins over the long term. Any org with a short term horizon will likely see experimentation as a cost without corresponding returns. But companies that really see long term - think Amazon - ground themselves in experimentation.
Of course, not every company can be Amazon but our belief is that more companies will start realizing that there's no alternative to developing a culture of experimentation. This is why we're excited about the market. For us, at VWO, it still seems day 1 :)
Many players have jumped into this space with their own A/B testing and Feature Flags solutions as part of their total offering, many of those offerings being free, open source or cheaper. Also potentially better in the concrete tasks they enable. I doubt that Optimizely's feature flag offering is superior to something more specialized like LaunchDarkly.
Also there are a couple of strong incumbents' solutions like Google Optimize and Adobe Target and it's always hard to go against incumbents specially when the incumbents are coming after you and not the other way around.
One clear problem for Optimizely in this space is that experience optimization became a function of marketing departments through out the years but for a while they were positioning themselves as a developer tool. This go-to-market strategy opened a lot of opportunities for other players who saw a bigger market when selling the same type of solution to Marketing Departments.
Maybe I'm wrong but it seems that they just stopped growing and have been experiencing a lot of customer churn since this is likely an expensive product with a hard to calculate ROI. They're probably still selling a lot but nowhere near to the original investor expectations / close to becoming profitable.
Maybe this is ad hominem, but it seems to me they must've raised a lot of money to prioritize that kind of setting, likely in the guise of recruiting. Crunchbase lists them as having raised $251.2M and their last round being debt financing.
If this is a down-round acquisition with most of the employees gaining very little I wonder if this is a lesson to founders to be more cost-effective and raise less money.
I can't speak for all industries, but for the ones I'm familiar with, marketing is always the product owner of websites. With two practical implications being
1. The entire website costs (development + software vendors) ultimately get booked against their budget, so they have the true purchasing authority for pretty much all the website tech that isn't centrally mandated/controlled
2. Website projects (including budget and requirements planning) tend to start in marketing loooonnnggg before a tech resource gets brought in. So developer awareness/familiarity ends up moot, since there's too much incremental effort and cost involved for it to be easily get buy-in and added to the plan at this stage.
It makes a ton of sense to target your solution at them instead of developers. They may not be able to use your product well or do the implementation, but they're the ones with the purchasing authority and ability to ensure the budget accounts for it. And can add it into the project requirements far earlier in the planning stage than when tech resources get involved.
No one I know has deployed Optimizely since 15/16.
Any roll-your-own experimentation platforms take considerable resources to make accessible to those in the organization interested in using it (product, marketing, etc.)
Then they went to the annual cost of $30K+ upfront and ended all monthly options. They had to move to high cost, high touch to compete with the free/cheap offerings to stay in business. This acquisition suggests that didn't work.
We ended up building randomization, remote config, and logging ourselves, and did the analysis with our existing stuff.
Whether or not this is a "good" exit, it's a great accomplishment for Dan and the team and can't wait to see what they do next!
It was discussed here: https://news.ycombinator.com/item?id=7287665
I always wondered how they got away with it for so long.
[1] http://www.datascienceassn.org/sites/default/files/Most%20Wi...
Experimentation-done-right is too expensive and too ambiguous to sell as a product. Every product experiment requires a complex set up, a lengthy running period across a huge base of users, and then heavy analysis in order to achieve statistical confidence over a specific feature's impact on a business metric. That "confidence" is often represented by a subpercentage point that may or may not be statistically significant. Fun problem for the data scientist, plain hell for the PM.
In my company which uses experimentation for everything, each A/B test requires two weeks before the Product Manager can even see the results. Two weeks of waiting for a confusing, contradictory dashboard that can't be taken at face value, that needs careful, human analysis before it can be called a "win".
That slowness is fine for high-traffic, high-risk & high-value lines of business. But it's not fine when you're releasing feature that aren't just optimizations.
Competitors like LaunchDarkly and Split.io have recognized that critical difference, I think. They know that causality is expensive, and are particularly aware that the fine line between feature release and metrics impact is tied too heavily with a company's politics, i.e. it chafes against the intuition of executives.
Instead, they offer experimentation as an add-on to their developer tools. You can experiment if you need to, but it doesn't obligate you to do so.
That goal is much more realistic than the Optimizely's current goal: "helping our customers win in a digital-first world".
But then they are asked to wait! Wait for the concept to mature within the org. Time being of critical importance most orgs end up getting frustrated as what they initially started with doesn't hold true now. And when you already have lots of money, it's easy to switch.
18 seems like an outlier, but for press releases I’ve read in the “we’ve been acquired” category, I’d guess that the median is > 10. Does anyone have first-hand knowledge about why these statements are released in this teasing way?
For obvious reasons it was tricky to run an A/B test just for testing the impact of Optimizely's script itself. But the key issue is that all the similar tools at the time (Optimizely not being the only culprit here) were determined to not required developer effort, which led to poor overall performance.
Then React et al came along and took ownership of the DOM, which meant adding tools which also manipulated the DOM became even more problematic.
Fortunately tools like Launch Darkly and Split solve this problem in a better way (high performance full-stack feature flags), even if it does mean developer effort to add tests. Optimizely did eventually launch their own version of this, but never really won back the developer mindshare.
Ultimately, it seems Optimizely enjoyed a few years of success, but a combination of developers getting more concerned with performance and the front-end world moving on to different architectures, seemed to lead to its decline.
I'm aware of Wasabi[1], but I believe it's abandonware at this point.
There is a limit of about six experiments at one time. Each experiment can have up to 8 variants. And if you want more, then be ready to shell out thousands per month (I forget the exact number, but that's the ballpark).
Overall, if I were to redo things, I'd probably be better off just using Google analytics events.
Edit: that said, I'd never use a piece of bloatware like Optimizely for just the split-testing feature. If your product would benefit from split testing, then include it as a feature. It's not a "hard" feature/problem to implement into just about any existing product that would benefit from it.
They're not the only ones with that problem - almost all competitors / actors in the field raised too much for the side of the market, from my opinion. Dynamic Yield is probably the one that did the best at that game, with a great exit when the hype was at its peak. But all the other ones are clearly in trouble (even Optimizely exit, I am sure, is clearly not a success for their investors, etc). The only actor that has a reasonable overall strategy is VWO (somebody asked if they're were profitable - of course they are, since they did not receive any funding, they have to. While Optimizely clearly never was), because it stayed lean and avoided the pitfalls of over investments (going to all continents on the planet, having 20 offices all over the world, etc). Hats off to Paras for that. We try to stick to the same strategy at Kameleoon, even if we had to raise a bit of capital. But you don't need to raise $200 millions to have a great CRO platform - we proved this for a fact.
About the switch to Enterprise - anyone in this industry will tell you it was absolutely necessary. Impossible to sell experimentation to SMB and expect to be profitable, you need larger customers. Because contrarily to Mailchimp for instance, a SMB customer paying $50 (or even $500) per month for a CRO platform won't be able to operate it on its own and will churn. The problem with Optimizely pivot to Enterprise is not the pivot in itself, it's how they did it (I heard some horrors stories from people inside Optimizely, not sure if they are true, but one thing is clear - it did not go well).
Anyway, it's still an exciting field, from the technical side of things, there are still many innovations to be done and we hope to spearhead that at Kameleoon :-)
If Dan and Pete were nice guys, they would have take care of the employees that built their company. The employees wouldn't have been millionaires, but they might be able to scrape together a down payment for a house. And not like the mansion that Dan has, just something that would move them out of the fear of rents increasing and losing their jobs.
They used to be nice guys. I don't wish ill towards them, but they do not deserve a good exit with this.
It is very easy to implement this in-house so long as you own the systems and don't outsource [too much].
They tried to move up-market and did it in an unreasonably difficult way in my opinion. It was easier to do business with ORACLE and then Google. The sales folks didn't listen, their proposals ignored our requirements, it was a mess.
I hope the staff got something.
Lot of big enterprises buy these kind of systems and pay through the nose for them.
I built sites with both back in the late '00 and while Sitecore was marginally better I don't miss working with either.
Adobe also bought Magento, while not as bad as Omniture, has it's own levels of hell.
I/we competed against these guys 21-23 years ago. We had real tech, they had brilliant powerpoint engineers.
I have no idea why they're buying Optimizely.
Reducing headcount or doing a hiring freeze is a way to improve your metrics ahead before you go shopping yourself for an acquisition.
I'll be most proud of the companies I sold to that then started doing internal webinars on how to do hypothesis-driven product development and really engaging in bottoms-up approach to experimentation. Those deals took 12-36 months to close, but you felt like you were changing how they ran their business.
I see a lot of folks throwing shade on Optimizely here. I think that is ridiculous. Not many companies can grow to 300 employees and over $75 million in annual revenues. Even fewer companies get to IPO. Everyone's thoughts about how Optimizely used to be worth more are BS. Valuations are perceptions of product/market strength rather than the reality. We shouldn't be looking at this as a failure. Rather we should look at this as a natural course of a business finding its product/market fit.
Form a market perspective, Optimizely was hemmed in by big players like Adobe and Oracle who bought competitors to round out their marketing suites. Adobe and Oracle do not care about their experimentation products and will give it away for pennies, so it isn't like they are driving innovation. On the low-end, Optimizely created a bunch of fast followers. VWO, ABTasty and Kameleoon, just to name a few copied not only the Optimizely tech but also their marketing in some cases and then just sold for cheaper. Ultimately, the market decided that Experimentation technology just wasn't super valuable (you still need to generate the test idea yourself). The technology could be copied easily and many people didn't care enough about it to change their cultures.
Outside of the market forces, Optimizely definitely made some mistakes. You could make the argument that pricing and product development had issues in the past few years.
I think those were not the core issues. I think the basic mistakes they made were in execution. It is less sexy to talk about but ultimately if you have a great product and a need in the market, you still can't win if you don't execute properly.
Marketing execution was poor, Sales enablement was haphazard, Partnerships with other tech firms or agencies changed almost daily, and culturally, the organisation struggled to maintain the original culture it had around transparency. It failed to realise that competitors had caught up to a bunch of core functionality it thought was unique. It failed to market the new features that were differentiable. It is easy to blame pricing or moving into "enterprise" as the issue because that is what people see on the outside. I think that is unfair. Many companies have done the same thing and it worked. It is all down to execution.
Ultimately, I think Optimizely built a real market and culture in the tech world that experimentation matters and data matters. The fact that you have a ton of copycat companies prove that others see it as valuable as well (I wonder how they'll fair in the future). Regardless of where it goes as part of Episerver, it will have brought loads of people into the world of experimentation, and I think that is a good thing for the world.
Calling Optimizely's competitors simple followers is clearly an over-simplification and a bias from the VC/SF crow indeed. As Paras pointed out, every actor in the field had their firsts in the market (some of Kameleoon's examples: our anti-flickering technology, our aggregation features in our reports / data engine, our automatic cross-device history reconciliation or our full ITP support). In many ways, Optimizely tech is now trailing behind today.
However it's true that Optimizely (and to a lesser measure, VWO) did a lot from the marketing side to develop the market, and we benefited quite a bit from that. We're definitely grateful to them :)
We started in late 2009, exactly the same time when Optimizely started and we had visual editor on day 1 and have many firsts in the market to our credit (integrating heatmaps into A/B tests, asynchronous code, Bayesian statistics and so on).
It's a perception among VC/SF crowd that Optimizely was the only innovator in the A/B testing market but that's not the truth if you actually put effort into researching. Many players, including VWO and of course Optimizely too, pushed the market in terms of innovation.
-Paras (founder of VWO)
