First time here? Get updates by RSS feed or follow me on Twitter.


BoS 2011 Lightning Talks: Corey Reid, Freshbooks; Karl Treier, Prospect Stream; Justin Goeres, JKI; Patrick Foley, Microsoft; Tyler Rooney, 4ormat.

Below are the lightning talk videos from five speakers at the Business of Software conference 2011, starting with the winning talk by Justin Goeres of JKI. Lightning Talks are terrifying and very hard to pull off - each speaker was allowed 15 slides with 30 seconds for each slide.

That's 7 minutes and 30 seconds in total. Done. 

Don't forget, registration for BoS 2012 is open now. We hope to see you in Boston October 1st-3rd 2012 and the first EARLY BIRD DISCOUNT tickets run out on March 30th. Save $900 on full registration.

Confirmed speakers this year include Professor Noam Wasserman, Gail Goodman, Dan Lyons, Jason Cohen, Mikey Traft, Adii Pienaar, Joel Spolksy, Peldi, Paul Kenny, Bob Dorf, Dharmesh Shah and others who spend their lives at the sharp end of software businesses around the world.

 Justin Goeres, JKI - Getting to Nowhere. the story of Richard Feyman's quest to travel to Tuva.

An epic tale of an extraordinary physicist, Richard Feynman, and some pointless goal-setting that leads to an inner truth. Might bring a tear to your eye - if you are human.


Karl Treier, Prospect Stream - 20 Tips on Starting a Business

20 tips, 15 slides. No mean feat. 

Patrick Foley, Microsoft - Confessions of a Wannapreneur

Try as we might, we couldn't get Patrick to resign live on stage. Next year... 

Corey Reid, Freshbooks - Be Afraid, Be Very Afraid

Of recruiting... 

Tyler Rooney, 4ormat - Things I Learned the Hard Way at

A smart techie in a cardigan telling tales of relentless scalabilty. Geek heaven. 


Don't forget, registration for BoS 2012 is open now. We hope to see you in Boston October 1st-3rd 2012 and the first EARLY BIRD DISCOUNT tickets run out on March 30th. Save $900 on full registration.

Confirmed speakers this year include Professor Noam Wasserman, Gail Goodman, Dan Lyons, Jason Cohen, Mikey Traft, Adii Pienaar, Joel Spolksy, Peldi, Paul Kenny, Bob Dorf, Dharmesh Shah and others who spend their lives at the sharp end of software businesses around the world.

We will organise the Lightning Talks later in the year. If you want to be considered, drop me a line - mark at businessofsoftware . org with your contact details, a possible topic and ideally a link to a video of you speaking somewhere else.

April 30, 2012

Something is going to change.

Can you guess what...?

April 03, 2012

What do Business of Software attendees want to talk about?

Want do Business of Software attendees want us to talk to them about this year? These are the most common words used by Business of Software delegates this year when asked, 'Talk to me about' to be put on their badges.

Here is a 'Wordle' of the first 150 responses...

BoS Talk to me
In other words, tea is a more popular subject than VC. I guess it is in Boston. We will have ourselves a tea party.

#BoS2012 takes place over 2 and a half days 1st-3rd October, 2012, at the Intercontinental Hotel, Boston. We advise booking accommodation early, whether you stay at the conference hotel - we hope you do and have a special conference rate - or choose another location. There is a $900 discount on the full ticket price till midnight 3rd April. If you didn't make it to last year's event, you will also get access to all of the talks from BoS 2011 when you register for BoS 2012.

Early Bird Registration Deadline midnight PST Tuesday 3rd April.


BoS 2012 Speaker updates & Early Bird Extension to midnight PST April 3rd

I want to apologize to you if you tried to register for Business of Software 2012 over the weekend to take advantage of our first Early Bird offer. Our event registration system was down for routine maintenance on Saturday and we have received a number of emails from people who were unable to register. Given the circumstances, it seems fair that we extend the deadline for registration and have done so to midnight PST on Tuesday April 3rd.

We also have some brilliant new speakers who have confirmed their participation at Business of Software 2012... 

  • Joel Spolsky - will decide what he will talk about nearer the time. Needless to say, it will be topical and relevant to BoSers!
  • Bob Dorf, co-author with Steve Blank of, The Startup Owner's Manual - The Step-By-Step Guide for Building a Great Company. Bob was founding CEO of Peppers and Rogers, the inventors of customer-centric marketing, before joining forces with Steve. He will be talking about fueling growth in enterprise software companies and the differences between web/mobile businesses where customer feedback and product iteration happen faster and in far greater volume.
  • Dan Lyons, Technology Columnist at Newsweek where he covers software, mobile and social media. Dan also wrote the popular (over 1.5 million views per month), 'Fake Steve Jobs' blog. He will be talking about what technology companies can learn from Apple, how to get the attention of the media and why Silicon Valley is not the center of the software universe.
  • Gail Goodman, CEO, Constant Contact. Constant Contact, provides software that helps small businesses manage CRM. She has led Constant Contact for over a decade through incredible growth and IPO to a point today where the company employs over 900 people and has 500,000 customers. Gail will talk about the challenges of targeting the SME market and share some of the things she has learned about how technology and people mix. 

Three other things for you...

  • You can view all of last year's Lightning Talks here including Justin Goere's very moving, 'Getting to Nowhere'. This talk considers Nobel-prize winning physicist Richard Feynman's decade long quest to visit Tannu Tuva. Some great lessons in software development and life along the way.
  • All the talks from last year can be accessed from here with a pass code. Last year's attendees have a passcode that lets them access all the talks from last year in their own steam. Attendees for BoS 2012 also receive a code so they can see the full set of talks when they are registered for BoS2012. Other talks are posted over the course of the year.
  • If you are London based, you might like to join us for an evening event, CEO Tales, with Peldi of Balsamiq Mockups who will be talking about customer-centric software development amongst other things on the evening of 17th April, 6-9pm.

#BoS2012 takes place over 2 and a half days 1st-3rd October, 2012, at the Intercontinental Hotel, Boston. We advise booking accommodation early, whether you stay at the conference hotel - we hope you do and have a special conference rate - or choose another location. There is a $900 discount on the full ticket price till midnight 3rd April. If you didn't make it to last year's event, you will also get access to all of the talks from BoS 2011 when you register for BoS 2012.

We hope to see you at Business of Software 2012.

Thank you.

March 30, 2012

We believe in sharing, we don't necessarily believe in paper

Nice to see some proof that sharing some of the BoS love brings more cool people to the physical event. Just got this note...

"Just ordered my ticket, and am really looking forward to it. I attended live stream from my living room couch last year. It was great, but felt like I was missing out on the networking and interaction. One question. It says the ticket should be printed. Will it be possible to show the ticket on my iPhone instead?


Look forward to seeing you there and, 'Yes', you can bring the paper ticket with you if you wish but as long as your phone has a screen that can display the registration details, that is good enough. See you in Boston!

Don't forget, registration for BoS 2012 is open now. We hope to see you in Boston October 1st-3rd 2012 and the first EARLY BIRD DISCOUNT tickets run out midnight, March 31st. Save $900 on full registration.

Confirmed speakers this year include Professor Noam Wasserman, Jason Cohen, Mikey Traft, Adii Pienaar, Joel Spolksy, Peldi, Paul Kenny, Bob Dorf, Dharmesh Shah and others who spend their lives at the sharp end of software businesses around the world. We hope to see you there.

March 20, 2012

BoS 2011: Business of Software Talks from 2011 all in one place.

If you came to Business of Software 2011, you can access all the talks from last year here using the passcode you were sent by email. Anyone registering for this year's event will also receive a code to access the talks prior to their general release. Click on the talk you want to view and enter your passcode.

The talks will be released, with transcripts, over the course of the year.

Don't forget, registration for BoS 2012 is open now. We hope to see you in Boston October 1st-3rd 2012 and the first EARLY BIRD DISCOUNT tickets run out on March 30th. Save $900 on full registration.

Confirmed speakers this year include Professor Noam Wasserman, Jason Cohen, Mikey Traft, Adii Pienaar, Joel Spolksy, Peldi, Paul Kenny, Bob Dorf, Dharmesh Shah and others who spend their lives at the sharp end of software businesses around the world. We hope to see you there.

March 18, 2012

Noam Wasserman, author of The Founder's Dilemmas hits Amazon best sellers list and wants to answer your questions

We were very excited when Professor Noam Wasserman, author of The Founder's Dilemmas - Anticipating and Avoiding the Pitfalls That Can Sink a Startup agreed to speak at Business of Software this year especially when he offered to get involved in a guest blog to see whether we could apply some of the lessons he has learned in his studies to some of the more grown-up businesses that come to Business of Software.

(Here he is speaking at Business of Software 2008).

His book was due for publication on 25th March, (here is the Publisher's Weekly Review) but he dropped me a note on Friday to let me know that not only had the publication date been bought forward, but the books was already on the Amazon best sellers list. Congratulations Noam!

We would love to get some sense of some of the things that you worry about as founders - not just so he can try to offer some answers, but to help him craft a talk at Business of Software this year that answers the issues that are relevant to you. 

If you get a moment, please complete this VERY short Survey Monkey Questionnaire. You can do it anonymously, but if you leave your email address, we will contact one person at random and send them a signed copy of the book. We have extended the deadline for submitting your question by two days so please submit by midnight Thursday 22nd March.

Link to that VERY short Survey Monkey Questionnaire.

Don't forget, registration for BoS 2012 is open now. We hope to see you in Boston October 1st-3rd 2012 and the first EARLY BIRD DISCOUNT tickets run out on March 30th. Save $900 on full registration.

Confirmed speakers this year include Professor Noam Wasserman, Jason Cohen, Mikey Traft, Adii Pienaar, Joel Spolksy, Peldi, Paul Kenny, Bob Dorf, Dharmesh Shah and others who spend their lives at the sharp end of software businesses around the world. We hope to see you there.

March 14, 2012

Patrick McKenzie (@patio11) on Engineering your marketing outcomes

So here is Patrick McKenzie's brilliant talk at Business of Software 2011 on engineering your marketing outcomes. Marketing comes naturally to some people. Unfortunately, few of them go on to found software companies. Patrick talks about types of marketing that use engineering skills and won't be rabidly opposed to your developers. Instead of spending a few weeks cranking out one more feature that won't be seen by 1% of your users and increase the value of your software by <1%, learn how you can use the same few weeks to create scalable systems that take much of the guesswork out of marketing.

Patrick talks A/B testing, funnel analysis and SEO through scalable content generation.

Don't forget, registration for BoS 2012 is open now. We hope to see you in Boston October 1st-3rd 2012 and the first EARLY BIRD DISCOUNT tickets run out on March 30th. Save $900 on full registration.

Confirmed speakers this year include Professor Noam Wasserman, Jason Cohen, Mikey Traft, Adii Pienaar, Joel Spolksy, Peldi, Paul Kenny, Bob Dorf, Dharmesh Shah and others who spend their lives at the sharp end of software businesse around the world. 

Watch the video here or skip below to see a full transcript of the talk.

Don't forget, registration for BoS 2012 is open now. We hope to see you in Boston October 1st-3rd 2012 and the first EARLY BIRD DISCOUNT tickets run out on March 30th. Save $900 on full registration.

Transcript of Patrick McKenzie at Business of Software 2011:

Patrick McKenzie: [Deep voice] Hello Ladies.

[Normal voice] And Gentlemen. [Laughter]

So, if you weren’t here to the talk last year I basically did about an hour-long takeoff on the Old Spice guy I was telling you about. Does everyone …OK …Raise your hands if you were here at the talk last year… [Everybody raising hands]

OK. Put your hands down if you have forgot… No, keep them up! [Laughter]

Alright. Now put your hands down if you have forgotten the topic of that talk. OK, Good. So some people remember even a year later.

So that's the reason why I like humor in presentations — because it helps people retain what you're speaking about. Now, right after Business of Software last year I flew back to Japan where I live, and I got off the plane and I gave a phone call to a lady who I met right before coming to Business of Software and she said, “Hey, what did you do on your business trip?” So I gave her the Old Spice speech. I said — and I got to get a new speech this year, and I probably can't sustain the Old Spice guy for sixty minutes and the joke will be old.

And she says: “That’s OK; I think you're cooler than the Old Spice guy anyhow.” So, that was a sign to me and we're now dating. [Laughter and applause]

So, the most common question I get asked is “Why are you in Japan?” And because it's kind of relevant to the topic of the talk, I’ll tell you. People always ask me “Is it because of Japanese girls?” And that did happen but it was, like, eight years later… The actual reason I’m in Japan is not because of Japanese girls, it’s because of Indian men. Specifically... [Laughter]

Ten years ago I was an engineer with very little self-confidence skills — and now I’m an engineer with very little self-confidence ten years older.

But I was in college and reading the Wall Street Journal. And the Wall Street Journal was saying “The Indians, they're going to steal all our programming jobs and we're going outsource everything to India for one quarter of the cost. And if you are getting a programmer right now it’s a mug’s game; you will never be able to compete with all of those folks.” And I'm, like, “Oh God, they're right!”

So, I thought: “How I am going to get a nice, safe job at a big mega corp like Microsoft like my parents have always wanted for me?” I thought: “I’ll play the Van diagram game: you take one thing that’s pretty hard like programming, take another thing that’s pretty hard like, say, speaking Japanese and you intersect the two.”

And the people who can do both — there must be, like, four of them, right? So, if I go to Japan and become bilingual then I can come back and get a job at Microsoft and no one will ever be able to take that thing here away from me until the day I die.

And that didn't really work out. But…

So, I’m in Japan and... We have lots of young engineers in the audience? Let me... Perhaps help put your mind at ease regarding the outsourcing issue.

I actually managed a Japanese company's outsourcing operations to our Indian subsidiary for three years.

And I will distill that into one anecdote which... This is absolutely true.

We made administration systems for universities and at one point we had... we sent the system over to the Indian subsidiary to get manually Q&A'd, you know, push all the buttons, make sure everything works. So, keep in mind in this conversation everything is happening via email with, like, a four-hour delay between each email.

Three days after we sent it over and expected to be, you know, mostly started and we asked: “What's the status?”

“We haven't started yet, the system isn't working.”

“Oh, really? Why isn't the system working?”

“We can't log in.”

“Oh, really? Did you follow the page of instructions that would get you from the subversion and the database dump into a working system and then be able to log in?”

“Yeah, we did that but the system isn't working.”

“Did you get an error message?”

“Yes, we got an error message.”

“What did the error message say?”

“I don't know, I can't read Japanese.” [Laughter]

“OK... Can you take a screenshot of the error message with the printscreen and paste it into, you know, MS Paint and send that over?”

And they do. And they show me a picture of the login screen and I write back: “That is not an error message, that is just a weather report, please continue with the instructions and push the buttons that we have marked for you and see if anything comes up.” And they say, “We don't agree that that is a weather report. [Laughter]

I'm like, “That is interesting because I can read the text and, according to our previous conversation, you can't.” [Laughter]

“Why don't you think it's a weather report?” And they said…they had two compelling pieces of argument on why it couldn't be a weather report: it was an on the login screen and written in flaming red text.

Why would you ever do that to a weather report? And I said, “Oh, it's on the login screen and written in flaming red text because, that weather report, if you ignore it, you'll die!” [Laughter]

And I told this story to Americans all the time, and one of them asked me: “What kind of system are you writing such that a) Weather is routinely lethal and b) It’s so routinely lethal that that's already special-cased?”

And all I can say is, “Welcome to Japan!”

There are 30 words of Japanese that have made it into the English language and 15 of them are for meteorological phenomena that can kill you. [Laughter]

So, what am I doing these days when I am thankfully quit off that day job and, will never do outsourcing management ever again? But these days my kind of claim to Internet fame, aside from being on “Hacker News” because it's my job — because I'm gainfully unemployed — is I'm kind of like the Internet's Bingo guy.

I made Bingo Card Creator, it’s a really simple software that makes bingo cards for elementary school teachers. I won't tell you too much about that specifically, but we are going to talk about some of the things that I have learned from it. Because I kind of get to use this as my laboratory where I can screw it up as much as I want, and aside from the 200K users, no one, like a boss, will yell at me. And, just so you get some perspective of where I am, Dharmesh fleshed his customer graph the other day so I thought: “Wow, he stole mine!” Like we started the same point we end near the same number, about 5000 customers and the curve looks almost exactly the same! But a key thing to keep in mind: my graph was cumulative and Dharmesh charges between like $1000 and $10,000 a month and I charged 30$ once. [Laughter]

So, key takeaway: if you want to make a billion-dollar business selling software as a service at the 10.000 a month price point is a great idea — and selling 30$ a month once, you have to do a little more.

So, if you remember Jason Cohen’s talk from last year, he said that you get many pieces of competing advice at events like this, and if you are reading the entrepreneurial blogosphere; and it's important for you to keep in mind where someone is coming from.

And I think that's important when you're listening to my presentation. So, I’ve prepared this lovely little four-quadrant graph on the axis of what market you're addressing and on the spectrum of funded to bootstrapped like me.

And I put up some companies that I know and respect and appear at other places on this graph and, of course, because the only purpose of having a four-quadrant graph is to put yourself up into the right. You can see that I'm more evolved by them. [Laughter]

Well, honestly, what you were you expecting from a guy named McKenzie, right? [Laughter]

But one of the reasons I got into engineering is this really simple and powerful idea that math and science always work.

And I think many of the engineers here really have an emotional attachment to this one. And we don't really have an emotional attachment to marketing, I mean…

Can I... just have a quick show of hands of who here is mostly an engineer? OK... hands down. Who here is mostly a marketer? OK, yeah. That's what I thought.

And who here is Paul Kenny? [Laughter]

OK. So for you marketers who haven't caught on to it yet, the stereotype of marketing in the engineering field is that, marketing is basically witchcraft. There's some, like, eye of newt and toe of dog, or whatever that goes into the pot — we have no clue how anything works and it's, like, vaguely evil. I mean, you're selling stuff to people...

And it's OK when people buy our stuff, but when people buy other stuff that's kind of unnatural so we're just creeped out by it.

But something that I've come to appreciate over the years of running my own business is that marketing is not really witchcraft, it's just about changing people's behavior. And this is not something that is totally unnoble. We know we can change people's behavior, we can do it all the time by stuff as simple as changing our tone of voice.

As any lumberjack will tell you, “All good things come from timber.” [Laughter]

OK, Tim told you about me.

And this is, you know... We change people's behavior in engineering too. That's why we build products, we want you to change from a manual process to this better process that we have; that's why Steve Jobs and an excellent product design, and can turn around how an entire world uses their cell phone.

And if we get into this we think that: “Wow, marketing is really sort of an engineering discipline.”

And so I want to talk about one particular type of way of changing people's behavior. If you think of it, your business is set up to cause a series of fortunate events.

There's billions and billions of people in the world and you really are trying to convince some small fraction of them to first come to your website. And of that small fraction an even smaller fraction are going to start your free trial, — if that's your business model. And of that small fraction an even smaller fraction are going to actually succeed in paying you money.

And this graph is called a funnel in traditional marketing and traditionally this is the point where I have like a clip art slide of a, you know, cooking funnel.

But I thought I would use something a little more appropriate because funnel-web spider. Yeah, bad joke, I know.

Anyhow, poor funnels kill companies because when you think of... When you are trying to influence people's behavior over a multiple stages of steps, if you execute on this poorly, your company will not succeed and, you know, you go home crying and your employees don't have jobs anymore.

Because think of where you find funnels in your business. You find funnels in email. If you remember Rob Walling’s presentation from last year, email’s one of the most important things you can be doing.

OK, I'm going to collect some percentage of people's emails but I'm going to email most of them, some percentage of them are going to open, some percentage are going to click on the link, and after clicking on the link some percentage will actually take an action that helps my business. That's a funnel.

If you don't do that right your business will not have that level of success from the email marketing.

For starting free trials, you know, some percentage will see your landing page, some percentage will convert into the trial, some percent will have a good experience with the trial, and some percent will go on to actually pay you money.

For your sales cycle, some percent of people will come from an undifferentiated visitor taking whatever action is required by your business to turn into a lead. Some percentage of those leads will talk to salespeople, some percentage will have a successful talk and you’ll get passed up to the decision maker, some percentage will signal actual buying intent and some percentage of those will actually convert into deals.

Even the core use of your product is often a funnel. Many of us make software that helps people do their jobs so there's some sort of a workflow involved in it. And there's many stages, you know, A to B to C to D, before a customer actually gets value out of it. And as people use the software, going from A to B, and B to C, and C to D, people stop using it — and people who don't get all the way to D don't get the value from your software.

And if they're not getting in the value than what, really, was the purpose of you writing it?

And check out, of course, put something in your cart, get to the check out page, put in the credit card details, actually click the Confirm button to let us charge your Visa, this is also a funnel.

And if you fail at execution of these things your business just will not work.

And there's many other reasons your business can't work. If you can't produce software that works or if you don't create actual value for users, yeah, that will kill your business too.

But everyone in this room has that down already. And I don't know if you all have funnels in our blood yet, so I'd like to change that.

Because what you do with a funnel: first you describe it, and that's as simple as saying “My steps are, in order, are this, this, this and this.”

Then you measure it, then there's that nebulous no-one-knows-what-comes-here step, and then you make a lot of money.

So if you're wondering: “How do I measure a funnel?” I'm not here to sell you any tools, but I use [Cosmetrics], personally, there's a bunch of things that you can use…

Google Analytics is free, any of these can get you up and running in, you know, minutes of engineering time and you can have information about your funnel in less than a day.

So the missing like “Aha!” step is that after you know what your funnel is and you know where people are falling out of it, what stages are the ones that aren't converting as well as you think they should be? You can optimize those.

So let me give you an example. Let me give you some general advice on optimizing: shorter is generally better.

If there's a step in your funnel that, when you think about it, doesn't add value to your business and doesn't add value for your customers, you should probably cut that step out.

That seems obvious. But there's a lot of companies that I’ve looked at, where’s there's just, you know, an extraneous…

Six years ago I worked with Fog Creek — we've talked about this previously — but like n years ago they sold manuals. And for a manual they actually need to, like, ship something to an actual human. So we asked for their shipping address, right?

And right now Fog Creek sells nothing that can actually ship to a human, but the shopping cart has been coded the same way — for the last ten years — so there was a step asking people for their shipping address, right?

And every step the funnel gets longer, even if it’s something simple, like, “Could you please write your address again?” Will cause less people to buy their very expensive, very good enterprise software.

And so, just taking that step out makes them an appreciable amount of money.

Improvements in a funnel compound multiplicatively. What does that mean?

It means that if you increase the number of people getting through step A by about 10%, and you increase the number of people getting through step B by about 10% you don't get 20% increase to the bottom line, you get about 21 %.

And yeah, the engineers in the room are going: “Wait XXXX there's an assumption you have to make that those are independent random events,” but I get it. You get an A+ in statistics but you can generally fudge this in real life and it still works.

So the nice part is that you get better results than expected the more and more you work on your funnels.

So when you first write out one of your funnels — and this is a funnel for Bingo Card Creator —  and I won't go through the actual steps — but it turns out there's a lot of work at making Bingo Cards, who know? You’re going to find there's some places that just really catch your eyes — so that really shouldn't be that difficult.

What are my customers failing there? What did I do wrong?

And so I just put red arrows on them, just like tie a string on your head: “OK, it seems like the open word list and creating cards steps were accessibly hard.”

And prior to measuring my funnels I thought: “Well, I think, like, my intuition is 70% of my users are actually —  who sign up for the service — are actually successfully creating bingo cards.”

And I found less than half were.

And that was kind of devastating to me. I did all this work creating software and spent all the money on AdWords to get them to my site, and over half of my users were not getting even the smallest scintilla of value out of my software.

I mean it’s not called Bingo Designer It’s Bingo Card Creator. If you're not actually creating Bingo cards — something is wrong!

And so the customization step — I was really losing a lot of people about that — and I thought: “Well, what could possibly be going wrong with this customization step?” and I actually looked at the screen and it started at two options and then, every time a customer had a feature request, I said, “Oh, that sounds like a great idea,” and I added it — and I added it, and I added it, and I added it…

And this was really intimidating people.

And so I thought: “Wait! Are people actually using these features that I’ve added over the years?”

I’ve had nothing but positive feedback about them. But if we check in the database — very few of my customers were actually using the features that I had spent most of my time writing.

And the added features were disimproving the experience for most of my customers. It was just making it less likely that they were going to succeed in their task of teaching children tomorrow.

I was hurting their lives with adding more features to this software.

So I thought: “Maybe there's something I can do about this that's kind of easy. Maybe I can just take the features, put them back to where they were four years ago, and for the people who actually love having those twelve knobs to fiddle, just click on this little “click here” link and they show on automatically.”

And you can imagine this isn't a lot of work —a wee bit of Javascript coding, less than an hour to collect these stats, less than an hour to do the implementation…

But then when you rerun people through the funnel… I had a radically better business result. 16% — and we called it a lift, — 16% of people who wouldn't have gotten through the funnel, actually managed to get through the funnel

Created more value to my customers and directly had a huge increase on my bottom line for only about an hour of work.

And that’s really powerful, right?

You’re thinking: “Yeah. That's really powerful; but you kind of run a little toy business. Does this actually work with real companies?”


Yes! So I've had the opportunity to consult with a few companies and I can’t tell you actual results from all of them but I’m going to tell you general results from some of them. And they just get me drunk later… [Laughter]

I don't drink! Ha-ha! [Laughter]

So, Fog Creek — can I get a shout-out from Fog Creek in here? Wheee!

sp2: Woo, woo! [from the audience]

Fog Creek are some of my favorite people; and we worked together last year and a little this year on revamping their website and kind of getting them more into being an engineering- and marketing-driven company. A company, as opposed to a company “by the engineers, for the engineers, of the engineers”… [audience mutters; chuckles]

Yeah. We all went through this. OK.

So, they had this page called “the details page” on the old version of the website. And it was a pretty important detail: the details page was the only way Fog Creek could actually get paid.

This is like the entrance to the checkout funnel. And you can just tell by looking at it, kind of difficult to get through there, right?

I actually — so they were paying me money to come use their website and attempt to buy their product. And I have an engineering degree, and I've been doing this for the last ten years or so and I couldn't figure out how. I was at the point where I nearly, like, got on the phone and called tech support to say, “Excuse me, I'm an outside web optimization consultant. How do I actually buy this?”

And it turns out you could actually buy it through the website.

We made that a little more explicit. This conveys in a much more comprehensible, you know, one look fashion, OK? Here’s what we offer, here’s what it costs, here is what you do to buy, you know, click on the nice big button.

And I can't give you an exact — like a numeric result from Fog Creek or else I'm persona non grata there, but suffice it to say the team that was involved in this project, like, their hands were stinging from the high fives for many, many weeks.

So I want to talk about another technique.

Everyone has a sickness. Some people have malaria, some people have TB, some people have “they don't buy our stuff.” [Laughter]

And we have a way of addressing this sickness. It’s called “Medicine”!

We give... And we know medicine works because we give two different groups of people two different treatments. We'll give one the placebo, and one the actual drug, and when we see one group perform markedly better than the other we know, “Aha!” that one difference that we controlled for was actually what caused the difference in behavior. This sort of thinking is kind of rare in the online blogosphere for, say, web design or marketing advice, because people really love their leeches. [Laughter]

It’s like, “Why do we use leeches?”— “I don't know! Everyone has always used leeches and smart people say, 'Use leeches,' so you should use leeches too.”

So, not to pick on “Smashing Magazine” because this is rampant in the UX designosphere, but they came out with a thing a couple of weeks ago that said, with reasoned arguments drawn from a lot of research from qualified people, that if you had less than sixteen points a point fund for your body copy you were costing yourself money.

Like that is a testable statement about the condition of the natural world — we should be able to test it, right?

So, I did!

I …it turned out my website was fourteen point copy. And I made one version of the website with the old fourteen point copy and one version of the website with the sixteen point copy, and showed half of the people who came one week one version and during the same the same week half of the people the other version.

And…(they have no idea this test is going on, and the only thing different about the website are themselves because it's a randomly controlled trial, is whether they are seeing 14 or 16 point.)

And it turns out — the most common result for A/B testing— there was not a lick of difference between the fourteen point and sixteen point.

So if you have something on your “to do” list — you know “Smashing Magazine” says you should get that done— you can take that one off probably.

But you have a lot of things like this, right? Like, what message are people going to respond to? What position are people going to respond to? What pricing are people going to respond to?

And our best answer for this, as a profession, is…leeches!

You can do better. Try two things. Test them, see which one actually works.

And there's very easy software for doing this these days. Your programming language of choice probably has an A/B testing library where you can code tests with just one line of code. If it doesn't, you can write one in a week — I did it.

Sidenote here: if you write one for a stack that doesn't already have it and you put it on your website, all of the OSS bloggers in the world are going blog about your nice new OSS thing being on your website…You will get SEO juice, the Googles will love you and you'll make a lot of money. End of sidenote. [Laughter]

OK. Do you think DHH gave a link to my blog because he really love teaching bingo cards? No. But he's a Rails guy and I wrote the Rails A/B testing library.

So, there you go!

So, write your own. If the people that are actually doing the A/B tests are not programmers, then there's two great options for doing them visually. One is Visual Website Optimizer, which I really like...— it’s, like... I very, very rarely get envious but this guy when he…— he sent me an email prior to launch and he said, “Hey, Patrick, I know you're the A/B testing guy. Can you to review this website?” And I was like, “My capsule review, if I had thought of this two weeks ago, I would be a millionaire a year from now.”

Can’t tell you what he’s making but he’s doing well. Anyhow... And then optimizing is another option.

So, there's many places you can test your website, right?

If you go back to your own company and start reading about all this stuff, start reading about what works for other people, you’re going to find that places early in the funnel, before people have strong emotional reason to be attached to you, are great places for A/B testing because you are on knife's edge there.

Maybe a little push to the left and they fall out, maybe a little push to the right, they fall in. Landing pages are a great reason for that. They have perhaps never heard of you, they just clicked on an ad, they've got, as XXXX says, they've got their finger hovered over the back button. And you have three seconds to make an impression.

And you should be testing for what three seconds gives you the right impression.

On your pricing page, like we saw earlier, in your shopping cart…

So one A/B test that I did, someone had an intuition, I was selling stuff via PayPal and I thought: “Maybe I should add Google Checkout or maybe I shouldn't add Google Checkout.”

And I tried to find querying the blogosphere and: “Leeches!” “No leeches!” ”Leeches!” “No leeches!”

So I said, “All right, I'll show half of the people PayPal and Google Checkout, — and to the other half just PayPal.”

And it turns out that people who saw both PayPal Google Checkout had a higher conversion rate total, and also a higher conversion rate through PayPal.

Like, just putting Google Checkout as an option, maybe if we can get inside the user’s head, maybe that makes me look more credible…

And so they were more likely to purchase even through PayPal than they would have been without seeing the Google button. So...who knows.

So it’s always dangerous to get into users' head. Look at their actual behavior rather then what you think they think…Or even what they say.

Well, one more thing you can A/B test is the actual behavior of your software. For example, if you're thinking, “Hey, we're investing in this amount of engineering resources into creating a new feature.”

If you create a new feature for your users, your goal is to change your users’ behavior into either making their lives better or making something better for your company by increasing the amount of software you sell to them.

If you create a new feature and give it to half of the users and don't give it to the other half — and nothing visibly changes — than there was no point in having that feature, right?

So particularly, things on these pages you can test. Headlines — people are really, really sensitive to them. Which is kind of screwy from an engineering point of view because I always skip over them to get to the good stuff, but apparently people read them, and it’s one of…

If you look at the eye tracking studies, it’s one of the few things peoples' eyes actually hit on a web page. Offers, sort of a value proposition to a user. You’re not saying, “Hey, you give us money, we give you software,” it’s: “Hey, give us money, and we give you this package of stuff. Particular software that you can use in a particular way, which include X amount of consulting services, yadda-yadda-yadda.” You can, like, tweak those knobs up and down and see what people really respond to.

One of the things that HubSpot said last year was that people really responded better to their core offering when they increased the amount of, like, hand holding that was involved in it, independent of the amount of hand holding that was actually consumed, because people just thought: “Ooh, software is scary.”

Calls to action, you know, the little text on your button of “sign up” or “start doing whatever now,” prominent graphical elements and important micro copy.

Thirty-seven signal pages are really good for this, you know, if you look at any of their signup pages right to the side of the actual form, they’ll have a micro copy that's designed to address your objections against doing that form.

So right next to any email submission they’ll say, “Don’t worry, we won’t spam you. Here’s why.” Right next to a credit card thing: “This is totally secure. We use bank level security,” which, as we all know, means absolutely nothing, but people really respond to it. [Laughter]

So, I've been doing this bingo thing for the last five years and last Saturday I taught a group of software engineers A/B test in forty-five minutes.

And because software engineers respond well to live coding, I said, “Hey, I'm going to pull up my website and we're going to live code and A/B test right now, just throw something out there — because I only have one minute available for this.”

“What's something we could change?”

And they said, “Hey, why don't we change this header, from ‘Make Your Own Bingo Cards’ to ‘Create Your Own Bingo Cards Now!’—exclamation point.

I said, “That's a great test.” And so, I can code in like two seconds, doo-doo-doo-doo, and a week from now, we'll have great information that says I've been doing this for the last five years and you've known about the existence of this website for the last two minutes, and therefore I did better. Right? [chuckles]

Well, that won’t make a good story at all.

So, the version that I've been working at for five years, I sent 20 thousand people to each of these versions, in full scientific glory, and 40% more of them bought…the only difference was something inserted by somebody who had no marketing background and who had no clue of who my customer even was… [inhales, exhales] [Laughter]

But this means that the Bingo Cards are now diamonds! Seriously. [Whistling]

So 37signals has done extensive A/B testing —  you can read some of their stuff online. And believe me, if you haven't started reading about A/B testing yet, you have no idea how deep the rabbit hole goes.

There's so many things to read about, but one of their things is, they just played around with the headline on their pricing page and 37signals, they're an Internet Megabrand — they've been around for the better part of the decade now — and just playing around with text on their pricing page has a huge, measurable, business result.

I can't give you actual numbers from Fog Creek, but Fog Creek has been around for the ten years; they make excellent software for making great developers better and their software probably has 100 man-years invested in it.

And that 100 man-years of effort doesn't matter in the first three seconds that somebody is at your website, so much as the headline does.

So you really need to put appropriate resources into making sure the headline and design and your value proposition really speak to that customer before they hit the Back button.

So this is the takeaway for this part of the presentation. Systematic A/B testing absolutely prints money. I've taken this to many different companies. The ones who actually get into the practice of: “OK, this sounds like freaky Voodoo. I mean, how can changing two words on your homepage actually increase sales by 40%?”

“But ah, we'll give it a try, and we'll run an A/B test, and see what happens.”

And if you get into actually doing this on a routine basis and making it part of your normal practice, it'll collect a 10% win there and a 20% win here, and pretty soon 5 and 10, 20%, add up to real money. And it... If you guys are already starting from... We have a software company that has 10 or 20 employees and millions of dollars of revenue — 5% adds up to real money really, really quickly.

And especially I'd like to talk to the folks in the room who have more mature products; you know, you've been doing this for 10 years, your customers seem pretty happy, your feature roadmap is getting kind of sparse because you've implemented most of what you set out to implement, and you think your software provides most of the value that you want your software to provide.

Think of how much extra work it is to make that mature product 1% better, you know. Maybe you'll get the 102nd feature into it, and it only has 101 right now. And maybe, you know, after two man-years of work implementing that feature you'll be able to say, “Yes!” to customers, and some small percentage of them, maybe 5%, will use it and it will make their lives fractionally better, and that will increase your sales by 1%. And that's for years of work.

Instead, you can make this A/B testing a routine thing in your play book. And you'll find, for mere hours or weeks of work, that you're getting, you know, a 5% improvement to the bottom line, a 5% improvement to customer success with the product, a 5% improvement to number of schoolkids who you helped to teach to read this year — for almost no effort at all, comparatively.

So, I really want to encourage you. If you do one thing from this presentation, go back and start trying it for a while. It's not much work to get started. Just try it. You, probably, will like it.

So, if this A/B testing is so magical and gets such awesome results from it, why doesn't everyone actually do this? Well, one reason is that, you know, we have cultural issues about it. We're engineers — this smells, vaguely, of marketing. Also, there's a tendency for A/B testing to tell you things that you really don't want to hear. Designers don't really want to hear that all the best design practices, well not all the best design practices. Many good, like... best practices are best practices for a reason. It's knowing which 80% of them are best practices for a reason, and which 20% are best practices because someone was able to state in a confident tone of voice they are best practices.

But, you know, that last 20% can really hurt you. Here's an example: many designers will tell you — and this is written in books — that your website design should be consistent on every page of the website because that makes it much easier for users to find their way around the website and do what they need to get done.

And what Amazon will tell you — because they've tested this extensively — and what everybody who I've tested with will tell you is that when people get into the directed mode of, like, trying to purchase your software, it's better to break your website's design and make it look not like the rest of the website. Just give them two options: either you go for it and purchasing, or you go back to the homepage, rather than: “OK, you can click out to the blog, you can click out to our documentation, blah, blah, blah.”

That will greatly increase the number of people who have successfully purchased the software, succeeding in their task, at the moment because they're on your purchasing page to purchase it, right? And if you ask customers... If you, like, collect metrics about what peoples' use of the software is after that, you know... Proxying for their happiness because your software is making them happier, right? Just that change will increase user happiness and user success with your software. “Well, I'm making you a pile of money, so you should probably do it.”

So, one of the things to make your testing — make your organization an A/B testing organization is, you really need to eliminate all friction about doing A/B testing. Some of my clients have, you know, workflows where it can take up to an hour for things to get from the point where it's on someone's computer to where it's visible on the actual Internet. And if it takes an hour for you to change copy on your homepage, you're going to do it like once a year whether it needs to be updated or not, right?

So, instead, you want that change to be virtually instantaneous. And you can do this with the A/B testing libraries we talked about earlier. If you're writing code for A/B tests, you want to write as little code as possible because engineers are lazy. That's like our job description. And you want to empower your engineers to be lazy by saying, “Hey, writing an A/B test is as easy as — take the code that's already there, put it in an “if” statement, write what you were considering doing, and then you're done!”

And when you do that, you'll find that you're not an A/B test every blue moon when, you know, a spirit moves you. You're running A/B tests every day. I was running... The reason I got into this whole low-touch sales processes towards large numbers of potential prospects was because I only had five hours a week to work on this business for most of the time I was running it because I had a day job.

And the... One of the few things I could actually do — because five hours is not enough to write new features — is, on Sunday evening I could push an A/B test live in a few minutes, and then the next Saturday I could look at the results for it and gradually improve my business over the years doing that. And, you know, you can pound a 5% improvement every... in a week. If you get an improvement one week out of every four you try, then 5% a month for a year is like a 70% improvement in the bottom line, which works pretty well.

You also need to have a culture of testing at your company. Many companies default to a culture of: “Let's ask what the boss thinks.” We once had a six-hour discussion at my company, my old Japanese company, about the difference between — and I'll translate it into English — the mailing list sign up, which we had 40 sign ups a quarter... that the mailing list sign up was either “Submit email,” or “Submit email address.” And this was, like, the six-hour meeting from heck, you know, just figuring out which of these two things we should use. And finally, we decided it the traditional way. The boss said, “I have heard all of your opinions. And I am the boss, I will make the important consequential decision to say, 'Submit email.'” If you have a culture of testing in your company, that discussion would have lasted for about five seconds. It's like: “Yeah, throw them both up at an A/B test. We'll see if it even matters.” And that will save you a lot of time.

So, the third major technique I'm going to be talking about is about the first run experience. And it's really important, because of all the people who have ever used your software, you can only say one thing about all of them: they've all run it the first time. How many, like... Do people... Raise your hand if you have a good answer for how many people run the free trial version or the actual version of your software the second time, percentage-wise? Like, raise your hand if that's something you measure.

OK. So, we got one or two hands.

This is another thing you can implement in very little time, and the number is going to shock and dismay you. 40% for my company, when I started measuring this, I got into most of my consulting companies, consulting clients, 30%, 40%, 50%, see the software the second time. You know, often there are, like, hard costs of several tens of thousands of dollars to get into a particular prospect's hands, and they play with it for three minutes and then stop using it, and that's the end of that customer relationship. That's clearly sub-optimal, right? So, you have to totally nail that first impression. And if you structure the first impression... the first use of the software as a funnel, where we think they are going to take activities A and B and C and D, and you measure where customers are falling out of it, they import their data, but they don't go up to setting up their admin users and they don't complete the stuff after that, then you can use A/B tests for improvements at that point in the funnel to make it work better.

And there are some examples of sites that do this to... some tactics that might work for you. Ah Peldi.. So, I really love Balsamiq Mockups. And this is not actually the Balsamiq Mockup screen, because it starts completely empty. And no one has any clue when they start with software that's completely empty. What do I do here? And no matter what your software does, if you start people at a complete, like, Garden of Eden state, most users are going to be: “Oh dear, what do I do first? I don't know. How do I get value out of this? It's not obvious; there's just a piece of white paper ahead of me.” If you're a designer, white paper is fun, but actual users have lives that they want to get back to and white paper is just: you have to do a lot of work prior to getting back to your life, and most of them are just going to crinkle it up and throw it away.

So, what Balsamiq Mockups actually look like when you start it is this. It has fake data imported into it which both tells you how you can get start with the software, you know, “Click here,” “Click here,” “Click here,” “Drag this,” “It's risk-free,” “There's a control Z available at any time,” and it starts to expose you to some of the richness of the software. You know, this thing that you're seeing and you're editing pane was clearly created just with the software. You can make something just, you know, just as vibrant and rich with just a few clicks of your mouse.

And so... You know, I don't have access to [Polly's] stats, but I'll bet if you gave people those two interfaces and saw which one got more uptake among customers, the one that showed them what they could do with the software would do much better. So, one thing that we don't do enough is customizing users' trial experience.

Raise your hands if every user who starts your free trial gets — they start at the same point.

Almost everybody, right? Raise your hands if prior to a user starting a free trial, there is some way you can know something about that user. I think that's true for almost everybody, because the user came from somewhere. They went to their local Googles and they described in excruciating detail what their paying point is. Many people this week are describing to their local Googles that their paying point is that they don't have Halloween bingo cards. And so, they come to my website, and I know, because Google would tell me, they searched for Halloween bingo cards.

So, if they sign up for a trial, should I give them a blank screen that doesn't give them any directions? No. I should look: “Hey, you want to make Halloween bingo cards? As a matter of fact, the Bingo Card Creator thing will actually do that. So, I'll give you easy options for starting that.” And those easy options for starting it increase the conversion to actually using the software among people who have... you know, who I can determine their intent by upwards of 20%. It's absolutely huge.

You can do that: know what search terms people are using, what advertising criteria do they respond to, what industry did they say they were in, what use case were they looking at on our website? You know, we have all this data sitting in our — sitting on the server somewhere. Use that data to make their lives better, because that's why you're here.

Anyone ever use this multibillion dollar software application? It's World of Warcraft. If you get dropped into World of Warcraft and... I've wasted a couple of years of my life on this. I used to be a (?) Guild leader — don't judge me. [Laughter]

But if you get dropped into it, the first thing they do is put a popup that says, “You right-click on stuff to interact.” And then you close the popup and there's this guy standing right in front of you with a big exclamation point on his head. And since you've only learned, like, one thing you can do in this world, you right-click on him and he says, “Hello. I am the Quest Giver. I am going to teach you about how you play this game to give us money.” [Laughter]

And he says, “Right behind you there is a wolf, you have one button, it's a Z, kill the wolf with Z, Z, Z, Z, Z, Z!” And you go Z, Z, Z, Z the wolf and it's like, “Bing! You succeeded in killing the wolf, great! Go back to the Quest Giver.” You go back to the Quest Giver and he says, “Great job on killing the wolf with Z. Now there's ten wolves, and I'll give you two buttons: Z and X. Kill the ten wolves and save our planet!”

And you go kill the ten wolves and you come back. And it's already... it's teaching you how to interact with the software at the same time while you are feeling, “Wow, I'm really getting success with this, I'm saving the world! I'm about to spend the next 720 hours of my life playing this game.” [Laughter]

You can do this in your software too, right? A guided tour in the middle of the software that shows them, you know, shows them the really fun bits and start — makes them feel like: “Wow, I'm already getting value this quickly!” Rip off this idea. Play World of Warcraft if you need inspiration. But stop after the first five minutes. [Chuckling]

So, a lot of us, you know, we spend so many years building the software, and we know that there's huge amounts of value hidden in the software but we hide it, you know. They might be a data-import step but you have to really start using the software for months until you start to really feel the productivity gains in your own life.

Many of us have this problem, right? It is? Yeah. So, you typically will not be able to convince your users: “OK, drop whatever you are doing. I want you to change the way you do business for the next few months. Use our software and come back to me and in the next few months, you'll, like, sort of feel life is better. And you'll know our software did it.”

That's a very hard sell. What instead you should do is spoil the good bits, figure out the, you know... You think your software is better. Why? It is still that to one, you know, the cappuccino of goodness and then give that to them right away. Like, I have this site called the “Appointment Reminder,” and it makes appointment reminder phone calls to, say, a doctor's office to his patients to tell them, “Hey, please come to your appointment at 2 o'clock.”

My value proposition to the user is two-fold: A, I'll make these reminder calls for you so that you don't have to do yourself and you don't have to hire a person to do it; and B, because people get the reminder call, your no-show rates for appointments is going to go down, and that's going to make you money.

Then those two things like save time, make more money are the generic... You can sell anything to enterprises if you can hit those two bullet points. So... but actually using the software it's kind of a hard sell because I have to tell people: “OK, first you need to stop whatever you're doing for scheduling appointments right now, start scheduling them with this software. Several weeks from now when your customers come to their appointments you are going to see that fractionally more of them actually come, and that's going make you huge amounts of money because fractional — like, fractional, decimal point of anything times cost of healthcare equals, like, a mountain of money.” [Laughter]

So, that's a hard sell, right? So, instead of doing that I have a quick little demo that I show people. And it's like as soon as you start using the software I say, “Hey, what's your phone number?” and people give in. And I give them a call on the phone which demonstrates that I can call phones and everyone... You are using a computer and you call a telephone. People feel that's a very magical interaction. “What, the computer called me? Hello?” [Laughter]

And a comely lady who goes to some college who I paid five dollars to and recorded this script gives them a sales pitch for the software, basically... in their ear and they listen to the sales pitch because whoa, a computer is talking to them on their telephone. And she says, “Yeah, this is, you know... We'll schedule appointments for your customers it's very, very easy, just as easy as putting your phone number was there.” And then we tell them that their appointment is and it says five minutes from now.

So, your fake appointment is five minutes from now. Are you coming to your fake appointment? Press 1. If you can't come to your fake appointment, press 5. And then they hit 1 and immediately on their screen, it pops up and says, “You confirmed your fake appointment,” and if they cancel, “You canceled your fake appointment,” but that's not a problem because we could email you right now and say, “Mrs. Smith is not coming to her 2 o'clock, so you can book someone for this and save the revenue for the 2 o'clock slot.” And now, instead of perceiving value six weeks after using the software, they perceive value roughly like one minute and 48 seconds after starting using the software.

And people who go through this process are much, much more likely to actually make the time and investment to start using it. And once they start using it and they see, because, you know, I send them stats every week: “Your prior no-show rate was 28.5%. We've got it down to 5%. The actual amount of money we made you is blah.”

People really respond possibly to that. A word of caution: we got into this business to make good software and help people's lives, right? This whole... the notion of using math, using science to improve our marketing such that we can... such that we can improve our businesses is a powerful tool, but it's a tool, not a cause.

The kind of pathological end game for, you know, making minor incremental improvements to your business is that you turn into Zynga, where you come up with a business model which is intentionally addictive to people and basically destroy your — their lives to line your own packet.

And none of the people in this room are going to have that problem, but just remember you have the great product or service first, and then you incrementally improve it towards even higher steps of greatness.

I often think that in this software industry we have wonderful, wonderful gifts with the power to change the world and we don't make great use of them. I mean, Harry Potter would have been a pretty bad book if, you know, Hermione came up with a new spell and was it “expecto,” poking. [Laughter]

And, you know, it would have been a pretty boring book too because the villain would have been: “All right, I've got this great, evil plan. I'm going to make everyone cultivate fake mandrake root, and them I'm going to cast a curse of withering on the fake Mandrake root so they have to come back and cast a counter-spell every four hours. If they don't cast the counter-spell, they have to plant more fake mandrake root.”

Terrible book, right? I'm sorry, I don't mean to hate on Zynga. It's just... I don't hate Zynga, I just love all things that are good and true in the world. [Laughter, clapping]

And so, you know, use your enormous powers for good, not evil. So, there's some more ideas for how you can use your engineers to improve your marketing outcomes on the bingo card, but don't try to read it because the text is way, way, way too small. You can find it on my blog later. I'm always happy to talk about this or any other subject with pretty much anybody. My contact information' s up there, I’m all over the internet so if you just post a story on Hacker News, I'll probably see it. People have done that to get in contact with me before. It's like, “Ask Paddy blah blah blah.” Email... Even my teachers can understand it — presumably know how to use it but... [Laughter]

It will work. Anyhow, thanks very much for being an attentive audience. We have about twelve minutes, and I would be happy to take your questions. [Applause]


sp3: When you’re doing A/B testing on offers.

sp1: We have mikes going around for the people who are watching this on the Livestream, so...

sp3: Sorry. Thank you. When you're doing A/B testing on offers, how do you avoid pissing people off by giving them different prices?

sp1: So, this is something that your A/B testing library has already figured out for you. It's usually going to track people with cookies and say that a particular browser will only ever see one of the offers or the other. But people have multiple devices these day, and sometimes that doesn't work. So rather than testing prices, you might test, like, subtly different products in that, you know, they're not seeing the difference between vanilla ice cream at $1 and vanilla ice cream at $2; they're seeing vanilla ice cream at $1, vanilla ice cream and this little, like, magic sprinkle at $1.25.

And when they see that, because they only get to see one of them, they'll see: “Oh yeah, it's $1.25 for the vanilla ice cream offer.”

But if someone says, “Hey, wait. These two are different.” “Oh, yeah, they're different. We have different product mixes, we test them out.” People very rarely get mad about this. In 5 years I have 200.000 users for the software. Only one person has ever caught on their being an A/B test. And that's partially because, you know, you implemented it successfully, and partially because people just don't care. You and your software all the time. It's one of the most important things to your professional life.

Your users, they have lives that, you know, 23 hours and 57 minutes of the day, are some place much better than your software. They do not notice the differences in it. So, the one person who did notice the difference said, you know, “I paid money for the software and I didn't get the color feature. And I went over my sister's house, and I logged in to her free version and she had color. What's up with that?” And I gave her her money back.

So, total cost of A/B testing for me over five years, 25 bucks. [Laughter]

Next to diamonds. Any other questions? Yes?

sp1: Gentleman down in the front?

sp4: Something I've struggled with is the first-time use experience.

sp1: Yes.

sp4: And the... Kind of want you to over-label the software. Like, on a field or whatever, you tend to put extra labels and extra help. They're really only useful to the user that first time. You know, after that first time, they know what it is, and you don't need it anymore. Probably get into overlabel for the first time and then you demo it to someone and it looks really complicated because you got so many labels.

sp1: Mm-hmm.

sp4: And kind of... How do you make all that work?

sp1: So, one of the things I like doing, there's no commandment to like, the engineering books of ethics that says your software has to perform the same every time it's run.

So, one thing you can do, you know if someone is using it for the first time, just put a notification on the top of the screen, or maybe a modal window that says, “Hey, we noticed you're using this for the first time. We've prepared a tour for you. Would you like to take the tour? Yes or no? You can always retake the tour by going to “settings” and clicking the “take the tour” button.”

And if they're in tour mode, you give them the, you know, either the simplified version of the experience, or the “We put labels on everything so that you know what it is” version of the experience. And when they're not in tour mode, you've give what you think as the usual use of the software.

And A, you'll find that people that do the tour convert much better in most cases, if you do the tour right; and B, that won't compromise the use of the software when you're an expert eight years down the line — you already know what all the features do. Does that help? Thanks much.

Any other questions? Yes, gentleman in the back?

sp5: You've got your feature when somebody's signing up for a trial.

sp1: Yes.

sp5: And... Like, for example, you detected that they were looking for Halloween cards.

sp1: Mm-hmm

sp5: Is there not a chance that the user will feel invaded, like you know what they were looking for? Because sometimes, I think, people have a sense of safety when they're browsing the Web...

sp1: Mm-hmm.

sp5: They just... nobody knows — they haven't filled in any forms yet, they're still safe. But then your website just is interacting with them. Which is cool.

sp1: Sure.

sp5: For some people it's probably — it's quite clever, there's another percentage of people who react negatively to that kind of attention.

sp1: OK. So the question is, if you're too clever about the user's needs, won't they think you're spying on them, and won't they react negatively? And I have multi-part answers to this question. One depends on your users. Engineers are weird, we care about things like that. Real people are less weird in that they don't care how software works. This is, you know, the notion that someone could see that I was searching for Halloween bingo cards and that could, like, be broadcast to the entire world. Totally not at the top of their list of concerns this day. They want to get back to their kids faster.

sp5: Mm-hmm.

sp1: You provide them with that, they will be totally happy with it. So, know your market on that. If you're worried about it, test in isolation. Give 10% of your people the more, you know, Facebook-y version of the world, and 90% of the people the traditional “we don't pretend to know anything about you,” and see which one works better. See if you get any complaints.

I have, you know, over 200.000 people have used this software, several tens of thousands have gone through this interaction with the software and anticipating their needs and I've never once had a complaint that said, “Whoa, whoa, whoa, when I went to your website to make the Halloween bingo cards, I don't want to tell you I was looking for Halloween bingo cards.”

Funny little sidenote on that. So, I did get one customer request support that said, “I couldn't make the bingo cards. Can you please help me with them?” And I logged in to her account, which was not something she expected me to be able to do, and I'm like, “Yes, was the bingo cards you were having trouble with the... the... the... 'sexual health for seniors' bingo cards?” And she says, “Ah, yes.” [Laughter]

And I said, “Ah. So, one of the fields... Have you ever had multiple pa... That one was a little too long for the software. If you truncate that one you can get the cards to print out the way you expected.” And she said, “Oh, thanks. And thanks for being so understanding about this.”

And I'm like: “[inhales sharply]” [Laughter]

“No problem, Ma'am.” But yeah, one awkward experience in five years — not too bad. Mm-hmm?

sp6: One idea about the Halloween. Maybe you could test, say, “Great for Halloween, Christmas...” No wait... Yeah, I know, but it's like more generic that… Maybe it would be worth testing to say, “Oh, it works for all sort of holidays, like Halloween,” or, you know, whatever they typed.

sp1: So, thank you for the idea. I've tested various variations on that theme. Not to bore you with, like, anecdata from my business. But one the things that I did was saying — the most popular uses of our software are X Y and Z, and customers didn't really respond to that.

When you put it in a discoverable thing on the left-hand side of the screen and say, “Hey, here are pre-made stuff,” and you populate the pre-made stuff with the ten most popular this, or you can search — people really love that — but when you kind of... you throw ten things to them like, you know, “People really love this for wedding showers, baby showers, Halloween, Thanksgiving, Christmas.”

“It's the middle of October, and I'm not getting married. I just want the Halloween stuff. Give me the Halloween stuff.”

So, if I have good data that says someone is really interested in Halloween this week, then they're probably interested in Halloween this week. I have a funny story there, but ask me for it later. Yes?

sp6: How much instrumentation is appropriate to put within an application to track features usage?

Again, getting back to the creepy aspects. You want to make your users successful; if you see what they're doing, you can perhaps make them more successful. But I don't know if you can comment on instrumentation within... Yeah.

sp1: Sure. So, keep in mind — and this is why we had the lovely four-quadrant graph — I sell bingo cards to elementary school teachers. I'm not in a regulated industry, so I can get away with, basically, logging everything you ever do. And if you are, you know, if you have XYZ compliance to worry about, you probably can't get away with doing that.

But, in general, I log everything and then I expose the data to myself and only myself, where I think it would be appropriate, and I make some data publicly available on my website, because I've published stats for years. And that stuff goes through a fairly rigorous... Like, I watch it for anything that could possibly be personally identifying, because I worry about that sometimes. But, you know, in general if it's just going to be sitting on a spinning platter somewhere, it's better to have more data than less.

And what you do, like... Customers aren't too annoyed with data being collected because people... Like, engineers care about that — the real world, not so much. What they're really annoyed is when it gets exposed in ways they didn't anticipate. So, you know, make sure that you're only seeing stuff that you should be seeing and that when you, you know, do sensitive things like logging into someone's account, you have their permission or you massage that messaging in a way that makes sense to them, like: “Oh, to help you fix that problem, I just took a look at what the computer had recorded about you making those cards, and I saw blah.” Or, you know: “We took a look at the...”

A great tip that I heard last week at Brooklyn Beta: you have exception notifications that get sent to you, right? And you can mail the people who are in charge of the account for that and say, “Hey, the automated system picked up that there was a problem with your account. And because I'm the head of engineering — or the head of the project — I took a look at it and I fixed the problem. And I just wanted to tell you that I fixed the problem.” Your customers will love that. And if you massage that messaging at the front like: “Well, yeah, I, you know... I see where the user experience falls, but it's not me, it's the computer.” Make the computer the bad guy? People don't usually have a problem.

Sure, Peldi?

sp7: I'd be careful with desktop software.

sp1: Mm-hmm.

sp7: There's a very different expectation there...

sp1: Right.

sp7: ...and it should be an opt-in if you want to send data to the server.

sp1: Peldi is absolutely right here. Peoples' user experience expectations for desktop software are a little different. I'd be really careful with desktop software too. If you're writing it, you should stop. [Laughter, clapping]

Sorry. So, there's a blog post... That's a pretty controversial thing to say. I have 2000 words on the blog post. Search for “Why I don't make desktop software anymore,” and it'll be in like the top five.

But yeah, your life and your customers' lives will get much better when you move into the SaaS model. The difference between HubSpot and me, they charge money every month. It's great stuff. And not only will — your life's better and they'll be better, but the expectation for everyone using the Internet is that Google Analytics is on like 40% of the pages on the Internet, and that's OK.

And that everyone has cookies on all the time, and that's OK. And, yeah, there's like a weird little microsegment of the population called engineers who actually know what a cookie is and don't want them on their hard drives. And that's OK too. They don't have to pay us money because they never will pay us money anyhow.

Alright. I think we're just about done. Feel free to come up to me with the rest of the conference or send me an email. Thanks very much for listening. [Applause]

Don't forget, registration for BoS 2012 is open now. We hope to see you in Boston October 1st-3rd 2012 and the first EARLY BIRD DISCOUNT tickets run out on March 30th. Save $900 on full registration.


March 12, 2012

Founder's Dilemmas - what do you worry about when growing a business?

I am still celebrating the news (you heard it first here!) that Professor Noam Wasserman has agreed to speak at this year's Business of Software on the subject of Founders Dilemmas.

Not only that, he has generously agreed to try to answer some of your questions in a guest blog to be published at the end of this month. How very, very cool is that?

Here is where you come in...

If you have a Founder Dilemma, leave us a brief summary of the question or issue you have (anonymous or named, entirely up to you), here by midnight on the 18th March. We pass on the best to Noam. Watch the blog for updates...

March 09, 2012

What did you take away from BoS2011?

We asked BoS2011 attendees what their key learnings were from the conference...

  • Greater honesty and transparency with our customers.
  • Having the right team is critical. Focus on internal tools is worthwhile.  Product is only a small part of the customer experience
  • Formalizing our plan to pay down our code debt. 
  • Pricing for the Lizard Brains of enterprise customers. Measuring A/B tests from the top to the bottom of the funnel.
  • To focus on the job our software is doing
  • Holy crap is social media ever important! Your presence is almost a product in itself.
  • Social media participation
  • Use more metrics in our customer acquisition process. Dedicate more time to marketing. Become closer to our top customers.
  • A/B testing, motivation to release soon and a new pricing page. Networking!
  • 1. Be more honest. 2. Change the copy on most of our web site. :)  3. Measure more.  A/B testing.
  • Too many to tell, top 3 are:  1. how important honesty and transparency is with customers  2. testing, testing, more testing and understanding the metrics  3. how to draw an owl, #JFDI
  • *Improve website copy  *Split testing  *(Continued) Passion for what we do  *Validated our 'basement-based' claim (Thanks Jason Cohen)
  • Flip the equation and put customers ahead of other stuff in decision-making.
  • There was so much, it's hard to pinpont a single thing. Communicate with your customers.
  • As a Dev Evangelist, the business plan modeling for working with customers and the sales/closing presentation for my own stuff.
  • To do a better job of managing the creative resources of my team.
  • The value of A/B Testing
  • Business Model canvassing
  • I took a lot away. Creativity, A/B testing, The Funnel, I need to do talks again, Sort out twitter, read a LOT more.
  • Start looking for problems that I have a passion to solve. That will be the SaaS application I will try to start this year (currently only do desktop apps)
  • Working on creative thinking towards software problems. The importance of A/B testing.
  • The social media approach. My next challenge is awareness, and I like the concepts presented. Not easy, but something I want to work through.
  • The biggest action point is the idea of being honest 'internally'. By which I mean being more transparent with management processes - eg. sharing of employee survey results - and that being honest about existing problems challenges with employees can lead to greater motivation and involvement. Immediate action will be sharing results of functional heads' work/surveys with the function we're surveying!
  • Look at all of our processes with a fresh eye.
  • Many things. Here are my first couple to focus on:  1_ Show our expertise via a book  2_ Use blogs/webinars around customer topics, not about us or about a specific customer (due to security/confidentiality)  3_ Change Corp Overview to ONLY tell stories to customers presentation
  • Measure more and report, in a timely way, more.
  • A much better understanding of SaaS based business models and the ease with which a new s/w business can be setup online.  Also I was unaware of the industry behind this (B2B companies like hubspot).
  • Too many to list. I've already started using the business model canvas to facilitate strategy discussions with our management team. I've also begun building hooks into our product to help with A/B testing and customer success measurement. Less concretely, but most profoundly, the conference has led me to make a commitment to start my own business in the next 2-3 years.
  • Capture better metrics around our Inbound Leads and application feature usage.
  • I will reach out to more people in my industry for customer discovery purposes.
  • Make an effort to perform more robust A/B testing. And think bigger. :)
  • Applying metrics (and perhaps some variant of A/B testing) to our desktop application.
  • Need more metrics in my SaaS app; going to change a marketing approach and try complete honesty to the point of pointing out flaws in my product.
  • Combining Clay's talk about focusing on the job + many of the talks about tracking folks religiously internally to make sure that its "mission accomplished"
  • 1. weekly sales meetings (Paul Kenny)  2. reposition our product in the marketplace (Clay Christensen)  3. Make sure everyone else does as much creative stuff as me (Josh Linkner)  4. Draw the rest of the fucking owl (Jeff Lawson)
  • Think disruptive! Be honest and transparent - it's the only way.
  • 1. more tightly focus our inbound marketing content  2. do more cohort analysis  3. start tracking user engagement by feature usage  4. do a better job in doing user-specific messaging  5. process for improving our website copy
  • I have 3 pages of to-dos. I will be focusing even more on customer interaction, on using metrics to make decisions, and thinking bigger.
  • Work on exercising our "creative muscle".
  • Too many to list here.
  • Do A/B Testing - possibly hire Patrick McKenzie. Tons of tips from Laura about blogging effectively. Tip from attendee on a good tradeshow to go to.  NPS surveys.  Might do rolestorming at next staff meeting
  • o A todo list 80 items long.  o I'm too soft charging for my software and maintenance.  o I should be more proactive (hate that word) about selling.  o I should be more proactive (hate that word) about ux research.  o When someone is ruining a speaker's talk by talking I should shut them up more quickly than I did.  o I should get out more. My socialising is not with tech people at all.
  • Loved Clayton's arguement about "What job is someone hiring your product for".
  • SAAS is the way to go, so we will create a Windows application, and we will add a/b-test to the application, and start with a tutorial.  We will add the middle alternative to buy.
  • To focus on the job rather than on the customer once people are using the product. Oddly enough it's the opposite (people's first) when trying to funnel them through the demo page.
  • Many things. I will summarize by saying we need to spend MUCH more time on our "pipe line" at every level.
  • How our web page can work harder for our business.
  • Trying to be more creative, finding out new ways to engage with customers. Being honest as well with customers. And maybe most important is that we do work with humans and not companies and we should work with that in mind.
  • For disruption, start small. JDI. Good enough is good enough. AB testing is important. Do something Social.  It Is OK To FAIL!!
  • Rolestorming. Finding our Purple Cow - from workshop. Sales Techniques - Asking for the sale.
  • Figure out what "job" our customers "hire" our software to do.
  • Come up with more metrics on the business. My gut feeling may have brought us where we are, but we must base more decisions on our data.
  • Business Modeling, Christensen's ideas on Innovation, RoleStorming, contacts made.  There is more, lots of good nuggets
  • A/B testing
  • I will re-visit company strategy having learned from Clay Christensen's talk
  • Only focus on the aspects of inbound marketing that you will excel at. Make our webpage simple. Find a way to get into the cloud (not easy for my niche).
  • We need to do a better job of measuring and testing
  • Testing Testing Testing. Loved the focus on it and David's discussion regarding data.
  • Lots of A/B testing!  ;)
  • ABD - Always Be Doing. Keep working/ Get Started Now
  • Metrics… Track everything.  And don't quit being creative.  Its the most important thing.
  • More focus on "Making the World Suck Less"
  • Seriously consider a SaaS model. Put more emphasis on non-feature improvements,
  • We'd just started doing usability testing on our software. We're going to ratchet that up and spend more time on the new-user experience.
  • Become a data driven company. Encourage creativity
  • Perseverance is key to building a business.
  • HR practices. some process improvements
  • Several things -   * need to be more intentional about defining our business model  * priority of investing in UI/UX  * paying attention to disruptors in our market and developing strategies to compete with them directly while still sustaining our current business  * practical tips on making our business data-driven  * solid recommendations from fellow attendees on better issue-tracking tools
  • Being brutally honest/transparent with the customers will help, not hurt the business.
  • My BoS commitment was to free myself up so I could dedicate as much time and energy to building my company and getting v1.0 of my flagship product ready for launch.    Realising this was going to be near impossible while working a fulltime job, I made a commitment at BoS to quit my job at FlexiGroup (who graciously helped me get to BoS, knowing my ambitions) as soon as I got home. It was very tough but after an incredible 7 year run working with some of the most talented people I've ever met, my tenure will come to an end next week.    What BoS has helped me with is to give me the courage to take this step, by introducing me to an incredibly supportive community of brilliant entrepreneurs. People who have been on a similar journey to me and are able to offer support or a friendly ear for the challenges I am facing. And it's very heartening, hearing about how others have similar challenges and being able to offer my own thoughts. Outside the walls of this conference there are very few other venues that offer this kind of experience.    Plus the fact that the delegates are just so friggin smart! I have been endlessly namedropping to the guys on my team all the great people I met but the highlight would have to be meeting Peldi from balsamiq. I love his style and how he built his fantastic company on his own mettle is truly inspiring.     Jason's talk was the highlight for me, and also his critique of my product pitch (which I already rabbited on about in my other survey response so I'll leave it at that :)).    My only regret from the conference was not saying hello to Mark! But I could see he had his hands extremely full. Oh and that I forgot to collect that bottle of champagne ;)
  • Work on operational metrics. Build internal tools.
  • 1. Measure my business in more and different ways than I'm currently do.   2. JFDI.

March 05, 2012

Here's to the third to a half of the world's population that are introverts.

Two great pieces about introverts from the past couple of weeks. Always makes me think of Business of Software!

Sacha Chua on introverts and how to be shy effectively.


Susan Cain, talking at this year's TED Conference in Long Beach on the power of introverts.

Three take aways from Susan's talk.

  • Stop the madness for constant group work - we need to teach people how to work on their own.
  • Go to the wilderness - we could all benefit from unplugging and getting inside our own heads a bit more often.
  • Take a good luck at what’s inside your own suitcase and why you put it there. When you have watched the talk, it makes perfect sense!).

Don't forget, registration for BoS 2012 is open now. Hope to see you in Boston October 1st-3rd 2012.

About Business of Software

THE conference for people who care about growing long-term, profitable, software businesses. Follow us on Twitter. BoS Blog.

About Neil Davidson

Joint CEO of Red Gate Software and Founder of the Business of Software conference. Follow him on Twitter. Neil's Blog.

About Mark Littlewood

Founder of the Business Leaders Network (TheBLN). Organizer of the Business of Software conference. Follow him on Twitter. Mark's Blog.

Free eBook

Don't just roll the dice How do you price your software? Is it art, science or magic? This usefully short book will help you get the theory, practical advice and case studies you need to stop you reaching for the dice.

Creative Commons

Creative Commons License

This work is licenced under a Creative Commons Licence.

Business of Software 2012

a Business Leaders Network & Red Gate Software conference

October 1st-3rd 2012

Subscribe to conference updates and get a free ebook!

View more posts