The Rule of Thirds: The Web Performance Analyst

Blurry Man - Brian Auer - http://www.flickr.com/photos/brianauer/2929494868/Recently, there has been a big push for the Dev/Ops culture, an integrated blending of development and operations who work closely together to ensure that poor performing web and mobile applications don’t make it out the door. They have become the rockstars of the conference circuit and the employment boards.
I fit into neither of these categories. I have never run anything more than a couple of linux servers with Apache and MySQL. I write code because I’m curious, not because I’m good at it – in fact, I write the worst code in the world and I am willing to prove it!
I am a member of a web and mobile performance culture that is language and platform independent, to use some buzzwords.
I am a web and mobile performance consultant and analyst.
I can take apart reams of data to find statistical patterns and anomalies. I believe that averages are evil, and have believed this for more than a decade. I have been using frequency and percentile distributions for almost as long and watched as the industry finally caught up.
I can link the business issue that faces your company with the technical concerns you are facing and help guide you to the middle ground where performance and the balance sheet are in careful equilibrium.
I don’t care what you write your code in. I don’t care what you run it on. Now, don’t get me wrong: I respect and admire the Dev/Ops folks I have met and know. I am just not in their tribe.

HTTP Compression – Have you checked ALL your browsers?

Apache has been my web server of choice for more than a decade. It was one of the first things I learned to compile and manage properly on linux, so I have a great affinity for it. However, there are still a few gotchas that are out there that make me grateful that I still know my way around the httpd.conf file.

HTTP compression is something I have advocated for a long time (just Googled my name and compression – I wrote some of that stuff?) as just basic common sense.
Make Stuff Smaller. Go Faster. Cost Less Bandwidth. Lower CDN Charges. [Ok, I can’t be sure of the last one.]
But, browsers haven’t always played nice. At least up until about 2008. After then, I can be pretty safe in saying that even the most brain-damaged web and mobile browsers could handle pretty much any compressed content we threw at them.
Oh, Apache! But where were you? There is an old rule that is still out there, buried deep in the httpd.conf file that can shoot you. I actually caught it yesterday when looking at a site using IE8 and Firefox 8 measurement agents at work. Firefox was about 570K while IE was nearly 980K. Turns out that server was not compressing CSS and JS files sent to IE due to this little gem:

 BrowserMatch \bMSIE !no-gzip gzip-only-text/html

This was in response to some issues with HTTP Compression in IE 5 and early versions of IE6 – remember them? – and was appropriate then. Guess what? If you still have this buried in your Apache configuration (or any web server or hardware device that does compression for you), break out the chisels: it’s likely your httpd.conf file hasn’t been touched since the stone age.
Take. It. Out.
Your site shouldn’t see traffic from any browsers that don’t support compression (unless they’re robots and then, oh well!) so having rules that might accidentally deny compression might cause troubles. Turn the old security ACL rule around for HTTP compression:

Allow everything, then explicitly disable compression.

That should help prevent any accidents. Or higher bandwidth bills due to IE traffic.

OCSP and the GoDaddy Event

Image by vissago - http://www.flickr.com/photos/vissago/
Image by vissago – http://www.flickr.com/photos/vissago/

The GoDaddy DNS event (which I wrote about here) has been the subject of many a post-mortem and water-cooler conversation in the web performance world for the last week. In addition to the many well-publicized issues that have been discussed, there was one more, hidden effect that most folks may not have noticed – unless you use Firefox.
Firefox uses OCSP lookups to validate the certificate of SSL certificates. If you go to a new site and connect using SSL, Firefox has a process to check the validity of SSL cert. The results are of the lookup cached and stored for some time (I have heard 3 days, this could be incorrect) before checking again.
Before the security wonks in the audience get upset, realize I’m not an OCSP or SSL expert, and would love some comments and feedback that help the rest of us understand exactly how this works. What I do know is that anyone who came to a site the relied on an SSL cert provided and/or signed by GoDaddy at some point in its cert validation path discovered a nasty side-effect of this really great idea when the GoDaddy DNS outage occurred: If you can’t reach the cert signer, the performance of your site will be significantly delayed.
Remember this: It was GoDaddy this time; next time, it could be your cert signing authority.
How did this happen? Performing an OCSP lookup requires a opening a new TCP connection so that an HTTP request can be made to the OCSP provider. A new TCP connection requires a DNS lookup. If you can’t perform a successful DNS lookup to find the IP address of the OCSP host…well, I think you can guess the rest.
Unlike other third-party outages, these are not ones that can be shrugged off. These are ones that will affect page rendering by blocking the downloading the mobile or web application content you present to customers.
I am not someone who can comment on the effectiveness of OCSP lookups in increasing web and mobile security. OCSP lookup for Firefox are simply one more indication of how complex the design and management of modern online applications is.
Learning from the near-disaster state and preventing it from happening again is more important that a disaster post-mortem. The signs of potential complexity collapse exist throughout your applications, if you take the time to look. And while something like OCSP may like like a minor inconvenience, when it affects a discernible portion of your Firefox users, it becomes a very large mouse scaring a very jumpy elephant.

Web Performance: Your opinion is only somewhat relevant

Project 365 - Year 2 : Day 004 : 04/01/10 - Peter GerdesContext is everything. Where you stand when reading or watching something shapes the way you experience it. Just as Einstein explained to us in the Train/Platform Thought Experiment, the position of the observer dictates how the event is described and recorded.
There is no difference with web performance. When a company develops an online application and presents it to customers (it doesn’t matter if they are outside/retail or inside/partner/employee), the perspective of the team that approved, created, tested, and released the application becomes, as a VP at a previous company explained to me, “interesting, but irrelevant”.
Step away from the world of online application performance for a minute, and put yourself in the shoes of the customer; become a consumer. How do you feel when a site, application, or mobile app is slow to give you what you want? I’ll give you some idea:

The stress levels of volunteers who took part in the study rose significantly when they were confronted with a poor online shopping experience, proving the existence of ‘Web Stress’. Brain wave analysis from the experiment revealed that participants had to concentrate up to 50% more when using badly performing websites, while EOG technology* and behavioural analysis of the subjects also revealed greater agitation and stress in these periods. (“Web Stress: A Wake Up Call for European Business”, emphasis mine)

I know it comes from a competitor, but it is true. It applies to me; it applies to you. And web performance professionals need to step away from the screens for a minute and put themselves in the shoes of the people standing on the platform.
Everyday, your online applications change, grow, fail, falter, and evolve – the train is always moving. To the people on the platform, all they see is your train and how it’s moving compared to the other trains they have watched go by. You worked hard on your train, polishing the brass, adding new cars, even upgrading the engine. To you, the train is a magnificent achievement that everyone should admire, especially now that the new engine makes it so much faster!
The customer on the platform is measuring how your updated train is moving compared to the MAGLEV bullet train on the super-conducting rail next to you and asking “How come this train is so slow?”
The complexity of a modern web site is astounding, and improving performance by 0.4 seconds is often a feat worthy of applause…among web performance professionals. From the perspective of your customers, that 0.4 second improvement is still not enough.
Web performance is a numbers game. As an industry, we have been focused on one set of numbers for too long. The customer experience, not the stopwatch, has to drive your company to the next level of performance maturity. To do that, you have to step off your online application train and take a cold hard look at what you deliver to your customers, alongside them down on the platform.

Company Culture is your Company Reputation

Building on the theme from yesterday, I am now more motivated than ever by an article on the Fast Company site today: Culture Eats Strategy For Lunch
A number of books on my list this past month (Tribal Leadership and Delivering Happiness to name two) showed me just how critical a true, strong, and real culture is in allowing any organization to step beyond the brand. When a company can step beyond its brand, it has the rare opportunity to demonstrate what it means to be a great, not merely a good, company.
How do you do it? The examples are everywhere, and they all show the same thing – the company comes last.
Ok, so maybe not last, but you get the point. Doing what’s right for the company (and in really bad companies, what’s right for me!) has turned organizations so many companies into examples of corporate inertia: If we keep doing this, maybe they won’t hate us.
How has your company REALLY (no lip service allowed!) put the customer first today?
Can you find an example where the whole company put the customer (not A CUSTOMER) first?
Image courtesy of Jacob Nielsen

Still trying to brand yourself? That must hurt…

I’ve been enjoying the articles posted by Matthew Prince on PandoDaily from the WEF in Davos over the last week. But the one that got me in the right place at the right time is the one where he described how Paddy Cosgrave, inspired by the desire to make something happen in Ireland, created the F.ounders conference.
How did this hit me? It focused on how someone stood up and created a reputation that he can carry anywhere he goes. Not a brand; a reputation.
As I have said before: Personal Branding is all about you, closed source. Everything has to come back to the “I” that’s not in team (although there is a “me”, so a person can still screw up a team).
Taking what you have, and giving it to others to advance everyone, that builds reputation.
Are you building a personal brand or a personal reputation?

The Three Pillars of Web Performance

Had a great conversation with a colleague today. She and I were bouncing around some ideas, and I listed my top 3 topics in Web performance as “Speed, Revenue, and Experience”. She was quick to correct me.
“No, not revenue, conversions”.
She was right. Just last week, I talked about how critical it is to convert visitors into customers. Doing this in some businesses doesn’t mean that there is any revenue, but the goal remains the same.
Speed is the one everything thinks is the same as Web Performance. It’s not. It’s the don’t be that guy measure of Web Performance, the one that can be easily quantified and put on display. But performance for an online application is so much more than raw speed.
Experience is the hardest of the three to measure, because what it is depends on who you ask. Is it design, flow, ease of use, clarity, or none of these things? But a fast application can still make people cranky. There are online applications that are clearly designed to make the customer do things the way the vendor demands and these are the ones that make you go “Why am I here?”.
Now, can all the metrics that measure Web Performance be distilled to Speed, Conversions, and Experience? If you stepped away from the very product specific terms the Web Performance industry uses every day, what would describe the final, bottled, and served essence of Web Performance?

Web Performance: The Myopia of Speed

In February 2010, Fred Wilson spoke to the Future of Web Apps Conference. He delivered a speech emphasizing 10 things that make a Web application successful.
The one that seems to have stuck in everyone’s mind is the first of these. People have focused, quoted, and written almost exclusively about number one:

First and foremost, we believe that speed is more than a feature. Speed is the most important feature.

Strong words.
Fred has worked with Web and mobile companies for many years, so he comes at this with a modicum of experience. And for years, I would have agreed with this. But Fred goes on to describe 9 other items that don’t get the same Google-juice that this one quote does. There are probably 10 more that companies could come up with.
But a maniacal focus on speed means that in some companies, all else is tossed in order for that goal of achieving some insane, straight-line, one-dimensional goal. Some companies are likely investigating faster than light technologies to make the delivery of online applications even faster.
Can you base your entire business on having the fastest online application? What do you have to do to be fast?
Strip it down. Lose the weight, the bloat, the features. And what’s left is a powerful beast designed to do one party trick, likely at the expense of some other aspect of the business that supports the application.
If a company focuses on a few metrics, a few key indicators, they might evolve up to NASCAR, where it is not just speed, but cornering, that matters. Only left-hand corners, mind you, but corners nonetheless. Here speed is important, but is balanced against availability and consistency to ensure that a complete view of the value of the site is understood.
But is that enough? Do your customers always want to go left in your application? What happens if you are asked to allow some customers to go right? Do all of the other performance factors that you have worked on suddenly collapse?
As you can tell, growing up means that my taste in fast cars and racing forms has evolved, become more complex. Straight-line speed, followed by multi-dimensional perspectives have led me to realize that speed is only one feature.
So, if top-fuel and stock-car racing aren’t my gig, what is?
For a number of years in the 1980s and again since 2008, I have had a love of Formula One. The complexity of what these machines are trying to achieve boggles the mind.
Formula One is speed, of that there is no doubt. But there is cornering (left and right), weight distribution, brake temperature, fuel mix, traffic, uphill (and downhill, sometimes with corners!), street courses and track courses. And there are 24 answers to the same question in every race.
And then, there is a driver. In Formula One, a driver with an “inferior” car can win the day, if that inferiority is what is particularly suited to that course, in the hands of a skilled manager.
There is no doubt that like Formula One, speed is key to coming out on top. But if the organization is focused solely on speed, then your view of performance will never evolve. The key to ensuring a complete Web performance experience is a maniacal focus on a matrix of items: speed, complexity, third-parties, availability, server uptime, network reliability, design, product, supply-chain, inventory management integration, authentication, security, and on and on.
The Web application is a just that: a web. Multi-dependent factor and performance indicators that must be weighed, balanced, and prioritized to succeed. No web application, no online application, fixed or mobile, will survive without speed.
However, if speed is all you have, is that enough to keep someone coming back?
Is your organization saying that speed is all there is to performance?

The Customer Investment

Who uses the products or services your company sells?
The usual answer, once you get through the marketing spin and positioning, is customers.
Companies spend a large amount of time, resources, and treasure converting prospects into customers, but where is the investment in keeping customers from becoming anti-customers?
The mobile phone business is an ideal example for this ebb and flow, a prime case study for customer investment.
I’m a T-Mobile USA customer; have been since 2004. This year, T-Mobile USA has decided that 2012 Is The Year T-Mobile Fixes Churn. Does this mean just the customers at the end of their contract or the one leaving because of the lack of the iDevice they want?
Or will T-Mobile USA extend this churn-loss plan (Go New Year’s Resolution!proactively to all customers.
Will T-Mobile bother to personally contact (hey, with a phone call?) every one of its current customers?
Will T-Mobile ask customers who are leaving why? Not in a stupid, aggressive way, but in a way that admits that they didn’t do enough for that person, but they really want to understand what went wrong.
Will T-Mobile USA take the time to invest in their customers?
Investing in customers means proactively working with them to ensure that the service they are getting:

  1. Meets the customer’s current needs
  2. Is flexible enough to adjust to the evolution of the customer’s business.

 
Joseph Michelli discusses the concepts of service velocity and service recovery in The Zappos Experience. These are items that companies need to consider. Customers want you to adapt and evolve to meet their post-sales needs (service velocity) and then be truthful, upfront, and solution-focused when there is a problem (service recovery). Customers want you to invest in them, in sickness and in health.
It’s so much easier to keep a customer than it is to get a new one to replace them. So why are so many companies lacking focus and discipline when investing in their customers.

Career Reform – The Changing Face of Expertise

Empty Road - William WarbyIn August 2011, I took the title “consultant” off my business card after having it for eight years. It was sad to see the old friend leave, but it was for the best – for both consultant and for me.
Two years ago (22 months for those of you who are more precise), I composed two pieces on what it meant to me as I evolved out of the role of “analyst” and into the role of “consultant” (here) and how this meant developing the skills of a “selling consultant” (here). It was a heady time. I was learning a lot of new skills, meeting the challenges of a post-technical role, managing to a new level of “success”.
Many things have changed since then. But the key lesson that I learned is that the career path that was in front of me was not headed in the direction I wanted to go. The true sign of this, that I ignored at the time, but which is so obvious to me now, is when I started counting down the days to my annual vacation.
Having just finished Onward and Delivering Happiness, I read that these moments come to all people. It’s how they choose to face them that determines their happiness after.
Due to a serious of weird misfortunes, fortune shined upon me. A new opportunity was presented to me, and I was able to use it to shape a new path forward, one I think that many maturing consults imagined that their roll would look like when they started their journey.
My new role is to act as a consultant to the entire organization. And what does that mean? My goal (and I get to invent the role as I go along) is to develop and share the knowledge of the strategic use of the product line, approached from a technical and sales perspective, to help current and new members of the company not only learn the How of the product line, but the Why that motivates prospects to become company customers. I also get to see how the product plan morphs, shifts to meet new information and new ideas.
Am I happy? Yes. When I began my change from analyst to consultant, I had hoped this is where I would end up.
If I stayed the course, would I have ended up here?
Despite being a counter-factual question, I think that the answer is no. I was being squeezed, shaped, and directed by the role of consultant. I had lost control of my own career and was being driven to the next destination in a blacked-out van.
Now I have gotten out of the van, checked my bearings, and started walking in the direction I want to go in.
What’s next? Well, I’m sure that in two years, I’ll have something to share.