One Year: Entrepreneurial Operating System (EOS)
All it took was some time, a few simple concepts, and practical tools. That’s the easy way to summarize EOS. Haven’t heard of this strange acronym? It stands for Entrepreneurial Operating System (EOS) which one of the many organized systems out there you can choose to run a business. For our software company, the EOS way of doing things certainly has had an impact and I’m here to tell you about EOS from a unique perspective: an engineering team’s perspective.
EOS and Engineers. Could it work?
I can say with full confidence that engineers are unique. If you know any engineers or if are an engineer yourself, you can understand what I mean. We tend to be skeptical about higher business processes because, well, we’ve got so many of our own we all already follow. So, when Pipeline CRM decided to adopt EOS we all raised a brow. Sure, it could work for the other departments but our department? We’re engineers! We do things differently. We think differently.
Our first impressions about EOS changed soon enough.
Engineering Challenges EOS
Here’s a bit of company background. Pipeline CRM rolled out this new way of doing things to set a new course for all teams to run the subfunctions of our software-as-a-service business including Product, Marketing, Sales and Customer Teams. Our team was included, of course, and if you are familiar with how engineering teams run, you can imagine the challenges.
Like I mentioned earlier, engineering teams often find it difficult to adopt higher business level processes as their processes are already well defined and very different, such as Agile (Scrum, Kanban, Lean, etc).
When EOS was presented to us, we wondered how we could adopt a business process given we had one that worked great for us already? And why should a process for running an entire business apply to software engineering?
With a team of skeptical engineering minds behind me, we looked forward with concern hardly expecting EOS would in fact make our engineering team stronger.
EOS Background
I’ll get more into what I mean by stronger. Who doesn’t want to run a strong engineering team, after all. First, let me briefly describe the EOS so you’ve got a good background of it.
EOS is a set of six concepts and a set of tools that come together to form a unified process for operating your business top to bottom. The six components are:
- Vision. Getting everyone in the organization 100 percent on the same page with where you’re going and how you’re going to get there.
- People. Simply put, we can’t do it without great people. This means surrounding yourself with great people, top to bottom, because you can’t achieve a great vision without a great team.
- Data. This means cutting through all the feelings, personalities, opinions and egos and boiling your organization down to a handful of objective numbers that give you an absolute pulse on where things are.
- Issues. Strengthening this component means becoming great at solving problems throughout the organization – setting them up, knocking them down and making them go away forever.
- Process. This is the secret ingredient in your organization. This means “systemizing” your business by identifying and documenting the core processes that define the way to run your business. You’ll need to get everyone on the same page with what the essential procedural steps are, and then get everyone to follow them to create consistency and scalability in your organization.
- Traction®. This means bringing discipline and accountability into the organization – becoming great at execution – taking the vision down to the ground and making it real.
The Pipeline CRM Engineering Team adopted each of those components. It was a journey. Let’s delve deeper.
Vision/Traction
These two components are very tightly coupled, so much so that EOS provides a tool called the Vision / Traction Organizer (V/TO) which is the primary tool we use to run the business and our teams. The core of the EOS process focuses on setting quarterly, yearly, three-year (or greater) goals.
First, the company sets its goals for each timeframe, then each department does the same, then each individual. The goals need to match the vision and align with the upper tier goals. This is where it gets hard for engineering teams to figure out how they fit into the company goals. Business goals, including at the company and individual level, in EOS speak are called Rocks.
We asked ourselves:
- How can we set these quarterly Rocks (business goals) when we work off of bi-weekly sprints that are planned on the fly?
- Or we work using Kanban, working from the top of the queue and never know what that may be?
- How does the React javascript dashboard component I am writing (because that’s what the product owners need me to work on) have anything to do with the company Rock of growing revenue?
- If I set my own Rocks, how do I find time to work on them if I am committed to the work of a project or sprint?
In our V/TO, we have a revenue growth goal in three years of a certain percentage. We also have company goals or Rocks (long and short term) around ARPA (Annual Revenue Per Account). But the V/TO asks not just what the number is, but what it looks like to the company.
Here is one of the items on a Scorecard:
“Modernized cloud infrastructure including a serverless architecture via AWS Lambda, Containers and Kubernetes, autoscaling, and re-architecting our app into more microservices”
Engineering Vision with EOS
We have taken this vision and used it as the basis of many of our department and individual Rocks.
Later, I’m going to talk about a tool called the Scorecard, which is another EOS process tool that is meant to measure the health of a company or team.
We also formulate engineering Rocks on items we find that need improvement based on the Scorecard.
First, let’s take a look at some highlights from our engineering Rocks that we’ve accomplished over the last year and categorize them:
Platform modernization and improvement:
- Deliver a significant platform upgrade (Infrastructure or Architecture).
- We upgraded all of our EC2 instance types
- We upgraded our Ubuntu version across the platform from 14.04 to 18.04
- And we completed a POC to move from EC2/Galera/MariaDB to Aurora
- Resolve a majority of Tier 2 service work in less than three days.
- Record five technical debugging videos for Customer Care
- Complete an application penetration test
- Disaster Recovery drill
- Take courses on Native Android Dev, Native iOS Dev, and ML
- Take Machine Learning courses
- Remove jQueryUI from PLD
- Rewrite Backend to React
EOS and Engineering Team Goals
How do these Rocks relate to revenue and ARPA goals? Well, higher revenue means more customers, stronger retention, and a larger worldwide footprint.
Engineering affects all of these through the performance of our applications and infrastructure, as well as the capabilities we offer technically. We must handle more load quickly. We need to beat our competitors with innovation and feature offerings and do things like invest in Machine Learning and AI knowledge.
Also, we need 99.99 percent uptime so we look to better reliability and scalability database solutions like AWS Aurora. We need to be fast to address our customers needs quickly, which is why refactoring Rocks are important. Along with it, we have always had strong attention to security, and larger accounts have more strict security policies and requirements, so engineering Rocks that continually improve our security bar are good.
We retain and keep our current customers by delivering great technical service, resolving support bugs quickly, and helping our Customer Care team however we can.
The Biggest Engineering EOS Managerial Challenge
The biggest challenge that engineers and engineering managers tend to ask or push back on when it comes to business management platforms merging down into engineering practices is this: How does time get allocated for it?
Legit question! After three quarters of EOS, we have found our own groove for this. It’s really not just a single way. It’s a combination of using slack time or downtime within projects, choosing manageable Rocks that don’t jeopardize project work, use time that we give all engineers, as well as incorporating it into our Kanban pull process of regular project work.
The Pipeline CRM Engineering Team is also a very senior, highly responsible and dedicated group. They are committed to improving engineering, improving their personal skills, and advancing our tech platform.
Ultimately, the team is motivated and not burdened by the EOS process.
EOS Meetings and Engineers
Who likes work meetings? I don’t think anyone does! They’ve got a bad reputation for being pointless and unproductive. Well, imagine if they are productive. Unheard of? Well, not with EOS. Let me explain.
Another big part of Traction is a regular cadence of meetings. Across the company, there are weekly meetings called Level 10’s (L10) to cover Scorecard, Rocks, and most to discuss important issues. The name comes from the fact that in the end, everyone rates the meeting on a scale of 1 to 10 for productivity and effectiveness. Let’s move on to those.
Issues: The Issues List and IDS
I – Identify. Truly identify the root cause of the issue.
D – Discuss. Openly discuss the issue with everyone sharing their opinion.
S – Solve. Come to an agreement on the next steps to make the issue go away forever.
Let’s Solve This
The “Solve” is critical to a good meeting. It’s a waste to have a long conversation with no outcome. We walk into these meetings with a list of issues that everybody contributes to during the week prior. We start from the top and give every issue the time it needs. The whole team is aware of the ranking of the issues before they walk into the meeting giving them time to reflect and prepare prior to the meeting.
If one issue takes all meeting, then the rest wait. But there are some rules in the conversation around going down rabbit holes, and not staying on topic, etc. This process was super easy to take on at the business level. But again, at first, it felt hard to map to an engineering team. Especially since engineers tend to hate meetings, especially recurring ones that fall outside the scope of their daily work.
After a few of these we started to get in a groove and the team feels the benefit of the meetings and the positive outcomes on our team and processes. For one, we are a remote-only engineering team. Everybody is in different locations, so the weekly cadence was good for our team communication in general. But what do we cover?
Here are some examples:
- App load time increase: support chat software – what can we do?
- How do we test the Ubuntu upgrade?
- Why do sidekiq jobs get stuck for many hours?
- MySQL monitoring tool trial – Should we buy?
- Google oAuth API security requirements – how does this affect us?
Data: Scorecard and Measurables
All companies should run their business based on data and make decisions based on what the data is telling them. Most companies use some form of Key Performance Indicators or OKRs to do this.
With EOS, the KPI comes in the form of a company Scorecard, as well as a Scorecard for each department. The Scorecard is a one page set of data values that provide a quick look at the health of your company or team. Once again, this felt easy at the business level, especially in SaaS where we have a hundred acronyms in which to measure our business: MRR, CAC, ARPA, LTV, ARR, etc. Our team wondered, how do we measure the health of engineering?
The Engineering Scorecard
Let’s take a closer look to answer this. Why does a software engineering team even exist? It provides the business with some kind of service. That service is software development and in our case includes the operations of running and serving that software to all of our customers. We look at quality, throughput, production reliability, and performance.
Estimates: We have a duty to provide the business with an estimate of the cost of work to be done, as well as to help understand delivery projections.
Rollbacks in production: We are a continuous delivery and deployment shop. We release to production multiple times per day. We have a simple Slack enabled deployment command capability, it’s pretty awesome. So deploys are safe. However, a rollback means something went wrong and it could have affected customers. We don’t want that. Our goal is zero rollbacks, with 1 being warning level.
Net Outstanding Bugs: These are bugs in production as reported by our thousands of users. Our Customer Care team fields these through support tickets, and we have processes to address them. Obviously, we want this number to track downward.
App Speed (Page Load Speed): This metric is controversial. Our measurement is an average of all pages in our application from every user in the world, including places with extremely slow internet. We use the average for the whole world and come up with a single number (4.3s) for non-cached page load time, and it includes all javascript and assets that do not affect page render. But as long as we use the exact same measurement week over week, we get to see a trend and fluctuations in performance. We’ve caught several problems by watching this number.
System Uptime: Our goal is 99.99 percent. Each week we report what it was for the week, mostly 100 percent. That’s pretty great.
Pull request time to review: Our goal in engineering is that a Pull Request (online code review in Github) is completed as quickly as possible. Waiting on pull requests is a delay in our workflow, so let’s measure how fast reviews are getting done.
People: The right people in the right seats
Our CEO, JP Werlin, is famous around here for saying, “There are only so many seats on this bus, make sure yours are filled with your best person for the job.”
This is very much the same as the EOS concept of “the right people in the right seats.” The EOS way simplifies performance reviews and uses a simple one-page chart to determine if you have the right people in the right seats.
We are Better with EOS. Could You Be?
EOS has helped Pipeline CRM focus on our mission, our core values, and has given us a way to sharpen our business in a predictable, repeatable way.
Upon first glance, it didn’t seem like it was possible to merge a business process into an engineering process. But we did it, and we are better for it.
We are improving our technology platform, our development process and skills, and our personal development through our adoption of EOS. We feel more connected to the operation of the business itself. It has been good for all.
