This is a live cheatsheet to give you an idea of what I offer and what I value as an engineering manager. I'll refine this post over time as I continue to grow.

---

What you can expect from me

My job as a manager is to enable you to do your best work so that we can grow the company and your career together. That means for me to understand your aspirations and to maximize your personal goals with that of the team's. I will support you by doing the chores that you don't want to do and push you hard on the work that you want to do.

There are many ways to getting things done. I don't expect you to adopt to my way but I do expect mutual understanding of how we each prefer to work. In particular, I push the team to be conscious of how they spend their time by focusing on what matters to the product and our customers. On any given week, I typically spend my time as follows:

• 40% on supporting the team. This could be anything from collaborating on technical designs, code reviews, completing unwanted tickets, or just getting to know you.
• 25% on development and operations. I pick up any gaps to keep our systems running smoothly.
• 25% on engineering management. I regularly consider these 3 aspects: organization, project, and technical. For organization management, that means working with the product team to ensure that our engineering strategy has alignment within the organization. For project management, ensuring that we are achieving our sprint goals. For technical management, keeping an eye on our technical debt and prioritizing longer term behind-the-scene work.
• 10% on personal growth. I am focusing on growing the skills that I need to be successful as a team lead and improving as a human being.

To enable product and customer mindfulness, it is my job to articulate the bigger picture to you so that you have the context to fully understand the problems that you're addressing. For example, I often like to collaborate on a one-pager problem statement to start a project before doing anything else.

I am tremendously focused and result-driven. I have an affinity for pragmatic, minimal effort to get just enough of a job done. I sometimes struggle when working with others who do not share these traits. Some things that are helpful for you to know about me:

• I have low ego. I don't know what I'm doing more often than I realize. Sometimes I may inadvertently inconvenience you. When that happens, please accept my apology and help me improve.
• I am driven by impact. Whether it is impact to the team, to customers, to the society, or to my family, I am proudest when seeing my effort make an impact in other people's lives, no matter how small.
• I value open and transparent communication. I am a straightforward communicator. You can count on me to tell you what's on my mind. I will not take it personally if you disagree with me, but make sure that it is done openly so that we can discuss it.
• I value a well-rested mind. Nothing can come between me and my sleep (except my kids 😅). When I don't sleep well, I don't perform well and get grumpy. I practice meditation, bodyweight training, and eating a pescatarian diet to keep my mind and body in shape.
• I am a slow thinker. I'll often ask to sleep on an idea before getting back to you with a response.
• I value understanding a problem deeply over clever solutions. I don't want to get ahead of ourselves and end up solving for the wrong thing.
• I value feedback. Let me know what I've done well and what I can improve. I try my best to give others feedback immediately in context as I find that it is the most relevant.

What I expect from you

I trust you with the work; we hire smart people to solve complex problems. I will give you context and feedback, but you need to become the expert on the problem and own your solution. I expect you to own the work, to build relationships with our team, and to come to work with an empathatic attitude.

• Support each other. Support can come in many forms, such as offering help on a difficult task, providing constructive feedback, or simply lending a listening ear. By supporting each other, we can develop a sense of trust and camaraderie.
• Build relationships. We seek people who can develop deep relationships with others. You can be introvert, extrovert, or anything else, but the ability to build and maintain human relationships is key.
• Humility. There are lots of smart, talented people in the world, and there’s usually someone smarter than you. If you can’t accept that, you’re going to have a tough time listening to others, developing healthy relationships, or understanding others.
• Make an impact. You're hired not just to code but to make a sizable business impact. Coding is a means to an end.
• Balance. I like my job but I don't tie my identity to it. If you're feeling sick or unwell, take some time off. If it's a nice day outside and you'd rather not work, take some time off.
• Make decisions. If you are stuck, figure out what you need to unblock yourself. When you make a decision, document your rationality and communicate to the rest of the team.
• Make the work visible. Make sure that there is a shared understanding of the problem that we are solving, any key deliverables have due dates established, and you are communicating the progress of the work. Capture feedback from stakeholders on your ticket and close the loop.
• Ask questions. If you don’t know something, ask. If something is unclear, ask. Asking questions is the best way for us to come to a shared understanding as a team.

Being on-call 24/7 for 18 months straight was one of the most anxiety-inducing responsibilities that I've ever had. I had to make sure that I always had my phone within reach, my laptop accessible, and my mind ready to fix any critical system failure. I sometimes jolted up in the middle of the night, believing there was a page on my phone when there wasn't. It wasn't until months later that I realized I had a problem, when my partner accused me of being irritable lately and it dawned on me that she was right.

If there's one piece of advice that I would give to fellow software developers hesitant about performing on-calls, it is this; don't do it! Don't take a job that requires it if you're feeling anxious about on-calls. We're blessed to be in a hot IT labour market now, so I'm sure that you have plenty of options available. However, if you're proceeding with being on-call (there are some good reasons to) and want to minimize the drain, continue reading.

What is it?

On-Call duty

I define on-call as being on pager duty for a period of time, expected to respond to urgent system issues. Typically, developers would be on rotation for 7 consecutive days of on-call duty out of every 4-6 weeks. On-call is a common practice with software teams that practice DevOps such that "you build it, you run it" is the expectation. In such teams, on-call duty is part of the development process, as opposed to a traditional separation of development team versus operations team.

Anxiety

The weird thing about my anxiety is that I find myself less anxious when there are more alerts happening regularly. One of the worst times was when I was on a family vacation for the first time in a long while, and there hadn't been any alert for weeks prior.

Anxiety is distress or uneasiness of mind caused by a fear of danger or misfortune. Wikipedia

According to the American Psychological Association, anxiety differs from stress in that anxiety is defined by a persistent worry that won't go away, whereas stress is typically caused by an external trigger. I am anxious about the prospect of our system going down. But once it's down, I get stressed about debugging the failed system while our customers are pounding on our support team. In fact, I gave a talk about the mental stress of debugging production applications.

Why on-call?

Motiva A.I. orchestrates digital marketing campaigns for our enterprise clients. Imagine a scenario where if you sign up for a newsletter at Verizon for Business (one of our customers), you'd expect to receive an email from them. Well, not if Motiva is down. Establishing an on-call practice ensures that we can immediately respond to any service interruptions.

"I can't wait to spend my weekend waiting for a system outage!", said nobody.

Keep in mind that on-call is a last line of defense to keep things running. If high-availability is a business requirement, then that consideration should be an integral part of the product development process right from the start.

For the individual software developer, on-call can be a helpful practice to hone your debugging and software engineering skills. Bugs happen when an unexpected event pushes your known system into an unknown state. Debugging in such an environment can be a helpful process to push your understanding of your systems and your tools. Moreover, I personally adhere to a pain-driven development mentality. Developers that feel the pain caused by the system they built will build better systems.

Dealing with on-call anxiety

As a developer

Seek professional help

If you're feeling anxious about your on-call duty, my first advice is to seek a workplace therapist. Second to that, I recommend working through this Anxiety & Worry Workbook by Clark and Beck.

Accept failures

What ultimately helped me mentally is accepting the fact that the worst that can happen if I miss an alarm or two isn't so bad afterall. We might lose a customer or two, which is pretty bad for an enterprise startup with only a handful of customers. But what's worse is getting burnt out and not being able to function anymore. Growing a startup is a marathon, and I shouldn't lose the race over any single bump. Having said that, I understand that the ability to step back is easier said than done. That's why I recommend seeking a therapist or working through that anxiety workbook to ground yourself first. It took me months of actively working on easing my mind before finding my peace.

Personal tips

While I worked on my mental health, here are some small adjustments that I made to improve my well-being.

• changed my pager tone to something more relaxing
• took it easy when I got paged, resolved it when I could and did not fret about jumping onto my laptop immediately
• blocked out certain hours in our on-call scheduling system during expected slower weeknights so that I could rest easy. Getting woken up in the middle of the night was my biggest anxiety. This gave me peace of mind.
• this wasn't an option for me, but I wish I could have opted out of on-call duties for a couple months when I was going crazy

Operational tips

In terms of operation, it would suck to keep getting paged for the same issue. Make sure that any incident is followed by a retrospective with actions to mitigate the problem in the future. We write up a one-page incident report for each incident, highlighting who it affected, why it happened, and suggestions for future prevention with actual tickets scheduled to be worked on. This give visibility to the product team around delivery expectation.

A second operational tip I find useful is that during an incident, don't try to do too much. Focus on getting the system back up, or at least the mission critical parts, even if they will be in a limping state. Leave the fixing for the team during work hours. For example, a couple times I would just pause our affected workers before spending any more time debugging. It's easier to explain to our customers that the system stopped working than to explain that it did the wrong thing. This is a debugging tip but it helped with my anxiety because I lowered the bar for what needs to be done substantially, i.e. from fixing to pausing the system gracefully.

As a manager

Be wary of your bus factor for mission critical components

As a serial technical co-founder, I often find myself taking on-call responsibility for months at a time, simply because there's no one else available. What is different this time is that we had been around for 4 years already when this long on-call stretch started unexpectedly. It happened because our only other senior backend developer left us in mid-2020.

People come and go. It was my fault for letting myself be stuck in this situation. We have other developers on the team, but they've been working on more interesting and valuable parts of our product. I've let myself be left behind as the only one responsible for our legacy, but still mission critical, components.

the more mission critical a component is, the more eyes should be on it

We categorize our application workflows as i) mission critical, ii) essential, or iii) everything else. Only a couple mission critical workflows on our system are allowed to raise on-call alarms outside of work hours. This reduces support fatigue and focuses our attention on what matters.

Distributed workforce

Motiva has been a remote-first company from the start. One thing we did well early on is to hire across time zones for our engineering team. My previous senior engineer was on the other side of the world from me (not deliberately to such an extreme; suitable partners are just hard to find). That worked wonders for our on-call and support needs. However, that obviously comes with its challenges as we had to rely heavily on asynchronous collaboration methods for actual development work.

Cover each other

I left myself behind in my case. But having lived through this trauma, I wouldn't put this on anyone else on my team. In addition to sharing knowledge across critical components, nobody should be on-call alone. I make sure that we have a clear escalation procedure in place so that whoever is on-call feels supported.

In reality for a startup, there's only so many people that can play musical chair in our team of ten. That's why we built semi-automated diagnostic tools for non-developers on our team to help out. The more issues that our customer support team can resolve, the fewer tickets will need to be escalated to our devs.

On the other end of the escalation ladder, I make sure that our team knows that I'm always available to whoever is on-call. That's the unfortunate responsibility of being the tech lead at a small startup. Luckily, my team only had to call me once outside of my on-call schedule this year.

Set clear expectations

Just like any team process, we make sure that a new engineering team member taking on-call duty knows that they are not expected to fix the world, knows that it's ok to take it easy, and knows that they have our support. If anything, err on the side of escalating early for their first few incidents so that we can pair on the incident with them.

Early on in this post, I mentioned that I wouldn't recommend taking on a job requiring on-call. If you don't think that on-call is for you, then don't do it. However, this is an unavoidable responsiblity if you want to work in a small startup. Something mission critical will eventually fail, and somebody needs to fix it. For such times, I'd rather that we have a clear on-call schedule with clear expectations than scramble to fight the fire.

Build robust and maintainable systems, but we don't have time!

The lasting remedy for my on-call anxiety was building confidence in our systems so that I know we'll be fine if I'm unavailable. We dug ourselves out of that technical debt. For two quarters in 2021, we directed all engineering resources to rebuilding the most vulnerable parts of our system as high-availability workflows.

Developers that feel the pain caused by the system they built will build better systems.

Hold on a second! Did I mention that I got stuck with this endless on-call stint because one of our two backend developers left? Between scaling our system to handle the 400% increase in system load this year, hiring for a replacement, building new features, reviewing code, mentoring, ... how did we find time to address this rotting technical debt?

Engineering management in a startup setting is still something that I'm learning even after 10 years. In this case, I couldn't take it anymore and had to convince the rest of the company that we had to stop everything to focus on addressing some technical debt known for years. The fact that it got to the point where my health was at stake until I prioritized is a failure on my part. Having said that, I learned a lot over these past few years in terms of maintaining an enterprise system and engineering management. I've been heads-down building Motiva for the past 6 years. If you find this post useful, take a look at my blog. I plan to write regularly over 2022 to capture some of my learnings.

The InGrower Mobile App has been put to the test in the hands of a few smallholder chicken farmers in Mozambique since January 2016. We have learnt what's needed and is proceeding to the next stage of development. That's why we're looking to hire a ClojureScript developer to build the next version of our mobile app.

One of my longer term goals is to grow a distributed technical team in East Africa that can tackle various social impact challenges in the region. This first hire will play an important role in setting the direction for the team.

Link to job description

InGrower Mobile is a side business that I've been working on since September 2015. We are an early stage social enterprise in the I.C.T. for Development (smallholder farming) space. InGrower Mobile came about from my co-founder's experience operating an agri-business incubator in Mozambique since 2010. We admit unemployed individuals, then provide training and support to help them start their own agriculture business to make money. For InGrower Mobile, our product is a mobile application that takes what we learned from the past 7 years running our agri-business incubator and applying them online to make our training and support available for a wider audience across East Africa.

We run out of drinking water one afternoon at our house on the 300-hectare farm. The region's power grid has been down for over a day. Our water tanks have run dry. It is 38 degrees Celsius, and the African summer sun is high in the sky. This happened on the first week of my month-long stay with our smallholder farmers in Moamba, Mozambique. It also sets the tone for what to expect. This is Africa. Nothing is easy.

Rewinding back five weeks, our first cohort of smallholder farmers were finishing trial runs of our product, InGrower App. We got lots of feedback. More inputs here. Move the box there. The usual pointing out the obvious. Users can tell you what's wrong, but they cannot tell you what to build. As we're an early stage startup that is still finding a product-market fit, we desperately need to figure out the latter. It is then that I see the only way to do this is for me to be in the same room as our users. And that is how I found myself living with our users on a chicken farm in Moamba.

One of the first questions that I asked my current co-founder when he pitched the InGrower idea to me was – do farmers in Mozambique use smartphones? I am now able to answer my own question. While living there, power and water often go out (though not usually at the same time), but the 3G mobile internet is always on. So what do you do when there is no power or water, and it's too hot to do anything? You browse Facebook on your $30 Android smartphone, like everybody else.

Of our first 14 trial users, two are heavy users of our product. These power users have been regular Facebook users for more than three years. They are used to navigating a mobile user interface, and understand the benefits of having data online. These are the users that we want as our initial champions for the app.

The farm is 10 km away from Moamba village. We often roar into the village sitting on the back of a tractor.

This is Moamba market:

This is my bed during my stay at the farm:

I specifically asked the manager before arriving to help me set up a mosquito net. Malaria is a real danger here, especially at the height of summer in January, when I was present. But what do you know? There were few mosquitoes in the house. Instead, I had giant bugs of all shapes and forms visiting me every night. Thank god I had the net!

I ate with the staff and farmers at the house. We all chipped in with US$20 per person each month. That was the budget for a month of groceries for them. Average monthly salary is US$100 per month in the country. A piece of bread cost about $0.15 and a whole chicken was $4 at the market. This was our typical lunch at the house:

There was a limit of two small pieces of meat per person for every meal when there was anything to eat at all. There were a couple of days when I had only a piece of bread and some crackers all day. I don’t know how the other guys who actually had to do physical labour managed to work on an empty stomach.

Sometimes, some of the guys went fishing or picked coconuts, and then our housekeeper would make something special.

At other times, I snuck out to the village and had a shameless feast. All of this grilled sausage and food is $6 and enough to serve four people:

A typical day goes like this for the chicken farmers of Moamba. They come in from the village at 7am. Tend to their chicken house. Breakfast at 10am. Back to their chicken house. Lunch at 2pm. By then, the sun and summer heat is in full blast. They chill in the staff house until 5pm, then head home back to the village.

I came to Moamba to do product development. In addition to all the learning and understanding that I received, the experience itself has been very special. Hanging out in the afternoon with the merry gang, and with the cool breeze blowing against your face. Riding into the village on a tractor. Brushing my teeth every evening out in the open, under the milky way. Nothing is easy here. But there's a certain romance in expecting the unexpected in daily life in Mozambique.

P.S. the title of this post is a play on Craig Mod’s article.