Lessons learned as a software developer turned project manager

The last couple of years I was the team lead for a massive automotive data backend. The backend featured streaming and batch data on a petabyte scale and had multiple teams using the backend for ingest and usage. The team was made of multiple skilled and competent engineers and developers spanning across multiple companies.

Becoming a project manager after being a software developer for many years changed me quite a bit. Below are my humble thoughts and breakdown of the experience.  

A technical solution is a journey and a goal

Technical problems worth addressing quickly become more than just the mere breakdown of their technical units of work. Very often I find myself in situations where I need to think less about the intricacies of an implementation and more about how to effectively distribute, communicate and evaluate the implementation effort at a varying detail levels.

Put simply, the most difficult challenge in a technical project is the communication between parties: it's the maximum speed at which your project can efficiently progress.

It's always easy to start thinking in terms of solutions straight away and it's a natural desire to do so as an engineer. We want to solve stuff or make stuff work. Tell the machine to do this and voilà, it's done. There's nothing inherently new in many initiatives or projects as far as the technical side is concerned. How many different API's, data models or ways of storing data are really out there?

What is new is the context in which this technical solution is applied and of course the constraints that come with that context. The context can certainly change. There can be new users, new requirements, new improvement ideas constantly popping up.

Another important metric is how fast new assumptions, new requirements or new problems around the project can be taken into consideration i.e. effectively communicated, discussed and addressed.  

I recommend identifying and documenting the context and constraints first, it'll save some time and surprises along the way.

Communicating technically is an art

This experience also helped me with improving my communication skills. Just like there are linters, tests and code for a programmer, there are tools you can use for effective project communication. The main point being that you need to adapt to the purpose: who you talk to and what needs to be talked about should be front and center.

The communication at a project manager level is a bit less codified and more subjective though: over communication and repetition help a lot with that but get boring quickly. It helps here, to automate the process with scheduled emails and email templates, just like you would automate sending out tweets or newsletter emails.

Don't under estimate the automation part! I am very confident that with sufficient input data and information about the context, most of the project management communication can and will be automated or at least codified enough to reduce the amount of work necessary. Auto generating PowerPoint's will be the subject of another article, the point being that it's very repetitive but necessary. Don't mechanically automate the content though though, people still want to hear from humans, not robots. Automate away what's keeping you from frequently communicating.

Sending out an email explaining what your team worked on and achieved the past weeks and what your plans for the coming weeks are goes already a long way.

Write everything down

Much of my time is spent writing, discussing and documenting findings. Writing requirements, design documents, project proposals and communicating those to both your team and the external interfaces to the projects is the key here.

This became a habit that I purposely block myself some time every week in my calendar to gather my thoughts and ideas. I use 1 gigantic Word document for random notes and journaling and a common wiki/internal blog for communicating news about the product, the development and other topics which might be relevant to the team.

I still keep some time for developing and participating. Most of my peers don't because they can't or because they have no interest in it. I see it as a great opportunity to build rapport and trust with developers as long as you don't get in their way and slow them down. It's about solving their problems too.

Agile is not efficient and doesn't scale

Agile rituals and other regular meetings are more like housekeeping tasks - or refactoring tasks - and don't count as project management per se. They are necessary from time to time but forcing them and spending more than half your time on them is a sign that something is going wrong somewhere.

The scope of the project and what needs to be done should be clear to everyone. If the methods used to support the mission get in the way or provide no immediate tangible benefits to the project, they should be removed.

I'm a big fan of of Marie Kondo and apply the learnings to the projects I manage, if something doesn't spark joy in the team it's removed.

This is akin to removing unused code and bloat. It's a liability if not dealt with at the beginning. Many will argue the long-term benefits of mandated agile but the benefits are really on the management side of the table. The accountability is shifted towards the developers and reset at every sprint since they become responsible for estimating, evaluating and are nailed down on those estimates.

This is bad. Developers are good at developing. Project managers should estimate. If a PM can't estimate maybe the technical task is better suited to a manager that actually understands the tech. This is not like in other industries where a manager just needs to be able to manage "grunts" for deterministic tasks. The planning and adjusting in the face of changing contexts requires a deep enough understanding of the technicalities and long term implications of technical choices to be efficient. You can't lead developers if you have no clue about developing. Sorry.

I believe a pragmatic middle ground is a mix between keeping the team size below 15 developers, delimiting the scope, and using Kanban with a strong documentation wiki. Ask yourself in true honesty, what are you building that requires 100/1000/10000 developers? I stipulate that with 20 to 25 great developers you can implement 80% of projects out there (mostly CRUD apps ^^)

As a matter of fact, for a lot of the development, we wrote the user documentation first and the developer documentation after the development.

A project is usually a business within the business

There are many aspects of managing a technical project and some are well, not technical. This quickly gets boring and it's tempting to just do it, right!? Who cares about the details? What matters at the end of the day is that it works right? Well not so fast.

While I tend to agree, forking out an MVP is different than building a business. As a project manager in a company you are essentially managing a business within the business.

The important points are that your project has to be sustainable and solve a business need. People who work on the project, budget, security and other things will pop up eventually. Prepare for the question "Why do we need this?" and "Our project does it better!" from other internal or external forces. Sometimes there's nothing you can do, but it's better to go prepared into battle. Get your arguments, KPI's and other tools ready and up to date for this eventuality.

It's easy to over-do it of course, to fall into the trap of making sure that everything works out perfectly and is planned to the second. Don't forget to breathe and mediate the different wastes of time that will necessarily pop up (politics...). Just like with gravity, anything with mass will attract a lot of stuff, some of which isn't necessarily good.

Your role is ultimately to cut through useless and irrelevant tasks and time wasters and focus on what will actually bring a benefit to your customers whomever they are. They should be happy and satisfied with the solution since they are your only quantitative and qualitative leverage you'd have against budget cuts or other head winds. They are your best allies, treat them like you'd want to be treated, clear, consistent and respectful communication of positive as well as negative aspects is essential. People are bored of corporate and PR speak so they're reacting positively to honesty and clear communication. It'll of course depend in each case, but I like to assume that people are well intentioned and I take it from there.

Don't forget though:

"Plan and prepare for every possibility, and you will never act."

Get people to use and try out the technical solution quickly and engage in dialogues wherever you can. The added utility of a technical project is to validate assumptions on top of solving a problem.

Incrementally iterate over feedback as well as on regular, short, time intervals (weekly, bi-weekly) wherever you can. Just be consistent.

You deal with humans: never forget that

Don't ever, ever, call your colleagues or developers resources or FTE's (full time equivalent), they're humans, not beans. Without them you wouldn't get anything reliably developed or deployed. I steer away from any recruiters that contact me with that word usage. People tend to forget that tech is built on consistent application of passion and effort from real humans that believe in a common vision. This is why outsourcing is very difficult...  

Keeping this in mind, don't assume things. There is as much being said in silence than in active communication, if not even more. Don't spin narratives and be crystal clear about expectations and outcomes.

Don't also "build" trust as a fake simulacre of due diligence. Manufacturing trust is an oxymoron and I think it's utterly stupid.

Be careful with tools

I get it, it's easy to vouch and advocate for a specific tool (Kubernetes? Nomad? Docker? Nix?) or for a specific methodology (Airflow? ArgoCD? WhateverOps?) but really, at the end of the day they don't matter that much as long as the team uses a stack that it is either confident in or a stack that it is confident (and has the time) to build up expertise in.

Keep in mind to always know the tradeoffs, from experience and not reading random blogs, of whatever tool or set of tools you'll decide to use. It's important to account for the failure modes of different technologies. Want to use Kafka (random example)? Have you seen it break and managed to fix the root cause? Are you aware of the open bugs and if they affect you somehow? Are you going to maintain it close to it's maximum efficiency?

Try a few out and see what works for you. You can develop a GUI with Python or write your API in PHP, it doesn't matter until you cooperate and coordinate the development effort and architectural decisions based on facts.

A relevant quote I like to think of is:

"When you adopt a tool you also adopt the embedded management philosophy within that tool"  - Clay shirky

This is very true. Don't pigeonhole yourself into one way of doing things, there are many different ways of getting stuff done, find the way that provides the maximum customer benefit and developer satisfaction for this specific context. Sometimes this can mean that the code is not perfectly "clean", whatever the meaning of "clean" is to you.

Branch your roadmap ahead of time

Sometimes things are not clear or the business isn't sure what to do. It's good to have a few alternative roadmaps planned out. It's easier to choose than to come up with a few things. Call it "something to hate".

If we know we're going to open an italian restaurant, you can prepare a few dishes as an inspiration and let the customers decide. Does that convey the idea right?

Change is the only constant. Plans change, people change, contexts change, having a few contingency plans ready for discussion is great.

Make it your mission to know your stuff

One big differentiator from my other project manager colleagues, is that I have been at the other side of the table. I know how to program API's, secure them, deploy them, scale them and everything in between. I mean the technical part of the job is not really a mystery.

This also means that I know, at least from my developer's perspective, when things like Agile make sense and when not. I know for example when pair programming makes sense and when not.

And the answer is the most important takeaway: leave your team the choice and trust them. Don't fall into the project management cargo-culting or the new management fad du jour. Just like in software development, there are practices in project management that are purely unjustified and not universal.

Some people thrive with standardization and rigid structure / org charts: "For any project we will do we need 1 UX designer, 2 developers and 1 Project manager". This is absolute madness to assume that any project will bend to a random organisation. Self organizing teams beat those rigid structures 10 times out 10. But that isn't much talked about since it might hint towards inefficiencies that are easily solved. When you hire people trust them to do what you hired them for, let go of extreme control, it's a distraction.

You read about those a lot in manager magazines and other "manager" type forums. Tips and tricks to manage developers... You can't grow and maintain a garden in agile sprints. A technical project is like a garden in quite a few ways. Plants and flowers won't bend to your will just because you wrote a great Initiative and broke it down into pretty Epics and User Stories. What matters is the constant and regular application of development effort at the cost of ultimate organization. As a matter of fact, I read books where the advice for how to treat flowers is more humanizing than other books about how to manage developers.

I recommend anyone reading up to now to check out the Developer Hegemony book by Erik Dietrich. It opened my eyes.

Few will agree with this but things are changing.

Watch out for liabilities

A common criteria I check when contacted by external companies or internal initiatives is to verify if projects are based around narratives or around real business needs.

The first type are purely career moves and might have a great upside since narratives can be easily adapted and changed, this will be very political and generally a waste of time for everyone involved.

The second type of project are the ones where I find myself comfortable with and aligns to my ethics. It also gives a great leverage for compensation discussions and progressions that are based on truths and not fabricated job titles.

Assume good intent but always watch out for outliers.

There is no complete recipe

There are just guides and hints. It's more like a buffet where you have to try things out and see what works for you.

You make your own journey and path adapted to your current context and circumstances. It's worthwile though and can be fun.

Don't forget to take care of yourself and don't make the issues in the project your own issues. After all, when all the code is written and done, what stays is the impressions you made on people and the relationships and trust you built.

Keep in mind that every single human experience is statistically unique, your experience might differ from mine or from someone else's. It is worth it however to discuss that and learn from each other because it's somewhere within that difference that lies the truth.

Have a good one!