When Twitter fired half of its employees in 2022, and most tech giants followed suit, I wasn’t surprised. In fact, I think little will change for those companies. After being employed in the tech sector for years, I have come to the conclusion that most people in tech don’t work. I don’t mean we don’t work hard; I mean we almost don’t work at all. Nada. Zilch. And when we do get to do some work, it often brings low added value to the company and its customers. All of this while being paid an amount of money some people wouldn’t even dream of.
What is happening right now in tech may be one of the greatest market inefficiencies—or even deceptions—in history. I am writing this article because I think outsiders deserve to know what’s really going on in the field.
I know my statement may sound a little bit hyperbolic—how could people be consistently paid a lot to do close to nothing? Surely that can’t be right! Well, let me share some examples from my own experience.
Five months ago, I was hired as a software developer by one of the world’s most prestigious investment banks. While I prefer to do freelance work because it involves real work, I was looking to have a bit more stability for a while, so I gave a chance to a normal corporate technology job. Since the beginning of my employment, five months ago, I’ve worked for around three hours in total (not counting non-focused Zoom meetings which I attended without paying much attention).
When I first joined the company, I was excited. However, since I joined they’ve only given me tasks that were exceedingly easy to complete, in just a few minutes, but allocating days or even weeks to them. At first, I wanted to speed things along. I genuinely wanted to build a cool product, so I connected with people across the organization to ask questions about our intended users, their needs and how our product would satisfy them. But it was soon made clear to me, a few times, that I shouldn’t do that. One person told me, “I don’t want to tell you not to ask too many questions, but…” and she basically told me not to ask too many questions.
I soon realized that the project was overstaffed and most people were pretending to work. And I also realized that was the job I was hired to do; my job was to pretend. If this had been the only time this ever happened to me, I would consider it an anomaly. Unfortunately, this has been the case with almost every tech job I’ve had for years.
Consider the case of my previous tech job, in which I was hired as a data engineer for one the world’s largest telecommunications companies. In the year and a half that I worked for them, there was only one two-week period during which I worked at full capacity. Other than that, I did almost nothing at all for the remaining 18 months. Outside those two productive weeks, most of my work involved attending irrelevant meetings, performing small tasks to pretend that a broken product worked well and even generate fake results. It felt ethically compromising and it was boring; I only worked half an hour a week or so, non-focused meetings aside.
I could go on with many other similar personal stories, but you get the gist.
This isn’t just me. All of the people I know who work in tech seem to be going through the same thing. One of my former colleagues, for example, told me all he does at work is watch Coursera courses. He’s considering resigning after the company-sponsored Coursera subscription ends. Another former colleague was hired a year ago as a data scientist for a large oil company. She’s making 200,000 pounds a year. All she does is prepare a PowerPoint presentation every week, and she’s utterly bored.
Another one of my friends was hired two years ago as a quant for one of the world’s most important investment banks (yes, that one). The interview process for that kind of job is among the hardest you can imagine—brain teasers, differential equations, graph algorithms. He was very excited at first, thinking he’d be building cutting-edge technology. However, while people always seem to appear busy from the outside, in reality he does almost nothing at all and is horribly bored but well paid.
It is hard to speak about this with others. Someone once told me that my frustration with the traditional tech workplace reminded him of Meghan Markle and Prince Harry because I kept complaining about stuff despite being in a highly desirable situation. Indeed, for a lot of people, the idea of being paid a lot to do nothing sounds like a dream come true. However, while we may not do almost any real work, we do have to constantly pretend that we do. That can be extremely frustrating and soul draining. Moreover, this shaky situation cannot last forever; it’s like a poorly balanced house of cards. With the recent massive layoffs and the collapse of SVB, the signs of strain are already there.
Let’s take a closer look at what’s going on in tech and why techies end up working so little.
Task bloating
Businesspeople love predictability. They like to estimate in advance how much a project would cost and how long it would take. This can be surprisingly difficult in some disciplines. Even when building a physical thing, like a railway, projects often overrun significantly. London’s latest train line, for instance, was delayed by two years two days before its inauguration. Imagine how much worse the situation gets with intangible products like software, where it’s hard to define exactly what it takes to deliver the product and even what the product is.
So, in the early 2000s, a philosophy called agile, which sought to improve the software development process, became really popular in tech. In agile, software is developed in very short cycles, as short as two weeks, and the result is validated and goals realigned between cycles.
Every couple of weeks, the team gathers to plan the tasks for the next cycle. But in this planning phase something really strange happens: It has become the standard practice to highly exaggerate the efforts required to perform a task, which we’ll call task bloating. In the planning session, hands-on people, managers and other stakeholders all agree that each task takes an abnormally high number of hours to complete and requires an abnormally high number of employees.
We’ll analyze the reasons for task bloating in a minute, but let me share a couple of examples so you can see the extent of the problem.
One of my recent tasks at the investment bank was to analyze what a couple of software code templates provided by Microsoft could be used for. Anyone familiar with software development would be able to do this in a couple of hours at most. However, in our planning session, it was collectively considered that this task required many days of work and two people. The difficulty of tasks is chosen by vote in this company and, in this case, the collective wisdom decided that the task was relatively hard. I voted that it was easy, which didn’t match the majority vote. When they inquired about this, I conceded that maybe it was more difficult than I thought after all. It is hard to disagree in those situations because it seems you’re going against the collective wisdom of the team. Also, when you notice that every single task is bloated this way and no one says anything, you don’t want to challenge the system. As two of us were assigned to this task, we ended up splitting it into two even easier parts, one for each. I completed my part in a few minutes and pretended it took much longer.
In the following cycle, I was assigned the task of writing a paragraph summarizing the results of the previous task. I voted for a difficulty score of 1, the lowest possible, as it was a 10-minute task. However, my colleagues didn’t agree; they voted that my task was harder than it looked and worthy of several days of work.
It is also customary to break up a simple task into a dozen pieces and then determine that each subtask is artificially difficult. For example, we had the task of deciding which software tool to use for a certain application. We had a list of features that were needed and four candidate tools, so we had to find out which of the tools covered more of the necessary features. This sounded to me like a simple task that could take one person a couple of days if done really thoroughly. But the task was broken up into four different subtasks to individually investigate each of the four candidate tools. Each of these four subtasks was assigned a window of many days to complete by two employees.
If I had to guess, I’d say a bloating factor of at least 10 is the norm. But 50 or even 100 isn’t uncommon. By following the practice of task bloating almost universally, the bar of what a tech employee is expected to do with their time has been collectively lowered.
What causes this bloating? It is tempting to become cynical and think that this is a major conspiracy that helps lazy employees not work and lets companies overcharge their clients. But I think that, although there may be some truth in that, there are more profound causes. Let’s have a look.
How the agile recipe kills productivity
The principles behind agile software development are commendable and much more suitable for the job than old-school ideas. So, many organizations have strived to “be agile.” However, they’ve done so by adopting agile “recipes,” which are step-by-step guidelines that are supposed to make a team agile. The most famous one is scrum. The adoption of a recipe results in a box-ticking exercise that makes companies believe they become agile just by strictly abiding by a set of inflexible rules. The effect is the opposite.
We already spoke about one of those rigid rules—that work must be divided into clearly defined cycles, usually called sprints, and that we must try to predict the difficulty of tasks to make sure they fit within a sprint. This structure encourages task bloating—employees want to deliver on the promises they make for the sprint, so they end up bloating the tasks to be sure to complete them within the sprint. So, productivity is sacrificed in the name of predictability.
The rigidity of sprints also creates other inefficiencies. For example, if a new employee joins the team in the middle of the sprint, more often than not they aren’t assigned any work until the beginning of the following sprint. So, due to glorification of the methodology, the new joiner is left with nothing to do. Everyone seems okay with it. Also, if someone announces they’ve completed an assigned task before the end of the sprint, in practice they are rarely assigned a new task until the beginning of the next sprint, leaving them completely idle, or they are assigned a filler task like “provide support to James on his task.”
The agile recipe also manages the day-to-day work, creating even more inefficiencies. For example, it prescribes exactly how often the team should meet and for how long. The recipe says, for instance, that the entire team is supposed to meet every day for a short debrief called a stand-up meeting. In this meeting, every team member gives an update on the current tasks and potential blockers. The idea of a stand-up meeting is to create some sort of team synergy, but I’ve never seen that happen. It soon becomes a box-ticking exercise. Most often, employees use this time to tell the manager what they’ve done and don’t communicate or listen to one another. It works for the manager, who benefits from being updated, but not for the others.
Some versions of the agile recipe also require employees to do a “demo” after they finish each task, in which they showcase to all other team members what they’ve done. And also, at the end of the sprint, all team members participate in a long joint session to do self-reflection, in which everyone speaks about everything that’s been done.
As you can see, a lot of the work in tech involved “meta work” that seeks to plan or discuss the actual work: You first speak about the work you will do, then you speak about the work every day for two weeks as you do it (since tasks are really bloated, this requires a lot of pretending), then you demo the result of your work, then you collectively reflect on it. Very often, the meta work is almost the only work you do, as the 15-minute debriefs and planning sessions take longer than completing the bloated tasks themselves. I once attended a planning meeting in which it was discussed for 20 minutes whether a task should be included in the next sprint or not. The task could be done in five minutes.
I have joked that the reason Facebook became “Meta” is that all its employees do is “meta work.”
The day-to-day inefficiencies introduced by the agile recipe have the effect of aggravating task bloating. For one thing, meta work ends up taking people’s time. I once asked a lawyer friend who’s very busy whether he does daily catch-ups with the associates. He told me, “No. We’re too busy for that. We cannot afford to be constantly meeting to catch up, as we have important things to do.” He was surprised to hear we do that in tech. But the constant meta work also kills productivity by interrupting an employee’s flow. For example, when a coder faces a difficult task, sometimes the best solution is to spend a couple of days thinking about it or doing research in a focused, freestyle way. It doesn’t help having to constantly interrupt work to speak about work and notify everyone of what you’re doing every step of the way.
Unfortunately, agile has become a cult; some even call it a religion. If you suggest that the methodology may be causing unproductivity, its proponents double down and say it’s because you haven’t been strict enough in following the recipe or that you have misunderstood the recipe (they may hire an agile consultant to help out). One of my friends mockingly used to say that agile is like communism: the reason it’s never worked is that it’s never been applied correctly.
Questioning the agile recipe is seen as heresy. For example, I once suggested that I wasn’t sure all the recurring meetings to catch up were the best use of our time. I was told, “Well, in this company we enjoy teamwork. Don’t you?”
The universal adoption of agile recipes has hijacked techies’ work. No one is incentivized to work at full capacity or keep the product and client as the top priority. Instead, employees’ main goal becomes to abide by the methodology. Amen.
How hype destroys motivation
In the world of tech, hype is rife. When a new technology emerges, people get overly enthusiastic and want to apply it everywhere—we’ve seen it with big data, AI, data science, blockchain, ChatGPT, and so on. I’ve seen companies hire as many as 70 people to build a product with the goal of following a trend without defining what the product would do or whether clients would want it.
Just a few weeks ago, for example, a company reached out to me looking for advice on how they could use ChatGPT in their accounting software. They said this was because they were trying to raise funding, and investors wouldn’t like it if they weren’t using ChatGPT for something. I’ve documented many examples of this phenomenon in the context of AI in my book.
This cart-before-horse approach makes companies overstaff teams to build unnecessary products. I think this is another major reason for the task bloating and perpetual thumb twiddling that I mentioned above, as techies become severely demotivated when they see no point in their work. One of my friends lost all interest in his job when an additional five people were hired for his team because the topic was deemed trendy by an upper manager, even though they were coping just fine with the workload. The expansion of the team diluted the work among too many cooks, so task bloating season started.
In some cases, employees soon realize the product they’re working on will never be used by real clients as it never really responded to a client’s need—it just followed a fad. But they still have to build the product. In those cases, why bother working hard instead of bloating the tasks and using the time for something more meaningful like searching for the next job, mowing the lawn or perhaps writing this article?
I once joined a team to build a product that was meant to help data scientists within the organization. As I’d worked as a data scientist for years, I was a bit surprised when I saw the product’s user interface, which didn’t seem to reflect the way data scientists really work. I enquired about this, and found out they hadn’t even interviewed data scientists across the organization to see what they needed—they’d blindly created the team and started the development of the product because it seemed trendy. When I inquired further, an upper manager told me, “We have decided not to speak with users; we will first build the product we think is right for them and see what they think.” He called it a “fail fast approach,” but I thought it was a “fail for sure” approach. Imagine the level of motivation and productivity in this team.
Why does this happen so much in tech?
Tech is not the only sector riddled with unproductivity. I’m sure some people would object that what I’ve said so far also plagues other industries. However, I think some characteristics of tech make it particularly prone to task bloating and unproductivity. I think one of the reasons for that is that technical work is poorly understood by business people. So, if techies tell a decisionmaker that reading a code template takes a week, instead of the more realistic 30 minutes, they are likely to believe it. It would be much harder for, say, a barber, to pretend that cutting someone’s hair takes days, as everyone has had a haircut before and roughly understands what a haircut entails.
Moreover, the benefits of many tech projects are intangible. If you promise to build an “AI-based insights engine,” it is hard for businesspeople to understand the return on investment. You can roughly measure the output of a barber in terms of number of haircuts, but how do you measure the value of a data science or an analytics project?
The adoption of agile recipes also helps conceal the problem, as it creates the illusion of agility and the illusion of teamwork. If you look at it from the outside, the team seems productive because every couple of weeks new tasks are completed and other tasks are collectively planned. It also seems that there is a lot of teamwork, as everyone meets all the time.
These factors combined have helped tech companies become unbelievably bloated without consequences. If you haven’t yet, you may want to check out this video, called “One day in the life of a Twitter employee.”
What does truly agile teamwork look like?
The title of this article says I’ve almost never worked when employed in tech. But that doesn’t mean I’ve never worked. In fact, I’ve worked really hard and been part of amazingly productive teams which delivered top-notch products in record time (and clients loved them). The thing is, this hasn’t happened in a traditional environment, that is, when employed full time by a mid-sized or large tech company. It was only when working as a freelancer or for early-stage start-ups that I’ve witnessed productive tech work. I’d like to share why I think that happened, as I hope some of the features could be exported to the broader tech industry.
The most common characteristic I see behind truly productive tech work is that the people involved prioritize building the right product above everything else. They build it in small cycles and validate it, which follows the agile philosophy, but they don’t limit themselves to a strict agile recipe. They sometimes borrow some elements of an agile recipe, but only because it helps them build the right product. For example, they have recurring meetings only if there’s a good justification for them. And, instead of letting the methodology dictate every single interaction, they hold impromptu meetings on an as-needed basis, even at five minute’s notice, all in the name of improving the product.
Also, in a company that prioritizes building the right product and serving its clients, employees outright refuse building something they consider inadequate just because the boss said it was a good idea. For example, in a start-up I worked for, two software developers and the CTO once had an intense verbal fight in the office about aspects of the product, and one of them ended up crying, with tears and all. I know that might be a bit too much, but it shows that they all cared passionately about the product and didn’t just do the assigned tasks.
This contrasts dramatically with how things work in the world of the traditional tech enterprise, in which the employees’ focus is all too often on doing their tasks as requested in the allotted time frame without questioning their business value and while following the methodology to the letter. In the book The Stupidity Paradox, Mats Alvesson and André Spicer describe this phenomenon, saying, “You avoid thinking too much about exactly what you are doing, why you are doing it, and its potential implications. You hope to avoid punishments and many worries that might come from deviation. You sidestep the burdens of having to think too much and upsetting others by asking difficult questions.”
It seems to me that most tech companies optimize for the wrong thing. They optimize for compliance with methodologies, for the adoption of trendy technology, for having a billion-dollar valuation, and so on, but they’ve forgotten about their clients. However, thinking about clients is what makes a business successful and a team truly productive.
A techy friend of mine told me today, “I’m not sure whether I should be pleased to save so much money and apply myself so little. It’s a trap really, as in this age of cost-of-living crisis it feels like a no-brainer to have access to this amount of savings. But shouldn’t I be striving to improve my skills for the future?” Scared to switch jobs, she’s clinging to her current one in which she is paid a lot, does very little, and learns very little. So is the case with many tech workers. If you are one of them, I encourage you to look for a new job as soon as you can; I hear there are lots of companies out there looking to build real stuff but struggling to find tech talent.
The freelancing mindset is great to have even as an employee. This read has been an incredible roast.
Now I see why there are so many people in forums 24/7.
Being in a desk for days without job to do is a torture
By the way, when they plan tasks they normally plan them from the point of view of somebody that has no idea of what is doing.
You has to understand that when you arrive a company you cannot enter like a boss. A company is a social complex mess, so you need observ people and make your position to have the right to do things. Seems stupid, but the oposite would be true. If it were your company you would constanty constrain pleople to follow your vision.
If you have the skills you should begin your own dream, not follow others'
s/herded/harder/
Good read, esp with the examples! Though never happened in my work experience to that extent… I've never been in large banks or government contractors, perhaps that's why but I can imagine some bloat there, from the anecdotes and the results delivered…
Fabulous article Emmanuel, I agree with a lot of what you said. I HATE agile or should I say I hate the amount of fluff (meetings) it generates. It reminds me of the Fawlty Towers sketch where Basil is trying to put a moose head up on the wall and can't because he keeps being interrupted by Sybil phoning him up and asking him if he's put the moose head up yet. And don't even get me started on sprint retrospectives!!
I agree Tom, the meetings are a killer. We didn't used to do things this way. Agile, sprints, daily stand-ups etc didn't exist. The sprint retro is another time wasting box ticker. Everyone is pressured into coming up with good and bad points that happened on the sprint. Even if nothing bad happened you usually have to think up some BS meaningless point. And very often, genuine problems aren't resolved by the scrum master, so the same thing comes up in the next sprint. Pointless!
And you can't even question the process, because your comments go into a black hole. I know, I've tried it before and got nowhere. It's all controlled by management and these Agile 'consultants' who make a fortune by coaching teams in this crap.
Also don't get me started on STORY POINTS. You get a different definition of that on every. single. project. Just use days FFS! And estimate the tasks properly instead of bloating them.
It all falls and stands with a good scrum master if you are doing scrum for example.
They need to be aware if the meetings make sense if the team needs that and what principles in general make sense from the scrum by the book. Because often the 'whole scrum thing' is too much. A scrum master should now the agile tools and know what to do with them, how to support the team so they can live up to their full potential.
I work in a team that lives scrum in a way. We are developing an in-house application. We have dailys, 2 week sprints, sprint reviews with the business, retros within the team, plannings (all at sprint end) and backlog refinements. I would not say we constantly blow up our estimates and never on purpose. We use them as a gut feeling how complex the ticket would be for ourselves. If there is someone with a huge difference compared to all other (we estimate in fibonaci) we talk a bit more on that so everybody can understand the complexity. We re-estimate then.
Sure this can be tiring sometimes. But I feel our estimates are fairly accurate overall.
In two weeks we have 3h of sprint end meetings in sum, and 150 min of daily. With option to extend the daily for a deep dive on a topic. Works pretty well for us.
The business test the application in beta and gives us feedback. In the reviews we present new stuff to the business and also talk about what urgent topics they brought in. We then take additional stuff in the planning into the sprint depending on developers availability.
This works for me and I am never out of work. If all tasks are done we pull in new tickets if that makes sense or do some clean up or training. But we do this decision as a team. We talk honestly about that in the daily.
So bashing agile tools in general is not the solution. Working with the agile tools the wrong way is easy.
So instead of getting only more developers maybe the companies should think about getting agile coaches/ scrum masters on board to get the most out of their existing developers. As you all stated here you hate to just wait for the 8 hours to get over. Maybe it would also be an option to align with your team (without your lead) if everybody feels that way and then ask for a person how is only dedicated to help you with agile.
One important foundation of working scrum is honesty and trust among the team and with the scrum master. Because our boss is rarely in the scrum meetings. Only if we need the boss or he needs us for additional requests he got.
And also be open to new stuff the scrum master brings up. It can be fun. And if not… as it is agile drop it again after one or two sprints. Everything that does not bring a benefit to us developers is wasted time in agile. But you should really try before abandoning stuff. Sometimes you will be surprised that it works anyways. When you do not try you have lost already.
From what you have write about Agile not working, What I could understand is that the "pillars" Transparency and Inspection and "values" Openness and Commitment has been compromised. This is a cultural issue to be addressed by the company and may be prevalent across the organization. Irrespective of any project management practice, the team will fail if these fundamental principles are not followed. Task bloating is a byproduct of lack of courage.
“The emperor has no clothes!”
Bravo.
It’s even worse, I fear, for remote workers. COVID really let the genie out of the bottle.
I totally sympathise with being given very little work and getting bored, but there are so many articles where something bad has happened and the person blames it on agile or scrum.
I've worked with multiple methodologies (Waterfall, RUP, Scrum, Kanban) and I've been perfectly happy with Scrum. I've not seen the exact issues you've talked about in the companies I've worked in, which includes big tech. Usually when these things happen it's because the company / department / team / person does not understand what the core concepts are and follows "recipes" like you say instead of allowing the team to customise and adapt using something as a framework. From your description, it seems like the company thinks it's being agile / doing scrum, but it's not.
Don't get me wrong, I've heard people complain about it and I'm sure it must happen a lot that things are implemented poorly. These cases seem to happen more often when someone up high up decides they need to be more "agile", they hire someone who can tell them "how to do scrum", they tick the box that "scrum is now implemented" and the company pats itself on the back and waits to reap the rewards.
If a company doesn't understand why something is useful, then it's much easier to fall into anti-patterns for anything. I certainly have always had plenty of work to do and have never been bored and I've mostly used "standard" Scrum now for about 13 years. My position has changed over the years so I naturally do less coding and more strategic stuff, but I've always had plenty to do that made a difference and had customer impact. Every process has its pros and cons. Even waterfall can be made to work with the right people. I've tried changing things up to see what works and what doesn't and really Scrum is pretty reasonable start. I certainly prefer it to the old Waterfall model. For all these processes though, the hardest thing is scaling them and that's especially true for anything at really big companies, so that's maybe where things fall down. But as I said, I've definitely seen it done well even there and more often than not.
I think ultimately companies that don't care about customers or think about what they're doing and why will always find ways to screw things up. Agile / Scrum is just an easy target / common symptom, rather than a root cause.
Interesting read though about how widespread the slacking is, good luck finding a better company :).
I'll second these thoughts. I've seen both sorts of teams implementing Scrum: those who go through the motions and those who use the structure to improve their work. Improving the work might be adjusting the events (meetings) defined in Scrum to accommodate their reality. Scrum is one way to practice things which help toward being agile. Once it is done the team should adapt to create processes which work for them.
It's different but familiar where I am. No time for Coursera or twiddling, but also endless tracking that drains productivity. Every transaction comes with a survey, because that's how everyone facing customers is measured now. There is a tension between wanting to deliver a lot fast (technical stakeholders love, good surveys) and to deliver the metawork like predictability (management stakeholders love, good surveys). It comes to the balance of stakeholders. Unfortunately, society has been taken over by MBAs and lawyers who construct process and guardrails, but never produce anything within them. Freelancers are a part of the system too; you are low cost with no benefits and produce code. That hasn't gone unnoticed by higher powers, I'll bet.
I largely agree with the assessment of agile. I'm a Lead Engineer, and the larger team I'm apart of wants to meet for an hour twice a week for standups, then an extra hour and a half every other week for reetrospective and risks, then senior leadership wants a 2 hour cal every week to review risks, and a 3 hour call every month to review risks at higher levels.
I've written in every risk that we have too many meetings. The excessive amount of meetings leads people to manufacture blockers and try to look busy, while also producing blockers because no one actually wants to work lest they get more work. Management's answer, "we need to schedule additional meetings between the twice weekly team meetings, so the smaller teams can meet and review status on a more localized level."
I'm going to quit in the near future. They don't want to get things done, leadership just stinks.
I have experienced this my entire 30+ year career. I quit my first corporate programming job out of sheer boredom, as I almost always completed my weekly assignments (bug free) by end of day Monday, then had to look busy the rest of the week. I get paid a small fortune now and literally do nothing but sit in Zoom meetings a few hours a day. Unbelievably soul-sucking.
This article is BS, and furthermore runs the risk of being harmful to the sector. Either the author won the lazy lottery with his traditionally posts (which he probably did intentionally, since these shortcomings would almost definitely be obvious during any team meet segment of an interview process), or he just has a general chip on his shoulder about agile. I have worked in multiple XP environments (which involve flavors of agile development) at midsize companies and all have involved constant, dedicated work during work hours with engagement toward clients’ needs and product improvement. Additionally, every piece of acquired startup software I’ve encountered (the author’s preferred work model) has been trash: horribly coded, broken, limping software built to look like it works, containing endless inner, hidden holes, hacks, and compromises, and written in some unreadable personal style, like the shell of a house built to sell sight unseen.
I once had a coworker who, like the author, came from a background of engineering a startup app, and always felt we could be moving faster. His code was consistently buggy and unreadable and the rest of the team was largely employed fixing and refactoring his garbage, just in time for him to churn out some new piece of trash for us to fix, all the while complaining about how slowly the rest of the team moved and blaming the process. Sound like anyone we know?
So sure, this guy has a personal bone to pick, but how is his lack of circumspection harmful to the rest of us? Simple, because purse strings are going to read this article and see an exciting opportunity to tighten up a big ticket item, the engineering budget, and teams like mine, the normal teams of the world, are going to have to spend more time fighting for resources and less time writing code. Thanks for shooting the field in the foot over your two dissatisfying experiences, Emmanuel.
I appreciate the honesty here, but it genuinely disgusts me. Infuriates me, even.
– Unemployed Tech worker that actually works.
Holy Technical Debt, friend! I've been saying similar stuff for years. Thank you for articulating and socializing so many of the frustrations I've had–not with Agile/Scrum, specifically. With the inflexible Scrum zealots I've had to deal with in my career, from tech leaders who arbitrarily mandate (without modeling out the actual numbers of transitioning Waterfall-to-Agile) that "our tech arm of the company" most do Scrum by the book; the endless parade of "Agile Coach" theorists that management engages for a period (which means we're forced to follow ONLY that one dude's Agile rulebook–and the team better adhere so the company can justify the expense of bringing this person in); and even the incompetent co-workers who are bad at their jobs, and then embrace Scrum by the letter to stave off their fraudulence and indicate that they're adding value. Look, I was Scrum Certified by Jeff Sutherland (one of the two co-founders of Scrum) years ago, and I have enthusiastically embraced certain aspects, as they fit the organization. But the trade-off is everything you described. Thank you!
WOW, what a great article. I've been in tech for a long time on the technology resellers side of things and I've always felt like there are some glaring problems in software dev. This really confirms my suspicions.
I've seen so many customers with projects they purchase infrastructure for, that just ends up on the cutting room floor. I'd expect some of that but good god it's like there are racks and racks full of failed projects out there.
This carries over to projects within IT departments as well. Just an amazing amount of failures mostly due to not understanding a business model and how it pertains to what they are developing.
Good article! I completely agree with you!
What other suggestions do you have for defeating the "norm"?
Hey Emmanuel,
Great article, thanks for writing it! A lot of what you said here needs to be understood by more people in our industry.
There's one suggestion / correction I'd make, though. It's about one tiny part of the article but I believe it makes a big difference in how people adopt some of these agile methodologies.
«that work must be divided into clearly defined cycles, usually called sprints, and that we must try to predict the difficulty of tasks to make sure they fit within a sprint.»
Sprints, as scrum proposes them, are meant to "contain damage." They're a way for us to restrict how much time we are willing to risk on something. At the end of that amount of time, we do one of three things: deliver that something if we finished it, decide we need and want to spend more time on it, or decide we don't have the budget to put more time and money into it (meaning we cut our losses and stop working on it).
In other words, sprints are not meant as estimation tools. Unfortunately, tons of organisations use them that way and it has become a cultural norm of sorts, and because of that, a lot of people think that that's what scrum is about.
In fact, we're not even supposed to estimate how long a task is going to take. Scrum doesn't talk about estimations anywhere. This is a requirement that comes from managers that don't understand they are creating something new (the type of project where scrum applies) and thus it's impossible to predict how long a task will take because it's usually something that was never done before. This is in stark opposition to something very well defined and known, like building a car: a car manufacturer can tell, with a very high degree of confidence and accuracy, how long it takes to get a car built from start to finish – because it's something they've done a million times and every single step of the process is well known.
When you have such a project, scrum is not the right tool. Scrum is best applied in projects where "complexity" is high. That's why scrum promotes "adaptation" – meaning you accept that you're not working with predictable things and you adapt as you go. And the unpredictability is not only about task estimation, it also encompasses market shifts, changes in the organisation, new learnings from clients, etc. All these things make it pretty much impossible to estimate something. That never stopped someone from trying, though, and there are tons of organisations who try to do it and fail miserably. When they don't fail, it's probably a matter of time. Or they're just pretending, as you so well describe.
I believe concepts / ideas like scrum can have their benefits but it's hard to implement them because it is very rare to have a whole team that fully understands the purpose of the tool and they end up being too rigid about it and mindlessly checking off boxes, instead of being intentional in what they do and use the tool to help them instead of becoming slaves of the tool. I think the best example of this is something you also mention: the daily stand up. People will mindlessly regurgitate "the three points": what they've done the day before, what they're planning to do today, and any blockers they may be facing (by the way, this is also made up, scrum doesn't say anywhere that this is what you need to do in a daily stand up). It's much, much better to be intentional about keeping the other people in your team up to date on things that are *relevant* to them *when* it matters, and we don't need a mandatory daily meeting for that.
I guess I got a little carried away, sorry 😛
Thanks again for the great article!
Very well formulated!
The problem is Agile is often seen as a process. Something you can pick and apply the pattern and then you are agile.
But that’s not the case.
Agile is a way of thinking: “Do something, check whether we got closer to the goal, think about the next step.” That’s all!
Nothing about estimations, nothing about meetings.
If you’ve done this cycle often enough, you’ll get some idea of how fast you approach your goal.
Then you can do a guess on when you could be there, based on what’s _probably_ ahead. But this “probably” also includes stuff you can’t see at the moment, which causes deviations, which take time. Meaning you reach the goal later than you have guessed.
If you try to factor in these deviations beforehand, you’ll end up in adding bonus time in each task, because “you’ll never know”
This usually happens when you have a culture which sees these delays as a problem. Which makes you justify if you’re late. When you can’t be honest. When you have to deliver fixed scope in fixed time. For sure you add extra time everywhere, because you never know.
That’s the problem with most agile implementations. They want to take a rigid base and apply agile cargo cult on top of it. Hoping this would magically solve all issues.
But you get quite the opposite. You don’t solve your problems you just add more meetings.
But you still have this rigid base. This keeps you from the core: “Do something, check outcome, plan next steps”
That’s why it’s so hard to implement agile. You have to change fundamental things.
If you are willing to do that, agile can be helpful and fun.
And then it will apply to everything. If daily stand ups don’t provide any benefit, toss them out. If you think estimating your tasks does help you then give it a try, if you think it just eats up time with no outcome, toss them out. Try stuff, get rid of things you don’t like, be bold. That’s what agile is about.
This the complicated, but necessary, stuff. So companies don’t do that, but rather add dozens of meetings.
Some criticism is warranted but most of it is a reflection of your own, personal experience and not necessarily applicable to all or even most. I've had times when I had 80 hour weeks and had times when I had 4 hour weeks.
My diagnosis for this post is – PROJECTION.
Interesting but doesn't match my experience working in software for 15 years. I've always worked in consulting not on a bank or product development or internal IT.
What usually happens in consulting is the opposite…
A client asks for proposals for some project. Lots of companies bid and severely underestimate how long things will take ( for various reasons, from software being hard to just cutting hours so the proposal costs are lower and they win the project).
Then during project execution everyone works ridiculous hours to try and meet the unrealistic deadline. There's no such thing as overtime. You book 8hours but work twice that. Happens a lot. On these projects, corners are cut, quality goes out the window, buggy software is produced. Then bugs are fixed slowly after go live usually under a longer timeframe and perhaps in a pay by bugfix model.
Fortunately not every project is like this but it happens too frequently.
Where I've seen inflated estimates is when the client turns around for an inflight project or a production system and asks for changes. As in that case, the client is usually stuck to that vendor and so it is in the vendor's interest to say the change is really complicated. The client then decides to go ahead or just drops the idea.
On a more general note, inefficiencies like this are not tech specific imho and I'm not sure it's any worse in tech than other sectors. Happens practically everywhere. Perhaps not on the barber, but you go for a routine check on your car and there's always some surprise costs you have no way to tell if they are bullshiting you.
This article must have written by someone who has never worked at a mission-driven company that works towards actually saving lives and helping people with technology. Too many false generalizations. Go work for an early stage healthcare company and make a difference if you're sick of jiggling your mouse for the big banks.
I like your analysis, but I think it's not the universal agile experience. I've worked in a Scrum environment at the same company for 15 years, developing real-time software on Linux clusters. I have seen good and bad use of Agile. I have worked hard and so have those around me. However, there were always engineers who did not fit the Agile mold. One very smart guy wanted to work for 2 months on a significant component. He did not want to report every two weeks, let alone every day. He quit when he was no longer allowed to skip the stand-up meetings. A massive amount of knowledge walked out the door.
What worked well in my experience was making small incremental changes to an existing product. A major new development which takes months or years to produce customer value is much worse for Agile. This exposed a massive amount of make-work.
To improve that, the role which is essential is the Product Owner. That role must be technical and insightful to make sure that every week code is being created to do something relevant and creative. The tendency to produce "research" is a huge error. At the same time, how can you work in parallel unless you have a high-level design and an interface spec?
In short, the benefits are real if you do it cleverly, and disappear if you do it stupidly.
“Since the beginning of my employment, five months ago, I’ve worked for around three hours in total (not counting non-focused Zoom meetings which I attended without paying much attention). “
Could this be part of the problem? That you are not paying attention during meetings to know the user’s needs and the work that needs to be done?
Me and the people I know in tech work our assess off, early mornings, late nights and we deliver solutions even in the face of little to no organizational alignment such as lack of user buy-in and adoption. The title of this article flies in the face of my experience and the experience of the people I know and work with. Though we know people like you and question how the hell you’re still here. The guys that show up like they know everything yet don’t really do anything and get all the praise.
There needs to be a clear vision, mission, and strategy from the C suite and leadership that is being continuously communicated and championed throughout the organization. Managers in the org have a responsibility to know, understand, and communicate the objectives and goals, and we, the analysts, project managers, developers, have a responsibility to know and understand these things do our jobs to reach those goals.
“She’s making 200,000 pounds a year. All she does is prepare a PowerPoint presentation every week, and she’s utterly bored.”
Is she getting paid for preparing the presentation or the years she spent in school learning and the time it took to build a career to add value to the company? If she’s bored, maybe she can spend her time taking training courses to learn something new, or find opportunities for continuous improvement in the org, something of value. She should work with her manager if she wants to take on more challenging work. This is a first world problem, getting paid a decent salary and having the capacity to do whatever you want. In my early 20’s, I had customer service jobs that were intolerably boring, and I got paid about $25,000. I did homework in-between calls. I wish I had her problem.
Task Bloating
“In the planning session, hands-on people, managers and other stakeholders all agree that each task takes an abnormally high number of hours to complete and requires an abnormally high number of employees.”
As an Agile servant leader, I protect the development team from the business and vice-versa. I get the requirements from the business, create the story and supporting documentation, then review the story with the business and development team together, make any needed updates, and then the dev team will estimate the work without the business present. That gives the dev team the space to discuss with the business and then have the protection to discuss openly amongst each other. They can then bring back any additional questions or clarifications or proceed with an estimate on the work that needs to be done to achieve the desired business outcome.
Response to Task Bloating examples: Document the planned work vs actual work and adjust future estimates accordingly. This provides evidence for you to point to when going against the “collective wisdom” rather than “pretending” that they were correct.
Meta Work
Again, protect the dev team. A good Scrum Master ensures the dev team’s capacity accounts for meetings, PTO, training, etc. No, the entire dev team cannot attend this scheduled meeting, but a lead or senior dev (or even a technical analysts) can cover for the team while the others focus on dev work.
Rigidity of Sprints
I’ve never worked on a team that did not bring in additional stories from the backlog if they had remaining capacity. You also have to consider how much the business can consume in a Sprint. That has been the challenge I’ve seen the most, where the business asks for a bunch of stuff, they want it right away, the dev team delivers, but then the business doesn’t have time to conduct user acceptance testing (UAT) and so the stories sit there almost done and have to move to the next sprint. This then skews the load for the next Sprint though a good team knows how to work through these obstacles.
Agile Demonstrates Productivity
I can appreciate the challenges and frustrations called out as they are common across Agile teams. The response to the challenges is where the flexibility of Agile comes into play. I agree that teams that try to apply a rigid style of Agile become locked into ticking a box, however flexible teams that do communicate well work through those challenges. Can we cancel this meeting since no one has updates? Yes! A new member joins the team – this should be covered in your onboarding process. Maybe they use that half-sprint to review the existing code and solutions, review the team documentation, get oriented to the organization. We agree with the challenges and solutions, we just don’t agree that it is the fault of Agile as a philosophy.
I think this comes back to the organization and its maturity level with Agile. I’ve been a full-time employee (FTE) on a productive team within a mature Agile org. I’ve also been a consultant on many implementation programs and projects where the Agile maturity was at crawl. As a consultant, me and my teams served as Agile champions to help in maturing the organization while implementing modernization and digital transformation solutions thus demonstrating how productive Agile teams can be.
The Problem
The problem is not Agile. The problem, based on the experiences you’ve shared, is not having a clear vision and roadmap, and poor organizational collaboration. In my experience, I’ve seen Agile, and my colleagues, work more often than not.
The difference, in my opinion, could be the technologies we are implementing. You on the build side, me on the buy (and configure/customize) side as most of my experience is implementing and continuously improving ServiceNow instances (as an FTE and as a consultant). I can imagine the effort it takes to build a homegrown solution from one past experience of mine and that was only building a custom database to integrate with Salesforce. The Agile philosophy and its many methodologies or flavors may not always be the best approach for every software development endeavor?
Alternatives
Is having a more flexible approach to Agile the secular alternative to a rigid application of the religious Agile recipe? Maybe an alternative is having a flexible organization that can adopt a software development methodology that is not Agile and that fits best with the objectives and technologies.
Criticisms
Nothing comes without criticisms, and it is good to voice them so that we can do better and learn from each other. That daily stand-up (DSU) can become mundane and meaningless especially if the right people are not there at the right time and if people are not paying attention. When I was an FTE, having the process owners (POs) at the DSU was often critical to remove blockers yet they attended less and less to the point where we wanted to cancel the physical meeting and just check in with each other, and the Scrum Master would reach out to the POs since they had to do this anyway. It was weird not having the fac to face even though most of the devs sat near each other and we all worked from home twice a week. It took something away from our collaboration. The point is, we criticized the ritual, tried something different, and learned from it. We also rotated Scrum Masters, and we had our own approach which helped mix things up while still having a good rhythm.
If you find yourself in a work environment where you cannot challenge ideas and you feel you are not delivering value, then surely this is a red flag that there is poor organizational alignment and like you said, maybe you should move on to other work. If you are finding that your Agile team is not productive, then this is a sign of many different things that are worth working through with your team. It could be a simple miscommunication, or it could be something deeper such as Agile not being the best fit for the team or the project.
Emmanuel, I would say you're onto something but this "thesis" does not comport with my experience, nor independently verified data, nor in some cases, simple logic. I'm probably twenty+ years older than you, with a PhD in Computer Science, and someone who chose to enter the IT field rather than become a full-time professor. I was a system administrator for quite a while (before DevOps and SREs were a thing) and even researched self-managing systems/autonomic computing in the hopes of helping to bring us to a place where sysadmin jobs would not be so difficult. I've been a toolsmith, a QA engineer, and a performance engineer as well.
Most of my colleagues over the years have worked very hard to make sure the quality of what they produce is as good as can be; tot exclusively for the company bottom line, a culture of rigidity, to receive recognition, or even to make more money but for personal integrity and self-betterment. Of course the foresight and flexibility of management factors into many decisions. For example, many of the companies at which I and other colleagues have worked have fostered discussion about higher level decisions, especially when it comes to architecture and implementation of new features.
This leads me to the next factor that I think dings your argument a little bit. The majority of keeping an IT product or service running properly (and selling) is maintenance. The current model of employment in the IT industry, where employers only invest in their employees as much as necessary to remain competetive in the marketplace and employees don't invest much in any company (average software engineer tenure in the US is 2.5 years but I suspect the median is even lower) makes maintenance *harder*. Institutional memory, continuity of design and implementation details, and the reasoning behind product and service changes get crushed in an environment where the customer is expecting quick and constant improvements. Your post here ignores the legions of employees who work hard to keep things humming because finding and fixing bugs for thousands of different environments requires focus and diligence. Implementing new features or making non-incremental improvements to features that work right out of the box require good design and frequent iteration.
I do not dispute some of your points about agile. But like any methodology, it depends on who's governing the processes, how they're willing to adapt to the addition or subtraction of resources, and how willing all the participants are to be honest about work estimates and their own skills. In the squads I've been in, most people hone the predictability of their tasks after about 8 to 12 two-week sprints. We *always* engage in a retrospective immediately prior to new planning so that we can give hints and tips to each other. In addition, we follow an interrupt-driven model that let's people overcome sprint obstacles more quickly. Management can see how we're doing at any point, as they should because they have a much better idea of how resources are being allocated at different levels. Just because it hasn't worked in most of the places you've worked doesn't mean it *can't*
Finally, software doesn't design, run, debug, and maintain itself (yet) and neither do companies. The proof is in the pudding: companies are succeeding so somebody is working. It's still impossible that only some 10-30% of the world's tech workers are doing the work of the other 70-90%. In fact, surveys conducted by organizations like Blind and TechRepublic suggest that IT professionals are still working harder (not just performatively) than professionals in many other industries.
I have a completely different picture of the world.
At all my jobs I work more than 8 hours.
At my recent job where I quit, I had a crazy CTO.
Who didn't tolerate arguments.
He gave me weeks and months of days and hours for tasks that took weeks and months to complete.
I was given seven tasks, seven hours and 30 minutes.
I did them for three weeks with terrible overtime.
And I am an engineer with over 15 years of experience.
But all I can say is you have a great place to work. And you're lucky.
Most programmers overwork and I'm 100% sure that companies like yours are very rare.
I had a knee-jerk and reading through your entire post — you make valid points. I had my suspicions you expressed them eloquently!
As someone from a long line of blue collar workers, I find this highly disturbing. To add insult to injury, staff at Apple tried to form a union for better pay and so they could work on meaningful projects. Please allow me to pick me jaw up from the floor. You get paid a shitload of money, do nothing and now bitch about it? Wow x100000.
Very relatable! 😅
Excellent read!
Great article. I was hired to do contract process improvement at a $99B software firm and have worked possible 4 hours in 4 weeks for about 90k USD salary.
There was 1 hr of training then a redo of that 1 hr because I did not have Oracle access and then 2 weeks later, an actual task. It took roughly 2 hours and that’s only because I checked my work.
I Will keep billing them, working out, and working Coursera classes in preparation for the next job.
It's been an eye-opener read. I've only just started working within the agile space and the waste of time was so obvious. But every time I was trying to question things, the tech people would look at me as if I were the enemy. So I guess there is a large number who are just happy being left alone. That does not make you point less valid and I do appreciate the honesty. The only thing I cannot relate to is the workload vs payment, because most of my career I was underpaid and overworked, I almost wish I didn't have to go through that. Almost 🙂
Well done! You have thrown down the gauntlet. One of my favorite sayings was from a professor who said, “all models are wrong; some models are useful.” In business, we love formulas and the mathematics of all things adding up… to something. Work should be a place that’s exciting where we are challenged. Your work will help people create a more creative and productive business environment.
Why not just stay at the boring job and use your time on open source/NGO projects so you can keep learning, help general interest and feel useful instead of focusing on only improving private interests?
You've truly mastered the crux of being in tech by writing this article which could have been summarised in a phrase! Well done 🙂
Can't relate at all, doesn't match my experience (the doing nothing part – the agile is ridiculous part, yeah that I can relate…). But I have difficulty understanding your position on this. On the one hand you seem to complain, as in you're bored, but when you finish a task fast, you "pretend" it took longer…? I've been in similar situations: I finished faster and tackled other tasks. If you really had been unhappy with it why didn't you act differently? And don't tell me social pressure: I've been there. You can finish tasks faster and tackle new ones. Even if the sprint is not finished.
Very interesting read. I always wonder how a company can layoff 10,000 employees. I assumed that these people had real functions and did real work…and the layoffs would be huge strains on the organization. I suppose if you're fat and happy with massive margins, efficiency doesnt matter. Only when it starts to cathc up to them do they realize lopping off a few hundred million in payroll might be wise. Cheers!
Brave. Accurate.
First we were misunderstood. Then we were revered. The keys should never been given to the kids that excel in optimisation.
Timely too, as I contemplate shelving 15 years of consultancy (last 8 years freelancing) for a nine-to-fiver.
I am a PM leader who took a nicely paid job at a top-3 US bank to decompress for 10y of sacrifice with great professional success at startups and tech, and this post genuinely felt like I wrote it myself. I want to thank you for dedicating time into putting your thoughts here and for letting many soul-shattered professionals like me reflect about the conundrum of balancing savings and job security with impact and personal+professional growth. Your post reaffirms my idea of looking for something more fulfilling ASAP, at the expense of less money and free time.
"The interview process for that kind of job is among the hardest you can imagine—brain teasers, differential equations, graph algorithms."
The above describes the human resources version of pretend work. Working in so-called organizational leadership opened my eyes to shocking amount of do nothing work across all levels of organizations. I learned the most "agile" were those most adept at politics, with some even going on to occupy positions in state and national government politics.
"It's a big club, and you ain't in it."
-George Carlin
This article is extremely distressing to me. I've been working – responsibly hard – in software development for large financial services firms as a developer, project manager and – gasp! – scrum master – for over 30 years combined. My salary was always slightly under-market in a hometown discount but it provided a good living and allowed for good asset accumulation through 401Ks.
My – and all of my involved teammates / teams – experience was NOTHING like what Emmanuel describes. Estimates were always inspected critically for excessive padding, there was constant release and delivery pressure with a focus on value for the big dollars being spent. Ineffective developers – especially contractors – were consistently let go. Initiative-level projects that didn't deliver on time caused management terminations and bad performance ratings with loss of bonus eligibility. How I know is, it happened to me, twice and I watched it happen to several other projects in my career. The pressure to deliver on time and on budget was constant, intense, and as my wife will tell you, out of balance with life.
The environment Emmanuel describes makes me envious, and quite frankly angry. Maybe I should take a position at an investment bank, be an experienced agile coach / product owner / scrummaster and let the team inflate their estimates to the point of ridicule, get paid over market to snore through boring scrum rituals and help management feather their nests along with ours.
Kid, two things. One, Agile is NOT the source of your organization's problems. Corrupted estimating and ineffective development management is an endemic problem regardless of project methodologies. Your organization is the problem. There's a saying, "laughing all the way to the bank". Two, you don't know how good you've got it making that kind of money. It's no wonder Musk fired 80% of Twitter. If you're that bored and wasting that much of your time and don't have the integrity to find more valuable work… retire early, get some perspective and find something socially valuable to invest your time in.
This was really good and very validating
Excelente lo expuesto.
Dear Emmanuel,
Thank you for the piece, I found it insightful and clear with concrete examples.
How do you think task bloating can be prevented in organizations that use agile recipes?
Best,
Birgul
There’s some merit to the opinions in this article, but I’m mostly disappointed.
Agile is not a silver bullet. The majority of teams using it execute it badly. If you don’t understand the engineering problems agile sets out to fix, there’s no chance you can use the method to improve your efficiency. Sadly few people do.
Emmanuel seems a talented engineer, he may also enjoy working solo. He definitely prefers early stage start up work or freelance work where relatively small organisations/teams can do large amounts of greenfield work. Don’t forget many startups have to start rebuilding tech from scratch when they grow to a certain size because what they cobbled together the first time just isn’t good enough to keep meeting increased demands. Sometimes the tech debt they incur “moving fast” cripples them. Also contractor work isn’t known to be the most maintainable high quality work in the industry. Another point worth mentioning is that a so called “10x developer” isn’t that useful to teams, they usually cause issues in the team longer term with maintainability of the code. He also seems inexperienced in agile. Often if a person has a good reason for their vote, the team will vote again to take the new information into account. You really need strong communication skills to work in agile teams.
The crux of the issue is not any failing of any individual engineer. You must learn to understanding the challenges of large complex engineering projects. This only comes from long hard experience in engineering management. The more people you have in a team the harder it is to manage that group. The larger and more complex an engineering challenge, it’s exponentially more difficult to manage. Mature engineering projects have to manage risk carefully, almost becoming risk averse. We’ve all heard that “small and nimble” and larger organisations have large inertia.
The example of physical engineering projects is a good one. In the example in the article the project overran two years. Mostly this kind of overrun is because of unanticipated problems cropping up, or underestimating complexity of tasks. It’s unusual for a project manager to over staff a project or commit some other kind of fraud because they’re incentivised to deliver the scope on time and budget. All these problems happen on physical engineering projects in spite of the fact that a projects scope is usually fixed and well understood before engineering starts.
Software engineering is a lot more fluid. Client expectations and engineering goals can change massively over the life of the project. This is why modern engineering teams plan and commit to sprints of work. The whole point of calling it a sprint is because you work hard to finish it and sprint to the end. Rinse and repeat. Hard to blame such an idea for efficiency issues.
excellent response
This is insightful. Found my way here from twitter. I'm a long-time tech who has struggled to find success (i. e. get hired). You can't imagine what it's like knowing I'm overqualified in almost every way, but watching others be rewarded handsomely for doing less than I do.
Omg – THIS!
I was wondering if it was just me, because I can't help but wonder – what keeps people so seemingly busy?
I think humans have a real problem with needing to prove their usefulness: it's universally considered "good" to appear busy.
We should all agree that we collectively don't need to work as much and try to make it socially acceptable to spend most of our time enjoying life, doing whatever pleases us. I'm all for 4-10h work weeks. I'm sure a lot of good would come from allowing people to spend time doing things they actually enjoy, instead of spending so much time pretending to work while contributing absolutely nothing to society.
Love this article and would definitely like to learn more
Hola Emmanuel
Ya que sos Argentino te escribo en castellano.
Me encanto tu articulo. Yo también soy contractor y comparto tu opinión acerca de "agile" y de como la predictibilidad sacrifico a la productividad.
Saludos desde Zurich
Kewin
Found your article with a pointer from (). Hey, the "it creates the illusion of X and the illusion of teamwork" is everywhere by now. It's a consequence of living in the times of "mature market economy," when much wealth is already accumulated and it's easier to grab/claim/buy a piece of it rather than creating new wealth via producing value.
That's why I would disagree with your explanation of why engineering productivity is so low in tech. BTW, as you point out yourself, it's not just tech, but most of mid-size and large companies. Business people are not blind. And they are not stupid. They choose not to see. They like agile for a reason. Why? I wrote about this based on my 20 years of working in Silicon Valley: http://oecisland.com/AJS.pdf
Take a look. You will find more relevant references there: including description of bull**it jobs by David Graeber.
A really good, if slightly depressing read. In the mid 90's I had freelance jobs very similar to the ones you describe and it was without doubt the unhappiest phase of my working career. I got out and have worked ever since on my own independently sourced if still essentially freelance software projects. My approach has sometimes been likened to that of Agile, despite my being a solo developer. Hearing it's cringe-y recipes pulled apart happily reaffirms why I've always avoided referring to anything as a scrum or sprint. I've probably missed out on potentially lucrative and comfortable contracts but I don't, for a second, regret taking the path I have. I still start each working day with a spring in my step anticipating the mental stimulus it potentially brings with it.
I must be doing it wrong! I’ve worked as a developer for almost 20 years in Japan, earning nowhere near Silicon Valley salaries, though I am well paid by Japanese standards.
I’ve also pretty much always been kept busy except maybe for periods in between projects. I certainly have never had times when I could cruise doing nothing for weeks on end!
Great article!
I'm conflicted about the message in this post. It sounds like the author enjoyed their mindless, easy tasks which they would drag over days and the meetings they deplore because they are they only way in which their "work" could be found out.
As I read, I was able to identify myself with every single line. Now, I work for a mid-size company, and I have tasks that are planned for 40 hours, but my coworkers and I can complete them in 8 to 16 hours of hard work, without our daily interruptions.
This text gave me a strange feeling that I don't know how to describe. Perhaps it's because I can relate to your experience of only working in startups. I worked for one once, and it was a real joy, chaotic, but extremely fulfilling.
Thanks for sharing your thoughts.
Good article. Too long to read through the whole thing because I have work to do now. I think the perspective, while mostly anecdotally accurate is largely too generalized. That said, I'm pretty sure the anecdotal perspective is accurate in at least some companies and vastly inaccurate for others. As is the case in everything, without empirical evidence, the claims in this paper are interesting at best. But I enjoyed what of the article I did read (about 75%). I've worked in tech for 25 years. I've observed the opposite of the claims in this paper in almost all places I've worked. I have and still do work +50 hours per week with meaningful and enjoyable tech work along with my co-workers who are generally working the same number of hours as I because our plates are full of stuff that need to get done. The paper is thought provoking, but perhaps mildly dangerous to some companies whose bean-counter's knee-jerk reactions will now be to inject themselves in environments where the opposite of the author's viewpoint is the case and will now cause problems with productivity by micro-managing work to make sure the points observed don't happen ironically causing progress impedance in turn causing the problem they were trying to mitigate in the first place, which didn't originally exist. 🙁
Great article
Spot on. Why I left my previous employer
This is a great real world demonstration of David Graeber's Bullshit Jobs. I also think it's not just tech working that way.
https://www.goodreads.com/book/show/34466958-bullshit-jobs
This is not just in the tech world, check out David Graeber's Bullshit Jobs, it's an eye opener.
Very brave & unspoken insights on the fat that's existed in tech for a decade.
Spending a large junk of my career on the sales side of tech from startup through to enterprise, I can speak to the huge inefficiencies that largely exist in established sales orgs.
Excess funding meant aggressive headcount growth that lead to minute territories causing reps to scramble to hit quota.
With enterprise being notoriously slow cycles, there is simply not enough to 'eat' or work on day-to-day and subsequently justify these highly paid salaries on a per week/month output basis.
This is evident in only 50% of reps now hit quota. With an average base salary of $150-200k+ and commissions being paid regardless of quota attainment, you have many reps cruising in terms of bandwidth making $250k-500k easily to essentially fail at their roles (missed quota).
Of course they're all claiming endless 'prospecting' activity but any diligent rep has pilfered through their accounts by Q1 and the vast majority these days (if they're honest) have little belief in their territory's quota and subsequently earning capacity that sales-ops forecasted in their previous Q4 planning.
They are also privy to the honest insights into their territory's potential via the handover from the previous rep.
I've even interviewed for numerous roles where the first year quota matches their top performing, tenured rep with the strongest territory. You don't need a sales management or operations background to see how broken this equation is.
Sure there are the busy spurts with an RFP deadline or that large deal's milestones landing at once causing those long days and late nights of pitch revisions & demo rehearsals, competitor war rooms, proposal builds, management oversight & approvals and interdepartmental 'touches' on the deal.
But it's nowhere near a 40hrs p/w output averaged over the year for most reps.
Like the dev side, 'being busy' is often spent on the new age paper tray shuffle: CRM field population like the deal's next steps, management & internal huddles to update the business and endless forecast reviews & tweaks for Wall St quarterly reporting to deliver the #1 focus being stock appreciation.
Let's not forget all the best practises endlessly recommended whether it's the sales book quotes & mottos, podcast recommendations, make your own luck, glass half full/, manifest your President's club stage entry collecting your attainment award etc etc. Sure – we get it.
Instead of preaching such insights likely practised by solid performers anyway, reduce the headcount, increase the territory size and provide more to do. Stop wasting investor's cheap debt on expensive, under utilised, unproductive & underachieving headcount. As the saying goes: want something done…
True sales people are simple creatures and would happily work around the clock on genuine, needle moving actions leading to closed deals & more commission cheques.
Perhaps why Emmanuel's article resonated for me so much having moved to startup world. At least I had real purpose being truly productive delivering real contribution that will hopefully make a difference.
Meantime there is the fast growing model of fractional execs where a CEO / CFO / CTO / CRO / COO etc can be hired part time.
With the every growing self educating buyer accessing endless resource in every part of their decision making journey, I absolutely see this fractional model spilling into the sales sector.
That actually sounded very interesting to me. How could a young Business Graduate with a Bachelor Degree find such a job opportunity. I would love to get such a job with Wallstreet woring with the Chicago Exchance or BP.
I have found it far more difficult to simply get my foot in the door of a company that would allow me to use my education. Any ideas of companies in either Northwest Indiana or Southern IL?
I find it odd that you mention task bloating using agile. In my experience the opposite happened when it came to estimating how long a task would take. The way it usually happened was because the subject matter expert on the team would come up with a really low estimate Everyone else would assume the expert's estimate was correct, because they obviously knew what they were talking about.
Then when it came to the sprint if anyone other than the SME ended up doing the task then it would usually take them a lot longer than the estimate said it should take. And then at the end of the sprint everyone wondered why half the tasks weren't completed.
Great piece of work!
Working in agile. Do feel the same as you. But I am just started with career that too having low salary grade. So until I reach ur salary grade say good luck to me. Anyway I would love to read ur book.
Coaching response:
As an Agile Coach, it is my role to enable the change and training necessary to undergo a successful agile transformation that will have positive lasting effects on the way people work and contribute value to a product.
I have also worked in tech for over 20 years, for some of the world’s largest Media companies and I can tell you we all work extremely hard and have a lot to do.
As an Agile Coach, what I read in the article was not agile or any incarnation of scrum. It is nothing less than bad agile. And it is a very common occurrence.
These types of misconceptions arise when a company decides to call itself agile without taking the necessary steps at the appropriate levels of a company to transform in a positive way. It also happens when people are not properly trained in their respective roles. Or retrained in their new role. Once these Antipatterns infect a team or an organization, they are very difficult to pry off.
Task Bloating: was described an occurrence in planning sessions when teams often exaggerate the time and effort required to complete tasks. This behavior is attributed to a lack of dissenting voices in the team and the rigidity of the sprint cycle, which encourages task bloating to meet the predictability demands of stakeholders.
The article also suggests that task bloating is a standard practice in agile, which is another wrong assumption. It is not. Task bloating is a dishonest way to attempt to game a system. Goodhart's Law says, “That every measure which becomes a target becomes a bad measure.” I coach leaders on some of the pitfalls of specific agile “metrics” and their dangers to a team or organization.
Agile methodology is value-based on delivering working software in short cycles, and the purpose of task estimation is to ensure that the team can deliver a potentially shippable product incrementally by the end of each iteration cycle. The estimation process is not intended to provide an accurate prediction of how long each task will take or how much effort is required. Instead, it is meant to provide a shared understanding of the work that needs to be done; the complexities, risks, and unknowns – and to facilitate collaboration among team members. This is one of the reasons we estimate in story points or other types of relative estimation.
A competent Scrum Master knows how to elicit participation from a team and not allow a single individual to control the outcomes when a team votes on sizing a user story.
If a task has no stated value to the product, the Product Owner should not be bringing it to the team.
The agile process involves breaking down work into smaller, manageable pieces, which are prioritized based on their value to the customer. The team estimates the effort required to complete each piece, but this estimation is not a commitment. The team adjusts their estimates and plan as they learn more about the work and as priorities change.
If a team is held accountable for value-based flow and delivery as well as their commitments, then task bloating effectively disappears.
The article also suggests that the rigidity of sprints creates inefficiencies, which is a misunderstanding of the purpose of sprints. Sprints are not meant to be rigid, inflexible boxes that limit the team's ability to deliver value. Instead, they provide a framework for the team to plan and execute their work in short cycles. The goal of each sprint is to deliver a potentially shippable product increment, and the team should be flexible in their approach to achieve that goal. The team should continuously inspect and adapt their approach to maximize their productivity and deliver value to their stakeholders.
Another fallacy: “if someone announces they’ve completed an assigned task before the end of the sprint, in practice they are rarely assigned a new task until the beginning of the next sprint, leaving them completely idle, or they are assigned a filler task like “provide support to James on his task.” – this is an antipattern. If a person completes their work item early, they should then pull from the next item in the sprint commitment. If there is not an obvious choice, the Scrum Master or Product Owner will pull the next priority work item from the backlog.
It is a fallacy that new team members have no work when joining a new team, mid sprint. A good scrum master or Product Owner will help the individual integrate quickly into the team and initiate pair programming or other collaborative onboarding activities.
Scrum ceremonies are a way for the team to interact with themselves and stakeholders. They foster teamwork, and transparency. If the meetings are not facilitated properly, or if the team does not understand why these meetings are important, then retraining by the coach or Scrum Master is needed to reiterate the importance of what the team is doing. And steps need to be taken so that the team is engaged, heard and understood.
Agile is not just a list of buzzwords, it is a framework by which we deliver value. If value is not being delivered, then the team needs to reevaluate their workstreams. With a competent Agile Coach and appropriately trained Scrum Masters, the unfortunate antipatterns and ineffective practices outlined by the author can be overcome.
There is no such thing as "agile" except the adjective. There is a "Manifesto for Agile Software Development" (which consist of four values and 12 principles). And it just happened to be the word "agile" chosen instead of "flexible." The problem is that agile has become a misnomer to keep classical Taylor-ish command and control project management in disguise.
The terrible practices you describe are coming from the scrum. At least the following quotes make me think so. And fun fact: agile is not mentioned in the scrum guide even once, nada, zilch.
"The agile recipe also manages the day-to-day work" – if it refers to stand-ups, it comes from the scrum. None of the manifesto's principles states how exactly to organize work.
"agile has become a cult" – if you pick a framework that states to be agile, then by definition, it's not agile because there is a framework and boundaries. Agile is a lifestyle, and if the lifestyle is religion, then yes, agile is probably religion.
"they may hire an agile consultant to help out" – yes, usually, they are scum trainers or scrum masters, usually without any technical experience in coding.
"Questioning the agile recipe is seen as heresy" – probably scrum or SAFE is a recipe, but not the manifesto. At least these two frameworks are evil. Overall, any out-of-the-box framework is terrible because it beats the purpose of being agile—change is the only constant. You need to adapt all the time. None of the frameworks comply with it. And this is more or less what you describe in the last chapter, "What does truly agile teamwork look like?". But everything above is not about agile. It's about misnomers and abuse of the manifesto of agile software development.
I'm highly influenced by Allen Holub, who criticizes fake agile and what you describe from your experience.
Let me fix your about:
Emmanuel Maggiori, PhD, is a software engineer who helps companies build challenging software products from scratch.
Should be:
Emmanuel Maggiori, PhD, is a software engineer who does nada, zilch but write this blog (which took 3 people 5 days)
It is quit simple if a method like agile is working or not. The method (tool) has to make the work more efficient and the environment has to support the tool. Example: An agile method doesn't fit to a traditional project contract, the most used incentive targets doesn't support collaboration. But the most difficult one is how to scale within a company and doesn't kill effiency ( Death valley curve). The real problem is, if a method becomes a hype or a religion we tend to make it more complex. Sometimes it is interesting to read about case studies from companies which share there experience https://www.forbes.com/sites/stevedenning/2022/01/30/can-firms-succeed-without-managers-the-case-of-haier/?sh=f4bf10594d47
Finally someone said it so damn right, well done Emmanuel!
I loved seeing someone say the quiet part out loud. As a product manager, I have noticed this in about 3/4 of tech companies where I have worked. And yes, it is not so bad at startups.
Once I told a company as I was leaving, "The emperor truly has no clothes" – which is exactly what another commenter said. I hated the acceptance, and often deliberate hiding, of low productivity. In my view, this DOES occur at more places than insiders will admit and outsiders would ever imagine.
Agile and Scrum are not at fault. The way compnies do them is. The most common being "cargo cult" Agile or Scrum. Also staff are kept or made to look busy by maximising utilisation instead of optimising the outcomes and values delivery of teams.
Another common anti-pattern or anti-practice is the claim of self-organising where which work is assigned instead of pulled. This compromises team selecting work increments into their sprints and ultimately the benefits and outcomes of Sprint Planning.
One could surmise that the bulk of the problems as mention here were due a genuine desire to be Agile but is comprised by a fear of losing control about what and how people work, e.g. in Scrum. The transitional to Agile and Scrum had not been clearly understood, especially the roles of Scrum, and therefore the practice of Agile and Scrum values and principles were constrained by residual resistance and by a fear of letting go of the command and control practice, e.g. step wise or Waterfall mindset.
Neither Agile nor Scrum are easy no difficult. Each is a discipline that has to be mastered through mindful practice of being sincere to their respective values and principles in practice. Most companies fail this by overlaying rigid command and control practices over them simply because they want the best of both worlds. This may work in some situations but mostly creates other constraints and problems, the experience of Emmanuel and other developers and software engineers.
#Scrum.asterMindset
There's an old adage that "a bad work person blames their tools". Agile and Scrum are just tools of the trade, originally, for software development. They've since also been applied to other business areas – rightly or wrongly.
The thing is, neither Agile nor Scrum are easy, although on the surface it seems easy enough to just get on with it. Agile and Scrum are mindset tools, very different from the tools that developers and software engineers are used to, very different to many of the tools that businesses and corporations have used. Agile and Scrum are disciplines that need to be mastered and it doesn't happen in a jiffy.
The unfortunate thing is in the chase and haste to be fast, to be first and to appear responsive (to whom??), too many have jumped onto the bandwagon paying mere lip service to Agile and Scrum. Many still won't make the necessary change away from command-and-control structures, from waterfall or stepwise ways of working and away from tasking work to pseudo self-organising teams.
Many of the issues discussed by the author above appears to stem from a FOGL – Fear of Letting Go – of what had worked for us but may not continue to do so and a "cargo-cult" practice of Agile and Scrum. Many managers within the settings of Agile and Scrum have not fully understood their roles – they have not given up managing people, demanding status updates just because they can. They do not see themselves as not owning but sharing a process of Agile and Scrum that's within a continuous cycle of transparency, inspection and adaptation – without authority. They fail to understand that instead of trying to maximise utilisation and outputs, they should instead be optimising outcomes and values.
In any Scrum team, there is only a Product Owners, Scrum Master and Team, nothing more – not project manager, not team leader, not whatever. To do so would be practice a cargo-cult version of Agile and Scrum. It's no wonder so many are led to the alters of and so few blessed with the success they expect and we haven't even touched on the values and principles, the anti-patterns to avoid, the pull-push conundrums of work, the pace and velocity of sprints, let alone the essential artefacts, ceremonies and rituals – all happily cast aside while the focus is on doing but not being Agile and practicing Scrum as the disciplines they are meant to be.
This reminds me of the race between the tortoise and the hare. The winner had the discipline to stick to the plan of winning the race.
#ScrumMasterMindset
I work in technology for one of the biggest banks and agree with what you wrote here. With that said, I think you are not seeing what this really is all about. I strongly believe, this is about money…BIG MONEY. Every low to middle manager has to justify their petty little existence. Everybody knows that if their staff will do tasks in efficient times, very quickly sr management above them will ask the dreaded question…."what value are they (managers) are adding?!". Having overinflated team is also good for them because they can lay off 15% of staff to "show budget savings" while not loosing any productivity. Again makes them look good.
My job is also boring and at times I feel I am wasting my life because I am capable of adding so much more and engaging my brain so much more. Instead I chose to play their corporate game, get paid well and spend the money on things I really love…all of which are outside my day job.
Same here especially since i am 50
See Graeber’s famous book _Bullshit Jobs_ to put this in a larger context. Great contribution to the phenomenon.
https://davidgraeber.org/books/bullshit-jobs/
This has been the best summary of my corporate tech experience which I always fail to communicate to folks in the outside.
If I can share the nightmare this has been for someone who came out of a boot camp and directly hired by a large financial company. How after 3.5 years as a developer I can say I only have about 10 months of inconsistent coding experience :
Interview :
My resume consists of a current “modern” tech stack. I am quizzed on this stack specifically. In addition, I have no concept of what on call is. I am under the illusion that I will be working in this tech stack.
Note : each team is really a reorganization of the same people in the same area with the addition of integrating with a large army of contractors.
Team 1 :
My mentor is too busy. He is not familiar with my tech stack. Everything is legacy code. I have no clue how this framework operates . No documentation. I spend the next 1.5 years doing testing work, changing production passwords. I lose the knowledge I paid so dearly to gain from boot camp.
Most of the team involve Mainframers. I am forced to support this technology once every 7 weeks for a full week 24/7. Processes randomly break at 1am, 2am, etc. No sleep during these weeks.
Team 2:
Really Team 1 but now in agile form. I become the maintenance guy once more. I work with a gifted dev on another team to write a program. By now, my coding skills have eroded to a point of embarrassment. This coding experience dries up in 3 weeks because we finish it. The remaining work is cutting the red tape of getting this to production which lasts 4 months. I am on this team for about a year.
Team 3:
Large vendor is hired to modernize our legacy code. Work that I should have gained experience from is handed to them. On this team, I am assigned real coding work. I am happy and sad. Happy because after 2 years I get to work on this code. Sad because I have forgotten the basics. I become the maintenance guy once more. No mentoring opportunities with our contractors.
Team 4:
Here I begin to work on more code with proper mentoring at first. Later, my mentor and fellow experienced devs grow increasingly frustrated with agile and the vendor company we are forced to work with. Actual coding work is scarce and inconsistent. I learn, then I forget. Mentor soon becomes too busy for me.
At this point, I still have the same on call as before. Now with the prospect of every 5 weeks.
Result of
3.5 years in : no opportunity for promotion. No compensation for on call that requires a sleepless 7 day house arrest.
I am at whits end. Every job I’ve had I have pulled myself up. Not in this industry. No matter how hard I try to hold the sand in my hands, it just slips through.
Now I feel like I must start over from the beginning as if I am looking for my first job. We all know that this first job as a dev is impossibly hard to acquire with little experience to show. I’ve been having large debilitating anxiety attacks that force me to use sick days.
I’ve never felt so trapped in my life.
Lessons learned : your top priority as a dev is to stay interview ready. Then you must ensure your current job allows you to gain the needed experience. No experience, no promotion. If not, you must seek work elsewhere. Stay interview ready please.
I wish I knew who to reach out to in this industry but I feel siloed. I know there must be more similar stories like mine but where are they?
Take care
Problem, as always, is crushing bureaucracy. Agile is just the label it wears on its forehead right now. Soon that label will say "devops". All these concepts start with some spectacular good result, that is then theorized and put into words, then eventually stirred into the massive soup of broken dreams and corrupted visions that is life in employment.
The agile principles are great, assuming the goal is working software. But if the organisation really has other goals (charge customers for bodycount working on projects, amass behavior data about some market sector, impress other parts of the organization) then agile principles and practices aren't going to make the company suddenly realise that your ideals are the right ones and pivot.
Emmanuel you are my hero! Finally someone calling out all this rubbish. The best time in my career was with a small startup company, trying to get a product off the ground. Small team, and everyone had a job to do. At one point, we called in an agile consultant to advise on using Scrum to run our development. After a while, the process was binned and we went back to doing it our previous way. The sprint ceremonies were seen as overbearing and too time consuming.
I've tried criticising this before, and got nowhere. Now I don't bother, and just go along with the show, picking up my pay each month.
The danger with staying idle is, you don't accumulate valuable experience. Always watch out for that, because at some point in your career you'll have to become a senior or principal developer. Just make sure you can back it up with years of experience.
Great piece but missed the most important cause: far too much money floating around mainly due to ad revenue.
Lack of productivity also comes from those numerous useless (to say the lest) excel-based mid-managers.