Companies Agility: A report by Scrum Alliance

A couple of days ago, the Scrum Alliance published a report called “The Elusive Agile Enterprise: How the Right Leadership Mindset, Workforce and Culture Can Transform Your Organization”.

In this report, Scrum Alliance and Forbes surveyed more than 1,000 executives to determine how important is the agility in an organization, its degree of success in transformation efforts and how much progress the companies have implementing this type of frameworks. Among all the respondents, there were 10% that are from Latin America (I’m from Costa Rica), so, I would have liked to read the results from this area, but the overall results are equally interesting. Also, those executives weren’t only from a technology oriented company, but also from other areas, however, I would like to comment on some aspects of the report from an IT perspective.

Geography distribution

Personal creation based on report data

One key thing to notice is that they want to measure the agility of a company, not the agile framework they are using (still, there is a 77% of these companies that leverage Scrum as their main framework). But, what is agility and what is being agile? agility is the property of an organization to respond to market changes and still deliver value to the customers, whereas agile is an organizational approach and mindset defined by the values and principles detailed in the Agile Manifesto.

Based on this premise, the report defines several benefits that are obtained after achieving a high level of agility within the company, such as:

  1. Faster time to market:

  2. Faster innovation

  3. Improved financial results

  4. Improved employee morale

Organizational changes

To achieve the mentioned benefits, the respondents affirmed that there were several changes that needed to be applied in order to redefine the company’s processes and reflect the agile mindset. In the report, Scrum Alliance mentioned the top 7 changes, but in order to not spoil the article, I would like to comment some of them with experiences that I’ve seen first-hand to be applied, and succeed.

  • Introduce an agile mindset

Being agile is not only about using Scrum (or any other flavor), perform some of the ceremonies or events that are required, and deliver software. The processes definitely are important, but the people are equally, or more important.

Having an upper management that believes in agile methodologies and promote them, facilitates the transition heavily. Implementing a change from the bottom-up, is nearly impossible, but when it comes from the higher grounds, it’s a quick, fluid and flexible process.

I worked for one organization that had Scrum implemented partially, but not the agile mindset. They performed some of the ceremonies when possible, defined their sprints to have incremental products, but in the other hand, had a requirements process that resembled a lot to the waterfall scheme and the releases were done until the end of the development. The development process was really flexible, and we were able to adapt to some changes, but still there were some gears that didn’t feel right.

The main problem was that the customers weren’t involved with the development process at all, and they were expecting that we exceeded their expectations. This wasn’t going to work, and in fact, it didn’t!

To remediate this, we tried to create backlogs that allowed customers to give their inputs and define what needed to be done. It worked in some things, but still there wasn’t much involvement from the customers, even though that we invited them. At this point, there wasn’t a single point of authority, someone that could work as the Product Owner (something fundamental), so we had to facilitate its creation.

To do this, we talked with the managers of each area that were part of the application and explained them what changes were needed and the possible benefits. They trusted us, and a new group was formed that worked as the Product Owner; this group consisted on a representative of each area, and even though this is not the regular Scrum process, it worked much better and we got much more feedback than before.

The agile mindset was introduced, little by little, to obtain success.

black-brain.jpg
  • Create incentives that promote agility

In another organization, the agile mindset was much better. Some processes were already defined there, customers agreed with the methodology and got involved in it, the ceremonies were executed and the benefits were visible. Even so, this organization needed some optimization because the processes weren’t applied uniformly across all development teams.

To solve this, a group of people from the organization decided to create a group to lead all the Agile efforts, and the first big task was to standardize the process across every team. Among many options, the one who won was to create a contest, but not a simple one.

The contest consisted in having all teams follow the process of the organization and the Scrum best practices. There were 4 phases and for every phase, a common goal. Each team earned points depending on how good the practices and the process were followed. For example: for the first phase the DSUs were the main goal, and a team earned one point for every DSU done in less than 10 minutes, using the parking lot technique granted extra points. For the second phase, backlog grooming, sprint planning and sprint retrospective events were evaluated. The next phases evaluated customer involvement and product deliveries.

At the end of the contest, a winner was selected from all teams and some prizes were given, but the real outcome was that all the teams managed to follow the same process and practices.

  • Train workforce

As I mentioned before, people are the most important factor when there are changes in any organization. People will determine how quick the change is applied, but there will always be blockers that are needed to be managed, for example: resistance to change, lack of communication, ignorance.

In my opinion, training people to work with Scrum is mandatory, and there are really clever activities that embodies the agile mindset, demonstrates how Scrum is supposed to work and make people enjoy the time spent learning about it. For example, I’ve been in trainings that uses Lego to create a city, building a tower with marshmallows and spaghetti, but the most recent training that I had was using stacks of cards to simulate a development process.

Key Findings in Report

In the report, there are some key findings after all the survey was executed and analyzed, all of them are interesting, and similar to the organizational changes, I’ll comment in just a few.

  • “Many organizations are adopting an ad-hoc approach to agile: 21% of respondents use Agile when/where needed, and 23% use it within specific functions. However, adoption needs to be enterprise-wide (and consistent) to realize real results.”

I agree that the adoption must be enterprise-wide, and I want to believe it, however the reality is not that. As the same survey expose, the number of companies that have adopted agile in every area of the company is less than 10%, and that’s because it’s not a simple process. Implementing an ad-hoc approach is a middle ground solution, that will reduce costs and obtain benefits.

Agile Adoption

Personal creation based on report data
  • “Not everyone eagerly embraces agility: Longtime employees (29%) are the biggest detractors of organizational agility and may stand in the way of widespread adoption. This is a prime opportunity for senior-level executives to address employee concerns and shift mindset.”

It’s true that longtime employees are one the of the biggest detractors, but that’s because the resistance to change is stronger in them (Star Wars jokes aside :). However, I wouldn’t limit this to just a group of people. There was one time that I had someone assigned that didn’t believed in Scrum, and it was due to a bad experience before where the execution was done incorrect; for example Sprint Plannings of 3 hours, DSUs of more than 30 minutes and other bad practices.

  • Many organizations eliminate hierarchy in the hopes of increasing agility: 44% of survey respondents have introduced a flatter structure to become more Agile. But that may be premature; Agile is about creating the right dynamics for teams to iterate quickly, not simply moving boxes around on organizational charts.

For this one, I believe that changing the structure is good, simplifying it and making it more easy to work with. However, I don’t believe that you need to flat every structure. There are frameworks like LeSS (Large Scale Scrum) that help making organizations more lean and scale scrum across all of it.

Summary

Moving to an agile process is not easy, evidenced by this survey. There will always be changes required, trainings needed and a really good management. If you are interested, read the whole report from the Scrum Alliance, there are really good insights to incorporate to your own company.

If you have any comment, don't hesitate in contacting me or leaving a comment below. And remember to follow me on twitter to get updated on every new post.

Technical Debt: A Practical Approach

Today, I'm gonna step away from the Amazon tutorials, I think I have spent enough words on it, so it's time to move to other areas, and as you can see from the title, this post is going to be all about technical debt.

For the last couple of years, this topic has been a rollercoaster (take a look at the google trends chart here), but rather than just writing about what is it, as many people already do, I will write about how it can be combined in a regular scrum development process. 

What's Technical Debt?

Before actually diving into the main topic, I need to give a little bit of background. If you are not familiar with the concept, this section might interest you, otherwise, it's still a good read.

The term technical debt is not something that started appearing in the web recently, nor is something I created. It was coined by Ward Cunningham around 1992 in his article titled "The WyCash portfolio management system" [1]. 

If I need to summarize the definition, I would say that it's the result of any decision that you make in software development that has a visible result and a secondary not-so-visible effect.

In the financial counterpart, any debt generates an interest that makes the debt increase over time. At some point, you might pay the debt, but not after incurring in a lot of expenses. In this scenario, the decision was to ask for a loan, and the secondary effect was the interest that made you pay more and more over time.

If we come back to software development, a perfect example is when you are comparing two solutions, the first one is a quick and dirty solution, and the other one takes more time but provide a cleaner result. If you choose the first one because it reduces the time to publish a product, or there is a hard deadline, then you are starting a loan and the interest starts to run. The interest can appear in many forms, for example, it could be the "Spaguetti code" that is created, or the high complexity of a method that will require extra time when doing any support activity, among other possible outcomes.

Incurring in technical debt is not bad by its definition, the bad part is when the debt is not handled properly. So, how can we handle it?

Managing Technical Debt

In 2014, a group of researchers performed a simulation to determine what's the best strategy to manage technical debt in an agile environment. They came up with a few strategies and decided that the best approach is to list the technical aspects that are most important to the developers, monitor those aspects using automated tools and remediate any failure based on thresholds limits. Sounds like a lot, I know, and that's why they explicitly state that, even though this is the best approach, it would require resources and time that are not always available. If you want to read the full article, you can purchase it from here or download it if you have an IEEE account.

Last year, to complete my master's degree, I had to lead a project and decided to focus it on managing technical debt. I already had a strategy in place, based on what these researchers simulated, I just had to prove that it works.

So, I started by doing some research to see if there were other efforts of people doing the same as me and chose a static code analysis tool among the ones in the market. We opted to use SonarQube to execute that part in a continuous way, and created some custom alerts whenever a metric failed based on some threshold limits. Sonarqube, in my opinion, leads the market not only because it is free and open source, but because the number of functionalities and languages t supports exceed any of its competitors.

After that, we altered our development process. Added weekly code review meetings to check the progress on the technical debt metrics, allocated time (no higher than 20% of 1 resource per week) to remediate failing metrics and established a more standardized code review strategy. 

We evaluated these changes during 5 sprints of 1 week, where we added new features and worked on remediating failing metrics. At the end of those sprints, we obtained the following results:

  1. Defects reduced in 30%, according to Sonarqube.
  2. Days of technical debt reduced in 25%.
  3. Security vulnerabilities reduced in 86%.
  4. Reduced cyclomatic complexity in 129 possible flows accross the application, resulting in simpler classes and functions.

Also, 4 months after the changes were applied, we noticed a reduction in the number of support tickets by 60% and the resolution time was improved in 50%, saving time in support activities and reducing costs. 

Even though the whole project took other areas into consideration, like process engineering and automated testing, we ended up adding technical debt management activities to our development process effectively. So, if you want to read the complete document about this project, in spanish although, it's publicly available from here.

If you reached here, I hope you enjoyed this post and if you are interested in applying something like this in your organization, don't hesitate in contacting me for more information.