Stormpath Kanban

Last year, Stormpath made the big shift from Scrum to Kanban. While we love Agile principles, the Scrum process wasn’t working for us. Kanban made our team more efficient, happier, and increased our focus on quality software. More importantly, it has become a core part of our company culture, and is now used by non-technical teams like Marketing and HR.

Kanban software development focuses on continuous delivery and drives high efficiency by limiting how much work can be done at once. Invented by Toyota and modified by David J. Anderson for software development, Kanban can have a huge impact on modern teams delivering cloud software in continuous environments.

Scrum, It’s Not You, It’s Me

Agile principles have always been important at Stormpath, and we use them on every team from Engineering to Marketing. Standups give our team transparency and eliminate blockers. Breaking work into minimum viable features gives our engineers a sense of progressing successfully. These small-step iterations – driven by user stories from our customers – help us improve the Stormpath API as we learn more about how people use it.

The sprint-based Scrum process, however, wasn’t working for us. It was too rigid, created too much overhead, and started to cause burnout on our team.

Overhead
We’re a start-up moving extremely fast with a very limited set of resources. Scrum had a lot of overhead: sprint planning was a half day event for our entire engineering team, plus preparation for the meeting for our product team. It made everyone feel involved and helped with coordination but that’s not a pain point for us. We’re already very collaborative. Sprint planning forced engineers focused on one component to sit through a discussion of unrelated components — for hours. Additionally, we would get into long (and often fruitless debates) about issue priority and scope that would drag in the whole team.

The Myth of Estimation
At its core, Scrum tries to drive productivity by imposing a time limit on work. A sprint is a bucket of available work hours that gets filled with issues based on time you think it will take to complete them. Like most teams, we have some over-estimators and many under-estimators. I can tell you with absolute certainty that 100% of us are bad estimators. Time spent trying to divine and debate a feature’s size felt like wasted time because we were probably wrong anyway. Moreover, what did it matter? If it’s a priority for our customers, it needs to get done regardless of size and, as a rule, we were always breaking it down to its smallest viable scope.

Scrum assumes you’re good at estimation

Bad estimation – rampant for many Agile engineering teams – caused a lot of problems. Issues turn out to be bigger than expected, now the sprint is overloaded. We kept finding ourselves jerry-rigging scope – even more planning meetings – to match the artificial construct of a sprint, gaming the process to very little business benefit.

Change Management Soup
Need to move fast and operate in an environment of uncertainty? Scrum complicates that. Once sprint planning is done, making changes to the sprint triggers more planning meetings. Sure, you want changes to work-in-progress to be painful as a deterrent to a meddling business team, but what about changes to work committed to but not yet in progress? What if a big bug is discovered? What if a customer call makes us rethink some part of a feature committed to in the sprint. These common situations created more meetings, frustration, and complexity in our process.

Morale and Burnout
Flawed estimation constantly led to unrealistic expectations and the team would miss their sprint commitments. The retrospectives would then turn into critiques of what we did wrong which often came back to bad estimation, priority changes, and/or feature specs. Business felt engineering missed their targets, Engineering felt business kept moving the ball. Everyone’s morale suffered.

Scrum Sprints

The two week iterations were great — we felt velocity because we were always deploying something. But our productivity quickly became spiky. Crazy hours at the end of the sprint in order to get it all in a release because “we committed to it” were followed by a productivity crash the next week so people could recover. Shortening the sprint cycles might smooth out productivity, but the planning overhead limited what could meaningfully get done in a shorter sprint. Lengthening sprints reduced overall pressure, but also exposed us to greater change management risk.

We started looking for alternatives.

Enter Kanban

While in business school at Stanford, I read a lot about Toyota’s “Just-in-Time” or Kanban manufacturing process. It really is a marvel of simplicity and more importantly, it supports our velocity goals. It’s designed to be flexible, so priorities can change pretty quickly. With one quick google search I found LOTS of articles, books, and videos on Kanban for software development. Our Atlasssian Suite even supported it.

What is Kanban?
Kanban is a continuous flow process: issues enter the queue and then get “pulled” through a series of steps in the development process. Kanban is often visualized on a Kanban board and each step is represented by a column. Issues can further be organized in rows, called “swim lanes”. Here at Stormpath, we chose to use swim lanes to represent issue priority – the further down in rows you go, the lower the issue priority. Core to Kanban are Work-in-Progress (WIP) limits that cap the number of items that can be simultaneously worked on within a given stage of development. Workers pull from left to right and can only move work onto the next stage if that column has an open WIP slot.

Kanban Swimlanes and WIP Limits

While Scrum drives productivity by limiting the work time available in a sprint, Kanban drives productivity and velocity by limiting the number of active, concurrent issues. Time estimation is no longer part of the process.

Velocity is still a major focus but now we optimize at an issue level, now called Cycle Time. Items to the right of the board, those furthest along, get higher priority. This creates an important core value on the team. Finish what we’re already working on, before you take on more work. Stop multi-tasking. Stop context shifting.

And all the churn on priorities and requirements from the business? Gone. With Kanban, the business owns the “To Do” column, but Engineering owns all the rest. Business can go hog wild with changes while things are in To Do, but they can’t change an issue without a lot of effort once it’s moved out into another column. As a result, the engineers now feel insulated from any changes from Business prior to work being picked up.

Happy Team
Kanban made our team very happy. Whereas Scrum felt like a prescriptive, inflexible formula imposed on the engineers by management, Kanban is flexible, process-driven and owned by engineering. Our team implemented and developed how we use Kanban at Stormpath, which gave them ownership over the process and its success. The transition was very smooth: about 2 days worth of discussion and planning.

What’s more, a continuous flow model means there’s no “deadline” but instead an ever-present pressure for “velocity.” This significantly reduced the burnout we were seeing. Even better, overall productivity went up. We weren’t spending half a day every other week with the entire team planning sprints; we weren’t gaming a feature’s scope to fit into a sprint; Kanban reduced context switching, which improved both productivity and code quality. Furthermore, the Kanban board itself reduced project management overhead because everyone could see it and know what was being worked on, by who, where it was, what needed to be worked on next, and where the bottlenecks were.

Structural Focus On Quality
A major, unplanned benefit of Kanban, was a focus on software quality. Under Scrum, there was always a temptation to cut corners in order to get things into the Sprint and then shove the missing work into the Tech Debt backlog, like skipping a code review, approving 70% code coverage when our standard is 95%, etc. Tech Debt kept getting bigger, and we missed some elements in designing new features. At Stormpath, we take code quality extremely seriously, so the constant risk made everyone uncomfortable.

Under Kanban, steps that ensure code quality are baked into the process. Since we’re rarely working against a deadline, there’s rarely pressure to skip these steps. In fact, because engineering owns the process, there’s a strong pressure to stick to the process. Engineers will fight to make sure we have 100% code review and 95%+ code coverage now.

Near Zero Overhead
Sprint planning? Gone. Project Management? Minimal. Estimation poker? None. Scrum Master? Nope. We still have stand-ups, but those don’t take up a lot of time. Engineering spends most of their time building product. Product owners decide what goes in “To Do” next, and engineering takes it from there.

Kaizen vs Retrospectives
In Kanban, Scrum retrospectives have been replaced with Kaizen meetings (kaizen in Japanese means “continuous improvement”). The difference is subtle but powerful. Here at Stormpath, we have Kaizen meetings every two weeks; engineers and product owners discuss how the process should be modified to improve velocity and quality, while reducing overhead. Instead of looking back and critiquing what we did, all the energy is forward-looking and focused on process improvement. Defensiveness down, collaboration up! Everything in the process is fair game and any new idea is treated as an experiment to be tested. Most ideas are given a try, and very few are shot down. As a result, kaizen meetings have an upbeat, creative feel and the process runs very smoothly.

Kanban isn’t perfect

We’ve successfully moved to Kanban and are really happy with it, but it’s by no means a perfect solution.

The Software Tools For Kanban Are Nascent
We use and love Atlassian Jira and Greenhopper. They have some nice Kanban features, but their Kanban board is relatively new and lacks the flexibility we expected (for example, having WIP limits in swimlanes). As a result, we’ve had to conform parts of our process to what was doable in Jira and Greenhopper. This is unfortunate, because to get the most benefits from Kanban, you should be able to model your board to reflect your team’s particular needs.

Loss Of A Systematic Focus On Urgency
With Scrum, that sprint clock ticks loudly. Everyone feels the pressure to get their work done before the sprint ends. Urgency is baked in. In Kanban, urgency is more abstract and doesn’t have quite the same emotion pressure. We’ve had to impose a sense of urgency in our team through culture and management—harder but hopefully healthier.

Keep Calm and Kanban

Kanban’s power comes from its focus on getting fewer items out faster through work in progress limits. Like anything, it’s not perfect and it may not be right for everyone in every situation but ultimately, we ended up with a happy team, higher productivity, less tension between the business and engineering, higher quality software, and A LOT less overhead. Win!

Interested in Kanban? Here are a few helpful links:

  1. Agile Chalk Talk: Kanban and Scrum https://www.youtube.com/watch?v=Jx6_E5XxqEo
  2. David J. Anderson’s Kanban Book: http://www.amazon.com/Kanban-Successful-Evolutionary-Technology-Business/dp/0984521402
  3. What is Kanban? Kanban + Scrum. http://www.ontimenow.com/training/what-is-kanban
  4. Do Agile Right (Kanban): https://www.atlassian.com/agile/kanban
  5. What is Kanban? http://www.kanbanblog.com/explained/

Like what you see? to keep up with the latest releases.

  • Alex Salazar

    Good catch! We’ll have to force our designer to watch episode 1 as punishment.

  • Alex Salazar

    PA, I agree with your comment. The big improvement was the move away from timeboxing and rigid process and into something more flexible and flow based. Kanban its self is not a silver bullet and we have issues with it too like managing urgency.

    However, Kanban gave us a tested framework for that kind of process as opposed to us trying to figure it out ourselves.

  • Alex Salazar

    Yep, we’re actively considering a move to another tool that supports Kanban a little better.

  • Alex Salazar

    Thank you for the post Juraj.

    I agree that our Scrum process wasn’t ideal and we tried to adhere to “proper” scrum many times.

    The some of the recommendations you list are great ones. However, our team was already small so splitting would have been counter productive. Adding a seasoned Scrum master would have either added a lot more overhead or taken up a hiring slot we wanted an engineer for.

    We tried to learn from bad estimation but it didn’t’ seem to get better. Maybe we’re dense 😉 We tried all kinds of tricks from point to t-shirt sizes. Hell I even a read a whole book on estimation and scoured the internet. To this day, I’ve not heard one person say they have a good estimation process. Just a lot of consultants talking about how they can help.

    So yes, there were a lot of things we could have done to fix scrum. But after a while it all felt like bandaids to a process that just wasn’t right for us.

  • Alex Salazar

    Cool. I’ll check it out.

  • Alex Salazar

    Checking it out as we speak.

    • Alex Salazar

      One quick point of feedback. Consider going to a “Freemium” model instead of “Trial” model. It’s the way of the world for devs these days.

  • Alex Salazar

    Hi Jonny,

    You know, we were really concerned about that too. In fact, still are. But it came down to this- Are we trying to become process experts or build a great user management product for devs. We went for the latter and decided to stand on the shoulders of others for the former.

    I’m sure something better will come along in the future and we’ll probably hop again. But it’s the price we pay for being lean and focused on our core business.

  • Alex Salazar

    On the WIP limits: You may consider putting them back it. We’ve received a lot of utility out of them. 1. They reduce overhead because no one has to police or, hell, even remember what the number is per column. 2. It highlights (literally) the bottlenecks. 3. Since we change our WIPs occasionally through Kaizen, it helps communicate the current process.

    On the quarterly objectives: Great point. We’re just now starting to do that. Feels a little bit like bringing Sprints back in but we’re fighting hard to keep it light and malleable.

  • Alex Salazar

    We opted out of a physical board for two reasons. 1) we have some remote engineers so a digital board was easier to collaborate on. 2) The issues represented were already in digital ticket form so it was easier to keep it all in one place.

    A digital board reduced the omni-presence that a physical board has so we’re going to play around with a few experiments.

    On the estimation: We tried that it and felt that it was more of a band aid than a solution. And created a few unintended consequences. For example, if we did complete the sprint early and had slack time, then proper Agile had our engineers fixing tech debt not building new features. Too much focus on tech debt and we become unbalanced on product dev. Sure, we could tweak our tech debt process but then its another band aid that now forces us to determine when slack time is “natural” or “articifial”. etc etc. Not long after, our Scrum process would be a patchwork and then the Scrum Masters would be telling us that all our problems are being we’re “doing it wrong”. This is the cycle we were living and decided to hop off.

  • Alex Salazar

    Hi Foster,

    Good questions.

    You know, we haven’t had a problem on the meeting side. Sure, someone has to actually put it on the calendar (our CTO), and someone has to break up rabbit hole conversations (who varies) but its pretty democratized and works well. The agenda setting is an interesting one. Standup agenda is well known. Kaizen agenda is set by the engineers themselves– we try to have them send in their topics a few days before.

    In terms of larger teams, I don’t know. We’re a small team all working on one product.

    The business piece is a little more nuanced. Since we’re a product company, the business exists to sell/market the products and features coming out of engineering– they’re downstream. As a result, all that really changed for them was expectation settings. Regardless of whatever date we gave business, they were going to wait until it was done anyway. Now, that reality is explicit instead of implicit. We were able to do this easily because we’re a product company and the two founders (Les and I) were advocates of the move.

  • JDT

    Agreed. All of the negatives you mention should have been addressed by your Scrum Master, and if not there was your problem. Perhaps you’re not in an environment that contributes and interfaces with other products and customers, but this is where estimating comes in to play. Committing work to those who depend on you via estimating and historical data…again, something your SM should have been guiding the team on. Don’t get me wrong, Kanban is a great framework also, as I’ve had teams run Scrum, Kanban, or Scrumban – but as someone else mentioned, you’re just not doing it right (and not in the typical way that phrase is used haha).

    • Trevor

      What do you do when you fail to finish tasks/stories for those other teams who depend on you? I’ve always had a hard time understanding sprint timeboxing because every group I’ve ever worked with has either moved unfinished work to the next sprint, thus making the timebox meaningless, or they cut-corners, thus reducing quality. What does your group do in this situation?

  • Gary Nickelson

    Great write up. Our team uses scrum and find it equally as frustrating. We’ll have to look into Kanban to see if it will help out with our workload. Thanks!

  • Robert Pipkin

    Alex, I was curious about your release model using the Kanban approach you describe above. Is it truly continuous deployment when items are done, or do they get queued up to a rhythmic release schedule (say, every week)? What we’ve found with our Scrum approach when a story doesn’t get finished, the minimal viable chunk is checked in, and the remaining work just goes into the next sprint. The developer just continues to work on the story for the next sprint.

    Anyway, thanks for sharing your experiences. I’m wondering whether the novelty or newness of the Kanban transition lent itself to the happier team. The mere fact that they felt more in control of their process could be a big part of that. How did the business folks react to the process changes?

    • Alex

      We’ve kept a Scrum style release schedule (roughly every two weeks). The good part about scrum is that we are able to decouple devs from deployment. We’ve held back from continuous deployment because we want there to be a hard QA review on everything before it hits production and doing it every time there’s an completed issue creates too much overhead for the team. 2 weeks seems to be working well enough at the moment but we sometimes do “hotfixes” if there’s a production issue or if we have a hot feature we want to get out.

      • Robert Pipkin

        Sounds very much like the place we now find ourselves. We may have to give the kanban approach a try. Thanks.

  • RMB

    Alex, you mention that you were “…rarely working against a deadline…”. What are your suggestions for Kanban in a deadline driven environment, or is Kanban even an option in such an environment? Also, is WIP generally driven by resource availability? Is it one WIP item per available resource?

    • IMO, a deadline driven environment is more of a problem than something to be adapted to (at least from a development standpoint). In my experience, most deadlines are totally arbitrary or the result of somebody in a marketing role providing a release date that nobody ever committed to or was able to guarantee.

      That a big part of the continuous delivery approach to things in the first place. New things come out all the time. There aren’t hard deadlines because updates and improvements are constant.

      The big question is simply this: If the deadline comes and the feature isn’t ready yet are you forced to release it in a broken state or do you push back until it’s ready?

      If the answer is that you release it in a broken state, I’ll challenge the competence of management. If the answer is that you push back until it’s ready then the reality is that there wasn’t a real deadline.

      “Real” deadlines are VERY rare. Arbitrary deadlines are extremely common. Eliminate those and you’ll improve your entire process.

    • Alex Salazar

      Hi RMB,

      Kanban in its purest sense has no notion of a deadline. But again, Agile Kanban is meant to be very flexible so if your team needs a deadline for a legitimate reason there’s nothing stopping you.

      But staying within Kanban, what happens if you deadline arrives and your project isn’t completed? Do you break your process and just ship what you have? That’s not realistic. Most likely you’ll push the deadline. So, what was the point of the deadline?

      Well, there is value in the form of pressure and urgency on the team. The team may work more hours or cut scope. It works but feels a little false to me. We struggle with this here at Stormpath, too. There must be a better way to instill urgency on the team… if you find something better let us know!

      On WIP, yes, it is based on resources. Now the correlation between the number of resources and WIP limits is art not science. A good benchmark is 1.5 per available resources but this is something you should re evaluate every two weeks until you’ve found a fit. Maybe your tasks are typically big epics in which case a lower WIP is appropriate or maybe they’re really small granular tasks so higher WIP.

      If you feel your team is too spread thin, then lower WIP. If you feel you could move an issue faster if you just dog piled more, lower WIP. If you’re having a lot of people idle and things don’t feel like they’re flowing, then raise WIP if you don’t actually have a resource bottleneck issue.

    • Most Scrum teams try to restrict both scope and timeline. This is unreasonable. You can fix scope and flex timeline (which is what most organizations actually end up doing if they’re honest with themselves) or fix timeline and flex scope. Kanban supports either of the latter approaches perfectly.

      In a deadline centric environment, you set a ship deadline and when the deadline arrives, you deliver what’s complete. There’s nothing in Kanban that prevents you from choosing a release date that everything gets pushed to production.

  • Michiel

    Take a look at the Theory of Constraints from Eli Goldratt. It’s based on Toyota’s production system and more optimised for processes. It forces you to work with goals and helps you decide what to work on to finish in time. It’s an evolution from the Kanban. http://en.wikipedia.org/wiki/Theory_of_constraints

  • Peter

    I think you should be careful in saying Scrum is braindead. All software development methods have a due date, Scrum is almost there (and passed for some), but when it came, it was a minor revolution. Same will happen with the next generation of “software process”…

  • Vijay

    Excellent article!! Calls out the specific problems with Scrum, especially the time boxing and uselessss breaking up of stories just to fit the box. Kanban seems fantastic for startup kind of environments and definitely a reiteration of Agile principles. Scrum has a bit too much of process and at time feels like tail wagging the dog.

  • Angelo

    HI Alex, first of all thanks for sharing your experience. But Using scrum every day the only big difference and obstacle for me it is still the top down hard deadlines. How would you apply Kanban with Deadlines ? Your team work like in continues delivery , but this is possible as well in Scrum. Thanks.

    • Alex Salazar

      I honestly don’t know how you would do but I’m sure someone has thought through it. It is probably the main challenge with moving to Kanban.

      We specifically, moved away from deadlines because they were ultimately fictitious. What happens if you’re going to miss your deadline? Do you just stop? Do you rescope? Or do you push the deadline? It’s never really the first so we just focus on reducing scope as much as humanly possible and planning all the downstream business tasks when we’re pretty clear on when it will be delivered (with some buffer).

      We do have target dates for things but its more about pressuring the team to put in more hours to try to get everything in under that date. But we manage that more through culture, motivation, and leadership than we do through hard, do or die deadlines. And more importantly we handle target dates outside of Kanban– its not a part of the process. Just something we do occasionally when we REALLY need to.

      That all being said, deadlines and target dates all exist to instill a sense of urgency. In pure Kanban, that can typically be achieved in the long-term by focusing on “Cycle Time”– how fast you’re moving issues through the pipeline on average. Easier said than done but worth looking in to.

      HTH.

      • Angelo

        Thanks for the helpful answer. Most of the time looks like many companies like the agile label on but not the effective process. We use Scrum but we have to deal with artificial pressure coming from the hard dead lines. No way to change this decision and still the wonder of work with agile methodology. The real challenge it s remove it and let apply agile. Have dead lines at the end kills Kanban more then Scrum where the planning session it is used to decide the level of complexity and what cab be fit in. For Kanban it s just mater of what put in WIP.

  • Elias

    i don’t agree with that whole ‘estimation is a myth’ theory…the point of estimation isn’t for putting a size or time on tasks…when estimating it asks the question ‘what is involved in doing this card?’…it may need to be broken down into smaller and simpler cards..with estimation you lower the chance of causing bottlenecks by reaching a card and saying ‘wow this is bigger then i thought it’d be’

    • Alex Salazar

      You’re 100% correct. It’s wrong to say that we don’t do estimate. We do. To your point, we use them to get a sense for how big a task is and how we can break it up if it’s “too big”. Our estimates help us do up front product management and feature definition.

      But we don’t plan our delivery schedules around those estimates because we’re often wrong. And being wrong when the development (and in turn business) pipeline are relying on those estimates is costly and painful. Moreever, we don’t set deadlines based on those estimates or hold people to those estimates because it ends up being a hollow enforcement mechanism– we just end up moving the date or get into crazy work hour spikes which risk burn out and morale.

      So it’s more accurate to say that we redesigned our processes, to remove/reduce our reliance on estimates on our pipeline. We instead use other forms of management (discussed in other comments here) to instill a sense of urgency. Also, Kanban itself recommends a focus on continually improving cycle time.

  • Ron Pastore

    we’re mid-transition and really enjoying the benefits of Kanban as described, but also experiencing the lost sense of urgency you mentioned. any advice on how to do that with culture/management?

    • Alex Salazar

      There’s no silver bullet but we achieve some of this by conveying “why” things are urgent and really trying to explain the business reasons and put a customer face on the item. In addition, we try to rally the whole company around it so the overall mood is aligned AND in high priority situations, we shut down all other activity so that the devs can focus. HTH.

      • Ron Pastore

        definitely helps. thanks Alex.

  • My old team struggled with Scrum for months before we switched.

    I read a post by Joe Stump (https://sprint.ly/blog/kanban/) on Kanban, and convince the team to try it. It made a huge difference for us: it just seemed to fit our workflow better.

  • Vladimir Zakharov

    Interesting to read, but the Scrum “drawbacks” that you described sound more like shortcomings of your actual implementation, not Scrum as a framework. Increasing tech backlog, time pressure to finish off the sprint, defensiveness during retrospectives – I don’t see any of this in the spirit of Scrum. Probably you could have improved your Scrum experience have you had more professional and/or insightful scrum masters; but I’m glad another option worked for you.

    • stormpath

      You’re absolutely right. But that’s kinda the point. Almost everyone we spoke to was having similar issues with Scrum and we didn’t really find anyone who was “doing it right”. We tried to fix it many, many times and short of paying a consultant to come in we just decided to switch. Took a few days and it’s been much easier ever since.

      • Vladimir Zakharov

        I’m really sorry you didn’t seek “professional help”. What you describe as scrum actually sounds like serial waterfall. Add to this that your page shows ups rather high under the “scrum vs kanban” google query, and you get quite a bunch of people who will think that “scrum isn’t right”, “scrum is a prescriptive formula” and “scrum creates time pressure”. Sad, sad, sad…

        Not that you need it now, but I’d really suggest that you read for instance “Agile Project Management with Scrum” by Ken Schwaber, which describes a lot of pitiful scrum implementations with problems not unlike yours.

      • Alex Salazar

        Thanks for the book recommendation. I’ll check it out.

      • Professional help is often key to success. But finding the right coach with an approach that suits your work and team is key. Many failings of agile methods are because people don’t have the right skills, experience, expertise and training.
        Too many of my coaching colleagues in this area ram Scrum down people’s throats and are too rigid with its interpretation. It’s true that Scrum works best when its done “as-is”, but it’s far from inflexible.

  • VerKl

    Long but great article, thanks! You’ve come to the point – Kanban means continous flow process and that’s why it is important to use the right tools to manage the daily tasks. We’ve moved to Kanban with Kanban Tool about 10 months ago and are really happy with it, it’s a perfect solution for us. Best. Done. 🙂

  • Dave Black

    Just about the best article I’ve read on the frustrations (and failures) I’ve seen on many different Agile/Scrum projects I’ve worked on at many different Clients and over many years! I can empathize wth so many of the statements you made:

    Sprint planning forced engineers focused on one component to sit through a discussion of unrelated components — for hours. Additionally, we would get into long (and often fruitless debates) about issue priority and scope that would drag in the whole team.

    Like most teams, we have some over-estimators and many under-estimators. I can tell you with absolute certainty that 100% of us are bad estimators.

    Many times over my Agile career it felt as if the team was just “going thru the motions” of Agile/Scrum just because they wanted to be labeled as being Agile instead of adapting for what was right for the team/project.

    I worked on a project that used Agile/Scrum where the scrummaster was extremely rigid to how things happened. He held very strongly to the Agile Manifesto and didn’t deviate from it even just a little bit – even if the deviation fit the needs of the team and project moreso than what is prescribed in the Agile Manifesto. Needless to say it was extremely frustrating and IMHO had a large part in the project failing. In the end, there was much turnover on the team (which I was included in) and ultimately the Client did away with Agile. They viewed it as too rigid and inflexible. I’d suggested to the scrummaster many times to “adapt” the methodology to the needs of the project and team. It is refreshing to see more teams moving to the Kanban approach which is much more flexible and less rigid.

  • Good read, we have tried JIRA, but it is way too complicated for what it does. We ended up with Kanzen ( https://mykanzen.com ) and it works great so far. Really flexible when it comes to Kanban boards.

  • I like Kanban. And I like your story of discovery.

    Many people forget Kanban is actually a Lean tool, and as such, flow and batch size are a critical part of removing wasted effort and optimising work. Kanban and Lean also work very well with Scrum.

    Scrum doesn’t ask you to do estimates. Scrum asks you to work at a sustainable pace. Urgency isn’t baked in. I’m very concerned that someone led you to believe that this is what Scrum is about.

    Misunderstandings about what Scrum is and isn’t, and lack of skills, experience, knowledge and training always hamper (if not kill) adoption of any agile method (including Kanban).

    At least your team found something that they feel works. The trick now is to embrace kaizen, muda, mura, and muri. An experienced coach can help you navigate this path and avoid the mistakes you made with Scrum.

    • Carlos Soares

      @Matthew Hodgson – I agree with your point of how lack of knowledge can destroy any chance of making a change. We had a case of a team lead ourselves, she decided herself to be our new Scrum Master, just because she read somewhere, that every Agile company needs one. She ended up making us do just stand-up meetings – as if this was what scrum is all about (yeah, we did nothing else differently!). Needles to say, she is no longer our SM. We have since created our own process, based somewhat on scrum, with added WIP limits, that we liked of kanban. I want to offer a good source of detailed information on agile methods: http://kanbantool.com/kanban-library/introduction, I especially like the section on Scrum and Kanban. Hope no-one gets mislead like we did!

  • Moving post-it notes is simply about kinaesthetic learning.
    It makes people get out of their seats and talk to each other about what work is coming up.
    A wall is also infinitely flexible in relation to visualisation of work, whereas software will always limit the way you want to work.
    Managers won’t log into your kanban system but they will definitely walk clients and stakeholders up to a physical wall to ‘show’ them what is going on.

  • It feels like software development has become a religion.

    No matter, just deliver what you customer is happy with.

  • Sounds like the problem is the organisation understanding with Scrum rather than Scrum itself. I can also run Scrum without using estimation. In fact Scrum Guide is very silent on estimation. The Sprint Planning is more about forecasting the amount of work. The notion of Sprint also creates regularity or rhythm but if the team see it as a deadline then it is a big problem how Scrum is communicated in the first place.

  • J Diggs

    Tell me how you applied Kanban to non-coding groups like Marketing and HR when Kanban doesn’t use due dates and yet many of these operational groups require due dates. Hybrid Scrum (sprints) + Kanban?

  • Chris Krause

    This is a truly exceptional article. Sharing this with some fellow producers here at BISIM. 🙂

  • One or the other may be better, but I don’t think you’ve made your case.

    If your Scrum teams are crunching at the end of each sprint to make their commitment they have two things to learn: 1. Commit to less, 2. Clarify product owner expectations about what the commitment means. It does not mean burnout. Sustainability is a key Agile value.

    And if you think Kanban discourages your team from cutting corners, I think you’re mistaken. One look at a big pile of todo’s in the IN box is a pretty strong force for rushing. If your team is now more quality-conscious you’ve done something right, but that something is not necessarily Kanban.

    From your short description I would guess you changed from a poor Scrum implementation to a fairly good Kanban implementation. And your improvement came from improving the implementation, not from changing frameworks.

    • Alex Salazar

      You’re correct that “you changed from a poor Scrum implementation to a fairly good Kanban implementation.” And that we could have probably improved our implementation of Scrum to achieve the same goals.

      My big point is that we invest well over a year into trying to improve our implementation of Scrum and frankly failed. And yet, we were able to solve the vast majority of our problems in ONE WEEK by switching over to Kanban– without any consultants or experts.

      I know there are great scrum implementations out there and I truly wish we had been one of them. But the effort to achieve that turned out to be orders of magnitude higher than it was to get to similar results in Kanban.

      Both can deliver similar results if done right, Kanban was just much easier to get right.

      • That’s great to hear Alex! Are you finding that your team is also very process conscious and is developing learning skills that enable them to guide their own process improvement, or is it a led team where you or someone else do that? One of my most favorite things about Scrum is that in a directive leadership culture it doesn’t work well, but in a collaborative leadership culture the team takes over responsibility for learning to improve itself over the long haul.

      • Alex Salazar

        Good point. One of the things we liked most about Kanban was that it’s 100% owned by the team, not management. And because the process is so light, the team has broad latitude on the processes it can add, change, or remove.

        As part of Kanban, the team has regular Kaizen (continuous improvement) meetings where we discuss what we learn since the last meeting and what improvements we’d like to try out.

      • Nice

      • Elizabeth Arroyave

        Alex Do you think Kanban can be applied not only to software development but to other processes? We are a digital agency and we are looking for a new methodology to work.

      • Elizabeth, Kanban was originally developed by Toyota for the car manufacturing process and is now being applied to software development, restaurant operations, marketing, etc. It’s one of the least prescriptive and most flexible frameworks out there.

        Check out this video for a quick overview of the principles – http://youtu.be/R8dYLbJiTUE

  • “Crazy hours at the end of the sprint in order to get it all in a release because “we committed to it” were followed by a productivity crash the next week so people could recover.” This isn’t a good way of doing scrum anyway if you dont finish an item put it back into the backlog re estimate and try again rushing and extending working hours just to meet a velocity target is not ideal especially when velocity is a tracking metric not a goal.

  • Rich Kucera

    Looking at your board, how about this: the division into phases between “analysis”, “develop”, “test”, “deploy” is perhaps unnecessary, leading to overly-detailed perhaps misleading metrics to pass up the chain.

    If you just conflate those phases into one column: “in progress” and put a WIP on that column. The other column to the left is still “To Do”.

    Then what you have is simply a day on a calendar, which you can then put on a calendar.

    Simple huh?

    The division into phases is questionable in a fast-paced environment, and past researchers have shown that there are no distinct phases, really, that lightbulbs go off about any level of design during any phase, because of feedback (see Parnas, “Why and How to Fake a Rational Process”).

    The division into phases is the only thing that seems to be driving this into a formal “methodology” like Kanban, instead of just putting it on a calendar.

  • pinkra

    Uhm.. it seems you haven’t applied Scrum well enough..

  • cynicsrising

    brilliant article, i’m in full agreement and i’m using this to improve the processes at my work place too

  • Whatever

    I hate kanban and it has failed me and everyone at my ocmpany

  • I found this post both amusing discouraging, and I have seen many like it. Teams complain that scrum is failing because engineers are inherently poor at estimating, because they have to scramble at the end of every sprint, because the scrum overhead is too burdensome, because priorities change too quickly. Rather than see these as the forcing functions they are intended to be… forcing functions for process improvement… many teams fall back on kanban, which seems to offer relief.

    Funny thing is, when correctly practiced, kanban is a forcing framework, too. Its forcing function is the drive to reduce cycle time and make it more predictable, usually by moving towards lean’s “one-piece flow”. That is not a walk in the park, by any means, but I don’t hear about that kind of discomfort in the “we abandoned scrum” posts. So I’m guessing that these new kanban teams, though happier, are neither finding nor removing the deepest impediments to sustainably high productivity. They have found a good way of managing work but are complacent about fundamental process improvement. And I think it’s a cop-out.

    Before we take the easy path of abandoning scrum, here are some strategies to consider: successful scrum teams use these to overcome the supposed burdens of scrum:

    If a team is poor at estimating… are they estimating in concrete units (days, hours) or relative units (bigger than a breadbox == story points); are they spending enough time understanding the problem domain, technology architecture, and design before they estimate? Are they afraid to be realistic? Is uncertainty and risk considered in the estimate (and what is being done to reduce those)?

    If they are scrambling at the end… what is the obstacle to simply promising less and meeting those expectations? is it organizational pressure? lack of trust? lack of devops tools? lack of test automation? quality assurance limited to testing? Does the team think the only productive use of time is coding? (many teams spend the last day or two of the sprint working on skunk-work or honey-do activities or in grooming or design for upcoming work.

    If the scrum overhead is too burdensome… why does it take a half-day of planning with the whole team? Why not one or two hours – what would have to happen to make that work? how can backlog grooming be more effective, efficient, and proactive? Is the team designing during sprint planning or simply deciding what it can commit to and what gaps there are in understanding? does the product owner need more involvement of technology spokesmen in advance of planning? what kind of release planning is there?

    If quickly changing priorities disrupt the sprint commitment… why is the sprint the length it is? can it be shortened — many teams shorten to as short as 1-week sprints to accommodate quickly changing priorities.

    Scrum forces your team to be disruptive. That’s the point, and that’s the challenge. You can accept the challenge or bail. The scrum master winds up being the primary driver of the external and cultural changes that make the disruption worthwhile.

    So if your team is considering moving from scrum to kanban, and you think that’s a good idea, I guess I would ask whether your scrum master (you?) is doing their job the way it was meant to be done.

  • Arnaud Viguié

    while reading the article, i have the feeling that you applied scrum process without its soul, and that s why it did not work, and that s why kanban worked. as you wrote, the team had the feeling that adopting kanban was their decision and that scrum had been kinda imposed top down. empowering teams is essential for scrum to succeed, as kanban. although the kaizen meeting you describe is how your scrum retrospective should have been since day one : no personal critic, all focused on improvement. i suggest you read jef sutherland 2015 book, scrum how to do twice the work in half the time. you ll see that he had been fighting, with scrum, all the default you described about… scrum! attitude and team spirit is even more important than the process/framework you choose

  • Kanban and Scrum fit well together. It’s not an either or but rather a oh-and. Scrum was actually created as an extension of the TPS principals. Don’t throw the baby out with the bath water here. It is not difficult to integrate the concepts.

  • Alexander Kryklia

    Are you still using Kanban?

    • Claire Hunsaker

      Yes, we are!

      • Alexander Kryklia

        Thanks, two years past after this blog post. Can you please share some insights how things and Kanban implementation changed comparing to this blog post. I switched my team from Scrum to Kanban recently and your blog post was valuable source of experience.

  • Alex Salazar

    You’re right. If anything we did away with anything associated with estimation. It was giving us a false sense of accuracy and then we’d base decisions, like the sprint plan, on those broken estimates. We’ve kept many other aspects of Scrum.

  • Alex Salazar

    Great point. I completely agree that we could have done Scrum better. In retrospec, I’d say the biggest fundamental improvement between the two processes for us was the move away from a time-based constraint based on good enough estimations of work to a work in progress constraint based on capacity. Since we were consistently off on our estimates, all our sprint plans were based on bad data. By moving to a model with less guess work, things more smoothly and we were able to cut out the spring plan meeting from our schedule.

    • Arnaud Viguié

      I understand that. Solution I personally apply is to track estimates all over the stories life: early, Ready, reality (time spent when Done). That requires the team to take note of what they spent time on everyday, which is not a burden at all, just need to get used to it. at the end of the sprint, you see what every one has been working on for how many story points. you both see what non expected work the team has worked on (which is super useful to know), and the reality vs early&ready estimates. this last info helps you to understand what you failed to consider while estimating. this helps to greatly improve teams’ estimates, and so reduce scope risks and the late appearance of unexpected dependencies. Also, as is often advised, teams fills their sprint backlog at 70% of velocity, no more. it leaves a buffer for unexpected demands or bugs, or to finish wrongly estimated stories. when the team has finished its sprint backlog and still have time, they split the rest of its time through learning and taking some predefined buffer stories (prepared in case the team finished early. This is a good practice part of the following paper from sutherland: http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=6759182

  • Alex Salazar

    100% agree. We didn’t. And that’s kind of the point of the article. Implementing Scrum well enough turned out to be something we weren’t able to attain, despite having a smart, hard working team. We couldnt find many people around us who were happy with it we could look to. And the only people telling us how awesome it could be were scrum masters trying to either sell us consulting or trying to join our team.

    I’m not saying we couldn’t have gotten there, but the move to Kanban was a breeze and worked really well for us. Productivity, quality, and morale all improved. All with much less effort and no experts on staff or in contract.

    Is Scrum better or worse overall? No clue. But Kanban won for us on our ability to easy implement and win with it.

  • Alex Salazar

    I like the way you’re thinking!

    Our actual phases are different than what’s on the image in this post. We simplified it to make it easy to follow.

    I’m all for collapsing phases. The way we divide phases is really based on where hand-offs happen. For example, code review is a critical part of any ticket. So by having it as a phase, we can set all of our tools to alert us on hand-off to another dev or when the work is complete. We can also point analytics at the board and see where we have bottlenecks in process.

    But yes, I’m all for collapsing where it doesn’t add value.

  • Alex Salazar

    True. I guess the issue is that any kind of forecasting is based on estimation so you can’t have one without the other. And if we suck at estimation, which I fully admit to, then all our forecasts were broken and in turn our sprint plans were based on bad data.

    On the deadline piece, you’re right. Our nature got in the way– a team of highly competitive, overachievers in a startup with an ever-shortening cash runway just wasn’t able to treat the sprint date like anything other than deadline, explicit or not. I’ll definitely take my share of the blame on that one.

  • Heidi Chan

    How do you determine what the project will cost and how long it will take? Seems good if you are working time and materials.

  • Komrad

    I’m using Scrum since I have to deliver content each week along with design, marketing, and other enhancements. It took several iterations to determine how much I could take on in a week, but now the process of weekly deliveries has become a habit. Content is king for my product, so if a have a large article to write, then I don’t include design or marketing, for example , in that week’s sprint. Meetings are no longer than 15 minutes, down from almost an hour initially.