Robyn Sands blogged the following article, entitled "a long overdue thank you," after Method R Corporation helped her complete the first multi-month phase of a complex application development project.
The past year has been, well, many words come to mind but let's go with challenging. It's also been interesting, frustrating, enlightening, exhausting, but right about now, it feels like it was a very, very good year.
Those of you that have read through the previous posts will remember that, right around the time I left for the Miracle Oracle Open World conference in October of 2008, I was officially labeled the 'architect' of one of the most dreadfully designed systems I'd ever seen. I was slightly dazed throughout my week in Denmark as I turned over all the problems with the system in my head. My panic levels increased when I got a call confirming that the system was now in 'production' mode—I'd been hoping someone would come to their senses and push the reset button. It was only prototype-production mode, but as we all know, once a system has been delivered, changes are that much more difficult to make and mistakes can be much more damaging.
One of the best things about having a network of good friends and colleagues is being able to share your misery and know that these people really do understand why the problem you're describing is actually as bad as you think it is. Or not. In either case, sometimes the only thing you can do about it is laugh and maybe drink, and this is exactly what I did on my last night in Denmark at the infamous Oak Table.
My story resulted in lots of incredulous looks and laughter and empathy, as usual, but this time it also resulted in a solution. Later in the evening (probably morning by then) Cary Millsap suggested that his new company, Method-R, might be able to provide some assistance. We talked a bit and I told him that I appreciated the offer, but this was much more than a performance problem—I needed to start at the very beginning and redesign the entire system. The main procedure was taking 40 seconds and it needed a sub-second completion time, plus this was happening with a fraction of a percent of the expected data. There was not one component that I was happy with: the hardware, the configuration, the schema design or the code. Cary thought about that one for a while, but eventually said that he'd be willing to take it on with the team that he had on staff. He had confidence that they were up to the task of supporting a redesign. We agreed to continue the conversation when we returned to the states, and left it at that.
Once I was home and we had an NDA in place, discussions with Cary continued and now included two of his employees, Ron Crisco and Ken Ferlita. From the very beginning, working with Method-R was an absolute pleasure and both Ron and Ken became an extension of the team. It’s almost funny: management will refer to the XYZ team (not the real acronym), yet that that team is just me and Method-R.
I noticed many differences in working with Method-R, and can't possibly capture them all in a blog post. However, these are the differences I found most significant:
Data-centric: We began by making sure we understood the data and the relationships. The data is provided by my end customer and it’s somewhat cryptic. Not all the fields were needed, but some of the unneeded fields helped to clarify the relationships in data we were using. Some of the data would be needed later as more functionality was added. We learned all we could and built a new schema, using third normal form for the fields in use, leaving unused data points as attributes to be more fully defined later.
Quality code: The code was excellent: well written, modular and reusable where appropriate, and it performed exceptionally well. Of course, there are detailed exception messages to capture any possible errors, and I still haven’t encountered an uncaptured error in testing. The code is also instrumented, providing better visibility of active processes and allowing trace files to be captured easily when needed. Method-R exceeded requirements and expectations in their delivered code, and the difference between the code as it existed a year ago and as it exists today is almost unbelievable.
Problem resolution: There were several components of the application that were still being defined by the end customer, and some of the unmade decisions impacted critical schema or code choices. However, we still needed to meet the release deadlines. In these cases, we would define the simplest effective solution, and keep our options open for the final solution. In at least one case, this required us to implement a table design that we knew we wanted to replace. But we also knew that if we coded a better solution at that point in time, we could end up spending a lot of effort on something that would be ripped out with the next release.
When you know better, it’s really difficult to accept the not-perfect solution. It’s even harder to make sure you go back and implement the right solution later. I’m happy to say that we did get rid of that nasty little compromise and I celebrated it's death with an better than average glass of scotch late one night. However, if we’d insisted on choosing a solution earlier, we would have come up with the wrong answer. I think this is one of the biggest challenges of–dare I say it–Agile design.
Cary and I have had a few conversations about Agile over the past year. He’s been pro-Agile for some time, as you know if you have read his ‘Measure Once, Cut Twice’ paper. I’d read it some time ago and I agreed with what he had to say, yet what I saw happening around me on other projects caused me to question the whole Agile process, as these other projects did not seem agile at all. As I noted in an earlier post, I think some of the problems arise from failing to understand and model the data, but I have come to the conclusion that the real problem is something much bigger.
Agile needs to be about streamlining the process and making the process support meeting the customer needs and requirements. Agile should keep the team focused on the problems to be solved in the immediate term while not losing sight of the longer term requirements. Agile should allow you to adapt when you need to while ensuring that those adaptations aren’t just shortcuts that weaken the final result. There is a mindset that needs to be part of Agile, similar to the mindset that led Toyota to implement Just-in-Time.
Here’s the punchline: if a waterfall minded manager decides to implement Agile, but he and his team don’t have an Agile mindset, they will simply trade one burdensome process for another.The process is not supposed to be the star of the show, it’s more like the score: it sets the tempo and the pace. It needs to be there but when it’s working well, it should be almost unnoticeable, except that you’d notice if it was missing. If your process is all about the lingo and form, you’re probably not doing it right. If your process detracts from your ability to meet the customer’s needs, to delight your customer, you are definitely not doing right.
Method-R is doing a lot of things right, including Agile. Did they stand up during our calls? I don’t know, but I doubt it. Did we scrum and sprint? Possibly, but we never used the words. Did our process support us and key our eye on the goals? Damn straight, it did. We met all our targeted dates, and had the fewest issues of any ‘team’ within the larger project. Did Cary and his guys delight their customer? Absolutely, and in doing so, they helped me to delight my customer. In fact, my customer is so delighted, they are expanding our role in within the system, requesting additional features and functionality.
To Cary, Ron, Ken and Jeff ... Thank you for everything you have done to help us meet our goals. Without your knowledge, skills and commitment to quality, plus the extraordinary effort you provided, we wouldn't have made it nearly this far. We also wouldn't have the head start on the new functionality we now have.
And btw, keep a spot or two on next year's dance card for me :)
Database Systems Architect, Cisco Systems