Developing on Windows: Reinstalling disabled services

As part of our project we have automated installing a service as part of our deployment script. However services go into a zombie state of 'disabled' if you have given the service a wrong password and tried to start it up.
Once a service gets disabled you cannot start, stop or uninstall it from the services console (services. msc). Rebooting the machine gets rid of the disabled service, and we can reinstall the service, however rebooting isn't an option for production machines.

We finally realized that we could reinstall the service over a disabled service as long as the "service console" (Service.msc) was closed.

It is annoying how having the GUI for service manager open, somehow stops me from reinstalling a service.

The Agile Maturity Model

Note: This blog is a companion to the talk given by Ravi Gujral and Shaun Jayaraj at Agile Philly http://www.agilephilly.com/Agile%20Maturity%20Model.ashx


The best way to start describing the Agile Maturity Model (AMM) is by talking about what it is not.

The AMM is not

• A mechanism to measure the performance of a team
• A pre-defined set of practices, whose adoption guarantees successful delivery of projects in time and within a budget
  

The AMM, in contrast, tries to quantify degree of responsiveness of a team to an organization's business needs. The philosophy being that increasing the discipline and proficiency of executing some best practices will increase the degree of responsiveness. The AMM primarily focuses on application development activities; the dimensions of the AMM are

• Testing
• Source code management
• Collective code ownership
• Collaboration
• Responsiveness to business
• Assurance and governance
• Story formation
• Design simplicity
• Build process
  

For each of the above dimensions, the AMM lists behavioral patterns exhibited by teams. Teams can use the AMM as a reference to identify and compare the behavior that they currently exhibit and plan to alter and tailor them.

For example the behaviors identified in the testing dimension are

• Acceptance testing in lieu of all other tests
• No unit testing framework, ad hoc testing (Manual testing of individual components /controls)
  
• Unit testing, manual functional testing; application has a testable architecture
• Unit testing integrated into build process, testing automated as much as reasonable given application
• Developers write unit tests before writing functional code
• Test are identified and produced as part of a story creation
• Automated functional testing (e.g.. GUI testing); stories remain in development until all bugs are fixed or deferred

Consider a team that develops websites. Their testing strategy is to only have manual testers test the application just before deploying it to production. All the tests performed by the tester are acceptance tests. This would put this team on the level of "Acceptance testing in lieu of all other tests"

Using the AMM this team can then introspect and choose to introduce unit testing as a best practice.


The AMM Dimensions

Let us now look at the various dimensions of the AMM and the behavioral patterns that are identified within each dimension. The behavior listed at the start of each dimension is an undesirable behavior and teams should try and move away from them as soon as possible. However to what extent a team would like to mature in a given dimension is the team's choice.

The practices can be classified as

1. Regressive
2. Neutral or Chaotic
3. Collaborative
   
4. Operating (Consistent exhibition of competance)
5. Adaptive (Expertise to adapt to change)
6. Innovating (Creative evolution of practice, and spread these practices throughout the organization)

Testing

1. Acceptance testing in lieu of all other tests
2. No unit testing framework, ad hoc testing (Manual testing of individual components/ controls)
   
3. Unit testing, manual functional testing; application has a testable architecture
4. Unit testing integrated into build process, testing automated as much as reasonable given application
5. Developers write unit tests before writing functional code
6. Test are identified and produced as part of a story creation
7. Automated functional testing (e.g.. GUI testing); stories remain in development until all bugs are fixed or deferred

Source Code Management

1. Traditional schemes based on fire-locking and time-sharing
2. SCM supports version of code
3. IDE(s) integrate with SCM; developers include meaningful comments in commits
4. SCM support merging; optimistic check-ins
5. SCM support atomic commits
6. All development collateral is in SCM
7. SCM is transparent to delivery team, behind build process
   

Collective code ownership

1. Knowledge held by specific team members; people work in isolation
2. No pairing, people work alone, some informal process for keeping people informed
3. Pairing; no code locking
4. Pairing scheme ensures rotation
5. Team signs up for functionality rather than assignment to individuals
6. Within an sprint, functionality delivery is signed up for just in time
7. Old (bugs) and new functionality is queued, development team pops the stack in real time

Collaboration

1. Regular progress updates from management to the delivery team (as opposed to the other way around); irregular team meetings
2. Project collaboration tools (wiki, mailing list, IM) in place and used throughout project team; project status published and visible to all
3. Daily stand-ups and sprint meetings; problem solving is bottom-up as opposed to top-down; team sets sprint objectives and agrees on estimates.
4. Integrated, continuous build process, with build status notification to the team and collective responsibility for state of the build
5. Business is part of the team, stakeholders accept working software at reviews in lieu of other tracking or progress metrics
6. Frequent (near-real time) prioritization of old and new functionality
7. Build automatically deploys to QA environment available to any interested party

Responsiveness to business

1. Frozen specification, unresponsive to business value
2. Team tries to accommodate new requirements by ad hoc change requests ("pile it on")
3. Iterative process with sprints of length short enough to respond to business change
4. Showcases per sprint; business prioritizes functionality per sprint
5. Continuous involvement of the business on the team: business certifies story complete out of dev; involved in (rather than approval of) story definition and acceptance tests
6. Business writes high-level stories for all requests; organizational story queue exists; prioritization decisions made from full story backlog
7. Development process is integral to business initiatives; development teams work as a part of the business unit rather than as a service to the business unit

Assurance and Governance

1. Status reports document progress; schedule acts as plan
2. Concept of release planning is introduced
3. Sprint planning is introduced
4. Plan becomes communication and tracking tool rather than an exception report
5. Every sprint review examines value delivered and assesses next sprint by need, priorities (and delivery)
6. Organizational adoption: Introduction of Portfolio Planning and global optimization of resources (use of metric and standard measures)
7. Business review of value and return helps teams on regular basis based upon status

Story Formation (Requirements)

1. Development tasks extracted from voluminous, frozen requirements artifacts
2. Production of lightweight artifacts (e.g. panel flows) to drive high-level requirements development
3. Mapping granular requirements (e.g., use cases) in high level user requirements
4. Marshaling use cases into discreet statements of functionality that can be delivered in time boxed development sprints
5. Stories are an expression of end-to-end functionality to be developed, including testable acceptance criteria and a statements of value
6. Spontaneous story development provided by business for in-flight projects; stories not derived from extant sources but are immediate expressions of customer demand/need
7. Global repository of functional requirements for stories developed by business for any business requirements, including a formal measure of business value delivered

Simplicity

1. Big up-front design that attempts to accommodate all potential future needs
2. Application of fundamental design patterns
3. Structural refactoring to decouple application architecture
4. Doing only what needs to be done
5. Aggressive and constant refactoring to improve code quality/simplicity extant code
6. Spiking solution with each release to introduce new ideas and challenge architectural decision
7. Technical design decisions taken with each story

Build

1. Ad-hoc build requests; scripting and component marshalling performed manually
2. Consistent, repeatable build process with durable artifacts executed manually with each release
3. Build is automated, executed on a timed basis
4. Unit tests are integrated with the automated build; team is constantly notified of the status of the build; build triggered by SCM updates
5. Test and metrics (e.g., code quality, complexity, check style, etc.) are integrated as gatekeeper events; build data archived and reported in build portal/dashboard
6. All product components advertise dependencies; master repository established
7. Product integration tests included; build process executes automatic deployment to QA testing environment

Conclusion
We will be talking about the above AMM model during the Agile Philly and discussing the interconnections between the various dimensions.

Ackowledgements
Ross Pettit , Gerg Reiser, ThoughtWorks Inc.

Related Blogs and Articles

1. An "Agile Maturity Model?" by Ross Pettit http://www.agilejournal.com/component/option,com_magazine/func,show_article/id,44/
2. Agile made us better but we signed up for great By Ross Pettit http://agilemanager.blogspot.com/2008/06/agile-made-us-better-but-we-signed-up.html

User stories in the Enterprise Integration space

In the 'user story' approach to requirements, it is considered a good practice to write your story's objective in the form "As a user I would like to perform some action so that I can get some business value/benefit. For example "As an insurance claims processor I would like to flag duplicate claims as 'duplicate' so that I don't process and pay for the claim more than once."

If you are developing software in your typical enterprise, soon or later you will come to a point where your application has to integrate with some external system. Writing user stories in such a scenario can be quite challenging.

Ideally the business should be able to assign some dollar amount to the business value called out in the 'so that" phrase. It is critical to call out the benefits of having the systems communicate with each other and share data.

Let us consider this example:
"As a user of system Foo I would like to create a new account and the new account creation message should be broadcast over a message queue."
The above story objective describes the implementation details and it does not call out the business value in this story. This is a poor objective for a user story.

An improvement will be to identify which systems will be consuming this message. Here is what the modified story would look like.
"As a new user of system Foo I would like to create a new account and the new account creation message should be broadcast over a message queue so that the SAP application can be informed of the new account."

This was slightly better than the earlier version, however it still isn't easy to associate a dollar amount to the 'so that' portion of the story title. To get to the business value in this story we need to look at what the SAP system does with the new account message.

Here is a final version of the story
"As new user of system Foo I would like to create a new account and inform the SAP system of the new account so that the SAP system can create a corresponding account for the user and bill the user for the services provided."

When you define integration stories in this manner, firstly it clarifies the business value obtained by implementing the story and it also makes it easier to test the application end to end. The other desirable side effect of this is to allow the business analyst to talk purely in terms of the expectations of the business and let the developers (who are more qualified than Business analysts when it comes to implementation) find optimal ways to make this integration work.

It could be argued that the architecture demands System Foo not be aware of the nature of the consuming application, and by calling SAP out by name we might tempt the developers to violate this requirement. Such details are important and need to be part of the acceptance criteria of the story.

Photo licensed under creative commons, photograph credited to Roger Smith

Stages of IT maturity

In my last post I mentioned that every development decision should be tested against the question "Is this helping my customer?".

Last week I came across a project team that did just that. They had asked the customer what she/he wanted and built the system exactly to those specifications.

I was called in to do an IT assessment of the progress of this project and I soon realised that this project was an abysmal failure. But how could that be, here was a project that was built exactly to specification and was delivered more or less on time.

The problem was that the project team built a system, based on what the business users wanted without considering if there is any business value in it. The business users were line managers and clerks and they had a very limited view of what the business was about, and how their day to day actions affected the top line/ bottom line performance of the company. So this resulted in a very narrow view of the requirements, and sure enough when the system went live it did not deliver as much value as top management expected it would.

So how does one get past a problem like this ? Here is where stories play a very crucial role, and the importance of the "As a role I would like to do some action so that I achieve some business value" format.

Some people feel that we are being pedantic if we insist on every story adhering to this format. But if the team had stuck to this format, they would have realised early in the game that the requirements being given to them were probably not adding any business value. This visibility would have allowed upper management to terminate the project early and reevaluate their strategy.

The other aspect to this problem is the maturity of an organization to consume a custom built software. For custom built software it is critical for the users to be able to clearly articulate the need of the business. They should be able to tie every requirement to the bottom line or top line performance of the company. This implicitly means that the business users have exhausted all means to improve performance by business process modifications. Only in such a situation should the business consider using technology to get them that extra bit of competitive advantage.

In my opinion the IT maturity of an organization is closely linked with the business maturity. I can think of four stages of IT maturity an organization can be in.

1. Chaotic tactical business processes (use of spread sheets and multiple point solution)
2. Business processes which provide accurate and up to date information to the upper management to make decisions. But the processes themselves don't adhere to a overarching strategy
   
3. Business processes tied to a strategy that is clearly articulated by the upper management
4. Business processes which are flexible and can easily shift and align themselves to the subtle strategy changes being employed by the upper management

To get to the Nirvana state of business, it is important for information systems to support them

Stage 1: To start with you have a bunch of spread sheets and custom point solutions spread out through the organization

Stage 2: Fewer spread sheets and some central solution which can provide some business intelligence

Stage 3: Collection of commercial off the shelf (COTS) best of breed solutions integrated with glue code

Stage 4: Collection of COTS as well as custom solutions in areas which give the organization a competitive advantage integrated with glue code




Pros and cons of each stage
Stage 1
Pros: Relatively easy to stand up spread sheets for your day to day work.

Cons: Virtually impossible to get any sort of business intelligence at an over all enterprise level

Stage 2
Pros: Easy to obtain business intelligence

Cons: If the team supporting the central application is small, they become the bottleneck.
The whole enterprise will over time become overly dependent on a small group of people who control the application.
The single application is critical to the functioning of the organization, this reduces the number of upgrades and changes that you can make to the application. Over time the application become out dated and crufty.


Stage 3
Pros: Allows flexibility
Can get the best of breed for each individual functional area

Cons : Can't get competitive advantage from such a solution (if you can glue together off the shelf products, then so can your competitor)
Will have greater IT spend because the skill level required to maintain this kind of an architecture is quite high


Stage 4
Pros: Competitive advantage

Cons: Increased IT spend

Responsible Developers: Does IBatis help my customer?

Agile (especially the XP flavor) empowers developers a great deal. Being an ex developer turned IM, I am starting to see that this is a double edged sword. It is dangerous to empower incompetent developers.
Incompetence can come in various flavors, You have your garden variety junior developer, who just wants to get his 40 hours in. Or the grand and wise 'architect' who wants to build a unified framework, which will encompass all knowledge. Finally you get the super dev, who wants to do everything his way, because his way is the coolest.
In an agile team, one of the most important characteristic expected from a developer is responsibility.
Responsibility towards his customer, every decision (which build tool to use, which persistence framework to use, which database to use, MVC or just write to the DB directly) taken by the development team, should have been tested against the question "How does this help my customer"?