In this blog we’ll take a look at examples from programmes that Pathway has been instrumental in managing or rescuing.

Our training and consultancy are based on lessons learnt from delivering demanding, business-critical programmes. Pathway has an enviable success rate, and we hope the information is useful for your projects.

Delivery date information when using Agile

It isn’t uncommon to hear that the only information available is the sprint completion dates, not the contents of the sprints. This isn’t correct. A sprint should contain features that will be delivered, and if a critical feature isn’t going to be included, the product owner should review this with the team, analyse the impact and update the relevant stakeholders. 

Project example – Delivery date information

• On a major transformation project for a media company, the sprints were planned in detail for eighteen months, with the critical features (required for each phase go-live) aligned to sprints. 
• The project included replacing legacy applications and building new infrastructure. A high level of assurance regarding the sprint schedule was critical due to the dependencies and business requirements.
• Some features/backlog items were moved (some brought forward and some moved to the next sprint), but the critical items were delivered in each sprint, and all cutover phases were delivered on time. 

Dress rehearsals 

You may have to explain dress rehearsals. Many people do not understand their objectives and activities. Dress rehearsals are not a formal part of Waterfall, PRINCE2 or Agile. (Unless they have been added since this was written.) They contribute more to risk reduction, successful cutover and increased productivity post-go-live than any other project assurance activity.

Dress rehearsals aren't a repeat of UAT or any other test phase. In dress rehearsals users complete the key business processes, replicating the first days, weeks or months after go-live. Test scripts are not used and dress rehearsals are based on business scenarios. 

The dress rehearsal environment utilises the go-live security settings, including users’ role-based access control. (RBAC) A copy of live data or data that mirrors the production data is used. The dress rehearsal environment is usually the To Be production environment or a dedicated dress rehearsal environment that mirrors production. If the environment build (including security settings) hasn’t been completed but other activities have progressed, it’s possible to do the first round of dress rehearsals, focusing on the business processes. The project team, users and stakeholders will benefit from this because dress rehearsals highlight issues that are not usually seen in test phases.

The support teams need to be involved in the dress rehearsals. Include a scenario where there is a system failure to provide assurance that the support procedures are working as expected.

Include superusers in the dress rehearsals. They will gain experience that will be invaluable when supporting their colleagues after go-live.

Project examples – Dress rehearsal benefits

Media company

On a major project for a media company, dress rehearsals were conducted after all test phases had been completed. This project involved replacing legacy editorial systems and the infrastructure. It was implemented in phases, with go-lives occurring over an eighteen-month period.

The project was managed using Agile with additional project management governance relating to the infrastructure transformation. The business processes were time-critical, with dependencies at each step. Each sprint had key features/deliverables that were mandatory for that phase.

The dress rehearsals highlighted 90 additional issues, and 45 were resolved before go-live. One of the issues relating to the timing of a critical BAU process. The deadline for completing all BAU activities would have been missed on the first day of go-live. The error didn’t show up in UAT because the starting point assumed timings. When the complete process was run end-to-end during a dress rehearsal, the issue was discovered. The matter was resolved, and there were no issues with the timings after go-live.

Insurance Company

A project for an insurance company included updating the finance system and the introduction of document scanning. The dress rehearsal activity resulted in a very uneventful go-live, and productivity increased from day one. (On most projects, there is a period of weeks or months with reduced productivity following go-live.)

Leading Charity

A leading charity introduced a new system to enhance support for hundreds of remote staff. This was a major cultural change that replaced manual processes. At the beginning of the project, there was considerable concern regarding the level of transformation. Effective communications, quality training, and dress rehearsals with 95 per cent participation resulted in a smooth go-live. A senior manager called a meeting to discuss his concerns about the ‘low level of system usage’. He based this assumption on expected support calls to the help desk. There were very few calls, despite everyone at work using the application. He was very pleased with the transition to BAU and said that the results proved the benefits of dress rehearsals.

Common benefits across the projects

Dress rehearsals were a key factor in the successful cutovers of the projects. The business teams were able to properly practise and experience BAU activities before going live. Real-life issues were identified and corrected before cutover.

The appropriate level of documentation, refined during dress rehearsals, allowed a smooth transition to BAU. This included documents relating to security, integration, system administration, user guides, etc.

Methodology Debate 

The great methodology debate: if you’ve been a project manager for any time, you will probably have been involved in discussions of this topic. People have strong views on this subject, and some of the voices can be very loud.

If you are starting out in your career as a project manager, I would like to give you some advice based on delivering many types of projects using different methodologies or combinations of methodologies. 

My thoughts on the methodology debate are the following:

• There is no best methodology. If anyone says that a given methodology is the best for everything, you can assume that they haven’t worked on a wide range of projects.
• Many organisations use a hybrid Agile/Waterfall methodology, taking the best from each one to create an approach that works for their organisation, business model, and type of project.
• People say that PRINCE2 is a Waterfall development methodology. This assumption isn’t correct; PRINCE2 is a project management methodology. Furthermore, a common misconception is that PRINCE2 is a rigid methodology. This isn’t the case, and understanding the tailoring principle of PRINCE2 is important.
• Waterfall is used on thousands of projects yearly, despite claims that it's obsolete. Studies have found that over half of projects in 2022 used this methodology. If you are managing a safety-critical project, Waterfall will usually be the default methodology. 
• Organisations use different methodologies based on the types of projects they are delivering. 
• Scrum for Safety can be considered in a safety-critical environment, but be aware that some regulators require separation of development and testing functions. If you plan to use Scrum for Safety, verify all requirements for the organisation, regulators and third-party stakeholders. It would be very disruptive (and expensive) to find this out during your final sprint.

Project example – Methodology selection

A train company provides an example of using multiple methodologies. They had a requirement for a reporting solution. The project was delivered successfully using DSDM. (Agile DSDM.) However, for systems that are related to passenger safety, waterfall methodology was used.

Use of Work Packages

Work packages are very useful for estimating effort and for managing progress on a project. A typical work package will contain the following information:

• Work package title/name.
• Code.
• Status.
• Total budget. (Based on days of effort.)
• Start date.
• Completion date.
• Personnel/resource.
• Project code.
• Comment.
• Purpose of the work package.
• Key objectives.
• Success measure.
• Tangible deliverables.
• Type of deliverable.
• Key dependencies.

Project example – Work packages 

Work packages were used in the project initiation for a Year 2000 programme for a major car manufacturer. Sixty work packages were prepared to cover all known activities. This included work packages for the IT consultancy, employees, and third-party suppliers.

The work packages and the project plan were prepared during the project initiation phase, and no work commenced until the plan and all work packages were reviewed and approved. This provided the foundation for the very strong governance of the programme, which started in 1997.

The programme was delivered on time (no option on this one!) and £10,000 under the original budget. The directors were pleased that the programme did not require additional budget. (On average, Year 2000 programmes went back to their boards four times for more funding.)

Cutover dress rehearsals

What are cutover dress rehearsals?

Dress rehearsals are the most effective activities for managing cutover risks that you can undertake on a project.

In cutover dress rehearsals the team completes all the steps in the cutover runbook to ensure that it is complete and all timings are correct. The process takes place in an isolated environment.

Cutover communication is included. Typically, a member of the project team assumes the role of a senior stakeholder and receives the communications. However, if senior stakeholders want to be involved, such an arrangement is beneficial.

Data assurance steps are included. This is a final check to ensure that the cutover will not impact data quality. Ideally, copies of live data should be used for the cutover dress rehearsals. 

Integrations are activated following the steps in the runbook. It is best practice to include everyone who will participate in the cutover during the dress rehearsal.

On some projects people have said that cutover dress rehearsals aren’t required. ‘We can do all the proving we need with paper-based exercises. We don’t need to do a cutover dress rehearsal.’

There are several issues with this approach:

• It is impossible to prove timings unless you complete the steps in the cutover runbook.
• In a paper-based exercise, it's possible to overlook activities and steps. You don’t want to discover the error when you are halfway through your runbook on Saturday afternoon.
• Environment constraints and dependencies are hard to identify by reviewing the steps in the runbook. 
• It is a sensible idea to double-check prerequisite activities relating to security settings and security configuration. Security issues can be difficult to resolve, and any problems related to security settings could delay the cutover and go-live process.
• In the communication leading up to the cutover, the project team would have confirmed the go-live date to stakeholders and possibly third parties. After all the effort to achieve a go decision, you do not want to have to stop the cutover. This disruption will negatively impact the organisation and, in some cases, result in substantial additional costs.

What are the principles of cutover dress rehearsals?

A cutover dress rehearsal follows all steps in the runbook to provide the highest level of assurance. If you can’t follow the timings in the runbook, stop and analyse the issues.

• Are the timings in the runbook optimistic?
• Are there issues that are preventing the task from being completed in the expected/required time?

Please rerun the cutover dress rehearsals until all the timings are accurate.

Data quality assurance is included in the cutover dress rehearsals.

How do cutover dress rehearsals differ from testing?

Does this duplicate any testing activities?

• No, cutover tests/proving exercises are not normally included in any test phases.

What are the objectives of the cutover dress rehearsals?

The objectives of the cutover dress rehearsals are to:

• Verify all aspects of the cutover, including runbook timings, data loads and the security configuration.
• Drive out any dependencies not uncovered during preparation of the cutover runbook.

What are the benefits of cutover dress rehearsals?

The main benefits of cutover dress rehearsals are the following:

• Lower level of cutover risk. Most cutovers have a limited time for completion, and this activity will provide a high level of assurance.
• Greater confidence and less stress leading up to the cutover.

What are the steps to run cutover dress rehearsals?

The prerequisite activities are:

• Development completed.
• The environment build is finished. (Including the security configuration.)
• Production data, or data that mirrors production, is loaded in the cutover dress rehearsal environment.
• Review of activities that include third parties is complete. Please agree to their participation or confirm that project team staff will run their tasks.
• Completion of the cutover runbook peer review and quality assurance. 

The main activities for cutover dress rehearsals are:

• Complete all activities in the cutover runbook in an isolated dress rehearsal environment or in the To Be environment.
• Complete data quality assurance. 

Please arrange a time to remove all the dress rehearsal data and substitute it with the production data for go-live.

Important: If you are completing the cutover dress rehearsal in the To Be production environment, consider how you can leverage this work to prepare for go-live. In other words, review and complete quality assurance activities to determine if components are production-ready. If so, this process can be confirmed and will provide a higher level of assurance and reduce effort over the cutover period.

How many people will take part?

The complete team that will be responsible for the cutover tasks is involved. It is acceptable for someone on the project team to assume the role of stakeholders and third parties. However, it is beneficial for third parties to take part in the project.

When do cutover dress rehearsals start?

It is beneficial practice to start the first cutover dress rehearsals as soon as the prerequisites are complete. If any issues with timings are highlighted, commencing the dress rehearsals early will give the team time to resolve them. (Some issues with data loads can take a considerable amount of effort to analyse and correct.)

How long will the cutover dress rehearsals take?

The time required to complete the cutover dress rehearsals will depend on the complexity of the implementation and the number of issues you encounter. 

Planning to run four cutover dress rehearsals is a good starting point. Also, plan at least two weeks' contingency between completing the final cutover dress rehearsal and the cutover. A month is better if this can be scheduled.  

Project example – Cutover dress rehearsals

Major transformation programme for an insurance company. 

• The cutover was from Friday at 17:00 to Sunday at 23:00.
• The window for the data load was eight hours.
• Based on previous projects, the third-party supplier estimated that the data load (that included a high volume of data) would be completed within eight hours.
• The recommendation to complete a dress rehearsal was rejected in the first instance due to assurances from the third party.
• Following additional risk assessment, it was decided that a cutover dress rehearsal would be completed.
• The data load was scheduled for eight hours, but it was aborted after eight hours with only 25 per cent of the data load completed.
• Resolving the issue required a redesign of the data migration approach. 

Running the cutover dress rehearsal early allowed the work to be completed, and the programme was delivered on time.

Parallel runs 

If you are implementing a new finance system, your plans should include parallel runs. There are options for this activity:

• Complete parallel runs in the dress rehearsal (or another isolated) environment before going live.
• Go live and continue to run the legacy system for a period of time.

Project example – Benefits of parallel runs

An organisation implemented a new payroll system and ran parallel runs before go-live. The parallel runs required live data; using obfuscated data was not an option.

Significant issues were encountered during parallel runs that were not identified in UAT or prior testing, for example:

• Pension calculation issues.
• Payment issues caused by a problem with the data load process.
• Application configuration issues.

By completing the parallel runs, the organisation avoided substantial BAU issues that would have caused incorrect payments or missed payroll runs.

Documentation on an Agile Project

The Agile Manifesto was created in 2001.

The Agile Manifesto describes the four principles of agile development:

1. Individuals and interactions over processes and tools.
2. Working software over comprehensive documentation.
3. Customer collaboration over contract negotiation.
4. Responding to change over following a plan.

Manifesto for Agile Software Development (agilemanifesto.org)

There’s an important point to discuss before we go into more detail on Agile. Many people misinterpret the principles, and these errors can have serious implications for projects. They mistakenly assume that the items on the right are not important (or don’t apply at all) on an Agile project, but their assumption isn’t correct.

The Agile Manifesto's principles prioritise the left items, but the right items are still included. They are important, and the degree of importance depends on:

• The complexity of the project.
• The type of project. (Does it include new infrastructure, dependencies with other teams and complex data mapping?)
• Scope of delivery.
• Number of people on the project and number of teams.
• The type of project.
• Third-party documentation requirements. For example, a project in a regulated environment.
• On the topic of development in a regulated environment, be sure to confirm that Agile will be accepted. Some regulators stipulate that development and testing must be separated.
• The contract negotiation requirements based on the scope of activities and responsibilities of third parties.
• Requirements for post-go-live documentation.
• Processes and tools definitely have a place in Agile. Test automation is one example.

A plan will be required if you are working on a major transformation programme that involves:

• Development of new applications.
• Data extract, transform and load. (ETL)
• Integration build.
• New infrastructure.
• Data warehouse.
• Data lake.
• New security configuration.
•  Third-party API integration.
• Significant business change.
• Communication to the entire organisation.
• Application updates from third-party suppliers.
• Data mapping.
• Decommissioning legacy systems.
• Business architecture.
• Technical architecture.
• Programme risk management.
• Third-party security testing.

This does not diminish the value of Agile on the programme, especially for the development, but an overall plan will usually be required to coordinate this level of activity.

Project example – Transformation project

An organisation implemented a major transformation project that included new integrations with associated data mappings. The team was reluctant to document the data mappings or integrations because they were following Agile and quoted the Agile Manifesto every time the issue was raised.

The integration took considerably longer than expected, with the resulting delay in cutover to the new system.

They were also reluctant to document the configuration for the support team and referred to the Manifesto again. The support team made their position very clear: ‘If you don’t deliver the information, how do you expect us to provide support?’ 

The key learnings for the organisation from this project were a deeper understanding of the Agile Manifesto and the importance of communicating with everyone involved in the project, especially the support team you are going to hand over to.

Note: When the manifesto refers to comprehensive documentation, many IT professionals believe this refers to the 300-page requirements documents that were common at one point on Waterfall projects. I think we’ll all agree that such documentation is not required on an Agile project, but it doesn’t mean that a project team doesn’t do any documentation.

Speaking about documentation: I took over a project for a bank and had to read a doorstop document. On page 175, the analyst wrote, ‘If anyone is still reading this, please let me know, and I’ll see you at the pub. I’ll be very glad to buy you a pint!’

DSDM planning

The DSDM framework uses timeboxes that fix the delivery dates. Therefore, it is critical to know which items must be delivered. MoSCoW is a key element of planning deliverable items.

MoSCoW is a very useful method for feature prioritisation. It stands for Must have, Should have, Could have and Won’t have. (Won’t have at this time.) The requirements that are must-haves equate to the Minimum Viable Product (MVP) or Minimal Usable SubseT (MUST).

MoSCow is used to agree on the product backlog and to ensure delivery of key stories. MoSCow, along with timeboxing, ensures a useable product at the end of a timebox.

It is recommended that a timebox includes no more than 60 per cent must-have items.

DSDM phases

The DSDM phases are:

• Pre-Project – This includes preparing pragmatic terms of reference.
• Feasibility – Confirmation that the proposed project is feasible from a technical delivery perspective and that it will fulfil the business requirements.
• Foundation – Document requirements to the Epic level at a minimum, appropriate design work is completed, and business change is confirmed. The typical deliverables from this phase are prioritised high-level requirements and appropriate governance documents that cover quality assurance, the iterative development approach, architecture design, risk management, the test approach, delivery, cutover, and post-go-live support. A dress rehearsal document is recommended, although it is not included in the framework.

At the end of the foundation stage, the project will have cost estimates, timescales for the complete project and plans that cover the first increment. (Please note that deployments may start from the first increment.)

• Evolutionary or Iterative Development – The development is undertaken in this phase using timeboxes.
• Deployment – Transferring the development to production at the conclusion of the timebox.
• Post Project – Review of the project to confirm that the agreed critical success factors have been achieved and to discuss lessons learnt. It is best practice to prepare a short document on lessons learnt, including steps that could be taken to improve quality.

Please note: DSDM uses the term 'timeboxes' for iterative development, and Scrum uses the term 'sprints'.

Process (agilebusiness.org)

DSDM summary

The strengths of DSDM:

The framework facilitates delivering projects on time and to high-quality standards.

• Reporting and project tracking are supported.
• The work in the foundation stage reduces overall project risk.
• Strong budget control.
• Alignment with strategic business objectives.

The weaknesses of DSDM:

• There is a learning curve with DSDM. (However, this applies to most frameworks and methodologies.) Without training before embarking on a project that uses the DSDM framework, the project may not be as efficient or gain all the advantages DSDM offers.

Project example – Use of DSDM

A major hospitality organisation used DSDM to develop a sales and marketing system. The company had previously used Waterfall for all projects. DSDM was selected after a workshop on the pros and cons of Waterfall and Agile. The project had strong support from senior managers, business SMEs and technical specialists. A third-party company was responsible for the development. 

Before starting the project, DSDM training was delivered to the project team. This approach was much more effective than training the team during the project. Everyone understood their role and the differences in activities compared to Waterfall projects.

The business had been considering the project for several months before it started and had a long list of features that they wanted to include. MoSCoW was used to prioritise the features.

The senior stakeholders welcomed the benefits of the DSDM structure, especially the deliverables from the timebox approach. Seeing the priority features delivered quickly instilled a high level of confidence in the project. Additionally, the approach alleviated initial concerns about the risk of going over budget. The core functionality was delivered on time and to budget. Features were added to the backlog, and they were delivered in subsequent phases. 

The review concluded that the project delivered the required business benefits and that the use of DSDM had been successful.

Waterfall Methodology Overview

What is Waterfall?

Waterfall is one of the oldest methodologies. Many factors contribute to its widespread use today.

• The methodology is easy to understand and implement.
• It has been the standard methodology in many organisations for decades, and it is trusted to deliver projects.
• Waterfall provides a high level of control.

There are some projects where Waterfall is the default choice:

• Projects that include regulatory requirements. For many, if not most, projects of this type, there is a requirement to segregate the development and testing functions.
• Projects that are safety-critical. Systems that are involved in railway passenger safety would be an example of this.
• Development projects that require extensive documentation.

History of Waterfall

The name of this methodology comes from the sequential nature of the phases. Winston W. Royce created the methodology in 1970, and he explained that it was like water flowing from one rock to another; each phase is completed in sequence before the next one starts.

Mr Royce was aware of the potential risks with the sequential approach. He included prototyping in his original document, but unfortunately most people didn’t read that far and missed this critical point. 

It is a common misconception that project managers have adhered strictly to the Waterfall methodology (or it would probably be more accurate to state that they adhered to the common understanding of the methodology) since its inception in 1970. Like all methodologies, project managers have tweaked it as required to deliver projects. 

Mr Royce was an advocate of computing with incomplete specifications. He proposed starting tests before the implementation phase was completed and bringing some activities from the other phases forward. Another recommendation was building an architecturally complete prototype with restricted functionality very early in the project. Many early adopters of Waterfall eventually reached the conclusion that planning to build a throwaway prototype was best practice. This reduced risk and saved effort later in the project. It’s a shame that people writing the early Waterfall methodology books didn’t read all of Mr Royce’s work or speak to him. 

Having covered this history, it is important to understand that Waterfall even if the numerous books written on the subject moved away from some of Mr Royce’s concepts, is a valuable methodology. The number of companies using it and the volume of projects delivered by Waterfall each year confirms this.

Waterfall Observations

An observation from the first project that I managed using Waterfall was that many people had difficulty visualising the system based on the requirements documents. Even if the requirements documents contain mock-ups of the screens, it may still be challenging to understand how the complete system will function.

To overcome this, I used prototyping on subsequent Waterfall projects. The response was very positive, with many comments along the lines of, ‘This is great; now I can see how the system is going to work.’ 

However, on some occasions, the response was, ‘I’m glad we are seeing the prototype now. This isn’t what I expected, and it won’t work.’ 

The prototypes were prepared during the requirements-gathering stage and therefore did not impact the Waterfall stages.

Prototyping had the following benefits:

• Higher level of collaboration between the team and the stakeholders.
• Increased level of confidence that the system would meet the business requirements.
• The prototypes assisted with the sign-off of the requirements documents because they could see how the requirements would be delivered.
• There are several reasons why there could be issues with requirements documents.
• The analyst may have misunderstood the requirements.
• The business requirements may be vague.
• The analyst may be referred to operational documentation that is out of date.
• The business may have a difficult time understanding the requirements document and not question things that would cause issues after go-live.
• Demonstrations with prototypes bring clarity to the way the business will use the new system. They are a great way to mitigate risks on a Waterfall project.

Project example – Delivering a safety-critical system using Waterfall

A major chemical company in the UK had a requirement to automate part of a manufacturing process that included precise mixing of potentially volatile chemicals. A team of highly skilled chemists managed the process, and there was no tolerance for error. The worst-case scenario was destruction of the plant.

The requirements document was prepared, and it was checked thoroughly by the team of chemists. The next step was to prepare the prototype to demonstrate how the complete process would work.

This phase provided a higher level of assurance to the business and highlighted some additional points that needed to be addressed. The requirements document was enhanced, an updated prototype was demonstrated, and the requirements were signed off.

Following development and testing, the system was accepted into the first phase of production. The team ran the system in parallel with their BAU process for an agreed-upon period to ensure that the mixtures were correct. This timeframe covered all mixture combinations in use. After three months, the system transitioned to full BAU. Comprehensive technical documentation was prepared and handed over to the support team.

For more information, please request a free copy of the Pathway Project Management Toolkit ebook.

Good luck on your projects!