Before executing a huge project, enterprises often undertake a pilot which can decide the course of the investment and deployment. Joe Lipscombe looks into the industry and identifies the factors that make for a successful pilot.
In an ever-growing industry with a vastly changing business landscape, ideas and projects can fall by the way side relatively unnoticed if the right preparation hasn’t been done by the responsible enterprise. In this respect, certainly one of the best options for businesses is to conduct a pilot for a proposed project in order to ‘test the water’ before diving in.
However, conducting a pilot will involve utilising many of the resources that you would for the subsequent project. What are the best ways to approach such an operation? And what considerations do you have to bear in mind when starting out?
After researching a wide range of business areas, we can start to get an idea of why companies employ the pilot strategy and what benefits can be found in the results.
“Risk and cost control are probably the most significant benefits of most pilot projects,” says Seyed Golkar, director of business solutions at GBM.
“The primary objective of a pilot project is to create a cut down version of a system in order to assess or analyse some of the salient functional or non-functional features of an application or a system. As such it will allow the users to realise and observe some of the key requirements,” he adds.
If the basis for some pilot projects is to create a reduced version of the major project, is it possible to conduct a pilot on a previously executed platform in order to save even more on costs and risks? Jimmy Joseph, project manager at Omnix International, believes so.
“We use an already executed project as a template for our pilots,” he says. “With this method we eliminate the fear and cost factors. We take a completed building, for example, and implement a new technology on the structure. That way, if we find faults or issues, we can address them cost free.
“It’s also a great platform for training. Experiencing the pitfalls of a major project in the pilot phase is priceless for training purposes, overcoming these pitfalls and learning how to adjust is a key factor.”
Concerns
One of the main concerns with a pilot is the involvement it must have with the client. How much input should an end-user have with this phase of the project? Should the pilot be left entirely to the hands of the distributer?
“The goal of the pilot must deliver a business result that project sponsors can relate to and also use to verify the value of the pilot,” says John Avery, partner for SunGard consulting services.
“For example, implementing a new technology or a new system with a simple ’hello world’-style test is typically not sufficient enough to pilot business impact, representative technology implementation effort or production deployment,” he adds.
As well as a cost saver, a pilot can also act as a time saver by granting end-users the opportunity to identify and assess issues which they may have encountered at a more critical time of a major deployment.
Golkar says a pilot may help an end-user organisation to analyse and evaluate features that are critical to the success of its operation.
“For instance, if the system that is being piloted is of a critical nature, then features such as availability, throughput or capacity can be sampled through the pilot system so that the end-users can gain sufficient confidence before embarking on the large scale implementation and deployment,” he says.
“Similarly, if a particular feature or function of the system is of critical nature to the effective use in an end-user organisation, then a pilot can be used to expose such feature or function to the end-users ahead of full implementation and roll-out. If successfully executed, a pilot will allow organisations to mitigate significant risks well in advance of big technology deployments,” he adds.
Embarking on a pilot, regardless of its time saving capabilities, does of course take time. When a sponsor is chomping at the bit to begin a technological deployment, how much time should they allow for a pilot to take place?
“Maximum time, ideally for us, would be a month,” says Joseph.
“You don’t need six months or a year if you’re employing our strategy of utilising a previously completed project as the platform for the pilot. You don’t need the entire project in order to fully conduct a pilot. Don’t build a 20 storey building, build a complex single floor, for example, and apply all the elements to that floor which may be a potential risk factor,” he adds.
Najla Naim, head of CPM RAN & IP BB Core at Ericsson ME, supports the view of a short and robust pilot plan.
“The pilot should not take more than four to six weeks. This is to make sure that technological advancements are happening at shorter and shorter lead times. Speed to implement a pilot, collect learnings, optimise service and then decide on commercial deployment is crucial,” she says.
Knowing how long it will take to address all potential risks and issues may be a headache waiting to happen, but enterprises cannot afford to remain in pilot mode forever. There has to be a clear plan which allows them sufficient time before eventually biting the bullet and deploying the live project.
“A clear definition of the objectives, scope and measurements of a pilot are critical in achieving a successful outcome. An agreed and understood timeline, stakeholder engagement and end-user involvement from the inception point all the way through to the execution of the pilot are also crucial,” says Golkar.
Considerations
Once an enterprise knows exactly why it is implementing its pilot and all its plans are in place, what considerations must it make before it officially begins?
“It is important for the pilot team to have a clear understanding of the internal and external factors that make the pilot different to the final target system. These factors and their effect must be considered when evaluating the pilot results,” Golkar states.
“It’s important to set up a representative environment that would loosely resemble the critical features being tested, proven and demonstrated,” he adds.
The location of the pilot is an important factor enterprises must consider, according to Joseph.
“For example, you cannot predict climate conditions. You can’t do a pilot for a project in one part of India then take it live in another part of India because of the unexpected climate conditions. In the Middle East it’s easier, because the climate conditions are very similar, that’s helpful for us here in this region. Pilots run more successfully here for that reason,” he says.
Naim adds: “The key considerations are to prepare for the pilot thoroughly for maximum efficiency during and after its deployment and to align fully with the vendor on objectives, scope, time frames, roles and responsibilities. Both entities should be on board working as one team.”
Pilots can often begin to feel very much like a live project. So what are the actual differences between the two?
“A pilot doesn’t have to be, and usually isn’t, a complete test or execution of all system functionality. Its primary focus must be on testing, proving and demonstrating areas of functionality or features of the system that are critical and sit at the highest levels of priority. If not, there is a big risk that a pilot would become a large-scale project and thus lose its effectiveness,” says Golkar.
It is limited in application to a lower number of users and demonstrated the features in a more controlled environment, Naim adds.
“This may limit the testing of the functionalities and features in terms of load tests and impact of a large volume of transactions on the performance of the network and the platforms operating the services.” she says.
A pilot, in a sense, may be out of the end-user’s hands. What kind of a guarantee can the sponsor receive before a pilot is implemented and what kind of involvement can they have during the process? The best option might be for the sponsor to invest in the pilot as a gesture of commitment to the project, Naim believes.
“I think the end-user needs to invest partially on the pilot as a sign of commitment to the business case. I think a model of shared investment with the vendor should be applied,” she says.
Declining
The end-user should, of course, be involved with the pilot to a degree. But does it ever get to a point where a company should decline a pilot? What sort of conditions do companies come across that may influence them to reject a pilot altogether?
“A pilot is not, and should not be seen as, the first phase of a large scale deployment. This is one of the most frequent mistakes made by organisations seeing a pilot as just the start of the large scale project. It is not, and under such situations one should avoid embarking on a pilot,” says Golkar.
Joseph adds: “If there is deadline, decline it, if the client is focused on just speed, it’s not worth it.
“If there is a tight deadline we would usually decline. It really depends on the client – if they embrace new technology and can offer us the time to implement it properly, then we would be happy to work with them.”
Once all the key elements of a pilot have been addressed, the pilot can get underway. However, even with substantial planning an enterprise cannot guarantee that you’re going to achieve what you set out to.
“Commercially paid pilot success rates are close to 100%. Sometimes they can be delayed but will still happen. Pilots done for experimentation and end-user experience only, which are free of charge, have a much lower success rate of between 0 and 30%,” Naim claims.
But who loses if a pilot doesn’t offer up the projected outcome, if anybody at all?
“If the pilot fails it’s good for us as we learn and develop new methods, but the client might be not happy, so it’s a difficult outcome to deal with. The standards and templates can always be reused, so the pilots can be deemed as successful throughout,” Joseph explains.
