Trials: Three common pitfalls
test driving a few alternatives first. As IT leaders, we often apply the same principles and run trails before a rollout decision. Yet most of us have owned cars we didn’t like and rolled out systems that didn’t deliver what we’d expected based on our trials. So why is that?
Of course, there’s no one reason for this; if there was, we’d understand it and avoid it.
The more common problems arise when we don’t test drive long enough or in a way that truly reflects our requirements – a quick scoot down the A4 isn’t the same as 5 years of school runs, commuting and family holidays.
Less obvious, but equally problematic, are the following three common mistakes which we try to help our clients avoid; Do any of these sound familiar?
Trial them both; then we’ll know
The idea of a multiple solution trial, like test driving several different vehicles, makes sense on paper and may well be necessary. It’s important that this isn’t done simply to push back critical appraisal – “we’ll find that out in the trial” can be a lazy way to avoid robust up-front research and questioning. Multiple solution trials are almost invariably more expensive and can leave a business confused as to which solution or supplier is best. Different solutions can create subjective arguments about the best approach. If you know your requirements and resources – Or if you don’t, why are you considering a trial? – what’s stopping you selecting the most likely best-fit product or supplier now?
The never ending trial
How long is long enough for a a trial? A week? A month? A year? It depends what you’re trialling, of course. Medical research requires drug trials that can sometimes run for decades. However if we’re dealing with technology and it’s still in trial after two years, something has probably gone wrong – the technology is probably half way to obsolete! The key here is in the planning and measurement of trials; what’s the minimum period we’ll need to prove, a minimum level of success for us to proceed with a product or supplier? If we don’t know, we’re probably not ready for a trial because we’re don’t know well enough what success looks like.
Trial Stockholm syndrome
Ok, so it doesn’t do what we wanted it to do and half the functionality is still “in the pipeline” but over time we’ve come to love this little sucker and we’ve invested so much love and effort into it… can we keep it? Pleeease?
Well – maybe… but let’s be really clear our original goals remain unachieved and do something about that – and let’s also be really clear what this little monster is good for.
This list is not definitive! There are hundreds of other things to think about when considering a trial. We’d love to hear your thoughts on this article and if any of the issues raised are on your agenda right now, we’d love to see if we can help. Contact us today.