19th July 2022
We often hear about A/B testing as a tool for driving change. But what is it and how can we use this approach to improve our L&D initiatives?
In this week’s Learning Data Explorers session, Steve and Natasha looked at the how A/B testing woks, why this can be a powerful tool and what you need to be aware of when implementing this approach.
A/B testing, is a way to effectively use data to evaluate the impact of a change. Leaders don’t want to make decisions unless they have evidence based on data..
Steve went on to speak about his personal experience of using A/B testing in the various firms he had worked at and stated that it can be a powerful tool to use. It’s about making a change and seeing the impact it has. Steve, went on to say that A/B testing isn’t always the right approach as the rigour does take a fair bit of work. But it will give you the gold standard result if you do a true random A/B test.
Steve, continued talking about of how A/B testing works. The essence of A/B testing, is that you take two groups that have been randomly assigned, make a change or add a variation and compare the results of Group A versus group B. If the difference is significant enough, then you say the changes have had an impact. If the two are very close, then you say there’s been no impact. There is a statistical test you can use to objectively say if the difference is actually ‘close enough’.
A key element of the process is to articulate out what your Null hypothesis is. You need to take care as a poorly thought out hypothesis will almost certainly break your experiment, When creating your hypothesis, you want to make sure that it is:
- Focused on one specific problem you want to solve or understand
- Able to be proven or disproven
- Focused on making an impact
If you have any questions about the A/B testing or would like to be on the sessions, get in touch with us hello@e3ct.com
