A Big Data Arms Race

There is no escaping it: ‘[w]e are living in the data age’ (House of Commons, 2016, p. 3) and the report goes on to state that global data is predicted to grow at 40% every year over the next decade, effectively doubling every other year. In today’s world, the need to consider what to do in the moment, but also, what may happen as a consequence to that action, becomes increasingly important. A hasty posting of a picture, an emotional text message or a new status update – the consequences of a fleeting moment can now last an eternity, but it can also build into predictive models of the future.

The way in which we engage with the collection of this explosion of data will be important, both in a growing perspective of societal awareness, as well as the subsequent impact that heightened awareness will have on the world. In fact, it has already shown a profound impact in the political arena, as the Paris Agreement on Climate Change (United Nations, 2015) demonstrated, by building its core arguments on masses of global datasets. The exponential growth of Big Data and its use around the world is only trending in one direction (not the band), such is its popularity, impacts and uses.

Some of the most compelling statistics provided to the House of Commons from the ‘The big data dilemma’ report the impact that big data could have on the UK economy.

The stakes for the UK economy are massive. Big data is already a UK success story but it has huge unrealised potential, both as a driver of productivity and as a way of offering better products and services to citizens. An analysis in 2012 calculated that big data could create 58,000 new jobs over five years, and contribute £216 billion to the UK economy, or 2.3% of GDP, over that period. In the public sector, big data can increase the operational efficiency and targeting of service delivery. (House of Commons, 2016, p. 3)

Big Data in Numbers
Figure 1. House of Commons, 2016, p. 6

From one perspective I can understand the concerns surrounding Big Data. The increasing digitalisation of individuals through their unique digital footprint or the Big Brother debate, including the questions whether anyone should have access to your data, without your consent? That feeling of your private sphere being invaded, analysed, and used to market to you ever more accurately. That being said, many of the services we enjoy online for free are often subsidised by the anonymised trend data; and in many cases, the data improves the service or experience. The inevitable (and quite depressing, or liberating) truth is… I am predictable! Yep, you heard it here first, my actions, my choices, my preferences and even my ‘likes’ – are all predictable (not 100%, but nearly). Even worse than this… yours are too! Scientists at Northeastern found that human movement behaviour was 93% predictable (Song, Qu, Blumm and Barabasi, 2010). Apart from the movements you are doing right now, just to prove you are unpredictable, such as: dancing in your office, waving your arms madly or switching your phone off. Maybe you too feel something about being predictable?

At first, I found it eerie and concerning that all my life I had felt in control and the perception of control came with the alleged unique choices I made. But when you consider how many choices you actually make each day, starting with which side of the bed to jump out of, you realise many are hardly choices at all. They are, in fact, habits. It is those predictable habits that Big Data can pick up on easily and as datasets become more open and can be cross-referenced more easily with other datasets, the predictability will go up. The ‘Data Arms Race’ has been written about by many authors (Steinberg, 2014 & Strong, 2015) but one of the areas where it is starting to gain serious traction is in Learner Analytics (Sclater, 2014) thanks to the world leading work being done by Jisc. This is clearly seen in evidence provided by Jisc into the Big Data Dilemma where they stated (in evidence BIG0027):

Appropriate engagement with the public in big data initiatives (particularly the segments of the public from whom data will be collected) is crucial. Indeed, in the case of the creation of learning analytics, Jisc recognised at an early stage that potential ethical and legal objections to learning analytics were barriers to development of the field, which could deny students and staff of higher education institutions the benefits of predictive analytics and adaptive learning.

In practice early indications are that users value the benefits which big data can provide. Survey data from Nottingham Trent University, an early adopter in the use of learning analytics, suggests students are “strongly positive” about analytics, with 93% of them stating they wanted to be warned if they’re at risk of failure. (Jisc, 2015, p. 2)

When I was training as a Sport Psychologist at the University of Chichester, I was taught two things that transformed how I looked at events in my life. The first was to ‘Control the Controllables’ (thanks to Dr Ian Greenlees and Prof Tim Holder) – focus on what you can control e.g. the time spent training or the analysis of an opponent’s previous performances. Secondly, the biggest predictor of future performance was previous performance, which haunted me as Director of Learning and Teaching at the University of Winchester. Knowing that improvements to an individual’s performance are often small and incremental, until concept thresholds are passed or key skills are learned and become autonomous, can leave many frustrated in their learning and perceptions of improvement. You then only need some marker variation in a department and the perception of progress against a standard can be destroyed, with it the motivation, confidence and sometimes persistence of the learner.


Big Data will change that profoundly! I believe that triangulated data sets in education will:

– create transformative levels of awareness in individuals, institutions and the sector, therefore allowing expedited achievement of concepts, efficiencies and greater achievement of human potential
– identify effective habits and encourage those to be practised and strengthened (possibly through gaming or micro-credentialing)
– provide institutions with evidence to better enhance support, create proactive prevention as opposed to the reactive cures, and enhance retention as well as student success
– allow students the capability to better understand the expectations of Higher Education via modelled exemplars and trend data regarding successful habits and performances to adopt

No doubt there will be many more; but the impact, excitement and potential of this field within education is possibly the most exciting development I have witnessed in my career to date. A move from proxies of teaching excellence to explicit data points and habits within the learning journeys!

The race is on and I believe the improvement of student learning will lead the pack – at least I have a fiver on it!

Reference List

Create citations in APA format, MLA format, Harvard referencing, and more!

House of Commons (2016) The big data dilemma: fourth report of session 2015–16 (HC 468) (Accessed: 29 February 2016).

Jisc (2015) Written evidence submitted by Jisc (BIG0027). (Accessed: 7 March 2016).

Sclater, N. (2014) Code of practice for learning analytics A literature review of the ethical and legal issues (Accessed: 29 February 2016).

Song, C., Qu, Z., Blumm, N. and Barabasi, A.L. (2010) ‘Limits of predictability in human mobility’, Science, 327(5968), pp. 1018–1021. doi: 10.1126/science.1177170.

Strong, C. (2015) The big data arms race part one: Marketers’ perceptions. (Accessed: 29 February 2016).

Steinberg, D.A. (2014) The big data arms race is on. (Accessed: 29 February 2016).

United Nations (2015) Adoption of the Paris Agreement (Accessed: 29 February 2016).