Can crowdsourcing and consensus targeting revolutionise discovery?
The Explorer Challenge was Australia’s largest open data crowdsourcing competition to date. OZ Minerals provided a $1 million prize pool and more than six terabytes of private and public exploration data to a globally distributed community of geologists and data scientists, who then developed groundbreaking approaches to predict mineralisation and identify exploration targets at the Mount Woods tenement in South Australia.
As economic mineral deposits have become increasingly difficult to find, explorers are seeking new approaches, innovative processes and ways of working that can drive up the discovery rate and speed up the exploration lifecycle, resulting in a more sustainable and efficient future for mineral exploration.
Despite exploring extensively around Prominent Hill over the past decade, OZ Minerals are yet to find another economic orebody in the area. The company wanted to find fresh insights and ways of working differently with their data. The Explorer Challenge provided approaches to mineral exploration that they never would have imagined internally, including ways to fuse datasets together, combining multiple layers of information, and making predictions based on the extensive datasets.
However, the Explorer Challenge was not the first crowdsourced exploration competition. Goldcorp ran the first iteration in 2000, followed by Integra Gold in 2015. The sporadic nature of these competitions makes them seem like one-off, independent events, with no clear links to driving widespread industry change or impact.
Is the Explorer Challenge any different? I certainly think so, for the following reasons.
‘Exploration geologists understand that the physical environment in which we work is incredibly complex on many levels.’
The concept of consensus targeting: increasing confidence, reducing risk
If you ask a geologist to tell you the level of uncertainty associated with their model and the targets they generate from it, you may get a blank look. What if an investor asked, ‘why are you drilling that target rather than this one; can you put numbers on why it is better?’
Exploration geologists understand that the physical environment in which we work is incredibly complex on many levels. This complexity means that there are millions of potential models that could legitimately represent the formation or signature of an orebody, but it is not feasible for the human brain, or even computers, to generate them, let alone test them. The result? We usually end up with one model that the exploration team broadly agrees on, that we know is wrong, yet think is good enough. We rarely have any understanding of the real uncertainty associated with this model and how it may compare to other alternative models. If we did, we would arguably significantly improve our ability to select targets and increase our discovery rates.
The consensus approach developed through the Explorer Challenge starts us down this path. This method allows us to compare and contrast alternative models and attach a level of confidence to our targets, just not in the way you might imagine.
As a scientist, if you think about comparing models, you might think about comparing their technical validity. Which one is more geologically sound and honours physics? Which one uses data more effectively? Perhaps you might also think, which model was created by the most expert geologist in this field, who has seen the most deposits? Again, quantitatively, it is very hard to compare in this way.
The consensus approach relies on relatively simple statistics – that the collective opinion is better than any one individual opinion or prediction (Surowiecki, 2004). This hypothesis is only valid for complex problems and environments like geology or hurricane prediction, where you can never truly determine that one model is better than any other because your environment constantly changes. Conversely, challenges that involve simple predictions do not benefit as much from aggregated predictions.
The consensus approach also relies on key inputs to be statistically relevant. Approaches entering the consensus must be independent, diverse, and trusted (ie technically valid).
In the practical case of the Explorer Challenge, this means combining the targets created by many different models into one. Figure 1 is a heat map of the 400+ targets generated in the Explorer Challenge, with clear hotspots where multiple models have predicted targets in the same area.
How would you feel if five geological models and five machine learning models all independently predicted the same target? I would argue that you would feel significantly more confidence in that target, as opposed to a target that was only predicted by one model.
The data can further be interrogated to see which approaches produced which targets. What did traditional geological approaches predominantly predict? Versus machine learning, versus specific machine learning techniques like neural networks and random trees?
We can now put a number on the confidence we have in our targets, based simply on how many valid approaches predicted them. This significant change can impact not just how we explore, but how we attract investment.
By using a crowdsourcing approach, the Explorer Challenge provided an environment that enabled consensus through several key factors:
• independence – the models were all developed completely independently and avoided groupthink
‘The consensus approach demonstrated in the Explorer Challenge provides a step change in the way we apply confidence to our targets.’
• synchronisation – hundreds of models were developed at the same time in parallel
• diversity – The models were created from different interpretations of data based on different, yet relevant, experience
• trust – the models were reliable as technically relevant and valid
• aggregation – the outputs of the models (the targets) were able to be aggregated meaningfully.
However, some key factors remain unanswered, namely:
• Conditions of entry – in the Explorer Challenge, submissions were subjectively judged on their technical robustness by experts in geology, mathematics and data science. Submissions deemed valid gained entry into the consensus model, with a total of thirty-seven models making the cut. The subjective nature of this approach introduces bias into the process. How could this be avoided? What does it mean to objectively qualify a model that can be included in the consensus as valid?
• Scale – what is a target on regional, local and deposit scale in a consensus model? How close do targets need to be to be considered the same? What is a target? In the case of the Explorer Challenge, we asked people to submit their top targets for economic mineralisation of base metals, with specifics on size and dimension.
• Delivering value to the expert community – for the consensus approach to really scale, meaningful incentives and rewards need to be provided and available for the expert community. If not, they will not continue to engage. This requires a new understanding of value transfer and a new business model.
Into the future
The consensus approach demonstrated in the Explorer Challenge provides a step change in the way we apply confidence to our targets. While more testing is needed, if proven, this method could drastically improve our discovery rates by enabling us to focus on quantifiably better targets.
The drill program at Mount Woods commenced in early Q4 2019, where the top consensus targets from the Explorer Challenge are being tested with an initial program of ~6 holes for 3000 m, to be followed by further drilling in H1 2020. During this program, OZ Minerals will live stream assay data to a select group of data scientists, which will enable predictions of drilling results to be made in near real time and significantly increase the amount of information available to geologists to make faster and more informed decisions.
The crowd approach has two other additional benefits, which may be more crucial drivers of adoption in the short term:
• Speed – the typical role of an exploration geologist is largely project management and execution. I would guess that 10-20 per cent of their time is spent on real interpretation and geological work. The crowd provides a way for quick interpretation and feedback on data, which is particularly relevant for small, time-constrained teams.
• Instant access to hundreds of experts – in an industry where we are used to hiring one or two consultants to support our internal teams, the crowd provides a unique way to get quick feedback from a much wider group of experts, all at the same time. This is particularly relevant when we are looking for multiple deposit styles or have higher uncertainty in the geological environment.
The crowdsourcing approach to exploration targeting generates hundreds of independent models and targets from data scientists and geoscientists around the world in just a few months. When the best of these are combined into one aggregated target map, this can reduce uncertainty, dramatically shorten the exploration lifecycle and may significantly increase mineral discovery rates.
So, what do you think? Do you agree that collective wisdom will shape the future of our industry? We would love to hear your point of view. Join Unearthed’s Exploration Newsletter to discover more: unearthed.link/FoEN
Thank you to all the next generation pioneers who took part in the Explorer Challenge. The knowledge and input you provided is driving signifi cant positive change to our industry.
Pioneers from the Explorer Challenge
First prize ($500,000) – Team Guru
Michael was the data scientist for the team, Jesse has a background in environmental science and GIS, and Glen has a background in process engineering in the fi eld of oil and gas and a keen interest in data science.
Team Guru deployed interpretable machine learning models for mineral exploration using geochemistry, geophysics and surface geology.
The team have invested their prize money in creating their business, Caldera Analytics, to build out their model and develop a global database of open data for use from tenement acquisition through resource definition.
Second prize ($200,000) – DeepSightX
The DeepSightX team exploited multi-disciplinary skills at the intersection of artificial intelligence and geoscience. Researchers from the Australian Institute for Machine Learning (AIML) and the Institute of Minerals and Energy Resources (IMER), both hosted by the University of Adelaide, collaborated with industry experts in minerals exploration (Austrike Resources) and geoscientific modelling (Gondwana Geoscience).
DeepSightX used a multidisciplinary approach to generate an AI model, DeepSight, which provides promising exploration targets in the Prominent Hill Region (PHR) supported by best practice geoscience.
DeepSightX has also created a business to continue focusing on this industry challenge. They will continue development on their solution, which uses machine learning to optimise drilling by reducing the number of drill holes and improving drill hole positioning.
Third prize ($100,000) – Cyency
Cyency has a strong data science and geoscience background. Hugh has been practising deep learning for several years, Derek has been involved with the technical and software side of mining for more than ten years, and Chris is an experienced geologist who is always looking for innovative ways of doing things.
The team started with their results from the Data Science Stream of the Explorer Challenge. They had a set of models that were good at predicting mineralisation across Australia, so they ran them over the tenement, applied several data science techniques to estimate a set of candidate points, and selected the top ten.
Cyency is currently working with a large Australian mining company to develop this technique and apply it to other areas. They use applied deep learning and image processing techniques in their approach.
Surowiecki J, 2004. The Wisdom of Crowds, 336 p (Doubleday).