Exploring Big Data Solutions for Southern Africa

In its role as coordinator of the Big Data Analytics and Transboundary Water Collaboration, the Sustainable Water Partnership (SWP) facilitates the integration of a group of public, private and nonprofit organizations researching the use of big data to improve management of the Ramotswa transboundary aquifer in Southern Africa.

The Collaboration partners have sponsored four research teams, each looking at different ways to use big data analysis to improve water security for the Ramotswa Aquifer, shared between Botswana and South Africa, and surrounding water systems. The projects have finished data collection and are now producing software architecture to analyze it. From April 20 to April 24, SWP and the University of the Western Cape (UWC) in South Africa, which leads the integration of the research outcomes, coordinated a workshop on Data Storage Solutions. It was attended by partners of the Collaboration and representatives of several regional and international entities that host data storage and visualization tools for water systems.

Partners in the Big Data Analytics and Transboundary Water Collaboration include USAID, SWP, the Southern African Development Community’s Groundwater Management Institute, IBM Research Africa, the U.S. Geological Survey, the Republic of South Africa’s Science and Technology Department and Water Research Commission.

During the workshop, SWP and UWC challenged partners to look beyond traditional research outputs and identify a collective data storage and visualization platform to showcase the Collaboration’s work and assist water resources managers in the region. The first session was led by Dr. Kevin Pietersen and Prof. Antoine Bagula from the UWC. They gave attendees an overview of the work done to date by the four research teams and opened discussion on the criteria of an ideal data exchange platform to serve as a medium-to long-term solution. Participants outlined the need to include (1) continuously updated data and (2) a visualization tool to be used by decision-makers for the Ramotswa aquifer.

The second and third sessions of the conference featured presentations on existing and inspiring solutions from Southern Africa and other regions (with both regional and global scopes). Both sessions were organized by Umvoto’s Andrew Gemmell, who leads the first research team for the Collaboration, which includes experts from the United Nations International Groundwater Resources Assessment Center (UN-IGRAC). Claudia Ruz Vargas and Arnaud Sterckx of UN-IGRAC presented two type of solutions. The Global Groundwater Monitoring Network (GGMN) Programme has created a global visualization tool for transboundary aquifers worldwide that relies on a variety of big data flows, like monitoring stations. The Southern Africa Development Community (SADC) Groundwater Information Portal (GIP) is based on a 2010 hydrogeological mapping project of the region. This portal is mostly historical in nature, but new layers can be added in the login portal. Also, it will now combine data of the 2010 project site with the Ramotswa Information Management System, which is another IGRAC project in the region.

From the International Centre for Water Resources in Germany, Dr. Philipp Saile presented on the Global Environment Monitoring System (GEMS) of the Groundwater Data Centre of the United Nations Environment Programme, which he heads. From the Global Runoff Data Centre in Germany, Ulrich Looser presented on the experience of managing large databases and related visualization tools. Both presentations provided insights to the partners on relevant issues like data collection, data convergence, system management and users’ rights.

Finally, Dr Anwar Vahed, Director of National Integrated Cyber Infrastructure System (NICIS), managed by the South African Centre for Scientific and Industrial Research (CSIR), presented the Data Intensive Research Initiative South Africa. NICIS allows national research institutions to use federated data solution systems, which combine data storage and visualization. Sponsored by the national government and compliant with the regional Cyber-Infrastructure Framework, access and use of the NICIS is entirely free and provides a medium-to long-term solution for activities such as the Collaboration’s research projects.

Ultimately, the best solution identified was a Federated Cloud Infrastructure, where the data that flows from the four projects can converge. With a way forward to converge the outputs of all projects into an integrated solution, SWP and the partners will begin to engage with CSIR to ensure the UWC will be able to consolidate the integrated tool for supporting decision-making in the Ramotswa Aquifer. Having seen the possibilities of similar work, the partners are excited to push past the traditional boundaries of their research and see the Collaboration produce a valuable resource to enable smart decision-making for transboundary aquifer management.

This article was written by Dr. Clara Bocchino, program coordinator for the Big Data Analytics & Transboundary Water Collaboration for Southern Africa.