Two New Zealand IT companies, Open Parallel and Catalyst, are at the forefront of the largest IT project in the world, the Square Kilometre Array (SKA) radio telescope.
The SKA is one of the most complex projects ever conceived; a global effort involving about 350 engineers, scientists and researchers from around the world working together on the SKA’s design and development, planning to build thousands of radio telescopes in Southern Africa and Australia between 2018 and 2024, to monitor and survey the sky.
The SKA will scan the skies thousands of times faster than previously possible, producing vast amounts of data.
Data rates involved will require real-time analysis of 10 terabytes per second; the equivalent of streaming one million HD movies at once.
The SKA project is a global enterprise involving eleven countries and represents one of the biggest IT challenges of our time.
Its Big Data requirements are a trigger for the development of new IT concepts, in both hardware and software.
Tender for the construction of the first phase of the SKA will be in 2017 for 650 million euros, a significant part of that being dedicated to computing.
In the lead up to that procurement process, Open Parallel's first contribution to the SKA was delivered last month.
The team, which included Catalyst engineers, devised and delivered the initial version of a Software Development Plan for how participants in the project will develop and deliver software and/or firmware to achieve design goals established for the SKA.
"Open Parallel's innovative and effective approach to software development is also applicable to other industries... and ensures that benefits from the SKA spill over into industry applications e.g. in the areas of optimisation and performance, cloud and green computing, the transport, storage and processing of Big Data,” says Dr. Agnes Mika from ASTRON, the Netherlands Institute for Radio Astronomy, and SKA's Science Data Processor Compute Platform work package leader.
“We are excited to support Open Parallel's participation in the SKA project," adds Don Christie, Director of Catalyst.
"The Catalyst Cloud, a flexible and powerful platform built on OpenStack, supports the sort of innovation, flexibility and efficiencies that these large scale projects need to succeed.”
Nicolás Erdödy, Director of Open Parallel adds: “SKA is an enabler of opportunities; the software stack for massive systems that we are designing can have applications to New Zealand problems, from primary industries to complex services and systems like the IRD transformation project."
Christie and Erdödy will be participating in the 2014 SKA Engineering Meeting in Perth, Australia at the end of September, to discuss the software platform and status of the SKA design with scientists and engineers who will converge from all over the world. Open Parallel and Catalyst are the only industry participants from New Zealand.
Open Parallel's team is lead by its founder Nicolás Erdödy and includes media platform pioneer and author of DirectX, Alex St John, and global expert in scalability and distributed systems, Rob O'Brien.
Open Parallel's team works in association with Catalyst, the largest open source software company in Australasia.
New Zealand's Open Parallel is the only IT company leading two work packages of the SKA: Open Parallel leads the Software Development Environment work package for the Central Signal Processor (CSP) and the Common Software work package for the Science Data Processor (SDP).
The Software Development Plan has been produced as a self-funded New Zealand industry contribution to the SKA project by members of Open Parallel team and associates from Catalyst.