From web apps to AI tools, and from scientific research to space missions, proper data management is critical. This significance is highlighted by the emergence of DataOps, Data Engineering, and other related disciplines that are specifically focused on the efficient, correct, and consistent handling of data.
While a great deal of attention has been paid to “Big Data” challenges, where solutions have to cope with quantity and/or flow rate of data large enough to overwhelm conventional tools, “quantity” is not the only aspect of data that can require special attention. Other particular needs can include reliability (low data point loss rate), accuracy (low data point error rate), uptime (low/zero interruption in data processing), and other metrics.
In this event, we’ll hear from Chase Million, founder and CEO of Million Concepts, a research and software consulting company with a deep technical bench, broad expertise, and a strong commitment to open source. He and the Million Concepts team have been involved in operations for NASA’s last four Mars rovers and the Galaxy Evolution Explorer astronomical telescope. They have also worked on data from the full range of astronomical and planetary missions.
Although one might expect the data needs of space missions to differ dramatically from commercial or enterprise applications, Chase and team have not found this to be the case. Although every operating environment is unique, managing and leveraging mission-critical data in any context requires the same skillset, care, and attention. This discussion will focus on Million Concepts’ current work to help develop a ground data processing workflow and orchestration system for the eight cameras aboard NASA’s upcoming VIPER mission to the south pole of the Moon. We will use this project as a case study to explore the considerations, trade-offs, and technical and non-technical approaches Million Concepts uses to evaluate the data needs for a project at hand and implement a suitable solution.
The recording for this event is available here.
