Data Management is one of the central challenges in developing modern software systems. The need for more sophisticated Data Management is even more emphasized in the current times of Artificial Intelligence and Big Data-based systems which have even more demanding data requirements than traditional Data Management had to consider.
In Data Engineering, we focus on preparing data for its deployment or usage in a complex AI/data-driven system. This covers for example discovering data, cleaning data, transforming data, or integrating data from heterogenous sources. Also, there is a focus on (domain-specific) meta-data creation and management. Furthermore, aspects of data biases and potentially arising societal issues like misrepresentation and unfairness become focus area. Data Engineering topics are often seen in the context of their application domains, like Digital Humanities, medicine, but also business application like banking.
In Scalable Data Management, the focus is on how to cope with the ever-increasing demand for storage and processing power by scaling data operations. This covers for example methods for stream-processing but also flexible distribution schemes or the deployment of scalable AI-models.