Skip to main content

Posts

Showing posts from 2022

From DBA to Data Engineer in Azure

I recently moved a role From being a DBA Manager, Who is responsible for the operational databases. I moved to manage the data engineering group. So what exactly is the difference between the two functions? DBA - Production Databases: SQL\ NoSQL- 24*7, powerful server on premise or on the cloud, managed or semi managed, security tasks, high performance is a target, multiregional, HA as top priority. Developers are using Microservices - so we have many applications many services and many many Databases. Many kinds of DB's like Cloud IAAS and PAAS. Secure and audit the data is must. The Clusters must have Uptime as long as we can achive. Data Modeling - is so important too. Challenges and Problems in the data bases systems Lots of DB’s Lots of creators / no standards Lots of Consumers (Query, tools, SLA) Raw data Lots of data resources Data silos In Data Engineering we have other challenges for example we have Data lake and Data Warehouses : Batch process. Stream Process. many data ...