Who We AreOur client is a world-leading reinsurance and risk management company, delivering comprehensive solutions across insurance, underwriting, and data-driven risk assessment. With a strong focus on innovation and long-term stability, they support clients in addressing complex risks and driving sustainable value in an ever-changing global landscape.We are seeking a highly skilled Data Engineer to enhance our data and analytics environments. The successful candidate will play a pivotal role in designing, developing, and managing data solutions that leverage cloud technologies and big data frameworks. This role will involve creating efficient data pipelines, optimizing data storage solutions, and implementing robust data processing workflows to ensure high-quality data availability for analytics and business intelligence.What You'll Be DoingCreate and manage ETL processes using Azure Data Factory to automate data flow between various systems and servicesImplement data storage solutions using Azure Data Lake and Cosmos DB, ensuring high availability and scalabilityDesign and optimize data processes in Databricks, implementing Spark jobs that utilize Delta Lake for efficient data handling and versioningDevelop distributed data processing workflows using PySparkBuild and maintain analytics models that can harness the capabilities of Delta Lake for better data quality and performanceDevelop scripts in Python and PowerShell to automate operational tasks, deployments, and monitoring of data workflowsCollaborate with development teams to integrate CI/CD practices using Azure DevOps and Git for version controlManage and optimize MS SQL Server databases, writing efficient T-SQL queries for data manipulation and reportingEnsure data integrity and security through proper database management practicesImplement logging and monitoring solutions to proactively address any issues impacting data applications and workflowsConduct root cause analysis on data-related problems and develop solutions to mitigate future occurrencesCollaborate with business intelligence teams to create and share reports using Power BIProvide insights and recommendations based on data analysis to guide business decisionsWhat You'll Bring AlongBachelor’s degree in Computer Science, Information Technology, or a related field is preferredMinimum 5 years of experience in a similar roleProficiency in Python and PySpark for data processing, optimization, and automationExperience with PowerShell scripting and Azure Data FactoryStrong understanding of MS SQL Server and T-SQL for database scriptingFamiliarity with Databricks, Delta Lake, Azure Data Lake, and Cosmos DBExperience with version control using Git and CI/CD frameworks in Azure DevOpsSolid problem-solving skills and ability to troubleshoot complex data issuesFamiliarity with DevOps methodologies including Scrum and Agile practices for software developmentExperience creating dashboards and reports using Power BI and Knowledge of data governance and data security best practices.Excellent command of both spoken and written English. Show more Show less