Undelucram.ro on behalf of: SoftServeAbout The RoleIn this role, you will drive a critical migration of AWS data pipelines from HDF5 files to Iceberg tables, working with C++, Python, and large datasets to modernize a high-performance data architecture. Our Client is a London-based leader in harmonized historical order book and market data, powering advanced trading analytics and AI-driven insights. You’ll collaborate in a focused team to deliver scalable, efficient data foundations that directly impact global capital markets.ResponsibilitiesDesign and implement new AWS data pipeline components in C++ and Python to switch output from HDF5 files to Iceberg tables, improving efficiency and modernizing the data architectureOwn end-to-end migration tasks within the AWS-based data pipelines, ensuring scalable handling of large datasets and minimizing risk to existing data flows and performanceCollaborate closely with a small, focused team to coordinate changes across C++ services, Python tooling, and AWS data workflows during the migration from HDF5 to IcebergAnalyze existing C++ logic that generates HDF5 files and refactor or extend it to produce Iceberg tables, maintaining data integrity and schema consistency across large datasetsIntegrate Python-based utilities and scripts into the AWS data pipelines to support data transformation, validation, and loading of large datasets into Iceberg tablesOptimize data processing paths in C++ and Python for large-scale datasets within AWS, aiming to enhance throughput, reliability, and scalability of the new Iceberg-based pipelinesTroubleshoot, debug, and resolve issues arising during the transition from HDF5 file generation to Iceberg table creation in AWS, ensuring stable and predictable pipeline behaviorRequirementsStrong proficiency with modern C++Practical experience with C++ development for data processing pipelines, including performance optimization and debuggingHands-on Python skills and experience with building and maintaining data processing scripts and utilitiesHands-on experience with designing, implementing, and maintaining data pipelines in AWS environmentsUpper-Intermediate English level or higherAnalytical and solution-oriented mindset with strong problem-solving and troubleshooting skills in complex data environmentsPractical experience with data formats and serialization, including HDF5/Iceberg, and migrating or transforming data between formats (nice-to-have)Experience with big data frameworks or ecosystems (as an advantage)SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law. Show more Show less