Sr. Data Engineer / Lead Data Engineer

Aurum Data Solutions

Onsite Opportunity Sr. Data Engineer / Lead Data Engineer

Client: MNC

Experience:8+years

Location: Ras Al-Khaimah, UAE

Job Mode: Permanent

Note:

• Selected candidates must hold a valid passport.
• Selected candidates should be open to travel to Ras Al-Khaimah, UAE.
• Candidates must possess strong communication skills.

Key skills: Python, Data Engineer, Databricks, SQL, Shell Scripting, Snowflake, PySpark, dataframes, Azure Data Factory, Apache Airflow, and Databricks workflows, FastAPI, Azure etc..

Job Description:

We are looking for a skilled Full Stack Developer with hands-on experience in building scalable web applications using Python (FastAPI/Flask/Django). You will work closely with the data engineering team to develop high-quality software products and features that delight our users. He/She also needs to ensure security is considered throughout the entire software development lifecycle. You will play a critical role in building scalable, resilient, and secure data platforms that power analytics, AI, and data-driven innovation at customer.

Education:

• A bachelor's degree in computer science, information technology, or a related field is required; a master's degree is preferred but not mandatory.
• Minimum age 21.

Experience:

• 5 to 7 years of hands-on experience in data engineering, demonstrating consistent career progression and technical growth.
• Proven ability to design, develop, and deploy highly scalable and efficient data solutions for complex business needs.
• Extensive experience managing and optimizing complex data integrations across diverse systems, platforms, and cloud environments.

Skills / Knowledge:

• Advanced proficiency in programming languages such as Python, SQL, and Shell Scripting, with the ability to implement optimized and scalable code solutions.
• Deep expertise in data platforms like Snowflake and Databricks, including extensive experience working with PySpark and distributed dataframes to process large-scale datasets.
• Advanced knowledge of orchestration tools such as Azure Data Factory, Apache Airflow, and Databricks workflows, including the ability to design and manage complex, multi-step workflows.
• Significant hands-on experience with tools like DBT for data transformation and replication solutions such as Qlik for efficient data migration and synchronization.
• Strong understanding of big data systems and frameworks, with practical experience in building and optimizing solutions for high-volume and high-velocity data.
• Extensive experience with version control tools such as GitHub or Azure DevOps, including implementing CI/CD pipelines for data engineering workflows.
• Advanced knowledge of serverless computing, including designing and deploying scalable solutions using Azure Functions with Python.
• Proficiency in API development frameworks such as FastAPI, with the ability to create robust, efficient, and secure data-driven APIs.
• Comprehensive expertise in designing and implementing ETL/ELT processes with a focus on performance, scalability, and maintainability.
• Proven experience in data warehouse development, including hands-on expertise with dimensional modeling and schema optimization for analytics.
• Solid English language communication skills, both written and verbal, for effective collaboration across teams and stakeholders.