Senior Data Engineer (Azure / Databricks / Python)

Location: Remote

Type: Full-time

Level: Senior

Duration: 4 months

Start: ASAP

About the role

We are looking for a Senior Data Engineer to design and implement scalable data pipelines supporting advanced, customizable data processing workflows.

You will be responsible for building end-to-end solutions that ingest data from streaming sources, process it in distributed analytics environments, and expose curated datasets through well-defined interfaces. The role also involves integrating external machine learning workloads into an existing cloud-native architecture.

This position is well-suited for engineers who enjoy working at the intersection of data engineering, cloud platforms, and applied machine learning.

Responsibilities

Design and implement scalable ETL/ELT data pipelines for streaming and batch workloads
Ingest data from cloud-based event streaming platforms into distributed processing environments
Build and maintain data processing workflows using Azure-native services
Expose processed and curated data according to architectural specifications
Integrate external machine learning or analytical workloads into existing data pipelines
Ensure reliability, performance, and scalability of data processing solutions

Core Skills

Extensive hands-on experience with Azure Databricks
Strong experience with Azure Event Hubs and Azure Functions
Proven expertise in designing and operating robust, scalable ETL/ELT pipelines
Strong Python programming skills for data processing and orchestration
Hands-on experience integrating LLM APIs into production systems

Highly Desirable

Experience integrating machine learning or custom analytics models into production data pipelines
Familiarity with streaming and time-series data processing
Knowledge of data formats commonly used in large-scale monitoring or analytics systems
Experience designing data solutions for high-volume, high-velocity data

What we offer

Work on complex, large-scale data processing systems
Real impact on architecture and data platform design
Collaboration with experienced engineers and architects
Competitive compensation

Senior Data Engineer with Python

Awareson Sp. z o.o.