technologies-expected

Expected, Snowflake, Python, Git, AWS, Microsoft Azure

technologies-optional

Optional, Apache Spark, AWS

technologies-os

Operating system, Linux

Opis stanowiska

About the project, You will work on a large-scale data platform built for a global consulting firm, supporting M&A (Mergers and Acquisitions) analysis., , The project consists of:, • A cloud-based platform for data visualization (APIs and services), • A large, integrated dataset (300M+ company records from 10+ sources), , Your primary focus will be on data engineering (approx. 80%), including building and improving data pipelines and dataset quality. Additionally, you will contribute to backend and platform development (approx. 20%), including features, APIs, and integration with the data layer., , Technology stack:, • Languages: Python, SQL, • Data: Snowflake, dbt, • Workflow Orchestration: Apache Airflow (complex DAGs), • Data Processing: Apache Spark - Platform, • Cloud: AWS (EKS, S3, Lambda, ECR, EMR, Opensearch) – Platform, Azure (AKS, Blob Storage, Azure Functions, ACR, Databricks, AI Search) - Dataset, • API: API Gateway, FastAPI (REST, async), • CI/CD: GitHub Actions, • Data Scale: 300M+ records across multiple sources, • Future Direction: Agentic AI systems, LangChain and cloud-native AI libraries and frameworks, Azure OpenAI integration, , Team: Data Architecture Lead, Data Engineers, DataOps Engineers, Frontend Engineer, Backend Engineer, Product Owner, , Distributed team across Europe and India, , Agile, collaborative, proactive and impact-driven environment, , We work on multiple projects at the same time, so we may suggest a different project if it better matches your experience and profile.

Twoje zadania

Your responsibilities, This is a hybrid role combining Data Engineering and Backend Engineering, with a stronger focus on the data layer., Build and maintain scalable ETL/data pipelines (Python, SQL, dbt, Snowflake, Airflow), Design and optimize data models, transformations and workflows, Ensure data quality, consistency, and performance, Integrate multiple data sources into a unified dataset, Contribute to backend development (APIs, data access layer, backend-frontend contracts), Support platform integrations and cloud-based solutions, Write clean, testable code and participate in code reviews, Document data processes and system architecture

Wymagania

Strong experience with Snowflake and dbt, Strong Python skills, Ability to write clean, scalable, and testable code (including unit tests), Solid understanding and applying object-oriented programming (OOP), Experience with version control systems: Git, Experience with AWS or Azure, Good knowledge of English (minimum C1 level)

Wymagania

Optional, Experience with Apache Spark (preferably Databricks), Experience with Azure AI Search or AWS OpenSearch, Experience designing ETL/ELT pipelines, Familiarity with LLMs, Azure OpenAI, or agentic AI systems

work-organization-work-style

This is how we work, at the client's site, agile

Co oferujemy

What we offer, Flexible working hours and approach to work: fully remotely, in the office or hybrid, Professional growth supported by internal training sessions and a training budget, Solid onboarding with a hands-on approach to give you an easy start, A great atmosphere among professionals who are passionate about their work

Co oferujemy

Benefits, sharing the costs of sports activities, sharing the costs of foreign language classes, sharing the costs of professional training & courses, remote work opportunities, flexible working time, fruits, integration events, no dress code, coffee / tea, drinks

recruitment-stages

Recruitment stages, Meeting with the Recruiter, Technical Interview&Meeting with the Engineering Manager, Interview with the Client

Opis stanowiska

CODILIME SPÓŁKA Z OGRANICZONĄ ODPOWIEDZIALNOŚCIĄ, CodiLime is a software and network engineering industry expert and the first-choice service partner for top global networking hardware providers, software providers and telecoms. We create proofs-of-concept, help our clients build new products, nurture existing ones and provide services in production environments. Our clients include both tech startups and big players in various industries and geographic locations (US, Japan, Israel, Europe)., , While no longer a startup - we have 250+ people on board and have been operating since 2011 we’ve kept our people-oriented culture. Our values are simple:, , • Act to deliver., • Disrupt to grow., • Team up to win.

Opis stanowiska

You will work on a large-scale data platform built for a global consulting firm, supporting M&A (Mergers and Acquisitions) analysis.

The project consists of:

• A cloud-based platform for data visualization (APIs and services)

• A large, integrated dataset (300M+ company records from 10+ sources)

Your primary focus will be on data engineering (approx. 80%), including building and improving data pipelines and dataset quality. Additionally, you will contribute to backend and platform development (approx. 20%), including features, APIs, and integration with the data layer.

Technology stack:

• Languages: Python, SQL

• Data: Snowflake, dbt

• Workflow Orchestration: Apache Airflow (complex DAGs)

• Data Processing: Apache Spark - Platform

• Cloud: AWS (EKS, S3, Lambda, ECR, EMR, Opensearch) – Platform, Azure (AKS, Blob Storage, Azure Functions, ACR, Databricks, AI Search) - Dataset

• API: API Gateway, FastAPI (REST, async)

• CI/CD: GitHub Actions

• Data Scale: 300M+ records across multiple sources

• Future Direction: Agentic AI systems, LangChain and cloud-native AI libraries and frameworks, Azure OpenAI integration

Team: Data Architecture Lead, Data Engineers, DataOps Engineers, Frontend Engineer, Backend Engineer, Product Owner

Distributed team across Europe and India

Agile, collaborative, proactive and impact-driven environment

We work on multiple projects at the same time, so we may suggest a different project if it better matches your experience and profile.

Twoje zadania

This is a hybrid role combining Data Engineering and Backend Engineering, with a stronger focus on the data layer.
Build and maintain scalable ETL/data pipelines (Python, SQL, dbt, Snowflake, Airflow)
Design and optimize data models, transformations and workflows
Ensure data quality, consistency, and performance
Integrate multiple data sources into a unified dataset
Contribute to backend development (APIs, data access layer, backend-frontend contracts)
Support platform integrations and cloud-based solutions
Write clean, testable code and participate in code reviews
Document data processes and system architecture

Co oferujemy

Flexible working hours and approach to work: fully remotely, in the office or hybrid
Professional growth supported by internal training sessions and a training budget
Solid onboarding with a hands-on approach to give you an easy start
A great atmosphere among professionals who are passionate about their work

Recruitment stages

Meeting with the Recruiter
Technical Interview&Meeting with the Engineering Manager
Interview with the Client

Senior Data Engineer (with Backend Experience)

CODILIME sp. z o.o.