Software Development Engineer II – Data Engineer

About this job

Gather AI is pioneering warehouse intelligence with a vision-powered platform using autonomous drones and existing equipment to digitize manual workflows, creating safer and more efficient facilities. As a Software Development Engineer II – Data Engineer joining the newly forming Full Stack group within Cloud Services, you will help build the company's data foundation from the ground up. In this role, you will separate analytical workloads from production PostgreSQL databases, develop shared transformation layers, and define consistent metrics across products. A normal day involves shipping well-scoped extraction pipelines, writing tested dbt models, implementing data quality tests, linking structured records to drone imagery, and collaborating closely with cross-functional teams like Full Stack, ML, Product, Security, and Customer Success in a remote setting.

Responsibilities

• Build extraction pipelines to move data from production PostgreSQL to the analytical warehouse using incremental loads.
• Build and extend shared dbt models including dimensions, metric building blocks, and serving tables.
• Implement consistent metrics in the semantic layer so measurements match across all dashboards.
• Ensure data trustworthiness by adding tests, freshness checks, alerts, and running safe backfills.
• Apply access-control and tenant-isolation patterns to every pipeline and model you ship.
• Maintain lineage to trace metrics back to their source records and drone images.
• Partner with integration teams to validate incoming WMS data against agreed contracts.
• Ship safely using CI/CD, join on-call rotations, and document models in the data catalog.

Requirements

• 2 to 5 years of experience building and running production data pipelines, or a Computer Science degree with equivalent experience.
• Strong SQL proficiency including window functions, joins, CTEs, and a working understanding of dimensional modeling.
• Hands-on experience building tested models in dbt or equivalent tools like Snowflake Dynamic Tables.
• Production Python pipeline development experience with tests and code reviews.
• Experience running orchestration tools such as Airflow, Dagster, or Snowflake Tasks, including handling failures and backfills.
• Experience moving data from operational databases into a warehouse or lakehouse like Snowflake or Databricks.
• Production cloud experience on Azure or another major cloud platform with Git, CI/CD, and Docker/Kubernetes.
• Clear written and spoken English communication skills suited for a distributed remote team.

Good to have

• Experience with change data capture tools like Debezium or Fivetran, or streaming tools like Kafka.
• Familiarity with PySpark, Snowpark, or Infrastructure as Code like Terraform.
• Experience with semantic layers or data catalogs such as DataHub and Purview.
• Knowledge of multi-tenant data platforms, row-level security, or working with image, video, and sensor data.
• Domain experience in logistics, warehousing, or robotics.

Who should apply

This role suits experienced data engineers with 2 to 5 years of background in building production pipelines who thrive in an early-stage environment and want to design a data platform from scratch. The position requires strong independence, clear written communication for a remote distributed team, and a solid grasp of SQL, Python, dbt, and cloud workflows. Freshers are not suited for this SDE II position as prior production experience is required.

Frequently asked questions