Data Engineer

About this job

Karya is an organization that offers AI-powered learning and earning options to economically underserved groups while supplying high-quality datasets to tech companies and research labs. As a Data Engineer based in Bengaluru, you will handle live client projects on the Karya platform. Your daily duties form the operational execution layer above the core product. You will set up project workflows, manage data jobs, perform quality checks, run reports, and handle final deliveries. A typical day involves running and monitoring data jobs on remote Linux servers, writing PostgreSQL and ClickHouse queries to track project metrics, and applying quality rules that impact worker payments and client deliverables. You will also collaborate closely with operations and client teams to answer questions regarding progress and file statuses, while coordinating with platform engineers when product fixes are necessary.

Responsibilities

• Run and monitor data jobs on remote Linux servers and recover failed jobs.
• Configure project workflows and track items in progress, held, or ready for delivery.
• Write PostgreSQL and ClickHouse queries for throughput, completion, quality, pay, and delivery status.
• Apply completion and quality rules for access, pay, and delivery decisions after sample checks.
• Write Python, SQL, or Bash scripts for checks and decisions and maintain them in the repository.
• Package datasets to client specifications and move them through cloud object storage.
• Read the TypeScript and Node.js codebase to understand tasks, assignments, payments, and file storage.
• Communicate technical updates and findings clearly to operations and client teams.

Requirements

• Proficiency in PostgreSQL including joins, aggregates, and JSON fields.
• Ability to write scripts in Python, Bash, or TypeScript checked into git.
• Ability to read a TypeScript codebase and a relational schema.
• Familiarity with Linux for running jobs including SSH, logs, and long-running processes.
• Skill in writing clear technical updates for non-technical teammates.
• Experience running queries or scripts against real project or production data.

Good to have

• Experience with cloud object storage like Google Cloud Storage, S3, or Azure Blob.
• Familiarity with collected datasets such as speech, text, or images.
• Experience with ClickHouse, Metabase, or similar reporting tools.

Who should apply

This role suits individuals with practical experience in Linux, database querying, and scripting languages like Python, Bash, or TypeScript. Candidates should be comfortable working at the intersection of data operations and client deliverables, ensuring that quality checks directly support worker earnings and project success. The posting does not state whether freshers can apply, so applicants should verify they meet the stated technical requirements involving real project or production data experience.

Frequently asked questions