Backend Data Engineer
KeySkills
Job Description
Job Description:
We are looking for a passionate and skilled Python Backend Data Engineer to join our team. In this role, you will design, build, and maintain a hybrid data platform across multi-cloud and on-premises environments. You?ll collaborate closely with AI engineers, product managers, and software developers to develop data-driven features and products that scale.
Key Responsibilities:
-
Design and develop scalable, high-performing backend applications using Python.
-
Manage Big Data application development across the full software development lifecycle.
-
Build and maintain RESTful APIs using FastAPI and integrate with real-time systems via Kafka.
-
Implement CI/CD pipelines for data workflows.
-
Develop and deploy solutions on cloud platforms (AWS, GCP, Azure).
-
Work with Spark and Apache Iceberg for big data processing and storage.
-
Handle both SQL and NoSQL databases such as PostgreSQL, MySQL, and MongoDB (GridFS).
-
Utilize modern authorization mechanisms including OAuth2 and JWT.
-
Perform data wrangling and manipulation using Pandas, Polaris, NumPy, and SQL.
-
Collaborate with cross-functional teams to ensure seamless integration and delivery.
Required Technical & Professional Expertise:
-
Strong experience in Python 3, FastAPI, Apache Spark, and Iceberg.
-
Expertise in developing RESTful web services.
-
Experience with Kafka messaging systems.
-
Data manipulation skills using Pandas, Polaris, NumPy, and SQL.
-
Proficiency in SQL and NoSQL databases: PostgreSQL, MySQL, MongoDB (with GridFS).
-
Understanding of OAuth2, JWT, and secure API development.
-
Familiarity with Gen AI ecosystems/tools is a plus.
Preferred Technical & Professional Experience:
-
Knowledge of object-oriented design, DDD, and Microservices architecture.
-
Exposure to BI tools and dashboarding platforms.
-
Experience with multi-cloud environments: AWS, GCP, Azure.
-
Strong problem-solving and analytical skills.
-
Excellent communication and team collaboration abilities.
-