Python pyspark Developer
Company Name
Infosys Ltd ( Bangalore )
Job Description
Job Description:
We are looking for an experienced Python PySpark Developer to join our Infosys consulting team. You will be responsible for designing, developing, and deploying data processing pipelines and solutions, collaborating with stakeholders to deliver high-quality, value-driven outcomes in digital transformation projects.
Responsibilities :
-
Collaborate with the consulting team in all project phases including problem definition, effort estimation, diagnosis, solution generation, design, and deployment.
-
Develop scalable PySpark applications for data processing and transformation.
-
Explore and evaluate alternative solutions through research, literature surveys, public domain resources, and vendor evaluations.
-
Build proof of concepts (POCs) to validate recommended solutions.
-
Create requirement specifications from business needs and define to-be processes with detailed functional designs.
-
Configure solutions, troubleshoot issues, identify root causes, and recommend effective resolutions.
-
Contribute to unit-level and organizational initiatives to deliver high-quality solutions to clients.
Key Skills:
-
Strong hands-on experience in Python and PySpark.
-
Proficiency in Spark Core and Spark SQL.
-
Experience in data processing, transformation, and optimization.
-
Knowledge of distributed computing and big data concepts.
-
Familiarity with Hadoop ecosystem tools (e.g., HDFS, Hive) is a plus.
-
Strong problem-solving and debugging skills.
-
Ability to work collaboratively in agile environments.
-
Excellent communication and analytical skills.
-