Data Engineer
KeySkills
Job Description
Job Description:
As an Associate Software Developer at IBM, you will leverage the power of data to uncover insights, patterns, and narratives. You?ll be involved in data collection, storage, and both batch and real-time processing. Working in a collaborative, Agile environment, you will help choose optimal data management systems and identify critical data for analysis. You'll also address challenges in database integration and manage complex, unstructured datasets.
Key Responsibilities:
-
Implement and validate predictive and statistical models using big data and machine learning techniques.
-
Design and implement enterprise search applications (e.g., Elasticsearch, Splunk).
-
Collaborate with engineers, scientists, and consultants to solve predictive modeling challenges.
-
Build tools or write programs for data cleansing, integration, and transformation.
-
Develop and evaluate predictive or prescriptive models.
-
Migrate databases (e.g., Hadoop to GCP), including objects like tables, views, procedures, and triggers.
-
Implement data replication (CDC, file, bulk transfer) and expose data as APIs.
-
Contribute to modernization roadmap and lead discovery and analysis workshops.
-
Assess application dependencies and address database incompatibilities.
-
Analyze non-functional requirements (e.g., security, HA, RTO/RPO, network, performance).
-
Prepare effort estimates, WBS, staffing plans, and risk documentation.
-
Guide the team in selecting appropriate migration and modernization tools.
Required Technical and Professional Expertise:
-
Experience with data migration (Hadoop to GCP), replication mechanisms, and database transformation.
-
Exposure to API development and enterprise-level data modernization.
-
Ability to analyze and address non-functional system requirements.
-
Strong project planning skills including effort estimation and risk assessment.
Preferred Technical and Professional Experience:
-
Excellent teamwork, verbal, and written communication skills.
-
Ability to understand client needs and deliver data-driven analytical solutions.
-
Capable of communicating technical results to both technical and non-technical audiences.
-