Senior Data Engineer at LTIMindtree
Architected and led development of an enterprise-grade ETL framework from scratch using Java Springboot and Python Flask, replacing licensed Talend infrastructure and eliminating annual licensing costs while processing 3+ million rows with complex transformations in under 45 seconds.
Company
LTIMindtree
Location
Pune, India
Period
Jun 2022 - present
Company Link
Highlights
- Developed Spring Boot Kafka services to stream over 1 million events daily, replacing a manual FTP process and reducing data processing latency by 95%.
- Built and scaled a suite of RESTful APIs supporting over 100,000 requests per minute, which improved system performance and data accessibility for end-users.
- Engineered complex Talend jobs to process and load over 50 GB of data daily into Oracle tables, integrating stored procedures and custom Java code for advanced transformations.
- Automated and optimized a multi-layer Talend data pipeline, which reduced the average development and deployment cycle time by a significant 80%.
- Leveraged Docker to containerize microservices, which streamlined the CI/CD pipeline and decreased average deployment time from hours to under 10 minutes.