Senior Data Engineer, NeenOpal Inc., Apr 2023 – Present
Key contributor in architecting end-to-end AWS infrastructures, strategically selecting services to optimize data warehousing and ETL solutions for diverse client projects
Significantly contributed to elevating the company’s AWS partnership tier by conducting AWS Well-Architected reviews for numerous clients
ETL and BI Implementation for a US-based Employee Educational Benefits Firm (Financial Sector)
Led data engineering initiatives, overseeing the entire project lifecycle
Engineered a highly efficient ETL solution by constructing a data lake on Amazon S3. Migrated and integrated data from MongoDB, MySQL, and PostgreSQL for advanced analytics using AWS Glue and Spark
Architected and deployed a robust CI/CD pipeline from scratch, leveraging AWS CodeCommit, CodePipeline, and CodeBuild to ensure seamless integration and continuous delivery
Data Engineer, NeenOpal Inc., Jun 2021 – Apr 2023
Enterprise-wide ETL + BI Implementation (AWS + PowerBI) for a multinational Logistics company
Engineered a robust, end-to-end ETL solution for 6 distinct SBUs within the company
Designed and implemented a resilient data model, creating a data warehouse on Amazon Redshift from the ground up.
Built and optimized ETL pipelines using Spark Glue for seamless data migration from diverse sources, including SAP, third-party data sources via APIs, and manual Excel files
Developed complex SQL queries to transform raw data tables into informative datasets, enabling PowerBI dashboards to deliver actionable insights
ETL + Forecatsing for a London based real-estate company
Took charge of the complete data engineering responsibilities for the entire project
Developed ETL pipelines for monthly data migration from government sources, extracting information from APIs and manual files stored in Amazon S3, and loading it into an RDS PostgreSQL database
Developed complex SQL queries for transforming raw property data into valuable insights
Partnered with data scientists to develop a sophisticated multivariate time-series forecasting model aimed at predicting average property price growth in various UK regions over the next decade
ETL+ BI Implementation (AWS + Tableau) for an Edtech Company
Led a team of 2 Data Engineers for development of automated ETL jobs
Orchestrated the development of ETL workflows to seamlessly migrate transactional data from AWS Aurora to Redshift, effectively utilizing Glue
Ensured the consistent quality control and performance monitoring of daily ETL scripts, promptly addressing any issues or errors that arose
BI Implementation (Azure + PowerBI) for a leading provider of employee scheduling software
Led a team of 2 Data Engineers for development of automated ETL jobs
Actively engaged in collaborative meetings with stakeholders to gain a deep understanding of the underlying logic behind various sales KPIs
Developed SQL queries and Python scripts to execute complex calculations for the KPIs
Created backend data models and views in Azure SQL database to dynamically transform raw Salesforce data into insightful data for powering 8 different dashboards consisting of 30+ sales KPIs
Database Optimization for a ed-tech firm
Boosted overall system efficiency, leading to optimized performance for both the database and frontend application
Fine-tuned backend data models and SQL queries, significantly improving the performance of the frontend application
Optimized the client’s Redis database to enhance data caching capabilities
Data Scientist Intern, NeenOpal Inc., Nov 2020 - May 2021
Sales and Inventory Forecasting for one of the largest Tiles producers in Sri Lanka
Assisted a Senior Data Scientist in developing a hybrid forecasting model leveraging ML algorithms and statistical models
Automated the manual effort of importing, cleaning, and preprocessing data, scheduled ML model autoruns
Used Tableau for presenting interesting insights to the stakeholders
Research Intern, HCI Lab, IIIT Allahabad, May 2019 - Jul 2019
Early detection of Autism in toddlers
Assisted Prof. Anupam Agarwal in his reserach
Performed Literature survey on early detection of Autism in toddlers
Worked on Autism Detection application using Image Processing and Machine Learning techniques
Worked on POCs on various new domains of machine learning like Transfer Learning