Syed Arshad

Data Migrations: Migrating KLM ground staff data to new shift planner application standards. Skilled in building reliable data pipelines and transformations using Python, DBT, and Dagster, ensuring accurate and seamless system integration.

• Automated manual data analysis and validation tasks using optimized SQL queries, improving efficiency and accuracy. • Improved query performance through effective use of partitioning techniques, resulting in faster data retrieval. • Contributed to the development and integration of OpsPilot AI agent, supporting data-driven functionalities and performance. • Played a key role in building and maintaining Airflow-based ETL pipelines, ensuring reliable data ingestion and transformation. • Developed interactive and insightful dashboards in Looker to support business analytics and decision-making.

Client: Eneco Netherlands: Senior Data Engineer At Eneco, a leading producer and supplier of natural gas, electricity, and heat in the Netherlands, I was involved in the development of a data pipeline that collected and processed real-time and historical weather data from various web sources and third party API providers. This data was then transformed into comprehensive reports and visualizations using Grafana and Power BI. • API data extraction. • Transformed multiple type of data like JSON/CSV and HTML • Design and implemented ELT/ETL pipeline. • Creating/Automating Kubernetes process • Created DBT Airflow ETL Process • Created pipeline using Azure Data Factory • Created reusable Databricks notebooks for different transformation steps and configured these notebooks to be scheduled as jobs within Databricks • Created Grafana dashboard. • Effectively communicated with stakeholders to gather and understand project requirements, ensuring alignment and timely delivery of solutions. • Collaborated with cross-functional teams • Facilitated requirements gathering sessions, translating business needs into actionable user stories and technical tasks.

Project Description: Data Input: As a Data Engineer, I leveraged the power of web scraping to build a robust Pricing Intelligence Tool for our stakeholders. By scraping competitor websites. I successfully gathered real- time pricing data and product information, enabling customers to track and compare prices across multiple online platforms. Responsibilities: • Designing and implementing web crawlers to extract competitive pricing, product features, and other key metrics from competitor sites. • Analyzing competitor websites to identify key data points for pricing intelligence, ensuring comprehensive coverage of relevant data for decision making. • Optimizing scraping processes for speed, accuracy, and scalability, ensuring real-time updates on product pricing. • Addressing anti-scraping measures such as CAPTCHAs and rate limiting, ensuring consistent and reliable data extraction. • Technology used –Python, Scrapy, AWS, Athena

Project Description: I used scrap data from competitor sites for our client (OLX). We used scrap full site listing this data help our customer understand the site popularity and growth in its region • Technology used - Python, Scrapy, AWS, Redshift

Project Description: Working for Client Mantra Designs Largest fashion commerce company in India, I used to scrap data from it competitor web site. • Technology used - Python, Scrapy, AWS, Kafka

Project Description: As part of the development team for a News Recommendation App, I was responsible for designing and implementing a system to collect, aggregate, and process news data from a variety of online sources. The goal was to provide users with personalized and up-to-date news recommendations based on their preferences and interests. • Technology used - C++, Python, Scrapy

Project Description: Multimedia Games US Inc. As a developer, I was responsible for migrating poker games from the legacy M1 platform (written in C) to the newer M2 platform using JavaScript. This migration project involved understanding both the technical and functional requirements of the existing poker games and ensuring a smooth transition to the new platform while preserving game functionality and performance Key responsibilities included: • Analysing legacy M1 platform code to understand core game logic and features built in C, ensuring compatibility with the new M2 platform. • Rewriting game mechanics and logic using JavaScript to replicate functionality while optimizing for the M2 platform’s architecture. • Collaborating with cross-functional teams to ensure seamless integration of game features, user interfaces, and backend systems within the M2 environment. • Testing and debugging the migrated poker games to ensure they functioned as expected, with minimal latency and high reliability across platforms.

Project Description: As a developer working on Promax, a Windows-based application designed to assist retailers with sales forecasting and promotion management, I contributed to the development of key features that empowered users to analyse sales data and optimize promotional strategies. • Technology used - C++, Visual Studio, SQL

Azure Data Fundamentals DP-900

Microsoft

Azure Data Engineer Associate DP-203

Microsoft