Vedansh Porwal

Vedansh Porwal

Senior Software Engineer – Big Data

Gurugram, Haryana, India

#OpenToWork

About

Results-driven Senior Big Data Engineer with 4.5+ years of experience (4 years full-time + 6 months internship) designing, developing, and optimizing large-scale ETL pipelines, data architectures, and distributed data processing solutions. Deep expertise in Apache Spark (PySpark), Apache NiFi, Hadoop, Hive, Trino/Presto, Apache Airflow, and Apache Kafka. Proficient in Python, Java, SQL, SparkSQL, and Shell scripting, including Java-based custom processors and record readers in NiFi. Proven ability to deliver high-impact, cross-functional data solutions in compliance-sensitive, multi-country environments. Three-time award winner recognized for collaboration and going the extra mile.

What I'm looking for

Big Data Engineer or Data Architect Role

Experience

Senior Software Engineer I – Big Data

Airtel Africa Digital Labs

Nov 2025 – Present

• Leading design and delivery of scalable big data solutions for compliance and business-critical use cases across 14+ African operational countries. • Driving architectural decisions for ETL pipelines and data validation frameworks built on PySpark and Apache Airflow. • Mentoring junior engineers and contributing to code reviews, improving team code quality and delivery velocity. Award: Going Extra Mile Award – Q4 2025 Project: ABACUS (Active Subscribers Base Compliance) • Developed and optimized PySpark-based data validation pipelines to process KYC data received from 14 operational countries, ensuring alignment with compliance and regulatory requirements. • Built anomaly detection logic to flag fraudulent and non-compliant records, directly contributing to fraud detection initiatives and regulatory audit readiness. • Created and published Trino/Presto SQL views for business reporting, enabling self-serve analytics for compliance teams. • Collaborated with cross-functional teams (compliance, engineering, SRE) to improve data ingestion, validation, and pipeline reliability. Award: Spirit of Collaboration Award – Q2 2024 Project: FINREP (Financial Reporting) • Designed and implemented an end-to-end Financial Reporting System deployed across 14 operational countries, enabling standardized financial data extraction and reporting. • Automated extraction of data from unstructured and semi-structured files using Apache NiFi, including custom Java-based processors and record readers, reducing manual effort and increasing pipeline efficiency. • Developed 50+ ETL pipelines with Java-based expression language and NiFi processors to streamline data transformations and improve processing accuracy. • Loaded and managed processed financial data into PostgreSQL tables, making data available for downstream analysis and reporting. Award: Going Extra Mile Award – Q3 2023

SQLJavaSparkPythonGcp

Education

S.G.S.I.T.S., Indore, Madhya Pradesh

Bachelor of Technology · Computer Science & Engineering

2018 – 2022

Skills

Mongo dbData architectureData pipelineData lakesData warehousingEtlHadoopHivePyaparkGcpAwsApache airflowPostgres sqlApache nifiJavaSparkPython

Languages

English (Full professional proficiency)Hindi (Native or bilingual proficiency)