Resume

Software Engineer and Technical Lead with 10+ years of experience building backend systems, distributed systems, and data platforms. I have worked across system architecture, large-scale data processing, cloud infrastructure, performance optimization, reliability, and production operations.

At Sophos, I work on the Data Platform team building systems for an AWS-based data lake and security analytics platform. My work includes Apache Iceberg and Hudi data pipelines, high-performance query systems, metadata indexing, batch detection frameworks, and data retrieval optimization. I also lead and mentor engineers, drive architectural decisions, and work across teams to improve engineering practices and system reliability.

I enjoy solving complex engineering problems where scalability, performance, reliability, and simplicity matter.

Before Sophos, I worked at o9 Solutions, Goldman Sachs, Eureka AI, and Samsung Research Institute, where I built proprietary databases and query systems, data pipelines, enterprise applications, analytics platforms, distributed services, and user-facing products.

I hold a Master’s degree in Computer Science from IIT Kanpur and a Bachelor’s degree in Computer Science and Engineering from IIT Guwahati.

Contact Information

• Email: [email protected]
• LinkedIn: itspawanhere
• GitHub: itspawanhere

Professional Experience

Sophos, Bangalore, India

Senior Software Engineer II
October 2023 - Present

At Sophos, I work on the Data Platform team responsible for the end-to-end lifecycle of security telemetry within an AWS-based data lake, including data ingestion, storage, processing, querying, and retrieval.

• Apache Iceberg Data Lake: Built a new Apache Iceberg-based data pipeline for XDR, MDR, firewall, and other telemetry sources, replacing a legacy Hive-based pipeline. Improved query performance by approximately 2x, reduced storage requirements by approximately 30%, and made data available in the S3 data lake 50%+ sooner.

  Added S3 write support, time-travel queries for auditing, and automated table maintenance including compaction, snapshot expiration, and data cleanup.

  Technologies: Java, Spring Boot, Kafka, Apache Iceberg, AWS Glue, Athena, Grafana, Logz, Metabase.

• Batch Detection & ETL Platform: Built a scalable batch detection framework on Amazon MWAA (Airflow) that enables threat researchers to onboard and execute custom detection workflows with minimal engineering effort.

  Designed support for multi-phase execution, customer segmentation by licensing tier, and ETL pipelines before detection runs. The framework reduced threat-detection turnaround time by nearly 10x.

  Technologies: AWS, Airflow, Kafka, Trino, Avro, Redis, Python, Terraform.

• Location Indexer - Sophos Search & Insights: Built a metadata indexing system for telemetry Hudi tables that maps tenant and time-range queries to the exact Parquet files containing the required data.

  Replaced expensive S3 fan-out scans with metadata-driven file resolution and Parquet footer range requests, reducing S3 reads by approximately 99%. Built Kafka-driven indexing pipelines into Mach5/OpenSearch and integrated DuckDB for query execution over resolved Parquet subsets.

  Technologies: Java, Kafka, AWS S3/SQS, Apache Hudi, OpenSearch/Mach5, DuckDB, Terraform.

• Data Lake Search Accelerator: Built a custom Trino plugin that uses Elasticsearch metadata to resolve file paths for query filters, including regex filters. The plugin prunes unnecessary splits and reads only matching Parquet files from the Iceberg-backed data lake, significantly reducing the amount of data scanned for selective queries.

  Technologies: Java, Trino, Elasticsearch, Apache Iceberg, Kafka, AWS.

• Technical Leadership: Lead and mentor 5+ engineers, driving architectural decisions, engineering standards, cross-team collaboration, and ownership of production systems. Focus on improving team productivity, code quality, and engineering practices.

o9 Solutions, Bangalore, India

Senior Software Engineer I
August 2021 - September 2023

• Integrated Business Planning Language (IBPL): Enhanced o9’s proprietary query language with enterprise capabilities including audit, sharing, and upsert operations. Built scalable ANTLR-based parsing and execution flows for complex queries while enforcing secure row-level authorization across millions of records.

  Improved query response time by approximately 40% using Redis-based caching and reduced repeated database load.

  Technologies: C#, SQL Server, Redis, RabbitMQ, ANTLR, Visual Studio.

• Lightweight Scenarios & Scenario Sharing: Designed and implemented lightweight what-if scenarios for o9’s proprietary database, minimizing data replication and memory consumption.

  Built scenario-sharing capabilities that allowed complete what-if versions of datasets to be shared with users and roles in read-only or read/write modes.

  Technologies: C#, SQL Server, Redis, RabbitMQ, Solr, MongoDB.

• Audit Trail: Built a comprehensive auditing framework for o9’s proprietary database, tracking transactions across the system and improving transparency, accountability, and data integrity.

  Reduced transaction tracking time by approximately 90%, enabling faster investigation and decision-making.

  Technologies: Java, Spring Boot, Hadoop, Lucene, Protobuf, HBase, SQL Server.

• Data Governance & Authorization: Designed measure-based and time-based ACL mechanisms that enabled dynamic, context-aware access control for enterprise-scale data operations.

• Service Health Monitoring: Designed and implemented a centralized service health monitoring system that provided real-time visibility into the performance and configuration of Java-based microservices.

  Improved SLA compliance by more than 20%, while reducing downtime and accelerating root-cause analysis.

  Technologies: Java, Spring Boot, RabbitMQ, Elasticsearch, Kibana.

Goldman Sachs, Bangalore, India

Engineering Associate
January 2021 - July 2021

• LUMOS: Enhanced an internal desktop application used by Goldman Sachs employees to centralize workflows, support tickets, and notifications.

  Added message boards, outage notifications, and ticket prioritization capabilities to reduce turnaround time for support requests. The improvements resulted in an efficiency increase of more than 30%.

  Technologies: Angular, Electron, TypeScript, SQLite, HTML, CSS, Java, Spring Boot.

Eureka AI, Bangalore, India

Senior Software Engineer
January 2020 - January 2021

Software Engineer
June 2018 - December 2019

• Data Pipeline - Pipecore: Built an advanced data transformation and integration platform capable of automating ingestion and transformation of datasets from multiple telecommunications providers into Eureka AI’s internal format.

  Supported efficient data movement between systems including Hive and Elasticsearch, improving overall data-pipeline efficiency by more than 3x.

  Technologies: Java, Hadoop, Oozie, Spark, Hive, Python, Kafka, Elasticsearch, Kibana.

• Eureka User Profile Store: Designed and developed the backend and UI for a platform that enabled exploration of telecommunications users through flexible filtering and REST APIs.

  Built dynamic data visualization capabilities that allowed clients to generate intelligence reports for selected user segments. The platform contributed to 6 telcos across 5 countries becoming customers.

  Technologies: Java, Spring Boot, Elasticsearch, React, Bootstrap, ChartJS.

• Contact Tracing: Developed a user-tracing platform using telecommunications data, with user consent, to identify potential sources and locations of COVID-19 exposure.

  Integrated Point-of-Interest data to help identify potential infection sites. The platform was used by government agencies to help track and minimize the spread of COVID-19.

  Technologies: Apache Druid, Spring Boot, React, Leaflet.

• Eureka Omni: Developed a customizable dashboard and reporting platform that allowed users to compose dashboards from existing or new widgets and generate reports in multiple formats.

  Technologies: Cube.js, Spring Boot, React.

• Credit Score Platform: Built an end-to-end system for ingesting credit-score data from the Eureka AI engine into Hive and Elasticsearch.

  Enabled clients to retrieve credit scores using mobile numbers or national IDs.

  Technologies: Java, Spring Boot, Hive, Elasticsearch, React.

• Campaign Management System: Designed and implemented a campaign-management platform that enabled Campaign Directors to create and manage campaigns for targeted customer segments.

  Solved the challenge of matching optimal users to campaigns during simultaneous executions and supported large-scale campaign execution with BI and analytics integration.

  Technologies: Spring Boot, MySQL, React.

Samsung Research Institute / Samsung Electronics, Bangalore, India

Senior Software Engineer
March 2016 - July 2016

Software Engineer
June 2014 - March 2016

• Samsung Email - Zero Inbox: Implemented an IFTT-style automation capability that generated events, tasks, and reminders from email content, helping users maintain a zero-inbox workflow.

  Also implemented a priority inbox capability using information-retrieval techniques to improve email productivity.

  Technologies: Android, Java, Information Retrieval.

• Google Drive Integration: Led development of a Google Drive integration for Samsung Email, allowing users to save email attachments directly to Google Drive.

  Technologies: Android, Java, Google Drive SDK.

• Splitbilling: Engineered a billing and usage-monitoring system that enabled administrators to track call, SMS, and data usage across personal and enterprise accounts through a unified dashboard.

  The system helped reduce company expenses associated with personal usage.

  Technologies: Python, MongoDB, Flask, JavaScript, Bootstrap.

Indian Institute of Technology, Kanpur

Graduate Teaching Assistant
July 2016 - May 2018

• Served as a Teaching Assistant for the Computer Science department at IIT Kanpur.
• Assisted students with coursework and academic activities.
• Worked as a Teaching Assistant for the Design and Analysis of Algorithms course.

Side Projects

PSFutureMail.com - Future Letter Platform

Designed and built a full-stack platform for preserving memories, goals, and reflections for future delivery.

Implemented secure authentication, long-term scheduling and delivery workflows, draft management, reminders, referrals, payments, and community features including public letters, likes, and engagement.

Technologies: Next.js, TypeScript, Firebase, Firestore, Resend, Tailwind CSS.

PSDiary.com - Personal Journaling SaaS

Built a full-stack journaling platform that helps users capture thoughts, develop writing habits, and gain insights through analytics and AI-powered summaries.

Implemented secure authentication, autosave, achievements, PDF export, and a responsive cross-device experience using a serverless cloud architecture.

Technologies: Next.js, TypeScript, Firebase, Firestore, Resend, Tailwind CSS.

Conduit - Peer-to-Peer AI Inference Gateway

Built a distributed AI inference gateway that routes requests from clients to GPU workers without relying on a central database or message broker.

Gateway instances share tenant state, API keys, quotas, and usage information peer-to-peer using the Holepunch stack.

Supports the OpenAI Chat Completions API with streaming, per-tenant rate limiting, and intelligent worker scheduling based on available VRAM and model affinity.

Deployed as a Dockerized multi-node cluster with a CLI for live interaction.

Technologies: Node.js, Autobase, Hyperswarm, Hyperbee, Fastify, Docker.

Education

Indian Institute of Technology, Kanpur

Master of Science (M.S.), Computer Science
July 2016 - May 2018

Indian Institute of Technology, Guwahati

Bachelor of Technology (B.Tech), Computer Science and Engineering
July 2010 - April 2014

Awards and Honors

• Jury’s Favorite Award - o9 Solutions Hackathon
  Won the Jury’s Favorite Award for building a customer-support bot using LangChain and OpenAI’s ChatGPT APIs.
  September 2023

• Employee of the Quarter - o9 Solutions
  Recognized as Employee of the Quarter for outstanding contributions and achievements.
  January 2023

• Tizen App Award - Samsung
  Won a Samsung Tizen phone for creating a holiday planner application published on the Tizen Store.
  May 2015

• Merit-cum-Means Scholarship - IIT Guwahati
  Awarded for outstanding academic performance over four consecutive years.
  2010 - 2014

• Top 0.18% - Joint Entrance Examination
  Achieved a rank within the top 0.18% among more than 500,000 candidates.
  May 2010

• Certificate of Merit - CBSE
  Recognized among the top 0.1% of students nationwide in Social Science.
  May 2008

• First Prize - State-Level Essay Competition
  Won first place in a state-level essay competition.
  May 2007

Skills

Programming

Java, Python, Go (Golang), JavaScript, TypeScript, SQL, Unix Shell/Scripting

Backend & Distributed Systems

Spring Boot, GraphQL, Kafka, RabbitMQ, Redis, Valkey, REST APIs

Data Engineering & Data Platforms

Apache Iceberg, Apache Hudi, Apache Spark, Airflow, Hadoop, Hive, Trino, Elasticsearch, OpenSearch, DuckDB

Databases & Storage

PostgreSQL, SQL Server, MongoDB, HBase, Elasticsearch, Redis, Valkey, Firestore

Cloud, DevOps & Infrastructure

AWS, GCP, Docker, Kubernetes, Terraform, Jenkins, Git, Nginx, Vercel

Frontend

React, Next.js, Tailwind CSS, shadcn/ui, Angular, Electron

Observability & Operations

Grafana, ELK Stack, Kibana, Logz, Metabase, CI/CD

Certifications

• AWS Essential Training for Developers
• Learning Kubernetes

Languages

• English: Full Professional Proficiency
• Hindi: Native or Bilingual