SAIKRISHNA SUBRAMANIAN

I make sure the data behind AI systems actually holds up in the real world.

Data Engineer·Bengaluru, India
About

I've spent the last three years elbow-deep in data pipelines — the unglamorous plumbing most people never think about until it breaks. Somewhere along the way that plumbing started carrying AI and ML workloads, and that's the part of the job that stuck with me.

These days I spend my time making sure the data feeding machine learning models is fast, clean, and something you can actually trust — whether that's a real-time safety alert or a chatbot answering questions in plain English.

The work

Real-time detection that replaced a rulebook

Built the pipeline that feeds live sensor data from 20+ industrial wells into a machine learning model, replacing a rigid rules-based system with one that actually learns.

5–6 min alert latency

A data foundation moving 100GB a day

Designed ETL pipelines pulling from APIs, databases, files, and event streams into a medallion architecture, feeding downstream teams doing analytics and AI.

100GB/day · 6+ consumers

Cutting cost without cutting corners

Rethought how messages were batched and deduplicated in an event-driven system, trimming both cloud spend and end-to-end latency.

−60% cost · −30% latency

Protecting proprietary logic at scale

Migrated production pipeline modules to a Cython-based architecture, securing IP across multi-tenant deployments without touching Spark performance.

40+ modules secured

Turning deployment into a non-event

Set up Databricks Asset Bundles as the standard deployment framework, embedding Git-based release management across the org.

−80% deploy time

Keeping history accurate under audit

Implemented SCD Type 3 and CDC patterns for large-scale historical backfills, keeping data accurate across audited customer environments.

SOC2 audited

Teaching a chatbot where its limits are

Built a conversational analytics assistant with a multi-agent routing layer and prompt templates per data domain, so it only ever answers from data it's allowed to see.

guardrailed · multi-agent

Standardizing how questions get asked

Designed prompt templates per domain to keep question framing consistent, improving routing accuracy across the whole conversational system.

per-domain prompt design
Toolkit
Languages & core
Python·PySpark·SQL·Spark internals·Delta Lake
Streaming & orchestration
Kafka·Event Hubs·Airflow·Kubernetes·Terraform
AI / ML
LLM guardrails·Prompt design·Multi-agent routing·Data quality checks
Security
SOC2·Checkmarx·Trivy·Code obfuscation
Also
Stakeholder management·Mentoring·Data storytelling
Background
Roots

A B.Tech in Computer Science and Business Systems from VIT — the overlap in that degree title turned out to be more prophetic than I expected, since most of my job now sits exactly at that intersection.

Still learning

Currently working through the Databricks Certified Data Engineer Professional and AWS Certified Data Engineer – Associate exams, mostly because production has a way of teaching you what you assumed instead of what you verified.

Say hello

Always glad to talk data, ML systems, or the space between them.