Eliminating Deadlocks in Batch Processing

Over the past few weeks, we significantly scaled our overnight batch imports. These imports were designed to pull daily records from multiple source systems and aggregate them into fewer consolidated records — a common requirement for analytics, reporting, and downstream processing. [Read More]

Influencing Team collaboration

In the past, I have frequently discussed collaboration models that suit our need for fast flow. This conversation keeps recurring as we build new shared services and collaborate more. Ownership is a key driver of motivation, and both team autonomy and team collaboration significantly influence that sense of ownership. [Read More]

Notes on Scaling Spark Jobs

Beginning in 2025, we set the goal of onboarding a large number of customers to our product. The product had been in development over the past year and was being demoed to prospective clients. The sales team started covering ground, and the first few customers were rolled out quickly. These... [Read More]
Tags: Spark Scaling

High output Management

I had shared that I was elated to management roles and my struggles with them. As a technical person and and IC my previous roles had been very crips. They completely outlined what was expected from me. But as a manager I had been struggling to define what my role... [Read More]

Runtime XML binding using EclipseLink

Recently I was doing a POC for a dynamic JXB user-case. So our customer was consuming a JAXB based schema from one of their upstreams. The schema published information fields of supported products. Since the upstream was also developing and adding more projects so they did not freeze their schema.... [Read More]