Jerome Rajan

0 %
Jerome Rajan
Data & AI Architect at Google
  • Residence:
    India
  • City:
    Mumbai
GCP Agent Platform
GenAI
SQL
Dataproc, EMR
Hadoop
BigQuery
PySpark, Python
Data Pipeline Design
Tableau, Redshift, Snowflake
IBM DataStage
  • AWS Lambda, S3, EMR, SQS, DynamoDB, Step Functions, Cloud Functions
  • Unix Shell Scripting, Python
  • Oracle, DB2, Redis
  • Alteryx, VBA, Blueprism, UiPath
English
Tamil
Hindi
Malayalam
Marathi
Spanish

My Blog

Real-Time Data Streaming into Kinesis & Ingestion into Postgres Using AWS Glue — Part 2 (Glue Catalog Setup)
October 6, 2020 / Technology
Real-Time Data Streaming into Kinesis & Ingestion into Postgres Using AWS Glue — Part 2 (Glue Catalog Setup)

Before we start building Glue jobs, we need to understand that one of the unique features of Glue is its…

Real-Time Data Streaming into Kinesis & Ingestion into Postgres Using AWS Glue — Part 1 (Setup)
October 5, 2020 / Technology
Real-Time Data Streaming into Kinesis & Ingestion into Postgres Using AWS Glue — Part 1 (Setup)

Earlier this year, Amazon announced serverless streaming ETL support in AWS Glue. Under the hood, AWS Glue runs managed Apache…

AWS Glue: Develop & Test PySpark ETL Jobs Locally Using Development Endpoints
October 3, 2020 / Technology
AWS Glue: Develop & Test PySpark ETL Jobs Locally Using Development Endpoints

AWS Glue jobs—especially Streaming ETL jobs—can incur cold-start spin-up times of several minutes per run. Setting up an AWS Glue…

The Juggernaut Resolution: Why You Should Start New Year Goals on December 1
January 5, 2020 / Experiences
The Juggernaut Resolution: Why You Should Start New Year Goals on December 1

One thing almost everyone has struggled with at some point is sticking to New Year’s resolutions. My resolutions have evolved…

DataStage Warning NGBR-00019: Input Data Set 0 Not Present in Design Time Information
February 25, 2018 / Technology
DataStage Warning NGBR-00019: Input Data Set 0 Not Present in Design Time Information

This IBM InfoSphere DataStage warning cost me two full days of investigation. Here was the exact scenario: I needed to…

How to Replace Substrings in an IBM DataStage Parallel Transformer
February 25, 2018 / Technology
How to Replace Substrings in an IBM DataStage Parallel Transformer

Suppose you want to replace a multi-character substring inside an IBM DataStage Parallel Transformer—for instance, replacing “choc” with “mock” in…

Generate a Unique Sequence Counter in a DataStage Parallel Transformer
February 25, 2018 / Technology
Generate a Unique Sequence Counter in a DataStage Parallel Transformer

Need a deterministic, collision-free sequential row number across all parallel partitions in an IBM DataStage Parallel Transformer without adding a…

Understanding LUTCreateOp and LUTProcessOp in IBM DataStage
February 25, 2018 / Technology
Understanding LUTCreateOp and LUTProcessOp in IBM DataStage

Every so often on DSXchange, DataStage veterans like Ray Wurlod, Craig Hulett, and ArndW share concise explanations of what the…

Aggregator Methods in DataStage: Hash Mode vs. Sort Mode (Courtesy: Ray Wurlod)
February 25, 2018 / Technology
Aggregator Methods in DataStage: Hash Mode vs. Sort Mode (Courtesy: Ray Wurlod)

Choosing between Hash mode and Sort mode in the IBM DataStage Parallel Aggregator stage is one of the highest-leverage performance…

What Exactly Are Hash Files in DataStage? (Courtesy: Ken Bland)
February 25, 2018 / Technology
What Exactly Are Hash Files in DataStage? (Courtesy: Ken Bland)

Editor’s Note: Archived here for quick reference from Ken Bland’s classic DSXchange newsletter tech tip (DSXchange). Full credit to Ken…

DataStage Parallel Transformer Date Derivations Cheat Sheet
February 25, 2018 / Technology
DataStage Parallel Transformer Date Derivations Cheat Sheet

When your pipeline cannot join against a calendar/time dimension table, these battle-tested IBM DataStage Parallel Transformer expressions compute common month-end,…

IBM DataStage orchadmin Command Reference Guide
February 25, 2018 / Technology
IBM DataStage orchadmin Command Reference Guide

In IBM InfoSphere DataStage, Parallel Datasets (.ds) store a lightweight descriptor file at the target path while distributing raw partition…

Notes on DataStage Bulk Load Tuning with the Oracle Connector
February 25, 2018 / Technology
Notes on DataStage Bulk Load Tuning with the Oracle Connector

Quick production checklist when configuring high-volume direct-path bulk loads into Oracle using the IBM InfoSphere DataStage Oracle Connector stage:

ETL — Code, Build, or Both?
February 25, 2018 / Technology
ETL — Code, Build, or Both?

Data has been growing at an exponential pace—not just in raw row volume, but across entirely new dimensions fueled by…

Business Alert & Exception Data Management Within an ETL Ecosystem
February 25, 2018 / Technology
Business Alert & Exception Data Management Within an ETL Ecosystem

Introduction Data transparency and traceability are foundational to any enterprise ETL ecosystem. Both business data stewards and platform support engineers…

1 2 3
Be Original
Would the boy you were be proud of the man you are?