Jerome Rajan

0 %
Jerome Rajan
Data & AI Architect at Google
  • Residence:
    India
  • City:
    Mumbai
GCP Agent Platform
GenAI
SQL
Dataproc, EMR
Hadoop
BigQuery
PySpark, Python
Data Pipeline Design
Tableau, Redshift, Snowflake
IBM DataStage
  • AWS Lambda, S3, EMR, SQS, DynamoDB, Step Functions, Cloud Functions
  • Unix Shell Scripting, Python
  • Oracle, DB2, Redis
  • Alteryx, VBA, Blueprism, UiPath
English
Tamil
Hindi
Malayalam
Marathi
Spanish

My Blog

A Detailed Guide to Key Generation in BigQuery
May 5, 2025 / Technology
A Detailed Guide to Key Generation in BigQuery

Migrating an enterprise data warehouse from traditional SMP or MPP databases (Oracle, Teradata, SQL Server, Netezza) to Google BigQuery inevitably…

Chatting With Dataproc Clusters Using an AI Agent
April 21, 2025 / Technology
Chatting With Dataproc Clusters Using an AI Agent

Managing fleet-scale Apache Spark and Hadoop environments on Google Cloud Dataproc routinely requires juggling gcloud dataproc clusters CLI commands, Cloud…

Implementing Your Logical Data Model in BigQuery: A Practical Guide
April 16, 2025 / Technology
Implementing Your Logical Data Model in BigQuery: A Practical Guide

A clean Logical Data Model (LDM) captures your enterprise’s business entities, attributes, grain, and relationships independent of any database engine….

Build a Fraud Detection Model in Minutes with BigQuery ML
April 10, 2025 / Technology
Build a Fraud Detection Model in Minutes with BigQuery ML

Traditionally, shipping a fraud-detection classifier meant exporting operational tables to object storage, spinning up a Python training cluster, wrestling with…

Parallel & Serverless CSV Ingestion to Cloud SQL Using Cloud Dataflow
August 31, 2024 / Technology
Parallel & Serverless CSV Ingestion to Cloud SQL Using Cloud Dataflow

Anyone who has tried bulk-loading hundreds of gigabytes of multi-file CSV exports from Google Cloud Storage (GCS) into Cloud SQL…

Spark Config Calculator
December 15, 2023 / Technology
Spark Config Calculator

The Spark Configuration Calculator is an interactive Streamlit web application designed to take the guesswork out of sizing and tuning…

Using Spark on Dataproc & Apache Iceberg to Build an Open Lakehouse
December 6, 2023 / Technology
Using Spark on Dataproc & Apache Iceberg to Build an Open Lakehouse

Raw Parquet lakes on object storage solved cheap, decoupled scalability—but left data engineers stitching together fragile directory-rename hacks whenever workloads…

A Guide to RAID Multiple Local SSDs & Mount Them on Dataproc
November 28, 2023 / Technology
A Guide to RAID Multiple Local SSDs & Mount Them on Dataproc

Heavy Apache Spark shuffles, wide joins, and terabyte-scale external sorts live and die by local scratch disk IOPS and throughput….

Understanding Driver Pools in Dataproc
August 22, 2023 / Technology
Understanding Driver Pools in Dataproc

Let’s learn about driver pools in Dataproc. But before we delve into the details of driver pools, it’s important to…

Build Your Own Ask-Me-Anything Search Engine Using Vertex AI, LangChain & Streamlit
July 29, 2023 / Technology
Build Your Own Ask-Me-Anything Search Engine Using Vertex AI, LangChain & Streamlit

Artificial Intelligence and more specifically, its “Generative” heir has experienced a golden year thus far. Generative AI has laid the…

Hadoop — Understanding Splits, Blocks & Everything In Between
April 21, 2023 / Technology
Hadoop — Understanding Splits, Blocks & Everything In Between

Understanding Hadoop is like trying to unravel a tangled ball of yarn while wearing oven mitts. I’ve had my fair…

Understanding CPU Oversubscription in Dataproc & Hadoop YARN
April 7, 2023 / Technology
Understanding CPU Oversubscription in Dataproc & Hadoop YARN

Let’s start by understanding 2 important properties in your Yarn configuration. For simple Hadoop workloads, it is best to leave…

Dataproc — Why Is My Cluster Not Scaling?
October 6, 2022 / Technology
Dataproc — Why Is My Cluster Not Scaling?

“Autoscaling” is a Dataproc API that automates the process of monitoring YARN memory utilisation and adding/removing capacity to achieve optimal…

GCP Cloud Logging: How to Enable Data Access Audit Logs for Selected GCS Buckets
October 4, 2022 / Technology
GCP Cloud Logging: How to Enable Data Access Audit Logs for Selected GCS Buckets

Introduction Data Access Audit Logs are used to trace and monitor API calls that create, modify, or read user data…

Hilton Shillim — Expensive, Expansive, Exclusive!
September 16, 2022 / Experiences
Hilton Shillim — Expensive, Expansive, Exclusive!

The Hilton Shillim Estate Retreat & Spa is a property nestled quietly into the western ghats. Taking that turn from…

1 2 3
Be Original
Would the boy you were be proud of the man you are?