AWS serverless streaming analytics
Posted inAWS Big data Kinesis

AWS serverless streaming analytics

Overview. This pipeline demonstrates a fully serverless streaming analytics architecture on AWS. High‑volume events flow into Amazon Kinesis Data Streams. An AWS Lambda function batches and transforms records in real time before…
Posted inBig data Spark

How to Install Apache Spark on Windows

Apache Spark is a powerful distributed computing system used for big data processing, machine learning, and real-time analytics. While it is often deployed on clusters, you can also install it…
Posted inAzure Big data CI/CD

Introduction to Git and GitLab

As a data engineer, managing versions of your code, data pipelines, and configuration files is crucial for efficient development and collaboration. Git and GitLab provide powerful tools to version, manage,…