Kashif SohailReal-time Airflow Alerts to Google ChatIntegrate airflow with Google Chat to send task status via callbacksJun 6, 20234Jun 6, 20234
Kashif SohailETL Jobs Made Easy: Using Meltano to Extract from GitHub and Load into BigQueryMeltano is an open-source platform that simplifies ETL jobs and provides an easy-to-use interface for data extraction, transformation, and…Mar 6, 2023Mar 6, 2023
Kashif SohailExport Query Result from BigQuery to Google Cloud Storage using PythonThis tutorial guides you to programmatically export your query results to Google Cloud Storage.Mar 2, 20232Mar 2, 20232
Kashif SohailDagster vs Airflow: Choosing the Right Workflow Management SystemWorkflow management systems are essential for data engineering and machine learning projects.Feb 28, 20231Feb 28, 20231
Kashif SohailChoosing the Right Data Management Tool: Comparing Apache Hudi and Delta LakeApache Hudi and Delta Lake are two open-source technologies designed to improve the performance and reliability of data lakes. While both…Feb 25, 20232Feb 25, 20232
Kashif SohailGet Free Microsoft Azure Certifications 2021This post has discussed the easiest and hassle-free way to get free vouchers to attempt Microsoft Azure Certification and add credentials…May 17, 20211May 17, 20211
Kashif SohailRead files from Google Cloud Storage Bucket using local PySpark and Jupyter NotebooksThis tutorial is a step by step guide for reading files from google cloud storage bucket in locally hosted spark instance using PySpark…Jun 28, 20205Jun 28, 20205
Kashif SohailHow to read Compressed CSV files from S3 using local PySpark and Jupyter notebookThis tutorial is a step by step guide for configuring your Spark instance deployed on EC2 instance, virtual machine hosted in cloud or in…Dec 9, 20193Dec 9, 20193