diff --git a/README.md b/README.md index 2451cb6..c945ef1 100644 --- a/README.md +++ b/README.md @@ -59,18 +59,7 @@ merica-pipelines/ --- -## ๐Ÿ“‹ 3. Katalog Pipeline Kanonikal - -| Nama Pipeline | Domain | Jadwal | Engine & Alur Medallion | -| :--- | :--- | :--- | :--- | -| `example_permits_medallion_dbt_postgres` | Perizinan Lingkungan | `0 8 * * *` | **E2E Medallion**: Sling CDC $\rightarrow$ GE Bronze $\rightarrow$ dbt Silver $\rightarrow$ GE Silver $\rightarrow$ dbt Gold $\rightarrow$ Callback. | -| `ingestion_db_environmental_permits_cdc` | AMDAL / UKL-UPL | `0 */2 * * *` | **Log CDC**: Replikasi intensif Postgres DB $\rightarrow$ S3 Bronze Parquet $\rightarrow$ DuckDB Silver $\rightarrow$ dbt Gold. | -| `ingestion_api_spku_air_quality` | Kualitas Udara (SPKU) | `0 4 * * *` | **Sliding Window**: REST API JSON $\rightarrow$ Bronze S3 $\rightarrow$ DuckDB Iceberg $\rightarrow$ GE Suite $\rightarrow$ dbt Gold Mart. | -| `ingestion_gws_waste_reports` | Sampah & Limbah B3 | `0 4 * * *` | **GWS Sanitizer**: Google Sheets $\rightarrow$ Sanitasi Regex $\rightarrow$ S3 Bronze $\rightarrow$ PyIceberg Silver $\rightarrow$ dbt Gold. | - ---- - -## โš™๏ธ 4. Cara Penggunaan Bersama MERICA Studio Appliance +## โš™๏ธ 3. Cara Penggunaan Bersama MERICA Studio Appliance Arahkan variabel lingkungan di `.env` file **MERICA Studio / Standalone** ke URL repositori ini: @@ -89,7 +78,7 @@ Ketika stack MERICA Studio dijalankan (`./scripts/01_install_stack.sh`), Airflow --- -## ๐Ÿ› ๏ธ 5. Menambahkan Pipeline Baru +## ๐Ÿ› ๏ธ 4. Menambahkan Pipeline Baru 1. **Tambahkan berkas konfigurasi YAML** di direktori `configs/.yaml`. 2. **(Opsional) Tambahkan model dbt** di direktori `dbt/models/silver/` atau `dbt/models/gold/`. @@ -103,7 +92,7 @@ Ketika stack MERICA Studio dijalankan (`./scripts/01_install_stack.sh`), Airflow --- -## ๐Ÿงช 6. Pengujian Unit Lokal +## ๐Ÿงช 5. Pengujian Unit Lokal ```bash pip install -r requirements.txt