merica-dlh/merica-pipelines
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(readme): remove canonical pipeline catalog section

Browse Source
This commit is contained in:
syahril
2026-10-22 10:17:51 +07:00
parent 887025db14
commit d7332b3f07
1 changed files with 3 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+3 -14
View File
@@ -59,18 +59,7 @@ merica-pipelines/
--- ---
## 📋 3. Katalog Pipeline Kanonikal ## ⚙️ 3. Cara Penggunaan Bersama MERICA Studio Appliance
| Nama Pipeline | Domain | Jadwal | Engine & Alur Medallion |
| :--- | :--- | :--- | :--- |
| `example_permits_medallion_dbt_postgres` | Perizinan Lingkungan | `0 8 * * *` | **E2E Medallion**: Sling CDC $\rightarrow$ GE Bronze $\rightarrow$ dbt Silver $\rightarrow$ GE Silver $\rightarrow$ dbt Gold $\rightarrow$ Callback. |
| `ingestion_db_environmental_permits_cdc` | AMDAL / UKL-UPL | `0 */2 * * *` | **Log CDC**: Replikasi intensif Postgres DB $\rightarrow$ S3 Bronze Parquet $\rightarrow$ DuckDB Silver $\rightarrow$ dbt Gold. |
| `ingestion_api_spku_air_quality` | Kualitas Udara (SPKU) | `0 4 * * *` | **Sliding Window**: REST API JSON $\rightarrow$ Bronze S3 $\rightarrow$ DuckDB Iceberg $\rightarrow$ GE Suite $\rightarrow$ dbt Gold Mart. |
| `ingestion_gws_waste_reports` | Sampah & Limbah B3 | `0 4 * * *` | **GWS Sanitizer**: Google Sheets $\rightarrow$ Sanitasi Regex $\rightarrow$ S3 Bronze $\rightarrow$ PyIceberg Silver $\rightarrow$ dbt Gold. |
---
## ⚙️ 4. Cara Penggunaan Bersama MERICA Studio Appliance
Arahkan variabel lingkungan di `.env` file **MERICA Studio / Standalone** ke URL repositori ini: Arahkan variabel lingkungan di `.env` file **MERICA Studio / Standalone** ke URL repositori ini:
@@ -89,7 +78,7 @@ Ketika stack MERICA Studio dijalankan (`./scripts/01_install_stack.sh`), Airflow
--- ---
## 🛠️ 5. Menambahkan Pipeline Baru ## 🛠️ 4. Menambahkan Pipeline Baru
1. **Tambahkan berkas konfigurasi YAML** di direktori `configs/<nama_pipeline>.yaml`. 1. **Tambahkan berkas konfigurasi YAML** di direktori `configs/<nama_pipeline>.yaml`.
2. **(Opsional) Tambahkan model dbt** di direktori `dbt/models/silver/` atau `dbt/models/gold/`. 2. **(Opsional) Tambahkan model dbt** di direktori `dbt/models/silver/` atau `dbt/models/gold/`.
@@ -103,7 +92,7 @@ Ketika stack MERICA Studio dijalankan (`./scripts/01_install_stack.sh`), Airflow
--- ---
## 🧪 6. Pengujian Unit Lokal ## 🧪 5. Pengujian Unit Lokal
```bash ```bash
pip install -r requirements.txt pip install -r requirements.txt