83 lines
3.6 KiB
Markdown
83 lines
3.6 KiB
Markdown
|
# Nessie — Konfigurasi `deploy/`
|
||
|
|
|
||
|
|
Nessie menggantikan Apache Polaris sebagai **Iceberg REST catalog** pada
|
||
|
|
`deploy/`. Konfigurasi diberikan sepenuhnya melalui **environment variable
|
||
|
|
di `docker-compose.yml`** (bukan file `application.properties`), dengan alasan:
|
||
|
|
|
||
|
|
- Kredensial (password PostgreSQL, access key MinIO) harus berasal dari `.env`
|
||
|
|
yang di-generate `install.sh` — mengikuti pola komponen lain di SiCABE.
|
||
|
|
- Env var adalah bentuk yang didokumentasikan resmi untuk image Nessie dan tidak
|
||
|
|
bergantung pada lokasi mount file config di dalam image.
|
||
|
|
|
||
|
|
## Versi
|
||
|
|
|
||
|
|
| Item | Nilai |
|
||
|
|
|---|---|
|
||
|
|
| Image | `ghcr.io/projectnessie/nessie:${NESSIE_VERSION:-0.108.4-java}` |
|
||
|
|
| Port API/Iceberg REST | `19120` (host `${NESSIE_PORT:-12120}`) |
|
||
|
|
| Port management (health/metrics) | `9000` (internal, tidak dipublikasikan) |
|
||
|
|
| Path Iceberg REST | `/iceberg/` (default warehouse, branch `main`) |
|
||
|
|
| Path Nessie REST v2 | `/api/v2` |
|
||
|
|
|
||
|
|
## Persistensi: JDBC2 di PostgreSQL
|
||
|
|
|
||
|
|
Mengikuti **konvensi shared PostgreSQL** SiCABE (lihat `config/postgres/init/01-create-dbs.sh`):
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
NESSIE_VERSION_STORE_TYPE: "JDBC2"
|
||
|
|
NESSIE_VERSION_STORE_PERSIST_JDBC_DATASOURCE: "postgresql"
|
||
|
|
QUARKUS_DATASOURCE_POSTGRESQL_DB_KIND: "postgresql"
|
||
|
|
QUARKUS_DATASOURCE_POSTGRESQL_ACTIVE: "true"
|
||
|
|
QUARKUS_DATASOURCE_POSTGRESQL_DEVSERVICES_ENABLED: "false"
|
||
|
|
QUARKUS_DATASOURCE_POSTGRESQL_JDBC_URL: "jdbc:postgresql://postgres:5432/nessie?currentSchema=nessie"
|
||
|
|
QUARKUS_DATASOURCE_POSTGRESQL_USERNAME: "nessie"
|
||
|
|
QUARKUS_DATASOURCE_POSTGRESQL_PASSWORD: "${NESSIE_DB_PASSWORD}"
|
||
|
|
```
|
||
|
|
|
||
|
|
> **Penting**: Nessie **tidak** memakai Flyway. Database `nessie` **dan** schema
|
||
|
|
> `nessie` harus dibuat lebih dulu (dilakukan `01-create-dbs.sh`); tabel-tabel
|
||
|
|
> Nessie di-auto-create saat startup. Driver PostgreSQL sudah ter-bundle di image.
|
||
|
|
|
||
|
|
> **Catatan versi**: `JDBC` (tanpa `2`) masih ada tetapi **deprecated** di 0.108.x —
|
||
|
|
> gunakan `JDBC2`.
|
||
|
|
|
||
|
|
## Warehouse & S3/MinIO (server-side)
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
NESSIE_CATALOG_DEFAULT_WAREHOUSE: "warehouse"
|
||
|
|
NESSIE_CATALOG_WAREHOUSES_WAREHOUSE_LOCATION: "s3://${MINIO_BUCKET_ICEBERG_DATA:-iceberg-data}/warehouse"
|
||
|
|
NESSIE_CATALOG_SERVICE_S3_DEFAULT_OPTIONS_ENDPOINT: "http://minio:9000"
|
||
|
|
NESSIE_CATALOG_SERVICE_S3_DEFAULT_OPTIONS_REGION: "us-east-1"
|
||
|
|
NESSIE_CATALOG_SERVICE_S3_DEFAULT_OPTIONS_AUTH_TYPE: "STATIC"
|
||
|
|
NESSIE_CATALOG_SERVICE_S3_DEFAULT_OPTIONS_PATH_STYLE_ACCESS: "true"
|
||
|
|
NESSIE_CATALOG_SERVICE_S3_DEFAULT_OPTIONS_ACCESS_KEY: "urn:nessie-secret:quarkus:minio.creds"
|
||
|
|
MINIO_CREDS_NAME: "${MINIO_ROOT_USER}"
|
||
|
|
MINIO_CREDS_SECRET: "${MINIO_ROOT_PASSWORD}"
|
||
|
|
```
|
||
|
|
|
||
|
|
> **Penting**: nama secret harus memakai **titik** (`minio.creds`), bukan hyphen.
|
||
|
|
> Quarkus env mapping tidak men-decode hyphen dengan benar (temuan eksperimen
|
||
|
|
> nessie-dremio) → error `Missing access key and secret for STATIC authentication mode`.
|
||
|
|
|
||
|
|
- Warehouse memakai bucket `iceberg-data` (konvensi base) dengan prefix `warehouse/`.
|
||
|
|
- `default-options.` **wajib** untuk Nessie ≥ 0.92.
|
||
|
|
- Kredensial S3 server-side **tidak diekspos ke klien**; engine (Trino/StarRocks/Dremio)
|
||
|
|
mengakses MinIO dengan kredensialnya sendiri.
|
||
|
|
- `external-endpoint` sengaja tidak di-set karena semua klien berada di dalam
|
||
|
|
network Docker yang sama (`minio:9000`).
|
||
|
|
|
||
|
|
## Healthcheck
|
||
|
|
|
||
|
|
Image Nessie tidak menyediakan `curl`/`wget`, sehingga healthcheck memakai
|
||
|
|
`bash /dev/tcp` ke **management port 9000**:
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
test: ["CMD", "bash", "-c", "exec 3<>/dev/tcp/localhost/9000 && printf 'GET /q/health/ready HTTP/1.0\\r\\n\\r\\n' >&3 && cat <&3 | grep -q '200 OK'"]
|
||
|
|
```
|
||
|
|
|
||
|
|
## Referensi
|
||
|
|
|
||
|
|
- https://projectnessie.org/nessie-0-108-4/configuration/
|
||
|
|
- https://projectnessie.org/guides/iceberg-rest/
|
||
|
|
- Cache riset: `.tmp/external-context/nessie/`
|