Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,4 @@ examples/**/docs/
tickets/**
site/dag/**
cache/**
articles/**
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@ MAKEFILE_DIR := $(dir $(abspath $(firstword $(MAKEFILE_LIST))))

include $(MAKEFILE_DIR)/Makefile.pipeline
include $(MAKEFILE_DIR)/Makefile.dev
include $(MAKEFILE_DIR)/Makefile.built
35 changes: 35 additions & 0 deletions Makefile.built
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Makefile for fastflowtransform

# Change this if your venv has a different name/path
VENV = .venv

.PHONY: help venv build check upload-test upload clean

help:
@echo "Useful commands:"
@echo " make venv - create virtualenv and install build"
@echo " make build - build wheel + sdist into dist/"
@echo " make check - run twine check on dist/*"
@echo " make upload-test - upload dist/* to TestPyPI (using uvx + twine)"
@echo " make upload - upload dist/* to PyPI (using uvx + twine)"
@echo " make clean - remove build artifacts"

venv:
python3 -m venv $(VENV)
$(VENV)/bin/python -m pip install --upgrade pip
$(VENV)/bin/uv pip install build

build: venv
$(VENV)/bin/python -m build

check:
uvx twine check dist/*

upload-test: build check
uvx twine upload --repository testpypi dist/*

upload: build check
uvx twine upload dist/*

clean:
rm -rf build dist *.egg-info
8 changes: 5 additions & 3 deletions docs/Profiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,14 @@ Supported engines and their expected sections:
|----------------------|--------------------|---------------------------------------------------|
| `duckdb` | `duckdb` | `path` (file path or `:memory:`) |
| `postgres` | `postgres` | `dsn`, `db_schema` |
| `bigquery` | `bigquery` | `project` (optional), `dataset`, `location` |
| `databricks_spark` | `databricks_spark` | `master`, `app_name`, optional `extra_conf`, `warehouse_dir`, `use_hive_metastore`, `database`, `table_format`, `table_options` |
| `snowflake_snowpark` | `snowflake_snowpark`| `account`, `user`, `password`, `warehouse`, `database`, `db_schema`, optional `role` |
| `bigquery` | `bigquery` | `project` (optional), `dataset`, `location`, `use_bigframes`, `allow_create_dataset` |
| `databricks_spark` | `databricks_spark` | `master`, `app_name`, optional `extra_conf`, `warehouse_dir`, `use_hive_metastore`, `catalog`, `database`, `table_format`, `table_options` |
| `snowflake_snowpark` | `snowflake_snowpark`| `account`, `user`, `password`, `warehouse`, `database`, `schema` (`db_schema` alias), optional `role`, `allow_create_schema` |

Each profile can define its own `vars:` block (values exposed via `var('key')` inside templates).

> Snowflake note: the CLI scaffold shows `schema:` while the docs often mention `db_schema:`. The configuration accepts either key because `schema` is an alias for `db_schema` in the settings model.

## Environment Variables

`profiles.yml` supports Jinja expressions. The helper `env('FF_VAR', 'fallback')` reads process environment variables and substitutes the default if unset. Examples:
Expand Down
Loading