faucet-stream wires 23 source and 18 sink connectors together with a single
faucet binary that runs pipelines declaratively from a YAML/JSON file — no Rust
code required. Or skip the binary and embed the same engine in your own service
through the typed Source / Sink traits.
cargo install faucet-cli
faucet init my_pipeline --source postgres --sink bigquery
faucet validate pipeline.yaml
faucet run pipeline.yaml
Why faucet-stream
Fast & reliable by default
Native streaming with bounded memory, connection pooling, multi-row inserts, bulk APIs, and parallel I/O — performance is the reason the library exists.
Config-driven or embeddable
Run faucet run pipeline.yaml, or call Pipeline::new(&source, &sink).run().await? from Rust. Same engine either way.
A runtime, not just connectors
Incremental + resumable replication, change-data-capture, exactly-once delivery, dead-letter queues, retries, quality checks, and built-in metrics + tracing — with zero per-connector code.
Pay only for what you use
Every connector is a Cargo feature. Build a slim binary with just the source and sink you need.
How this book is organized
- Getting Started — install, run your first pipeline in five minutes, and (if you like) learn the whole architecture as a story.
- Tutorials — end-to-end walkthroughs of real pipelines (incremental REST → BigQuery, Postgres CDC, DAGs, embedding).
- Cookbook — short, task-oriented recipes for pagination, auth, state, upserts, dead-letter queues, secrets, and more.
- Reference — the connector catalog, CLI commands, and config-file grammar.
- Operations — deploying, observability, performance tuning, and troubleshooting.
- Extending — author and publish your
own
faucet-source-*/faucet-sink-*crate.
Where else to look
- API docs: every crate is on docs.rs, rendered with all features so optional connectors are visible.
- Source & issues: github.com/PawanSikawat/faucet-stream.
- Runnable examples: the
cli/examples/directory ships a config for nearly every connector pair, andexamples/has adocker-composestack so they run locally.