Pipelines

Pipelines define what data is extracted and how it is shaped for the warehouse — which source tables map to which fact and dimension tables, and on what schedule. Analysts configure the dimensions, measures and refresh cadence here. This is the design layer that turns transactional records into a clean analytics model for BI tools.

Workflow

Open Pipelines from the Data Warehouse export menu.
Create a pipeline and name the dataset it produces.
Map source models to fact and dimension tables.
Select the destination it writes to.
Set the schedule (for example nightly) and activate it.

Fields reference

Every field on this screen, drawn from the live data model.

Field	Type	Required	Description
Website Messages `website_message_ids`	List	—	Website communication history
Columns `column_whitelist`	Text	—	Comma-separated stored field names, or "*" for all stored fields on the source model.
Filter Domain `filter_domain`	Text	—	BridgeERP domain (Python list literal). Combined with incremental watermark when present.
Partition Field `partition_field`	Text	—	Field used by destination to partition (e.g. by date).
Incremental Field `incremental_field`	Text	—	Field whose value advances the watermark. Records with value > watermark are exported.
Watermark `watermark_value`	Text	—	Last successfully exported incremental_field value.
Schema Version `schema_version`	Number	—	Bumps when column set changes.
Last Columns Hash `last_columns_hash`	Text	—	Hash of last exported column set; change triggers schema_version+1.
Name `name`	Text	Yes	Name
Code `code`	Text	Yes	Code
Destination `destination_id`	Link → mfi.dw.destination	Yes	Destination
Source Model `source_model_id`	Link → ir.model	Yes	Source Model
Frequency `frequency`	Choice: Manual, Hourly, Daily, Weekly	Yes	Frequency
Compression `compression`	Choice: None, gzip, Parquet	Yes	Compression
Batch Size `batch_size`	Number	Yes	Batch Size
Retry Count `retry_count`	Number	Yes	Retry Count
Has Message `has_message`	Yes/No	—	Has Message
Ratings `rating_ids`	List	—	Ratings
Sequence `sequence`	Number	—	Sequence
Active `active`	Yes/No	—	Active
Company `company_id`	Link → res.company	—	Company
Provider `provider`	Choice: Google BigQuery, Snowflake, Amazon Redshift, Amazon S3 (Parquet), SFTP (CSV gzip), Generic HTTP POST	—	Provider
Model `source_model`	Text	—	Model
Last Run At `last_run_at`	Date & time	—	Last Run At
Last Run State `last_run_state`	Choice: Never Run, Running, Success, Failed, Partial	—	Last Run State
Last Error `last_error`	Text	—	Last Error
Runs `run_ids`	List	—	Runs
Run Count `run_count`	Number	—	Run Count

Actions & buttons

Buttons available on this screen and what they do:

Run Now
Reset Watermark

Notes & rules

Model mfi.dw.pipeline; the extract-and-shape definition.
Configures dimension/measure mapping toward a star schema.
Bound to a destination and a refresh schedule.
Each enabled run produces an Export Run record.

Technical model: mfi.dw.pipeline · Record: Data Warehouse Export Pipeline