fix: FeatureView serialization with cycle detection by HaoXuAI · Pull Request #5502 · feast-dev/feast

HaoXuAI · 2025-07-10T05:43:29Z

What this PR does / why we need it:

Follow up on PR: #5482.

Which issue(s) this PR fixes:

Misc

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

Copilot

Pull Request Overview

Adds support for on-demand feature sources, enforces cycle detection in FeatureView (de)serialization, standardizes the topological sort API, and refreshes related documentation.

Introduce OnDemandSourceType alias and refactor sources signature in OnDemandFeatureView.
Rename all topo_sort functions/methods to topological_sort and update callers.
Enhance FeatureView.to_proto/from_proto with cycle detection and update __copy__/__eq__.
Add DAG module README and update compute-engine reference docs.

Reviewed Changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
sdk/python/feast/on_demand_feature_view.py	Add `OnDemandSourceType` alias and clean up `sources` hints
sdk/python/feast/infra/compute_engines/feature_resolver.py	Rename resolver method from `topo_sort` to `topological_sort`
sdk/python/feast/infra/compute_engines/feature_builder.py	Update builder calls to `topological_sort`
sdk/python/feast/infra/compute_engines/algorithms/topo.py	Rename functions `topo_sort[_multiple]` to `topological_sort[_multiple]`
sdk/python/feast/infra/compute_engines/dag/README.md	Add high-level DAG documentation
sdk/python/feast/feature_view.py	Implement cycle detection in (de)serialization, update copy/eq
docs/reference/compute-engine/README.md	Refresh compute-engine table and links
docs/getting-started/concepts/batch-feature-view.md	New guide for `BatchFeatureView`

Comments suppressed due to low confidence (1)

docs/reference/compute-engine/README.md:22

The markdown link for ExecutionPlan is nested as [link]([link](...)). It should be formatted as a single link, e.g. [link](URL).

| `ExecutionPlan`    | Executes nodes in dependency order and stores intermediate outputs   | [link]([link](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/compute_engines/dag/README.md))         |

sdk/python/feast/infra/compute_engines/feature_resolver.py

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

HaoXuAI · 2025-07-13T17:07:07Z

sdk/python/feast/on_demand_feature_view.py

                for feature in on_demand_feature_view_proto.spec.features
            ],
-            sources=sources,
+            sources=cast(List[OnDemandSourceType], sources),


Somehow this was breaking mypy, so put a fix on this.

franciscojavierarceo · 2025-07-14T01:47:08Z

docs/getting-started/concepts/batch-feature-view.md

+## ✅ Key Capabilities
+
+- **Composable DAG of FeatureViews**: Supports defining a `BatchFeatureView` on top of one or more other `FeatureView`s.
+- **Transformations**: Apply PySpark-based transformation logic (`feature_transformation` or `udf`) to raw data source, can also be used to deal with multiple data sources.


is this exclusively limited to PySpark?

not exclusively to Pyspark, will update it

franciscojavierarceo · 2025-07-14T01:47:48Z

docs/getting-started/concepts/batch-feature-view.md

+        *,
+        name: str,
+        source: Union[DataSource, FeatureView, List[FeatureView]],
+        sink_source: Optional[DataSource] = None,


for some reason i thought we agreed on naming it sink.

Yeah, on a second thought, I think sink_source is more explicit for the user to know it is passing a data source to this config.

franciscojavierarceo · 2025-07-14T01:48:24Z

docs/getting-started/concepts/batch-feature-view.md

+        Field(name="conv_rate", dtype=Float32),
+    ],
+    aggregations=[
+        Aggregation(column="conv_rate", function="sum", time_window=timedelta(days=1)),


nit: maybe we should make it avg since summing a conversion rate is weird

docs/getting-started/concepts/batch-feature-view.md

docs/reference/compute-engine/README.md

franciscojavierarceo · 2025-07-14T01:51:49Z

docs/reference/compute-engine/README.md

+## Feature resolver and builder
+The `FeatureBuilder` initialize a `FeatureResolver` that extracts a DAG from the `FeatureView` definitions, resolving dependencies and ensuring correct execution order. \
+The FeatureView represents a logical data source, while DataSource represents the physical data source (e.g., BigQuery, Spark, etc.). \
+When defines the FeatureView, the source can be a physical DataSource, a derived FeatureView, or a list of FeatureViews.


missing a word here. ?

docs/reference/compute-engine/README.md

franciscojavierarceo

lgtm, had some small suggestions that would be great to incorporate though. thanks for the docs!!!

Co-authored-by: Francisco Arceo <arceofrancisco@gmail.com>

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

# [0.51.0](v0.50.0...v0.51.0) (2025-07-21) ### Bug Fixes * FeatureView serialization with cycle detection ([#5502](#5502)) ([f287ca5](f287ca5)) * Fix current version in publish workflow ([#5499](#5499)) ([0af6e94](0af6e94)) * Fix NPM authentication ([#5506](#5506)) ([9f85892](9f85892)) * Fix verify wheels workflow for macos14 ([#5486](#5486)) ([07174cc](07174cc)) * Fixed error thrown for invalid project name on features api ([#5525](#5525)) ([4a9a5d0](4a9a5d0)) * Fixed ODFV on-write transformations ([271ef74](271ef74)) * Move Install OS X dependencies before python setup ([#5488](#5488)) ([35f211c](35f211c)) * Normalize current version by removing 'v' prefix if present ([#5500](#5500)) ([43f3d52](43f3d52)) * Skip macOS 14 with Python 3.10 due to gettext library ([#5490](#5490)) ([41d4977](41d4977)) * Standalone Web UI Publish Workflow ([#5498](#5498)) ([c47b134](c47b134)) ### Features * Added endpoints to allow user to get data for all projects ([4e06965](4e06965)) * Added grpc and rest endpoint for features ([#5519](#5519)) ([0a75696](0a75696)) * Added relationship support to all API endpoints ([#5496](#5496)) ([bea83e7](bea83e7)) * Continue updating doc ([#5523](#5523)) ([ea53b2b](ea53b2b)) * Hybrid offline store ([#5510](#5510)) ([8f1af55](8f1af55)) * Populate created and updated timestamp on data sources ([af3056b](af3056b)) * Provide ready-to-use Python definitions in api ([37628d9](37628d9)) * Snowflake source. fetch MAX in a single query ([#5387](#5387)) ([b49cea1](b49cea1)) * Support compute engine to use multi feature views as source ([#5482](#5482)) ([b9ac90b](b9ac90b)) * Support pagination and sorting on registry apis ([#5495](#5495)) ([c4b6fbe](c4b6fbe)) * Update doc ([#5521](#5521)) ([2808ce1](2808ce1))

HaoXuAI added 2 commits July 9, 2025 22:41

update

cdda0a0

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

update

a482e1c

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

HaoXuAI requested a review from a team as a code owner July 10, 2025 05:43

HaoXuAI requested a review from Copilot July 10, 2025 05:43

This comment was marked as outdated.

Sign in to view

HaoXuAI added 2 commits July 9, 2025 23:46

fix linting

d6c2704

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

fix doc

3962b73

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

HaoXuAI requested a review from Copilot July 10, 2025 06:48

HaoXuAI added the ok-to-test label Jul 10, 2025

HaoXuAI changed the title ~~Compute engine feaeture view serde~~ fix: FeatureView serialization with cycle detection Jul 10, 2025

Copilot AI reviewed Jul 10, 2025

View reviewed changes

sdk/python/feast/infra/compute_engines/feature_resolver.py Show resolved Hide resolved

HaoXuAI added 4 commits July 12, 2025 21:29

fix doc

0237706

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

fix doc

af000e7

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

fix linting

20182b7

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

Merge branch 'master' into compute-engine-feaeture-view-serde

ca075bc

HaoXuAI commented Jul 13, 2025

View reviewed changes

franciscojavierarceo reviewed Jul 14, 2025

View reviewed changes

docs/getting-started/concepts/batch-feature-view.md Outdated Show resolved Hide resolved

franciscojavierarceo reviewed Jul 14, 2025

View reviewed changes

docs/reference/compute-engine/README.md Outdated Show resolved Hide resolved

franciscojavierarceo reviewed Jul 14, 2025

View reviewed changes

docs/reference/compute-engine/README.md Outdated Show resolved Hide resolved

franciscojavierarceo approved these changes Jul 14, 2025

View reviewed changes

HaoXuAI and others added 5 commits July 13, 2025 23:52

Update docs/reference/compute-engine/README.md

2803839

Co-authored-by: Francisco Arceo <arceofrancisco@gmail.com>

Update docs/reference/compute-engine/README.md

a554718

Co-authored-by: Francisco Arceo <arceofrancisco@gmail.com>

Update docs/getting-started/concepts/batch-feature-view.md

9919bcd

Co-authored-by: Francisco Arceo <arceofrancisco@gmail.com>

update doc

4725d1b

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

update doc

dde5028

Signed-off-by: HaoXuAI <sduxuhao@gmail.com>

HaoXuAI merged commit f287ca5 into master Jul 14, 2025
18 of 19 checks passed

franciscojavierarceo mentioned this pull request Jan 14, 2026

Add blog post on new transformation framework #5856

Open

Copilot AI mentioned this pull request Jan 14, 2026

docs: Add blog post on unified transformation framework #5858

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: FeatureView serialization with cycle detection#5502

fix: FeatureView serialization with cycle detection#5502
HaoXuAI merged 13 commits intomasterfrom
compute-engine-feaeture-view-serde

HaoXuAI commented Jul 10, 2025 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

HaoXuAI Jul 13, 2025

Uh oh!

franciscojavierarceo Jul 14, 2025

Uh oh!

HaoXuAI Jul 14, 2025

Uh oh!

franciscojavierarceo Jul 14, 2025

Uh oh!

HaoXuAI Jul 14, 2025

Uh oh!

franciscojavierarceo Jul 14, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

franciscojavierarceo Jul 14, 2025

Uh oh!

Uh oh!

franciscojavierarceo left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

HaoXuAI commented Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Which issue(s) this PR fixes:

Misc

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

HaoXuAI Jul 13, 2025

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

HaoXuAI Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

HaoXuAI Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

franciscojavierarceo Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

franciscojavierarceo Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

franciscojavierarceo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

HaoXuAI commented Jul 10, 2025 •

edited

Loading

franciscojavierarceo Jul 14, 2025 •

edited

Loading