-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Summary
FeatureStore._materialize_odfv uses source_fv.entities (entity names) instead of source_fv.join_keys (join key column names) when building BigQuery queries. This causes materialization to fail when an entity's name differs from its join key.
Steps to Reproduce
- Define an entity where name ≠ join key:
creator = Entity(
name="creator",
join_keys=["creator_id"],
value_type=ValueType.STRING,
)-
Define a FeatureView using that entity, and an OnDemandFeatureView sourcing from it with
write_to_online_store=True. -
Run
store.materialize_incremental(...).
Expected Behavior
The generated BigQuery SQL should reference creator_id (the join key column).
Actual Behavior
The SQL references creator (the entity name), which doesn't exist as a column:
google.api_core.exceptions.BadRequest: 400 Unrecognized name: creator;
Did you mean creator_id? at [6:24]
Root Cause
In feature_store.py _materialize_odfv, two lines use .entities (names) where they should use .join_keys (column names):
- Line 1354:
all_join_keys.update(source_fv.entities)→ should besource_fv.join_keys - Line 1394:
join_key_columns=source_fv.entities→ should besource_fv.join_keys
FeatureView.entities returns [e.name for e in entities] (entity names), while FeatureView.join_keys returns [e.name for e in self.entity_columns] (actual column names). When these differ, the query fails.
Note: regular materialize_single_feature_view does not have this bug because it resolves join keys correctly through the provider layer.
Suggested Fix
for source_fv in source_fvs:
- all_join_keys.update(source_fv.entities)
+ all_join_keys.update(source_fv.join_keys)
if source_fv.batch_source:
entity_timestamp_col_names.add(source_fv.batch_source.timestamp_field) job = provider.offline_store.pull_latest_from_table_or_query(
config=self.config,
data_source=source_fv.batch_source,
- join_key_columns=source_fv.entities,
+ join_key_columns=source_fv.join_keys,
feature_name_columns=[f.name for f in source_fv.features],Environment
- Feast version: 0.58.0
- Offline store: BigQuery