Enterprise product catalog search on Google Cloud

Enterprise product catalog search on Google Cloud Enterprise product catalog search on Google Cloud Hybrid keyword + vector search with Vertex AI, a Cloud Run search API, and a Pub/Sub + Dataflow pipeline that keeps the index fresh Google Cloud Project: catalog-prod Region us-central1 Retrieval and ranking search 1. A shopper query reaches the global load balancer through Cloud Armor1 2. Cloud Armor → Load Balancer2 hybrid query 5. The query embedding and the keywords are sent to the search index together and results are blended5 7. Pub/Sub → Dataflow7 archive 3. Load Balancer → Search API3 embed query 4. Search API → Embeddings (embed query)4 changes 6. PIM / ERP → Pub/Sub (changes)6 upsert 8. Dataflow → Vector and keyword search (upsert)8 tune query Shoppers (Users) Shoppers web and app Merchandisers (Users) Merchandisers synonyms, boosts Cloud Armor (Cloud Armor) Cloud Armor Load Balancer (Cloud Load Balancing) Load Balancer global external Search API (Cloud Run) Search API Cloud Run Vector and keyword search (Vertex AI) Vector and keyword search Vertex AI Search / Vector Search Embeddings (Vertex AI) Embeddings text embedding model Pub/Sub (Pub/Sub) Pub/Sub catalog change events Dataflow (Dataflow) Dataflow enrich and embed BigQuery (BigQuery) BigQuery catalog and clickstream PIM / ERP (Server Farm) PIM / ERP product master Cloud Storage (Cloud Storage) Cloud Storage images and feeds Looker (Looker) Looker search analytics Cloud Monitoring (Cloud Monitoring) Cloud Monitoring

Flow

  1. 1A shopper query reaches the global load balancer through Cloud Armor
  2. 2Cloud Armor → Load Balancer
  3. 3Load Balancer → Search API
  4. 4Search API → Embeddings (embed query)
  5. 5The query embedding and the keywords are sent to the search index together and results are blended
  6. 6PIM / ERP → Pub/Sub (changes)
  7. 7Pub/Sub → Dataflow
  8. 8Dataflow → Vector and keyword search (upsert)