A team must land Salesforce objects in Unity Catalog tables and is deciding which ingestion layer to start from. What does Databricks guidance recommend?
Choose an answer to check it. Afterward, select any option to review its explanation.
Databricks recommends starting with the most managed layer and dropping down only if it does not meet requirements, for example when the source is unsupported. Salesforce has a fully-managed connector, so that is the starting point.
It is true that SQL and Spark APIs in Lakeflow pipelines give more customization, which is why this is tempting. But customization is the reason to drop down a layer, not where to start, and nothing here says the managed connector falls short.
DIY connectors are possible on Databricks, but the premise is wrong: fully-managed connectors include automated schema evolution along with authentication, CDC and retries.
Federation avoids copying data, but SaaS data can be ingested into tables; Lakeflow Connect has SaaS connectors for Salesforce specifically.
id q0ddaff6c80