Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/lakebridge/docs/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,11 @@ For migrating your SQL workloads we provide transpilers that can:
- Translate SQL code from a variety of source platforms to Databricks SQL.
- Translate some orchestration and ETL code to Databricks SQL.

Internally, Lakebridge can use two different transpilers:
Internally, Lakebridge can use three different transpilers:

- *BladeBridge*, a mature transpiler that can handle a wide range of source dialects as well as some ETL/orchestration.
- *Morpheus*, a next-generation transpiler that currently handles a smaller set of dialects, but includes experimental support for dbt.
- *Switch*, an LLM-powered transpiler that uses Large Language Models to convert SQL and other source formats to Databricks notebooks.

The table below summarizes the source platforms that we currently support:

Expand Down
17 changes: 15 additions & 2 deletions docs/lakebridge/docs/transpile/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ It is important to note that the transpiler isn't always able to know whether it

## Plugins overview

Out of the box, Lakebridge comes with two transpilation plugins: Morpheus and BladeBridge. Each one has its own set of capabilities and levels of guarantee that we describe below.
Out of the box, Lakebridge comes with three transpilation plugins: Morpheus, BladeBridge, and Switch. Each one has its own set of capabilities and levels of guarantee that we describe below.

### Morpheus

Expand Down Expand Up @@ -133,6 +133,20 @@ Whether you're migrating a single job or thousands, BladeBridge delivers predict

More details about the BladeBridge converter [here](/docs/transpile/pluggable_transpilers/bladebridge_overview)

### Switch

Switch is an LLM-powered Lakebridge transpiler that extends beyond SQL to convert various source formats into Databricks-compatible outputs. Using [Mosaic AI Model Serving](https://docs.databricks.com/aws/en/machine-learning/model-serving/), it understands code intent and semantics to handle complex transformations.

Key characteristics:
- **LLM-powered extensibility** - Built-in prompts for multiple SQL dialects (T-SQL, Snowflake, Teradata, Oracle, etc.) and generic formats (Python, Scala, Airflow)
- **Custom prompt support** - Add YAML prompts to handle any source format not covered by built-in options
- **Flexible outputs** - Python notebooks with Spark SQL (default), SQL notebooks (experimental), or any text-based format
- **Databricks-native processing** - Executes as scalable Lakeflow Jobs with serverless compute
- **Model flexibility** - Choose any Model Serving endpoint to match your organization's requirements
- **Complex logic handling** - Excels at stored procedures and business logic beyond ANSI SQL/PSM standards

Switch's LLM-based approach enables support for any source format through custom prompts. For built-in prompt details and custom prompt creation, see the [Switch documentation](/docs/transpile/pluggable_transpilers/switch).

### Supported dialects

| Source Platform | Converter: BladeBridge | Converter: Morpheus | Output: SQL | Output: SparkSql | Output: PySpark | Other features available |
Expand All @@ -144,4 +158,3 @@ More details about the BladeBridge converter [here](/docs/transpile/pluggable_tr
| Snowflake | | ✅ | ✅ | | | dbt Repointing (Experimental) |
| MS SQL Server (incl. Synapse)| ✅ | ✅ | ✅ | | | |
| Teradata | ✅ | | ✅ | | | |

Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,11 @@ custom: # this section is optional, it is passed to the transpiler at startup
<key 1>: <value 1> # can be pretty much anything
```

Databricks provides 2 transpilers: _Morpheus_, its AST-based transpiler, and _BladeBridge_, a pattern-based transpiler.
Databricks provides 3 transpilers: _Morpheus_, its AST-based transpiler, _BladeBridge_, a pattern-based transpiler, and _Switch_, an LLM-powered transpiler.
These transpilers are installed by `Lakebridge` itself as part of running the `install-transpile` command, as follows:
- the latest _Morpheus_ is fetched from [Maven Central](https://central.sonatype.com/), and installed at `.databricks/labs/remorph-transpilers/databricks-morph-plugin/`.
- the latest _BladeBridge_ is fetched from [PyPi](https://pypi.org/), and installed at `.databricks/labs/remorph-transpilers/bladebridge/`.
- the latest _Switch_ is fetched from [PyPi](https://pypi.org/), and installed at `.databricks/labs/remorph-transpilers/switch/`.

Installing 3rd party transpilers is the responsibility of their provider.

Expand Down
Loading
Loading