With Apache Spark Declarative Pipelines, engineers describe what their pipeline should do using SQL or Python, and Apache Spark handles the execution.

dlt supports Python 3.9+. Python 3.13 is supported but considered experimental at this time as not all of dlts extras have python 3.13. support. We additionally maintain a forked version of pendulum ...

You can also edit or refactor them in an IDE - the .py notebook above is a regular Python file. If your notebook is documentation ... which interoperates very well with Jupyter Book, or Quarto ...