How do you guys debug a workflow in databricks? Notebooks now have integrated debugging tooling, but how do you debug the Notebook the workflow is calling with parameters etc? Can I make a test notebook that call the notebook with parameters like the workflow, and somehow debug into my notebook, or how are you in general getting a debugging experience with breakpoints in a notebook that is called from a workflow?

How do you guys debug a workflow in databricks? Notebooks now have integrated debugging tooling, but how do you debug the Notebook the workflow is calling with parameters etc? Can I make a test notebook that call the notebook with parameters like the workflow, and somehow debug into my notebook, or how are you in general getting a debugging experience with breakpoints in a notebook that is called from a workflow?

• databricks
• azure-databricks

• Could you please provide your approach, if you have tried anything? – Bhavani Commented Nov 18, 2024 at 11:15
• Well I have not found any ways to get a breakpoint in a workflow it self. So I was thinking whether I could make a debug notebook that call the working notebook with same parameters as the workflow. I tried with %run "./debug-tester-nb" dbutils.notebook.run("debug-tester-nb", 60, {"Key1": "data", "Key2": "data2"}). And then I hoped I could do a step into with debugger, but did not work. I need some way to be able to call a notebook in the same way as the workflow do and get a full debug developer experience. – Thomas Segato Commented Nov 18, 2024 at 11:20

1 Answer 1

Currently, the debugging the workflows is not supported. To debug the workflow, you can make use of the notebook debugger as a workaround, but you need to do this process manually.

After the workflow run, take the failed code block and use another notebook cell and use the notebook debugger here.

If you are using any workflow parameters in that code, you can make use of the notebook widgets in place of those workflow parameters and pass the required values to the widgets as shown in below sample.

print("hi")
# Define widget
dbutils.widgets.text("param1","")
# use it as workflow parameter
print(dbutils.widgets.get("param1"))
print("bye")

You can set the default value as well like below.