{"id":6658,"date":"2020-05-29T14:25:47","date_gmt":"2020-05-29T12:25:47","guid":{"rendered":"https:\/\/blog.via-internet.de\/?p=6658"},"modified":"2020-05-29T14:25:47","modified_gmt":"2020-05-29T12:25:47","slug":"azure-databricks-working-with-widgets","status":"publish","type":"post","link":"https:\/\/via-internet.de\/blog\/2020\/05\/29\/azure-databricks-working-with-widgets\/","title":{"rendered":"Azure | Working with Widgets"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">TL;DR<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Don&#8217;t want to read the post, then explore this <a href=\"https:\/\/community.cloud.databricks.com\/?o=4113294248817247#notebook\/2346634357411131\/command\/2346634357411141\">Azure Notebook<\/a><br><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Requirements<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Define needed moduls and functions<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">from datetime import datetime\n\nimport pyspark.sql.functions as F<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Create DataFrame for this post:<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">df = spark.sql(\"select * from diamonds\")\ndf.show()<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Working with Widgets<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Default Widgets<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"python\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">dbutils.widgets.removeAll()\n\ndbutils.widgets.text(\"W1\", \"1\", \"Text\")\ndbutils.widgets.combobox(\"W2\", \"3\", [str(x) for x in range(1, 10)], \"Combobox\")\ndbutils.widgets.dropdown(\"W3\", \"4\", [str(x) for x in range(1, 10)], \"Dropdown\")<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Multiselect Widgets<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">list = [ f\"Square of {x} is {x*x}\" for x in range(1, 10)]\ndbutils.widgets.multiselect(\"W4\", list[0], list, \"Multi-Select\")<\/pre>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/i1.wp.com\/blog.via-internet.de\/wp-content\/uploads\/2020\/05\/azure-databricks-working-with-widgets-01.png?fit=700%2C124&amp;ssl=1\" alt=\"\" class=\"wp-image-6665\"\/><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">Monitor the changes when selection values<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">print(\"Selection: \", dbutils.widgets.get(\"W4\"))\nprint(\"Current Time =\", datetime.now().strftime(\"\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/i2.wp.com\/blog.via-internet.de\/wp-content\/uploads\/2020\/05\/azure-databricks-working-with-widgets-11.png?fit=700%2C61&amp;ssl=1\" alt=\"\" class=\"wp-image-6668\"\/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/i1.wp.com\/blog.via-internet.de\/wp-content\/uploads\/2020\/05\/azure-databricks-working-with-widgets-13.png?fit=700%2C207&amp;ssl=1\" alt=\"\" class=\"wp-image-6671\"\/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">Filter Query by widgets<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Prepare widgets<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">dbutils.widgets.removeAll()\n\ndf = spark.sql(\"select * from diamonds\")\n\nvals = [ str(x[0]) for x in df.select(\"cut\").orderBy(\"cut\").distinct().collect() ]\ndbutils.widgets.dropdown(\"Cuts\", vals[0], vals)\n\nvals = [ str(x[0]) for x in df.select(\"carat\").orderBy(\"carat\").distinct().collect() ]\ndbutils.widgets.dropdown(\"Carat\", vals[0], vals)<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Now, change some values<\/h3>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">filter_cut = dbutils.widgets.get(\"Cuts\")\ndf=spark.sql(f\"select * from diamonds where cut='{filter_cut}'\").show()<\/pre>\n\n\n\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-8f761849 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/blog.via-internet.de\/wp-content\/uploads\/2020\/05\/azure-databricks-working-with-widgets-21.png\" alt=\"\" class=\"wp-image-6676\"\/><\/figure>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" src=\"https:\/\/blog.via-internet.de\/wp-content\/uploads\/2020\/05\/azure-databricks-working-with-widgets-22.png\" alt=\"\" class=\"wp-image-6677\"\/><\/figure>\n<\/div>\n<\/div>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>TL;DR Don&#8217;t want to read the post, then explore this Azure Notebook Requirements Define needed moduls and functions Create DataFrame for this post: Working with Widgets Default Widgets Multiselect Widgets Monitor the changes when selection values Filter Query by widgets Prepare widgets Now, change some values<\/p>\n","protected":false},"author":1,"featured_media":6659,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9,20,87],"tags":[],"class_list":["post-6658","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-databricks","category-widgets"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts\/6658","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/comments?post=6658"}],"version-history":[{"count":0,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/posts\/6658\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/"}],"wp:attachment":[{"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/media?parent=6658"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/categories?post=6658"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/via-internet.de\/blog\/wp-json\/wp\/v2\/tags?post=6658"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}