Grow Your Skills and Earn Rewards! Mark your calendar: 16 March – 03 April, 2026 Join us for a three-week event dedicated to learning, upskilling, and advancing your career in data engineering, analytics, machine learning, and generative AI. Whet...
- 1424 Views
- 16 replies
- 6 kudos
Sunday
Join us for a new Community [Industry] BrickTalk: Scaling BioPharma Intelligence With the Databricks Agentic Clinical Ops Ecosystem! In this session, we are coming at you with a super cool BioPharma use case in which AI-powered Databricks agents tran...
- 209 Views
- 0 replies
- 2 kudos
Monday
We are back with instructor-led Databricks Learning Festivals for APAC & EMEA! Both Udemy and LinkedIn recognized Databricks as one of the top fastest growing skills for 2026 — Udemy's The 100 Fastest-Growing Global Skills for 2026 and LinkedIn's T...
- 839 Views
- 2 replies
- 3 kudos
a week ago
June 15–18, 2026 Register Now! Join thousands of data analytics and AI professionals at Data+ AI Summit 2026, the world’s largest conference for data, analytics and AI in San Francisco and virtually. Don’t miss early-bird pricing! Early-bird discoun...
- 1792 Views
- 1 replies
- 1 kudos
3 weeks ago
OverviewTo enhance security and reliability, Databricks is updating the trusted TLS Certificate Authorities (CAs) that issue certificates for our public-facing websites and API endpoints. This change ensures continued compliance with industry best pr...
- 1093 Views
- 3 replies
- 2 kudos
01-30-2026
Our new Claims Management Automation Solution Accelerator helps insurance teams streamline and modernize their claims workflows. This accelerator comes with pre-built code, sample data and step-by-step instructions packaged in a Databricks notebook, ...
- 433 Views
- 0 replies
- 2 kudos
a week ago
Our Community Champion Program celebrates members who go the extra mile to share knowledge, support fellow practitioners, and help the Databricks Community grow stronger every day. These champions bring curiosity, experience, and a genuine willin...
- 424 Views
- 5 replies
- 7 kudos
yesterday
This week, the Community once again turned curiosity into conversations and conversations into solutions. From insightful blogs to technical troubleshooting, there was plenty to learn from: Spotlight Contributors These members kept the convers...
- 135 Views
- 0 replies
- 5 kudos
yesterday
Community Activity
- 244 Views
- 1 replies
- 2 kudos
FAQ for Virtual Learning Festival: 16 March - 03 April 2026
General Q: Where can I ask questions if I have questions while going through the self-paced material? The Databricks Academy Learners Group page is a forum within the Databricks Community where you can ask questions and get them answered by your fe...
- 244 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Then waiting to Databricks Certified Professional Cert on 9 April.
- 2 kudos
- 228 Views
- 2 replies
- 0 kudos
Resolved! Migrating notebooks from Old databricks community instance
I didn't migrate my notebooks from the old community instance. Now I can not login. How can I access the old instance or the materials therein? Please help, I had tons of notebooks in there
- 228 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @RichardWanjohi ,Unfortunately, Databricks Community Editon has been shutdown at January 1, 2026. So there's no way to restore your content. From now on you should use Free Edition.This was announced several times and they asked every user to back...
- 0 kudos
- 97 Views
- 3 replies
- 0 kudos
Complex Json file Flatten Dynamically
Hi ,We have multiple complex JSON files and need to flatten them, especially handling array data types. Whenever an array is present, we need to create a new child table and establish a relationship between the master and child tables. This will foll...
- 97 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @SantiNath_Dey , Given you’re dealing with multiple complex JSON structures, I agree with a metadata‑driven approach rather than hard‑coding the flattening logic. At a high level: Define a small set of config tables: One for patterns... as in whi...
- 0 kudos
- 11 Views
- 0 replies
- 0 kudos
Data linkage and analysis with masked data
HelloLooking for guidance to keep data masked if it needs to link with unmasked data, or analysis requires a combination of masked and unmasked data. How have you tackle this challenge?Thank you.
- 11 Views
- 0 replies
- 0 kudos
- 17 Views
- 1 replies
- 0 kudos
Apply expectations conditionally in SLDP
Hi folks, The following code runs as expected, and all three rules are validated. @dp.view(name=f"v_validate_source_{table}") @dp.expect_all_or_drop({"201-Data row":"row_cnt > 0"}) @dp.expect_all_or_drop({ "101-One footer row" : "foo...
- 17 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Great question. With decorators, you cannot place them inside an if block the way you wrote. Decorators are applied when the function is defined.The clean way is to build the expectations dictionary first, then apply one decorator:rules = { "101-...
- 0 kudos
- 35 Views
- 2 replies
- 0 kudos
Issue in receiving my databricks Certificate to wrong email becouse wrong email
Hello team,I passed the Data Engineer associate exam two days ago and still didn't receive my certification till now cause the email provided to Kryterion was wrong, the wrong email was ataamohmed554@gmail.com, but my correct mail is ataamohamed554@g...
- 35 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @Ataa_mohamed11 , Please file a ticket with our support team, support team will respond shortly. Please also provide them with your email address (it is not advised that you post your email address here in community). Please note that we cannot...
- 0 kudos
- 59 Views
- 4 replies
- 8 kudos
How Much Has AI Actually Changed Your Day to Day?
Community, I'm genuinely curious: Describe your workday two years ago vs. today in a sentence or two. I'll go first: then, I spent half my day context-switching between Drive, Sheets, Docs, and Slack just trying to find what I needed. Now, I vibe cod...
- 59 Views
- 4 replies
- 8 kudos
Latest Reply
- 8 kudos
A couple of years ago, I used to hesitate a lot when it came to quick experiments or small spikes. I felt that even small tasks had to go through tickets, planning, investigation, and documentation. All of that takes time, so trying things quickly wa...
- 8 kudos
- 424 Views
- 5 replies
- 14 kudos
Databricks Community Champion - March 2026 - Pradeep Singh
Our Community Champion Program celebrates members who go the extra mile to share knowledge, support fellow practitioners, and help the Databricks Community grow stronger every day. These champions bring curiosity, experience, and a genuine willin...
- 424 Views
- 5 replies
- 14 kudos
- 238 Views
- 14 replies
- 3 kudos
Lakeflow SDP failed with DELTA_STREAMING_INCOMPATIBLE_SCHEMA_CHANGE_USE_LOG
Hi,A column was deleted on the source table, when I ran LSDP it failed with error DELTA_STREAMING_INCOMPATIBLE_SCHEMA_CHANGE_USE_LOG : Streaming read is not supported on tables with read-incompatible schema changes( e.g: rename or drop or datatype ch...
- 238 Views
- 14 replies
- 3 kudos
Latest Reply
- 3 kudos
@Kirankumarbs I have tried doing entire refresh of tables it worked thank you so much but on UC ui I still see the old column with void as datatypeand on querying the table it throws error saying incompatible schema change. could you please help with...
- 3 kudos
- 22 Views
- 0 replies
- 0 kudos
setup justfile command in order to launch your spark application
Hello Guys, Actually, I build a just file for my project which will be execute my wheel job task using command line, but when i run my wheel task i have encountered this error. from pyspark.sql.connect.expressions import PythonUDFEnvironmentImportErr...
- 22 Views
- 0 replies
- 0 kudos
- 475 Views
- 9 replies
- 0 kudos
Merge with schema evolution fails because of upper case columns
The following is a minimal reproducible example of what I'm facing right now.%sql CREATE OR REPLACE TABLE edw_nprd_aen.bronze.test_table ( id INT ); INSERT INTO edw_nprd_aen.bronze.test_table VALUES (1); SELECT * FROM edw_nprd_aen.bronze.test_tab...
- 475 Views
- 9 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Dhruv-22 , I did check with our product teams and they agree with what I wrote above, and that if you have a support contract to open a ticket about it. They are aware of this behavior and the workaround needed. However, they haven't seen this af...
- 0 kudos
- 57 Views
- 4 replies
- 1 kudos
Resolved! I am waiting the Verify code but not happens
I am waiting the Verify code but not happens .Is the free edition Down?
- 57 Views
- 4 replies
- 1 kudos
- 80 Views
- 0 replies
- 1 kudos
Microsoft just announced a new certification: Microsoft Certified: Azure Databricks Data Engineer
Microsoft has just announced a new certification: Microsoft Certified: Azure Databricks Data Engineer Associate (Exam DP-750)This certification is entirely focused on building data engineering solutions using Azure Databricks, with deep integration i...
- 80 Views
- 0 replies
- 1 kudos
- 525 Views
- 13 replies
- 4 kudos
Resolved! unionbyname several streaming dataframes of different sources
Is the following type of union safe with spark structured streaming?union multiple streaming dataframes, and each from a different source.Anything better solution ?for example, df1 = spark.readStream.table(f"{bronze_catalog}.{bronze_schema}.table1") ...
- 525 Views
- 13 replies
- 4 kudos
Latest Reply
- 4 kudos
Thanks @Kirankumarbs, Thanks @SteveOstrowski You have provided very useful information.
- 4 kudos
- 40 Views
- 2 replies
- 0 kudos
How to stop Databricks adding quotes to multi-line selections
I'm using a query to generate some YML code from my tables, and running into an annoying behaviour. Here's a simplified example...Run this query in a notebook or the SQL editor:SELECT 'foo\nbar' FROM system.information_schema.tables LIMIT 10You get a...
- 40 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @ChrisHunt, As Ale_Armillotta mentioned, any field that contains a newline is wrapped in double quotes so that each row still represents a single CSV record. I think that is a logical and expected behaviour. There is currently no setting to turn t...
- 0 kudos
Announcements
![Community [Industry] BrickTalk: Scaling BioPharma Intelligence and Agentic Clinical Ops Ecosystem](https://calendarstemplates.com/t5/image/serverpage/image-id/24660iA21075242DC0E0B9/image-size/large?v=v2&px=999)
| User | Count |
|---|---|
| 2220 | |
| 1185 | |
| 1152 | |
| 1138 | |
| 952 |
| User | Count |
|---|---|
| 407 | |
| 286 | |
| 220 | |
| 63 | |
| 45 |
Latest from our Blog
Lakebase Branching Meets Docker: The Migration Safety Net I Wish I Had Years Ago
Lakebase Branching Meets Docker: The Migration Safety Net I Wish I Had Years Ago Before I became a Solutions Architect at Databricks, I was a backend software engineer and for a few years of my past l...
Design Filters and Parameters That Scale in Databricks AI/BI Dashboards
Why Filters matter Databricks AI/BI Dashboards become truly useful when users can slice data by date, segment, status, or region without editing SQL. Filters make this possible by separating what use...
Spark Declarative Pipelines “How-To” Series. Part 1: How to Save Results Into A Table
Spark Declarative Pipelines “How-To” Series Introduction Lakeflow Spark Declarative Pipelines (SDP) is a framework designed for building scalable, maintainable ETL pipelines. Everyone should want to m...
AI Newsletter Series (Part 2): Designing a Useful LLM Pipeline Across PDFs, Use Cases, and Accounts
In Part 1 of this series, we focused on parsing enterprise PDFs, and this blog we build on that work and show how those chunks are combined with structured use-case data to produce consistent, account...
