Hey Databricks Community! Join us for a free, community-sponsored BrickTalk on Thursday, November 5! BrickTalks is a community event series where Databricks experts share real-world use cases, demos, and practical insights for building with data and ...
- 658 Views
- 1 replies
- 2 kudos
2 weeks ago
NOVEMBER · THE TOUR IS ON THE MOVE Nine stops. Three regions.One epic November. THIS MONTH'S ROUTE Munich → Atlanta → Amsterdam → London → Toronto → Los Angeles (Anaheim) → New York City → Milan → Tokyo A free, one-day event on how organizations...
- 1469 Views
- 1 replies
- 0 kudos
a week ago
DATA + AI LIVE Time: AMER Nov 11 / 9 AM PT / 12 PM ET APAC Nov 12 / 9:30 AM IST / 12 PM SGT / 1 PM JST-KST / 3 PM AEDT EMEA Nov 17 / 9 AM GMT / 10 AM CET Register for the Workshop Note: Marking RSVP on this Community event does not register you fo...
- 1142 Views
- 0 replies
- 2 kudos
2 weeks ago
REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...
- 3776 Views
- 0 replies
- 4 kudos
09-01-2026
Free · Hands-on · Instructor-led AI/BI + Genie Launchpad From dashboards your team can trust to answers in plain language with Genie. A three-day arc of live, hands-on sessions, each built for a different role. Join only the day or days that fit you...
- 840 Views
- 0 replies
- 1 kudos
2 weeks ago
Responsible gaming initiatives are becoming increasingly important as regulation, gameplay experiences and the betting and gaming industry continue to expand. The Responsible Gaming Solution Accelerator, developed in partnership with AWS, shows how t...
- 575 Views
- 1 replies
- 5 kudos
2 weeks ago
Recap • September 14 – October 11 Community Pulse Four weeks of articles, learning content, and conversations – the highlights, all in one read. Community Stars The members who showed up and made a real difference : @ivanvyd @data_pulse @Tho...
- 767 Views
- 0 replies
- 4 kudos
3 weeks ago
Community Activity
- 14 Views
- 1 replies
- 0 kudos
guidance on cleainig up after long cdc failure
Hi All,We had a CDC failure which took far to long to resolve. A conversation with vendors on that.Does anyone have any guidance on the process steps to identifying the missed changes and applying them to the tables.I have so far run a query to ident...
- 14 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hello @turagittech !, Your approach of comparing the source and destination tables to identify discrepancies is a good starting point. A MERGE would be a reasonable next step to correct the mismatched records. A few recommendations before proceeding:...
- 0 kudos
- 678 Views
- 1 replies
- 2 kudos
AI Functions for Synthetic Test Data: Generate Realistic Datasets in Pure SQL
The views expressed in this post are my own and do not represent those of my employer. Not long ago, generating realistic synthetic data meant building custom pipelines with GANs, Transformers, or RNN-based models. It worked, but the engineering ove...
- 678 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Thanks for writing this up, @DouglasGarcia. The split you draw, "AI is best for text and profiles; use standard SQL for numbers and dates," was interesting enough that I ran an experiment on it.I built customer, order and ticket tables with Databrick...
- 2 kudos
- 10 Views
- 0 replies
- 0 kudos
Databricks SQL: Check Join Row Counts Before Resizing the Warehouse
When a sales query runs slowly, inspect how many rows each join produces before changing compute size.Consider a hypothetical retailer joining order lines to payments using order_id. One customer’s order has three product lines and two payment recor...
- 10 Views
- 0 replies
- 0 kudos
- 86 Views
- 2 replies
- 2 kudos
Resolved! Update Ingestion pipleline to use pager duty with API
Hi,I have been working to update a cdc ingestion pipeline to add a notification to it on failure. We have had two instances where a maintenance task by vendor as tripped up CDC with some grief.I reviewed all the information and I am getting an error ...
- 86 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Agree with 3 and we have had far too much experience with issues from that and the latest is why I am beefing up notifications. PagerDuty has an email configured for the service, so maybe that one has to go via email.My UI tends to be up to date, and...
- 2 kudos
- 56 Views
- 1 replies
- 0 kudos
403: Permission Denied for certain foundational models
Hello,I have an individual account (Azure Databricks, Premier, East US)without any support contracts, and I am using it solely for personal projects. I am getting this error whenever I am trying to use the GPT and/or Claude models (system.ai.claude-s...
- 56 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @gautamrjt, I don't think this is a configuration issue on your side. Open models like Meta Llama 3.3 70B Instruct are available to pay-per-token workspaces by default. Premium partner models such as Anthropic Claude and the OpenAI GPT models have...
- 0 kudos
- 1107 Views
- 17 replies
- 4 kudos
MS Fabric Direct Lake, works with delta tables only?
Lately we started using Direct Lake to use the results of our ingestion pipelines (gold tables) in PBI dashboards. This is working ok but has one annoying and troublesome issue. Which is: the gold tables are the end result of declarative pipelines so...
- 1107 Views
- 17 replies
- 4 kudos
Latest Reply
- 4 kudos
Hi @dbernstein_tp,Good to hear the MERGE is only touching a few thousand rows now. That's the part that matters for Direct Lake. On the job vs manual timing, the same code running at very different speeds almost always means it isn't running on the s...
- 4 kudos
- 43 Views
- 0 replies
- 0 kudos
Gating DEEP CLONE backups: what saved time, what didn’t, and the bug that almost slipped by
I’ve spent a good part of this year on Unity Catalog backups at scale: thousands of Delta tables, nightly DEEP CLONE into a separate backup catalog, and a backup window that kept getting longer. The obvious fix was “only clone tables that actually ch...
- 43 Views
- 0 replies
- 0 kudos
- 44 Views
- 0 replies
- 0 kudos
One PDAL init script for every Databricks Runtime (and no apt)
I needed PDAL on Databricks for a LiDAR processing workload. PDAL is a C++ library with a long native dependency chain (GDAL, PROJ, HDF5, a recent libstdc++), and the Python bindings have to match the interpreter exactly. The usual approach, an apt-g...
- 44 Views
- 0 replies
- 0 kudos
- 48 Views
- 0 replies
- 0 kudos
Backing up Unity Catalog volumes without melting the driver
Tables are the easy part of a Unity Catalog backup, because you have DEEP CLONE. Volumes have no equivalent. When I started on a self-service backup tool this year, the first version backed volumes up the way I suspect most people do: list the files ...
- 48 Views
- 0 replies
- 0 kudos
- 41 Views
- 0 replies
- 1 kudos
Lakeflow Expectations: Dropping Bad Rows Can Hide Real Sales
A practical data quality lesson: choose the response to a failed rule based on its business impact.Consider a hypothetical retailer using Azure Databricks. An incoming order has a valid order ID and sales amount, but its customer ID is missing.A rul...
- 41 Views
- 0 replies
- 1 kudos
- 27 Views
- 0 replies
- 0 kudos
Building on Lakebase | Episode 3: Branch, Test, and Ship: Release Changes Safely using Branching
Welcome to Building on Lakebase, a four-part series on building transactional apps and AI agents on Lakebase, a fully managed serverless PostgreSQL database built by Databricks. Over the four episodes led by Michelle Deng and Benjamin Nwokeleme, Solu...
- 27 Views
- 0 replies
- 0 kudos
- 42 Views
- 0 replies
- 0 kudos
Models Should Be Config, Not Copies: A Config-Driven MLOps Blueprint on Databricks
Banking · Insurance · Credit Risk · Any Team Running More Than One Model — From Copy-Pasted Pipelines to One Shared Pipeline and One YAML per ModelBy @SrinuB in collaboration with @sridharplv Your first model went to production beautifully. Your fif...
- 42 Views
- 0 replies
- 0 kudos
- 190 Views
- 3 replies
- 0 kudos
Real time Streaming to AI Engine at Databricks
Hi there,Usercase: Fraud Detection Technology : Databricks, Kafka, Microsoft Foundry, Dashboard showing real-time reportsAsk: Please let me know if any reference or technology details to achieve 1-2 milli second (Max) latency using above technology s...
- 190 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @abhietc31, the fraud detection Solution Accelerator scores Kafka transactions in real-time mode and feeds a live Databricks Apps dashboard from Lakebase. For your own pipeline, the Lakebase sink writes real-time mode output to a Lakebase table wi...
- 0 kudos
- 462 Views
- 3 replies
- 0 kudos
Unable to Serve Gemini 3.8 Flash on Azure Databricks – East US
I am working with an Azure Databricks workspace in the East US region and am trying to use the Gemini 3.8 Flash model through Databricks Model Serving.The model is visible in the workspace under the system AI models as:system.ai.databricks-gemini-3-8...
- 462 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @krSindhu, on Azure Databricks, Gemini 3.8 Flash needs both ADI Services, which require eligibility and enablement, and cross-geography routing. Check that the Integration with ADI Services toggle is enabled. Confirm that Enforce data processing...
- 0 kudos
- 40 Views
- 0 replies
- 0 kudos
Exam Suspended before start (Kryterion) - Ticket #01054624
Hi everyone,I'm reaching out here hoping a community moderator or Databricks staff member can help escalate my support ticket.On October 25, 2026, I was scheduled to take the Databricks Certified Data Engineer Associate exam. I paid the 246 USD certi...
- 40 Views
- 0 replies
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2293 | |
| 1506 | |
| 1255 | |
| 1189 | |
| 961 |
Latest from our Blog
Governed Writes with SQL SECURITY DEFINER for Stored Procedures
SQL Stored Procedures in Databricks now support SQL SECURITY DEFINER, and it is Generally Available on SQL warehouses, Serverless compute and All-Purpose clusters (DBR 19+). A SECURITY DEFINER procedu...
Assume, don't inherit. Putting Databricks RBAC to work.
Access problems in Databricks are rarely about missing permissions. More often they are about people carrying every permission they have ever been given into every session. Someone gets added to a fin...
Geospatial Reasoning with Genie Agents: Competitor Impact in Your Trade Area (Part 1)
As we start building agents that deliver business value, it is inevitable that subsets of the data an agent needs are geospatial in nature. When agents are left to make geospatial decisions on their o...
Serving the Lakebase Data API Through Azure API Management
Why use APIM with Data API? The Data API is one of the underrated features of Lakebase. Traditionally, teams that wanted to share a database over HTTP built a layer of REST services in front of it. Th...
