Hey Databricks Community! Join us for a free, community-sponsored BrickTalk on Thursday, November 5! BrickTalks is a community event series where Databricks experts share real-world use cases, demos, and practical insights for building with data and ...
- 542 Views
- 0 replies
- 2 kudos
2 weeks ago
NOVEMBER · THE TOUR IS ON THE MOVE Nine stops. Three regions.One epic November. THIS MONTH'S ROUTE Munich → Atlanta → Amsterdam → London → Toronto → Los Angeles (Anaheim) → New York City → Milan → Tokyo A free, one-day event on how organizations...
- 1177 Views
- 1 replies
- 0 kudos
a week ago
DATA + AI LIVE Time: AMER Nov 11 / 9 AM PT / 12 PM ET APAC Nov 12 / 9:30 AM IST / 12 PM SGT / 1 PM JST-KST / 3 PM AEDT EMEA Nov 17 / 9 AM GMT / 10 AM CET Register for the Workshop Note: Marking RSVP on this Community event does not register you fo...
- 999 Views
- 0 replies
- 2 kudos
2 weeks ago
REGISTRATION IS NOW OPEN The Data + AI World Tour is heading your way A free, one-day event on how organizations are building AI they can trust to deliver real outcomes - by grounding models in their own enterprise data and business context. THE...
- 3600 Views
- 0 replies
- 4 kudos
09-01-2026
Free · Hands-on · Instructor-led AI/BI + Genie Launchpad From dashboards your team can trust to answers in plain language with Genie. A three-day arc of live, hands-on sessions, each built for a different role. Join only the day or days that fit you...
- 679 Views
- 0 replies
- 1 kudos
a week ago
Responsible gaming initiatives are becoming increasingly important as regulation, gameplay experiences and the betting and gaming industry continue to expand. The Responsible Gaming Solution Accelerator, developed in partnership with AWS, shows how t...
- 512 Views
- 1 replies
- 4 kudos
2 weeks ago
Recap • September 14 – October 11 Community Pulse Four weeks of articles, learning content, and conversations – the highlights, all in one read. Community Stars The members who showed up and made a real difference : @ivanvyd @data_pulse @Tho...
- 713 Views
- 0 replies
- 4 kudos
3 weeks ago
Community Activity
- 31 Views
- 1 replies
- 0 kudos
Genie Space Token Limit & Metric View Description
Hi,I want to get clarity around 2 things related to Genie Agents/Spaces-1. Is their any token limit defined for each space?2. How much is the character limit for description section within any Metric View?
- 31 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Siddharth,1. Token limit per agentThere is one, but the docs don't publish a number. Per the troubleshooting page, when an agent gets close to the limit a warning appears and Genie starts context filtering, keeping the tokens it thinks are most re...
- 0 kudos
- 33 Views
- 1 replies
- 0 kudos
Lakebase synced tables: how can a database owner/admin configure default SELECT privileges for the m
QuestionWe use Databricks Lakebase synced tables to sync Unity Catalog tables into Lakebase. We want a reader role to automatically receive SELECT access on newly created synced tables, without having to rerun grants whenever another table is added.G...
- 33 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi Pat,That error is expected. ALTER DEFAULT PRIVILEGES FOR ROLE can only be run by members of that role.The Lakebase docs say it in their own example: "only members of table_creator_role_name can run this command". databricks_writer_16840 is the int...
- 0 kudos
- 135 Views
- 2 replies
- 0 kudos
Why I started evaluating the pipeline instead of choosing the best model at each stage
I recently worked on a multi-label classification pipeline on Databricks, using embeddings, scikit-learn classifiers, Delta Lake, and MLflow to track experiments.One result caught my attention: the embedding model that ranked fourth in my initial scr...
- 135 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Frank, here’s a third answer to your joint-search vs. prune-first question, building on @Isaac_18’s.Instead of running a new search every time, I build an offline evaluation suite from historical run data. I replay candidate pipelines against those c...
- 0 kudos
- 350 Views
- 2 replies
- 0 kudos
Lessons from a Power BI to Databricks migration: metric views first, then Genie
We recently moved a 20-report Power BI estate to Databricks in 6 weeks, against a 23-week plan for a traditional rewrite. Sharing what worked and what didn't, in case it helps others planning the same move.What made the differenceMetric views before ...
- 350 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Fiscal calendar time intelligence in metric views is tricky. I used a similar approach but had to hardcode certain shifts. For mastering DAX and metrics, I attended the Power BI Masterclass at https://redeagle.tech/training/power-bi-masterclass and i...
- 0 kudos
- 448 Views
- 3 replies
- 0 kudos
Resolved! "Organization permissions" in DAB's
Hi all,I've created a Databricks app in git, and it is deployed through DAB's. I want my team to have access to it and manage it and view insights and logs thus i have added in the dab CAN_MANAGE permission to my App DAB to the appropriate group.When...
- 448 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
@Isaac_18 Thanks for this, that makes sense. Now I believe my confusion stems from navigating the documentation. What's the difference between azure docs and aws docs? We are hosting databricks on azure, so have been navigating the azure docs, and ig...
- 0 kudos
- 75 Views
- 2 replies
- 0 kudos
Resolved! Unable to create metastore in account console with my Azure Account
Azure Managed Identity Credential with Access Connector Id /subscriptions/c03723bd-79a8-401a-8392-a6aef2b0d7b9/resourceGroups/RG_databricks_masterclass/providers/Microsoft.Databricks/accessConnectors/ac_meta_new and Managed Identity ID: for Account I...
- 75 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Resourceid: /subscriptions/c03723bd-79a8-401a-8392-a6aef2b0d7b9/resourceGroups/RG_databricks_masterclass/providers/Microsoft.Databricks/accessConnectors/ac_meta_new Databricks Accountid:https://accounts.azuredatabricks.net/?account_id=79762b8f-7...
- 0 kudos
- 86 Views
- 2 replies
- 0 kudos
Zerobus support for private-endpoint storage (MLflow traces to Unity Catalog)
We'd like to store MLflow 3 traces in Unity Catalog, using an experiment linked to a UC table-prefix trace location. In our Azure UK South workspace, linking the location fails with:PERMISSION_DENIED: Zerobus failed to open the table. Make sure the t...
- 86 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
Private endpoint storage for traces remains a listed limitation. Store Open Telemetry traces in Unity Catalog lists this as a limitation among other constraints, Traces cannot yet be written to storage protected by Private Link. It is stated as a kno...
- 0 kudos
- 180272 Views
- 245 replies
- 46 kudos
Resolved! Databricks Certification Exam Suspended? Our Support Team Can Help
Taking exams is so stressful already, and if your exam gets suspended it is extra stressful. Don't worry! Our support team is awesome and will help you get back on track. The most important step is to file a ticket in our Help Center. Be sure to inc...
- 180272 Views
- 245 replies
- 46 kudos
Latest Reply
- 46 kudos
Hello @Cert-Team @cert-ops,My exam was suspended during the initial checks, before the exam started.I submitted an appeal (#01053519) on 23 October after my original request (#01052483) was closed. It has been a week since my scheduled exam, and I ha...
- 46 kudos
- 64 Views
- 2 replies
- 0 kudos
Best approach and tools for migrating from Oracle to Databricks?
Hey everyone,We're planning to migrate our data warehouse from Oracle to Databricks. We have a mix of standard tables and a good amount of legacy PL/SQL code to deal with.What is the best way to approach this migration? specifically:1) How do you usu...
- 64 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
For converting legacy PL/SQL, begin with Genie Migrate (agentic SQL converter). It supports the Oracle dialect and outputs either Databricks SQL or notebooks using the DataFrame API, while iteratively validating and fixing the converted code before w...
- 0 kudos
- 155 Views
- 2 replies
- 0 kudos
ai_decide() failing with Endpoint is not enabled for batch inference in Free Edition
I'm running ai_decide() in a notebook (SQL cell, serverless compute) and it's now consistently failing with:[INTERNAL_ERROR] Remote session creation failed: INVALID_ARGUMENT: BAD_REQUEST: Endpoint ai_decide is not enabled for batch inference in this ...
- 155 Views
- 2 replies
- 0 kudos
Latest Reply
- 0 kudos
The issue is tied to how Databricks Free Edition handles model serving resources. Specifically, the Free Edition imposes hard restrictions on these capabilities - there is no support for batch inference endpoints, no custom models and certain foundat...
- 0 kudos
- 351 Views
- 4 replies
- 0 kudos
Cluster environment variables not visible to Java Dataset.foreachPartition / mapPartitions
We run a Spark JAR task (Java) on a standard access mode cluster. Credentials come from cluster environmentvariables set as secret references in the cluster's Environment variables field, e.g.MY_PASSWORD={{secrets/<scope>/<key>}}.The variables are vi...
- 351 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @capoorhimanshu,Your worry about the closure is fair. A secret that rides inside the serialized task is harder to control than one that never leaves a secret store.As far as I can tell, there's no supported setting on standard access mode that pus...
- 0 kudos
- 118 Views
- 3 replies
- 0 kudos
Free Edition — S3 storage credential validation fails at "Assume Role" though IAM config is verified
Workspace ID 7474651448614968 (Free Edition, AWS). Credential databricks-dataengineer-v2 (IAM role arn:aws:iam::278137443935:role/DATABRICKS-DATAENGINEER) fails validation at the Assume Role step; CREATE EXTERNAL LOCATION returns PERMISSION_DENIED .....
- 118 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Arisk, I agree with @DoTA that your AWS setup looks right. The master role ARN you're using matches the one in the AWS setup docs (https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-manual), and I coul...
- 0 kudos
- 46 Views
- 0 replies
- 0 kudos
Moving a partitioned Delta table to liquid clustering without rebuilding it
If you have a big Delta table that was partitioned years ago, you probably know the feeling. Half the partitions are tiny, a few are huge, and queries that filter on anything other than the partition column scan far more than they should. Liquid clus...
- 46 Views
- 0 replies
- 0 kudos
- 601 Views
- 2 replies
- 2 kudos
Resolved! Streaming read fails with "Detected a data update" after a restatement job touches the source
Hi everyone,Looking for the right pattern here rather than a workaround.Setup. DBR 15.4 LTS, Unity Catalog. A silver streaming table reads from a bronze Delta table with a normal streaming read. Bronze is append only in the ordinary course of busines...
- 601 Views
- 2 replies
- 2 kudos
Latest Reply
- 2 kudos
Hi @Islam_hoti, good instinct not to just slap skipChangeCommits on and walk away. Both of your ideas work. Which one I'd pick depends on how often corrections happen.Option A: rare, batched corrections (what you have). The Delta streaming docs descr...
- 2 kudos
- 66 Views
- 1 replies
- 0 kudos
Managed table conversion and vacuum
Hello everyone,I am looking for clarification about the 14-day rollback period after converting an external table to a managed table using: ALTER TABLE <table_name> SET MANAGEDDatabricks document...
- 66 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Davide, this thread never got a reply, but you asked the same thing in SET MANAGED and rollback in relation to VACUUM and posted your test result there, so I'm linking it for anyone who lands here first.Short version: the 14-day rollback window i...
- 0 kudos
Announcements
| User | Count |
|---|---|
| 2293 | |
| 1505 | |
| 1255 | |
| 1189 | |
| 961 |
Latest from our Blog
Serving the Lakebase Data API Through Azure API Management
Why use APIM with Data API? The Data API is one of the underrated features of Lakebase. Traditionally, teams that wanted to share a database over HTTP built a layer of REST services in front of it. Th...
[PARTNER BLOG] Scaling Auto Loader on AWS: Moving to File Notification Mode with Managed File Events
Contents IntroductionUnderstanding the Key ConceptsWhy File Notification with Managed File Events?Key Benefits of Managed File EventsArchitecture OverviewPrerequisitesPhase 1: AWS IAM SetupPhase 2: Da...
End-to-End Cyber Lakehouse Streaming via Zerobus Ingest, Spark Declarative Pipelines, and OCSF
Streaming raw events into a cyber lakehouse and normalizing them to a standard schema is a common pattern for security analytics. This blog walks through an end-to-end architecture that uses Databrick...
AI Functions for Synthetic Test Data: Generate Realistic Datasets in Pure SQL
The views expressed in this post are my own and do not represent those of my employer. Not long ago, generating realistic synthetic data meant building custom pipelines with GANs, Transformers, or RN...
