Databricks Community

Hi Venugopal,Which Associate exam are you going for? (Data Engineer, Data Analyst, ML or Spark Developer.) The prep differs a bit between them, but the core approach is the same:Download the exam guide PDF from the certification page for your exam. T...

Great question, and you're right that it's not a storage limitation — the variant encoding does have a null type (ID 0), as your Parquet link shows. The answer lives in Spark's type system, and there's actual evidence in the source and JIRAs. Fair wa...

We hit exactly this, offline feature store on Redshift, ~200-column denormalized table fed by domain models. Adding a feature meant NULLs for history, and full recompute was too expensive, so we landed on: recompute the domain for the backfill range ...

The pattern is documented and you are right not to full refresh pipeline 2, but as written the deletion detection can never fire, and the full refresh is not why. spark.readStream.table("raw.contact") makes contact append-only, so no key ever leaves ...

Answers: Automated curation trade-off: the upside is low setup; the downside is less explicit control. Genie ranks and conflict-resolves snippets automatically, so if your sources are inconsistent or ambiguous, you still need to review and clean up ...

Adding one option on top of the two-column approach above: depending on how attached you are to the exact progress-bar look, you may not need hand-rolled HTML at all anymore.Table visualizations in AI/BI dashboards picked up richer native formatting ...

Great topic! Databricks machine creation and setup can seem complex at first, but once you understand clusters, compute resources, and workspace configuration, it becomes much easier. A step-by-step guide with screenshots and best practices for begin...

@Advika I was searching for the last week Pulse July 20-26. Please help me with a link.

@Rishabh_Tiwari  How to earn the DAIS2026 batch ?

If your action (count(), collect(), show(), etc.) is already inside the try block, this sounds like a false positive. The warning is likely from a Databricks inspection rather than the Flint time-series library. Check if inspection SCPAP005 can be di...

Hello,I’m also interested in knowing the timeline for this. I was in the middle of the Building Retrieval Agents on Databricks course and noticed the transition/maintenance status. It looks like the course is replacing the older Generative AI Solutio...

Thanks @Advika - I am now into the labs. It look an additional 20 minutes through and was not helped by the out of date incorrect guidance in the course on how to access Labs. In general the integration with Labs is clunky, slow, lots of white pages ...

@ShivannaGundana wrote:This is a helpful troubleshooting discussion for teams working with Databricks and AWS environments. Checking IAM permissions, network connectivity, and instance configuration are important steps when diagnosing SSM registratio...