Like most engineers, I’ve had moments where I trusted automation a little more than I should have.

One of those moments came when I set up an auto-fix bot for a CI pipeline.

It found a race condition and “fixed” it by removing the lock. The deadlock that followed made its way into production and surfaced at 2am on a Friday.

The AI wasn’t really the problem. It did exactly what I asked it to do.

The problem was that I never defined what a safe fix looked like.

And the agent had no way to challenge that assumption.

AI-assisted DevOps is not Agentic DevOps

People use “AI-assisted DevOps” and “Agentic DevOps” interchangeably.

They are not the same thing.

AI-assisted DevOps is a text generator with a human execution layer.

• Copilot suggests. You run.
• Claude Code proposes. You accept.
• The agent generates. The human ships.

Agentic DevOps is an execution model.

• Agent scaffolds infrastructure.
• Provisions identities.
• Scans code, opens issues, proposes fixes.
• Cleans up when the work is complete.

You state the outcome. The agent executes the workflow.

That is the pattern I have been exploring with Snowflake CoCo.

CoCo gives me an AI-native execution layer. The CI/CD workflow described here is an implementation pattern:

• Use intent to define the desired outcome
• Use policy to define the agent’s authority
• Use human review when the agent is not confident enough to act.

The important part is not the repository.

The important part is the operating model.

What changed when I applied IDD to CI/CD

I have written before about Intent-Driven Development. The idea is simple: structure prompts around outcomes instead of procedural steps.

The four blocks are:

• Goal
• Requirements
• Constraints
• Output

The agent figures out how.

You define the what and the why.

When I started building this CI scaffold, the scan prompt initially said something like:

Find bugs and open issues

Simple. Direct. Not enough.

The agent found bugs.

It opened issues.

It also tried to auto-fix a 400-line authentication module with high confidence and wrong assumptions.

The problem was the missing constraint.

“Fix my bugs” carries an implicit constraint : “Do it safely.”

IDD forces that constraint into the prompt and the workflow.

Not as a comment.

Not as a README note.

As a policy that the agent reads before acting.

The Decision Tree

The constraint became a scoring system.

Before opening any issue or proposing a fix, the scan agent evaluates three dimensions.

• SEVERITY: how dangerous the issue is if left unfixed.
• COMPLEXITY: how much the fix touches.
• CONFIDENCE: how certain the agent is that its fix is correct.

Then it routes the issue.

conservative (default)

auto-fix only if severity=low AND complexity=low AND confidence=high everything else -> needs-review

aggressive

auto-fix if confidence >= medium

off

never auto-fix, always needs-review

This is not a magic rules engine.

It is a simple judgment model with explicit inputs.

The race-condition fix from my Friday story would have scored:

severity: high
complexity: high
confidence: medium
route: needs-review

A human would have seen the issue, read the context, and decided whether to run the fix.

That is the difference I care about.

An agent that always acts is dangerous.

An agent that never acts is just autocomplete with a workflow file.

The useful middle is an agent that can decide when not to act.

Intent defines the goal. Policy defines the boundary.

Infrastructure-as-Intent does not mean the agent gets unlimited freedom.

It means the desired outcome is expressed clearly enough for the system to act.

But action still needs a boundary.

For this workflow, that boundary lives in configuration.

For example a policy document could be:

fix_mode:
  max_auto: conservative

The intent says what the agent should achieve.

The policy says how far the agent is allowed to go without human review.

Change it through a PR.

The commit history becomes the governance record.

When your security team asks who allowed the agent to auto-fix beyond low-risk changes, you do not need to reconstruct a Slack thread.

You show the PR that changed conservative to aggressive, the reviewer, and the merge date.

That is the record I want Agentic DevOps systems to preserve.

Not just what the agent did.

Who allowed it to act at that level.

For runtime experiments, there can still be a variable override.

Runtime overrides are necessary, but they introduce drift. Every temporary experiment requires a sunset tag and a hard expiration date. If the override outlives its sprint, the agent must revert to the version-controlled default and trigger a CI alert. This is how we ensure ‘temporary’ hacks don’t calcify into unreviewed permanent policy.

Maybe the team wants to try aggressive mode for a sprint. Set the variable. When the experiment ends, delete it. The policy falls back to the reviewed default.

Every scan run should log the active ceiling and its source.

Fix ceiling: conservative
Source: repository configuration

The point is not to slow the agent down.

To make authority visible, embed the active ceiling status directly into your PR templates. Don’t just show the outcome of the agent’s work; show the governance state that authorized it. A persistent ‘Governance Status’ block is the audit evidence your security team actually wants.

This is the engine of CoCo Automations. As agents move from reactive tasks to proactive operations — like daily drift detection — the ‘conservative’ vs. ‘aggressive’ dial becomes our primary safety check. Setting these modes at the repository level ensures that even when the agent operates autonomously, it is tethered to a pre-approved ceiling of authority.

Visibility alone, however, is not a substitute for intervention.

Human override should always exist

The needs-review route is not a dead end.

It is the place where human judgment enters the workflow.

A developer can review the issue, inspect the context, and decide whether the agent should proceed with the fix.

Even for a complex issue.

Even for a low-confidence issue.

The override exists by design.

Agent judgment is the default.

Human judgment is the final decision.

agent proposes -> human decides -> agent executes

A system where the agent can never be overridden is broken.

A system where the agent always acts is also broken.

The override is the acknowledgement that developers know their codebases better than any model.

They should always have the last word.

This is where Agentic DevOps becomes useful.

The agent is not replacing the engineer. It is doing the execution work after the engineer has set the boundary.

ICR did not drop. It got smarter.

The Intent Compression Ratio of this scaffold workflow is 48.

One conversation instruction.

48 distinct state-changing operations.

That number did not change when I added the decision tree.

What changed is that each operation now carries a risk assessment.

The agent that runs 48 operations conservatively is more useful than the agent that runs 48 operations blindly.

The metric I track is not only ICR.

I also track the agent-raised PR merge rate:

The percentage of agent-created PRs that engineers merge without modification.

That number measures trust.

A team at 60%+ merge rate has accepted the agent as part of the workflow.

A team at 10% has a noisy bot.

Same ICR. Different values.

Trust takes time.

The conservative ceiling builds it.

What I mean by Agentic DevOps with CoCo

Not: replace engineers with AI.

Yes: give engineers a CI pipeline that knows its limits.

The value is not faster code generation by itself.

Faster generation without better control creates faster review queues and faster mistakes.

The value is compressing the loop.

That compression matters because the agent is calibrated.

It does not open a PR for every needs-review issue.

It does not silently change complex code because the prompt sounded confident.

It does not confuse autonomy with permission.

Agentic DevOps means the agent can say:

I am not sure enough to act on this one.

That is not a limitation.

That is the design.

Why this matters

Most AI engineering conversations still treat autonomy as the finish line.

Can the agent write the code?

Can it open the PR?

Can it deploy the change?

Those are useful questions, but they are not the first questions.

The first question should be:

What has to be true before the agent is allowed to act?

That is where intent becomes operational.

That is where policy becomes part of the development loop.

That is where CoCo, CI, and IDD start to become more useful than a smarter autocomplete.

Agentic DevOps is not about giving agents unlimited power.

It is about giving them:

• explicit intent
• bounded authority
• visible decisions
• human override

That is the CI agent I want.

One that knows how to act.

And one that knows when not to.

By making agentic authority visible and configurable, we are not just automating tasks; we are codifying intent. This is the final piece of the Intent-Driven Development puzzle: ensuring that the ‘evidence’ generated by our agents maps directly back to human-authorized intent.

Implementation References

The article describes the Agentic DevOps pattern rather than depending on a single repository. The following public references provide implementation context for the GitHub, GitLab, and Snowflake CoCo workflows discussed in this post.

CoCo skill
Snowflake-Labs/devops-snowflake-coco-agents
CoCo skill implementation for the DevOps agent workflow. This provides the core skill context behind scaffold, scan, issue routing, fix behavior, and the human-review loop.

GitHub template
Snowflake-Labs/github-coco-agent
Template for exploring CoCo-driven Agentic DevOps workflows with GitHub Actions. It shows how the pattern maps to GitHub-native CI/CD, repository configuration, issue routing, and PR-based governance.

GitLab template
snowflake-dev/gitlab-coco-agent
Template for exploring the same Agentic DevOps workflow in GitLab CI. It shows that the pattern is not tied only to GitHub Actions and can be adapted across CI/CD systems.

Further Reading

This post continues my Intent-Driven Development series:

• Infrastructure-as-Intent: The Field Velocity Blueprint
• The Ghost in the Machine: Why AI Needs the Spirit of UML
• Intent-Driven Development: The Shift Developers Can’t Ignore
• Intent Compression Ratio: Measuring the Power of Intent
• ICR and Token Economics
• From Intent to Evidence: Why AI Skills Need Tests

For related thinking, Andrew Ng’s recent writing on Three Key Loops for Building Great Software and his Agentic AI course reinforce the same direction: reliable AI systems are built around iterative loops, not one-shot prompts.

That is the bridge this article explores.

Intent gives the loop purpose.
Evidence gives the loop trust.
Policy gives the loop boundaries.
Human review gives the loop judgment.

Disclosure: Feature image generated with AI and reviewed by the author.

About the Author

Kamesh Sampath is a Lead Developer Advocate at Snowflake, author, and long-time open-source contributor with 25+ years in enterprise software. He works across data engineering and AI with developer communities, helping practitioners turn modern data platforms into systems that hold up in production.

Through talks, writing, and hands-on demos, Kamesh makes cloud, data, and AI topics easier to understand and apply — grounded in real-world constraints. His sessions mix deep technical detail with practical patterns that developers and data teams can apply right away.

Lately, he’s been speaking about Apache NiFi (Snowflake Openflow), AI (Snowflake Cortex), and PostgreSQL.

He believes technology becomes powerful when it is shared, taught, and built together.

GitHub | LinkedIn | Blog | X

AI skills should not be trusted because they worked once. They should be trusted because they can reach the intended state repeatedly, with executable tests proving the outcome.

AI agents are easy to celebrate when they work once, but that is also the trap. The demo runs, the code looks right, the file appears in the right place, and everyone nods. Then you run it again. This time a file is missing, a parameter is read differently, or a command is skipped. The final result is close, but not correct. This is where AI-native development gets real: we do not only need agents that can generate; we need skills that can be trusted.

In my earlier posts on Infrastructure-as-Intent, Intent-Driven Development, Intent Compression Ratio, and token economics, I argued that developers are moving away from writing every step by hand and toward expressing outcomes. Not every command, not every click, and not every line of glue code — just the outcome. But there is a missing layer in that story. Intent needs evidence.

Without evidence, intent is only a better prompt

“It Worked Once” Is Not Engineering

Every developer knows this. We do not trust code because it compiled once, we do not trust infrastructure because one deployment succeeded, and we do not trust an API because one request returned 200 OK. We trust systems when we can verify them again and again.

AI skills should be held to the same standard. Today, many AI workflows still feel like demo-driven development: we run an agent, inspect the output, and if it looks good, we move on. That is fine for exploration, but it is not enough for reusable skills.

A reusable skill needs a stronger question. Not: did the agent produce something useful? But rather: did the system reach the intended state, and can it do that again?

The Gap Between Output and Outcome

There is a gap between a plausible result and a proven result. I call this the Verification Gap. On one hand, the agent gives an answer that looks right. On the other hand, the system is in the correct state. That gap matters because AI agents operate through probabilistic reasoning. The same instruction can lead to different paths, tools, or decisions depending on context and model behavior. Often those paths converge on the same outcome. Sometimes they do not.

The mistake is not using AI; the mistake is assuming one successful run is proof. A better question is: how do we make non-deterministic behavior measurable enough for developers to work with? An answer that looks right is not the same as a system that is right.

CoCo as the Proving Ground

I encountered this problem firsthand while building skills for Snowflake Cortex Code, also known as CoCo. Because CoCo is an agentic coding assistant that combines reasoning, tool use, and code execution, it acts as a practical environment for testing AI skills in real-world workflows. I needed a way to prove that an agent’s answer wasn’t just plausible, but that the system was actually in the right state. That practical need led me to build inspect-coco.

A CoCo skill can package developer intent into a reusable capability. Instead of asking an agent to reason from scratch every time, we can give it a skill that captures a workflow, convention, tool pattern, or common engineering task. This is the shift I described in Infrastructure-as-Intent: we move from prescribing steps to expressing outcomes, turning a skill into a reusable unit of intent.

That is powerful, but it is also risky if we do not test it. A skill can hide many actions behind one instruction — it may create files, change configurations, call tools, or update a project. The user sees a simple request, and the skill performs the work. This is exactly what I mean by Intent Compression Ratio. A high-value skill compresses many steps into one intent, but high compression requires high confidence. If one intent represents ten steps, we need to know whether all ten steps completed correctly.

The same challenge exists whether you are building skills for CoCo, Claude Code, Codex, Gemini CLI, OpenHands, or any other agentic development environment. Once an agent can inspect repositories, modify files, execute commands, and automate parts of the software lifecycle, the question becomes the same: how do we know it did the right thing?

CoCo exposed the gap early. It became the first proving ground because it offered a practical environment where skills could interact with real projects, tools, and workflows. That made it easier to test a simple question: can intent be translated into repeatable, verifiable outcomes?

While the pattern is not limited to CoCo, CoCo is the concrete runtime. Inspect AI is the evaluation foundation. inspect-coco connects the two.

What inspect-coco Does ?

inspect-coco is a developer-first test harness for the instructions and workflows that become AI skills. The flow is intentionally familiar:

• It uses Inspect AI to run evaluations.
• It uses CoCo to execute the skill.
• It uses Docker to isolate the environment.
• It uses a test script to verify the result.

The key is the test. inspect-coco does not ask another model whether the answer looks good; it runs the skill and checks the system state. Did the file exist? Did the content match? Did the command succeed, and did the project end up in the expected shape?

If yes, the test passes; if not, it fails. That is a language developers already understand. If you prefer, the verification layer can also use familiar testing frameworks such as pytest, allowing teams to reuse existing assertions, fixtures, and testing practices instead of learning a new evaluation model. The important idea is not the framework itself, but that the outcome is verified by executable tests rather than judged by appearance.

Three Files

An evaluation can start with just three files. That is intentional; no heavy platform is required to understand the idea.

• instruction.md (Markdown) describes the intent.
• tests/test.sh (Shell script) verifies the outcome.
• task.toml (TOML) tells the evaluation how to run.

The simplest example is the hello-world evaluation in the repository. The instruction asks the skill to create /workspace/hello.txt. The agent can choose its path, but the test objectively checks the result: does the file exist, and does the content match exactly? Exit 0 means pass; anything else means failure.

That is boring in the best possible way.

Unit Tests for Skill Instructions

One design choice in inspect-coco is worth calling out: it does not require starting with a fully packaged skill and running the whole lifecycle end to end. It can start with an instruction. That instruction may later become part of a CoCo skill, a Claude Code command, a Codex workflow, or another agentic environment. But before it becomes reusable automation, the instruction itself should be tested.

A skill is not only code; it is packaged intent. If the intent is vague, the skill will be vague. If the instruction is hard to verify, the skill will be hard to trust. If the constraints are weak, the agent has too much room to improvise. Because of this, inspect-coco treats the instruction as a testable artifact, making it closer to unit testing for instructions than demo testing for agents.

The question is not just whether the complete agent workflow can succeed. The earlier question is whether the instruction is clear, constrained, and verifiable enough to become a reliable skill in the first place. Instruction-level tests tell us if the unit of intent is well-formed before packaging it, while end-to-end tests tell us if the whole workflow actually worked. Both are useful, but they answer fundamentally different questions.

Intent Needs Structure

The instruction follows a simple structure, where each section has a specific job:

• Goal: Says what we want.
• Requirements: Say what must be true.
• Constraints: Reduce unwanted choices.
• Output: Tells us what success looks like.

This structure matters because vague intent creates drift. If the instruction is loose, the agent has more room to guess. It may still produce something useful, but useful is not the same as correct. A good instruction narrows the space. It does not remove all non-determinism, but it makes the outcome easier to test.

To enforce this, inspect-coco also includes an IDD-style check for instruction quality, catching weak instructions early. The framework does not only ask if the skill worked; it also asks if the intent was clear enough to test.

Repeated Runs Matter

One passing run is useful; it tells us the skill can work. But it does not tell us whether the skill is reliable. Because AI systems vary between runs, inspect-coco can execute the same task multiple times, measuring results across epochs and reporting pass rates such as pass@k.

A skill that passes once out of three is not the same as a skill that passes ten out of ten. Both may look good in a demo, but only one is ready to trust. The goal is not to pretend AI is deterministic; rather, it is to measure how stable the outcome is. This is where evaluation becomes more than a checklist — it becomes a feedback loop.

Why Not Just Use LLM-as-Judge?

LLM-as-judge has value when the output is subjective. A summary may need to be judged for clarity, a response checked for tone, or a support answer reviewed for usefulness.

But many developer tasks are not subjective. A file exists or it does not. A test passes or it does not. A command exits cleanly or it does not. A generated project runs or it does not. For those cases, we should not ask for an opinion; we should run a test.

That is the design choice behind inspect-coco: when the outcome can be verified by the system, the system should verify it.

Why Inspect AI ?

I built inspect-coco on Inspect AI because agent evaluation needs more than prompt scoring. Developer skills often act in a real environment: they create files, modify projects, run commands, and use tools. So, the evaluation framework needs to support real execution.

Inspect AI provides that foundation, giving us tasks, solvers, scorers, sandboxes, repeated runs, and logs. inspect-coco adds a CoCo-specific layer on top:

• CoCo execution
• CoCo skill scaffolding
• IDD instruction checks
• Docker-based environments
• Snowflake authentication via OAuth, personal access tokens, and JWT
• Script-based verification

That is the important split. Inspect AI is the general evaluation foundation, CoCo is the concrete runtime where I am proving the pattern, and inspect-coco connects the two for developers who want to build reliable CoCo skills today.

Developer-First by Design

I wanted this to feel like normal development. Not like a research project, not like a dashboard-first platform, and not like another abstract AI evaluation layer. A developer should be able to look at the files and understand what is happening:

• Markdown for intent
• Shell for verification
• TOML for configuration
• Docker for isolation
• CLI for execution
• Pass or fail for confidence

That is the whole point. AI changes how we build, but it should not remove the habits that made software engineering work.

Evaluation Should Be Close to the Skill

inspect-coco can scaffold evaluations from an existing CoCo plugin, meaning a skill can grow a test right beside it. This is important because testing should not feel like a separate ceremony. If writing evaluations is too far away from writing skills, developers will skip it.

The closer the test is to the skill, the more likely it becomes part of the workflow. That is how software teams learned to treat tests, and AI skill development should learn the same lesson.

From Prompt Engineering to Intent Engineering

Prompt engineering taught us how to talk to models. Intent engineering asks a harder question: can we express the outcome clearly enough for a system to act on it? But there is an even more important question after that: can we prove the system reached that outcome?

This is where AI-native development has to mature. The future will not belong to teams that generate the most code; it will belong to teams that can express intent clearly, compress complexity safely, and verify outcomes continuously. CoCo skills give us a way to package intent, and inspect-coco gives us a way to test it. That is the bridge from intent to evidence.

In Intent-Driven Development, the question is not only whether the agent produced something. The better question is whether the system reached the intended state, and whether it can do it again. That is what we should measure, and that is what we should improve. That is what turns AI skills from impressive demos into reliable developer tools.

Intent is the source. Evidence is the contract. Every serious AI skill needs tests.

References and Further Reading

Project

• GitHub repository
• Documentation
• hello-worldExample

Snowflake and CoCo

• Snowflake Cortex Code documentation
• Snowflake CoCo Quickstart

My IDD Series

• Infrastructure-as-Intent: The Field Velocity Blueprint
• The Ghost in the Machine: Why AI Needs the Spirit of UML
• Intent-Driven Development: The Shift Developers Can’t Ignore
• Intent Compression Ratio: Measuring the Power of Intent
• ICR and Token Economics

Related Reading

• Inspect AI
• OpenAI Evals
• Promptfoo
• DeepEval
• SWE-bench

About the Author

Kamesh Sampath is a Lead Developer Advocate at Snowflake, author, and long-time open-source contributor with 25+ years in enterprise software. He works across data engineering and AI with developer communities, helping practitioners turn modern data platforms into systems that hold up in production.

Through talks, writing, and hands-on demos, Kamesh makes cloud, data, and AI topics easier to understand and apply — grounded in real-world constraints. His sessions mix deep technical detail with practical patterns that developers and data teams can apply right away.

Lately, he’s been speaking about Apache NiFi (Snowflake Openflow), AI (Snowflake Cortex), and PostgreSQL.

He believes technology becomes powerful when it is shared, taught, and built together.

GitHub | LinkedIn | Blog | X

You think you know which role is running your query. You probably don’t.

Your stored procedure failed.

You checked the role.

The role has the privilege.

It still failed.

Your task will not resume.

You own it.

You still can’t run it.

The bug is not in your SQL. It is in your mental model.

In Snowflake, the role that calls an object is not the role that runs it.

That role is the executor role. It explains a whole class of errors that look like permission bugs but are really execution-context bugs.

The Mental Model

When you run a SELECT, the role doing the work is your session role. Simple.

When you CALL a stored procedure, it depends. The role might be the procedure owner. It might be you. It might be something else entirely.

That “role doing the work” is the executor role.

Snowflake exposes two functions to see it:

• CURRENT_ROLE(): your session role, right now.
• INVOKER_ROLE(): the role Snowflake is using to evaluate the currently executing object.

At the session level, they return the same value. Inside a stored procedure, view, or task, they can diverge completely. That divergence is intentional. It is the whole point.

Most objects default to the owner. Most surprises come from not knowing what “owner” means in each context.

Follow Along

New to Snowflake? Start a free trial. Your account is ready in minutes. Everything below uses SNOWFLAKE_SAMPLE_DATA, available in every account.

Option A: Guided with Cortex Code Sign up with Cortex Code for Developers, clone the companion repo, and run $executor-role-workshop. The skill walks through each concept, shows what will fail before it runs, and cleans up after itself.

Option B: DIY with Cortex Code Open the companion repo in Cortex Code and follow this post section by section. The repo has the full runnable SQL for each example. This post is the guide.

Decision Matrix

The Privilege Illusion: Stored Procedures

This is the executor role pattern I reach for most in the field.

Owner’s Rights: The Firewall

Rule: A stored procedure with EXECUTE AS OWNER runs with the procedure owner’s role. The caller’s session role has no effect on what SQL inside the procedure can access. The SQL below proves it.

• Roles: ACCOUNTADMIN to set up. data_engineer_role to create the procedure. analyst_role to call it. The analyst has no table access at all.

You have a data_engineer_role that owns sensitive order data. You have an analyst_role that needs a specific, controlled view of that data. Direct table access is too broad.

Owner’s rights procedures solve exactly this. The procedure runs with the owner’s grants, not the caller’s. The analyst calls it. The data engineer role’s grants do the work. The analyst never touches the table directly.

-- Full setup + teardown:
-- https://github.com/kameshsampath/executor-role-blog-demo/blob/main/sql/01-owners-rights-proc.sql

-- Create the proc as data_engineer_role
USE ROLE data_engineer_role;
USE WAREHOUSE COMPUTE_WH;

CREATE OR REPLACE PROCEDURE executor_demo.tpch.high_value_orders(min_price FLOAT)
  RETURNS TABLE (order_key NUMBER, total_price FLOAT, status VARCHAR)
  LANGUAGE SQL
  EXECUTE AS OWNER
AS
BEGIN
  LET rs RESULTSET := (
    SELECT o_orderkey::NUMBER, o_totalprice::FLOAT, o_orderstatus::VARCHAR
    FROM SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.ORDERS
    WHERE o_totalprice > :min_price
    ORDER BY o_totalprice DESC LIMIT 10
  );
  RETURN TABLE(rs);
END;

GRANT USAGE ON PROCEDURE executor_demo.tpch.high_value_orders(FLOAT) TO ROLE analyst_role;

-- Analyst calls the proc: data returned despite having no table access
USE ROLE analyst_role;
CALL executor_demo.tpch.high_value_orders(500000);
-- Expected: 10 rows of high-value orders

-- Analyst queries the table directly: blocked
SELECT o_orderkey, o_totalprice
FROM SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.ORDERS LIMIT 5;
-- Expected: SQL access control error -- privilege [SELECT] not granted

Owner’s rights mode is not a workaround. It is the access model.

Caller’s Rights: The Window

Rule: A stored procedure with EXECUTE AS CALLER runs in the caller’s session context. It can read session variables the caller set. An owner’s rights procedure cannot. The SQL below proves it.

• Roles: analyst_role owns and calls this procedure. No privilege delegation needed.

Owner’s rights mode has one hard constraint: it cannot see your session.

No caller session variables.

No caller database context.

If your procedure needs to know who is calling it, or needs a value the caller set in the session, you need caller’s rights.

-- Full setup + teardown:
-- https://github.com/kameshsampath/executor-role-blog-demo/blob/main/sql/01-owners-rights-proc.sql

USE ROLE analyst_role;
-- Set the variable BEFORE CREATE PROCEDURE: it is read at compile time
SET SEGMENT_FILTER = 'BUILDING';

CREATE OR REPLACE PROCEDURE executor_demo.tpch.customers_by_segment()
  RETURNS TABLE (cust_key NUMBER, name VARCHAR, segment VARCHAR, nation VARCHAR)
  LANGUAGE SQL
  EXECUTE AS CALLER
AS
DECLARE
  seg VARCHAR DEFAULT $SEGMENT_FILTER;
BEGIN
  RETURN TABLE(
    SELECT c_custkey, c_name, c_mktsegment, c_nationkey::VARCHAR
    FROM SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.CUSTOMER
    WHERE c_mktsegment = :seg LIMIT 10
  );
END;

-- Caller sets the variable, proc reads it
SET SEGMENT_FILTER = 'BUILDING';
CALL executor_demo.tpch.customers_by_segment();
-- Expected: 10 rows where c_mktsegment = 'BUILDING'

SET SEGMENT_FILTER = 'AUTOMOBILE';
CALL executor_demo.tpch.customers_by_segment();
-- Expected: 10 rows where c_mktsegment = 'AUTOMOBILE'

Owner’s rights mode buys you privilege isolation. Caller’s rights mode buys you session context. You cannot have both in a single procedure.

The choice between owner’s rights and caller’s rights is the choice between what the proc can see and what the caller can see.

(Note: There is one more modeEXECUTE AS RESTRICTED CALLER (Native Apps only). When a Native App provider needs access to consumer account objects, the consumer authorizes it explicitly: GRANT CALLER USAGE ON DATABASE consumer_db TO APPLICATION my_app;. For standard procedures, the choice is OWNER or CALLER.)

The Silent Task: EXECUTE TASK

This one trips up everyone at least once. Including me.

Rule: OWNERSHIP on a task lets you change its state (ALTER TASK RESUME/SUSPEND). EXECUTE TASK ON ACCOUNT lets the scheduler actually fire the task body. These are two different privileges. You need both. The SQL below proves it.

• Roles: ACCOUNTADMIN to grant EXECUTE TASK ON ACCOUNT. pipeline_role to own and resume the task.

You create a task.

You own it.

You resume it.

Snowflake says NO.

Cannot execute task, EXECUTE TASK privilege must be granted to owner role.

OWNERSHIP is not enough. EXECUTE TASK is account-level, separate from any object grant. You must grant it explicitly. Always.

There is also a subtler version. You grant EXECUTE TASK ON ACCOUNT. The task resumes. The state shows started. The task body never executes. OWNERSHIP/OPERATE handles state changes. EXECUTE TASK ON ACCOUNT is what lets the scheduler fire the body.

One task.

Two privileges.

-- Full setup + teardown:
-- https://github.com/kameshsampath/executor-role-blog-demo/blob/main/sql/02-execute-task.sql

USE ROLE pipeline_role;
CREATE OR REPLACE TASK executor_demo.tpch.daily_order_stats
  WAREHOUSE = COMPUTE_WH
  SCHEDULE = 'USING CRON 0 6 * * * UTC'
AS
  INSERT INTO executor_demo.tpch.order_stats (total_orders, avg_price)
  SELECT COUNT(*), AVG(o_totalprice) FROM SNOWFLAKE_SAMPLE_DATA.TPCH_SF1.ORDERS;

-- Without EXECUTE TASK: fails
ALTER TASK executor_demo.tpch.daily_order_stats RESUME;
-- Expected: Cannot execute task, EXECUTE TASK privilege must be granted to owner role

-- Grant it, then resume
USE ROLE ACCOUNTADMIN;
GRANT EXECUTE TASK ON ACCOUNT TO ROLE pipeline_role;

USE ROLE pipeline_role;
ALTER TASK executor_demo.tpch.daily_order_stats RESUME;
-- Expected: Statement executed successfully

Ownership means you control the object. EXECUTE TASK means you are allowed to fire it. Snowflake separates those deliberately.

The Ghost User: Dynamic Tables

The data looks correct. The numbers are wrong. The DT refreshed without error.

Rule: A dynamic table’s background refresh runs as an internal SYSTEM user by default. A masking policy that checks CURRENT_USER() will never see a named user during that refresh. The data is silently masked. EXECUTE AS USER pins the refresh to a named identity. The SQL below shows the broken case first, then the fix.

• Roles: ACCOUNTADMIN to create the service user and grant IMPERSONATE. transform_role to own the table and the dynamic table.

Dynamic tables refresh in the background. The refresh runs as the owner role. That is fine for most transformations.

It becomes a problem when your masking or row access policies check CURRENT_USER(). By default, the refresh executor is an internal SYSTEM user. Your policy does not match it. The data comes back masked. The DT refreshes without error.

EXECUTE AS USER, lets you pin the refresh to a named service user.

-- Full setup + teardown:
-- https://github.com/kameshsampath/executor-role-blog-demo/blob/main/sql/03-dynamic-table.sql

USE ROLE ACCOUNTADMIN;
CREATE USER IF NOT EXISTS svc_transform
  DEFAULT_ROLE = transform_role MUST_CHANGE_PASSWORD = FALSE;
GRANT ROLE transform_role TO USER svc_transform;
GRANT IMPERSONATE ON USER svc_transform TO ROLE transform_role;

USE ROLE transform_role;

-- Masking policy: show real values only when refresh runs as SVC_TRANSFORM
CREATE OR REPLACE MASKING POLICY executor_demo.tpch.mask_price AS
(val NUMBER) RETURNS NUMBER ->
CASE WHEN CURRENT_USER() = 'SVC_TRANSFORM' THEN val ELSE -1 END;

-- Base table with policy applied (see full SQL for setup)
-- executor_demo.tpch.lineitem_prices already has mask_price on l_extendedprice

-- STEP 1: Create DT WITHOUT EXECUTE AS USER (the broken case)
CREATE OR REPLACE DYNAMIC TABLE executor_demo.tpch.dt_revenue_broken
  TARGET_LAG = '1 hour'
  WAREHOUSE = COMPUTE_WH
AS
  SELECT l_orderkey,
         SUM(l_extendedprice * (1 - l_discount)) AS net_revenue
  FROM executor_demo.tpch.lineitem_prices GROUP BY l_orderkey;

ALTER DYNAMIC TABLE executor_demo.tpch.dt_revenue_broken REFRESH;
SELECT * FROM executor_demo.tpch.dt_revenue_broken LIMIT 5;
-- Expected: net_revenue = -1 for all rows
-- CURRENT_USER() during refresh = internal SYSTEM user, not SVC_TRANSFORM
-- Policy masks everything. Refresh succeeds. Data is silently wrong.

-- STEP 2: Create DT WITH EXECUTE AS USER (the fix)
CREATE OR REPLACE DYNAMIC TABLE executor_demo.tpch.dt_revenue
  TARGET_LAG = '1 hour'
  WAREHOUSE = COMPUTE_WH
  EXECUTE AS USER svc_transform
AS
  SELECT l_orderkey,
         SUM(l_extendedprice * (1 - l_discount)) AS net_revenue
  FROM executor_demo.tpch.lineitem_prices GROUP BY l_orderkey;

ALTER DYNAMIC TABLE executor_demo.tpch.dt_revenue REFRESH;
SELECT * FROM executor_demo.tpch.dt_revenue LIMIT 5;
-- Expected: real net_revenue values
-- CURRENT_USER() during refresh = SVC_TRANSFORM. Policy unmasks. Data is correct.

EXECUTE AS USER is not about security. It is about identity. Policies need to know who is running the refresh. You have to tell Snowflake.

The View Trap: Masking Policies

This is the one that quietly breaks the most carefully designed security models.

Rule: INVOKER_ROLE() inside a masking policy returns the role of the currently executing object’s owner, not the session role. Through a view, that is the view owner’s role, not the querying user’s role. The SQL below proves it.

Roles: ACCOUNTADMIN for policy creation and grants. view_creator_role to own the view and base table. analyst_role to query both.

You apply a masking policy using INVOKER_ROLE(). The rule says: if the invoker is ANALYST_ROLE, show the data. You run as ANALYST_ROLE. You query the table. The data is visible. Correct.

Then you query the same data through a view. Masked.

INVOKER_ROLE() does not return your session role here. When you query through a view, it returns the view owner’s role. Not yours.

-- Full setup + teardown:
-- https://github.com/kameshsampath/executor-role-blog-demo/blob/main/sql/04-masking-invoker-role.sql
-- https://github.com/kameshsampath/executor-role-blog-demo/blob/main/sql/05-masking-caller-role.sql

-- Policy using INVOKER_ROLE
USE ROLE ACCOUNTADMIN;
CREATE OR REPLACE MASKING POLICY executor_demo.tpch.mask_order_price AS
(val NUMBER) RETURNS NUMBER ->
CASE WHEN INVOKER_ROLE() = 'ANALYST_ROLE' THEN val ELSE -1 END;

-- View owned by view_creator_role (not analyst_role)
USE ROLE view_creator_role;
CREATE OR REPLACE VIEW executor_demo.tpch.orders_view AS
SELECT * FROM executor_demo.tpch.orders_sample;

-- Analyst queries the table directly
USE ROLE analyst_role;
SELECT o_orderkey, o_totalprice FROM executor_demo.tpch.orders_sample LIMIT 5;
-- Expected: real prices  (INVOKER_ROLE = ANALYST_ROLE)

-- Analyst queries through the view
SELECT o_orderkey, o_totalprice FROM executor_demo.tpch.orders_view LIMIT 5;
-- Expected: -1  (INVOKER_ROLE = VIEW_CREATOR_ROLE, not ANALYST_ROLE)
-- Your session is ANALYST_ROLE. INVOKER_ROLE() does not care.

-- Fix: use CURRENT_ROLE() instead
USE ROLE ACCOUNTADMIN;
ALTER TABLE executor_demo.tpch.orders_sample
  MODIFY COLUMN o_totalprice UNSET MASKING POLICY;
CREATE OR REPLACE MASKING POLICY executor_demo.tpch.mask_order_price AS
(val NUMBER) RETURNS NUMBER ->
CASE WHEN CURRENT_ROLE() = 'ANALYST_ROLE' THEN val ELSE -1 END;
ALTER TABLE executor_demo.tpch.orders_sample
  MODIFY COLUMN o_totalprice
  SET MASKING POLICY executor_demo.tpch.mask_order_price;

USE ROLE analyst_role;
SELECT o_orderkey, o_totalprice FROM executor_demo.tpch.orders_view LIMIT 5;
-- Expected: real prices  (CURRENT_ROLE = ANALYST_ROLE regardless of view ownership)

The policy was logically correct. The mental model was wrong.

Use INVOKER_ROLE() to enforce by object ownership. Use CURRENT_ROLE() to enforce by session identity. They are not interchangeable.

When to Use What

What To Do Next

Run the examples. Or let the skill do it for you. $executor-role-workshop in Cortex Code walks through each concept with live SQL, shows what will fail before it runs, and cleans up after itself. Companion repo.

When your next stored procedure fails with a permission error you cannot explain, check which EXECUTE AS mode it is running in. When your task refuses to resume, check for EXECUTE TASK at the account level. When your masking policy behaves differently through a view, check which context function you are using.

The executor role is not a feature. It is a model. Once you have the model, the errors explain themselves.

Resources

• Companion repo + Cortex Code skill: full runnable SQL + interactive workshop
• Executor Role Developer Cheatsheet: decision matrix, tips, troubleshooting table
• Caller’s and Owner’s Rights Stored Procedures
• Dynamic Tables: EXECUTE AS USER: February 2026 release
• INVOKER_ROLE Function

About the Author

Kamesh Sampath is a Lead Developer Advocate at Snowflake, author, and long-time open-source contributor with 25+ years in enterprise software. He works across data engineering and AI with developer communities, helping practitioners turn modern data platforms into systems that hold up in production.

Through talks, writing, and hands-on demos, Kamesh makes cloud, data, and AI topics easier to understand and apply — grounded in real-world constraints. His sessions mix deep technical detail with practical patterns that developers and data teams can apply right away.

Lately, he’s been speaking about Apache NiFi (Snowflake Openflow), AI (Snowflake Cortex), and PostgreSQL.

He believes technology becomes powerful when it is shared, taught, and built together.

GitHub | LinkedIn | Blog | X

Introduction

We talk a lot about tokens.

Input tokens.
Output tokens.
Context windows.
Pricing.
Latency.
Caching.

But I think we are missing the deeper question.

Why are we using so many tokens in the first place?

Most conversations about token optimization start too late. They focus on trimming prompts, compressing context, caching responses, or choosing cheaper models.

Those things matter.

But they are symptoms.

The deeper issue is often not the model.

It is the interaction architecture.

A weak interaction architecture forces the user to explain too much, repeat too much, clarify too much, and carry too much system knowledge in every request.

That is not just a usability problem.

It is an economic problem.

Because in AI-native systems, tokens are no longer just text.

Tokens are infrastructure.

They affect cost.
They affect latency.
They affect context capacity.
They affect reliability.
They affect how much useful work an AI system can perform before the interaction collapses under its own verbosity.

This is where Intent Compression Ratio (ICR) becomes useful.

In my previous article, Intent Compression Ratio: Measuring the Power of Intent, I looked at ICR as a way to understand how much procedural complexity moves from the developer to the system.

In this post, I want to look at the same idea from another angle:

What happens to token economics when intent is poorly compressed?

The Token Problem Is Not Just Token Count

It is tempting to think token optimization means making prompts shorter.

That is only partially true.

A short prompt can be bad if it is ambiguous.

A long prompt can be necessary if the system has no context.

The real question is simpler:

How many tokens does the system need to faithfully understand and fulfill the user’s intent?

That is the economic unit that matters.

Not prompt length alone.

Not output length alone.

But the total token footprint required to move from intent to outcome.

That includes:

• The first prompt.
• The clarification turns.
• The repeated context.
• The hidden system instructions.
• The tool call explanations.
• The retries.
• The corrections.
• The familiar “no, that is not what I meant” loop.

When you add all of that up, a small human request can become a very large machine interaction.

I call this token amplification.

Token Amplification

Token amplification happens when compact human intent expands into a disproportionately large token footprint.

For example, a developer may say:

Deploy the payment service with autoscaling and observability.

That is a compact intent.

But in a low-ICR system, the developer may need to explain:

• What service means.
• Which environment.
• Which runtime.
• Which scaling rules.
• Which observability stack.
• Which naming conventions.
• Which security constraints.
• Which deployment policy.
• Which rollback behavior.
• Which team standards.

Then the system asks clarifying questions.

Then the user repeats context.

Then the system generates something close, but not quite right.

Then the user corrects it.

By the end, the original intent has become a long token trail.

The system did not use more tokens because it understood more.

It used more tokens because it understood less.

That distinction matters.

ICR as an Economic Signal

For this discussion, I use a deliberately simple version of ICR:

ICR = Intent Fulfilled / Total Tokens

Higher ICR means more useful work completed per token consumed.

Lower ICR means the system burns more tokens to achieve the same outcome.

This makes ICR more than a developer experience idea.

It is also a cost metric.

A high-ICR system does not merely feel better to use.

It is more economical.

It reduces repeated explanations.
It reduces clarification loops.
It reduces redundant context transfer.
It reduces orchestration overhead.
It reduces the probability of retries.

That means token optimization is not only a prompt engineering problem.

It is an intent design problem.

Prompt Verbosity Is Often a Tax on Missing Intent

We have all seen this pattern.

A prompt starts simple.

Then it grows.

“Act as a senior engineer…”

“Follow these conventions…”

“Use this format…”

“Do not forget…”

“Before answering…”

“Assume the following context…”

“Here are the constraints…”

“Here are examples…”

“Here are exceptions to the examples…”

Some of this is useful.

But often, prompt verbosity is compensating for missing system memory, missing context, missing abstractions, or missing intent contracts.

The user is not just expressing the goal.

The user is rebuilding the world around the goal.

Every time.

That is expensive.

Not only in dollars, but in cognition.

The developer becomes responsible for carrying the system model in their head and serializing it into text repeatedly.

That is not intelligence.

That is manual context transport.

Skills Are Where Intent Becomes Executable

Intent by itself is not enough.

If I say:

Deploy the payment service.

The system still needs to know what “deploy” means in my environment.

Does it mean Kubernetes? Snowpark Container Services? Terraform? A CI/CD pipeline? A staging rollout first? Which observability defaults? Which governance rules? Which rollback policy?

This is why skills are becoming such an important primitive in agentic systems.

Claude has Skills. Cortex Code has Skills. Gemini has Gems. Codex has Skills.

Different names, but they point in a similar direction.

A skill is not just a saved prompt.

At its best, a skill is compressed operational knowledge.

It can encode the steps, constraints, conventions, tools, expected output, safety boundaries, and domain-specific judgment required to complete a recurring task.

Without skills, the user has to serialize all of that context into the prompt.

With skills, the system can load the right operational pattern and execute the intent with less repeated explanation.

That is ICR in practice.

The human expression gets smaller, but the executable meaning gets richer.

The Economics of Repetition

The point is not to make every prompt shorter.

The point is to stop repeating what the system should already know.

A lot of token waste does not come from the user’s intent.

It comes from everything wrapped around the intent.

The same architectural context.
The same coding standards.
The same deployment rules.
The same governance constraints.
The same output format.
The same safety instructions.

Again and again.

In a single prompt, this feels harmless.

Across a workflow, it becomes expensive.

Across agents, tools, and retries, it becomes amplification.

This is the economics of repetition.

Low-ICR systems make humans restate context.

High-ICR systems turn repeated context into reusable capability.

That is why skills matter.

A skill is one way to stop paying the repetition tax every time. It packages recurring operational knowledge so the user can express the outcome instead of re-explaining the procedure.

The token savings are not just from fewer words.

They come from moving repeated meaning out of the prompt path and into the system.

But compression creates a second responsibility: transparency.

A Small Experiment: ICR Lab

To make this visible, I built a small open-source Streamlit app called ICR Lab.

The app is intentionally simple. It does not call a live LLM, benchmark models, or try to be an agent framework.

It simulates token behavior across different interaction styles so the economics are easier to see.

The current version compares five modes:

• Verbose Prompting: large prompts with repeated context.
• Clarification Heavy: multiple rounds of back-and-forth before the system can act.
• Context-Aware: the system reuses some context and reduces repeated explanation.
• Intent-Optimized: the user expresses compact semantic intent and the system assumes more structured responsibility.
• Over-Compressed: the user strips so much context that the system misinterprets the intent, triggering correction rounds that amplify tokens beyond what a verbose prompt would have cost.

The line I keep coming back to is this:

Same intent. Different interaction architecture. Different token economics.

That is the entire point of the demo.

The task does not change.

The model does not change.

The interaction pattern changes.

And the token footprint changes with it.

The fifth mode exists to prevent the wrong takeaway. Over-compression is not optimization. When the system lacks enough shared context to decompress a terse intent, it guesses wrong — and the correction cost exceeds the savings.

ICR is not “make prompts shorter”. It is “make meaning denser”.

The Wrong Lesson: Just Make Prompts Shorter

There is a dangerous interpretation here.

Someone might say:

So high ICR means shorter prompts?

Not exactly.

A short prompt with no shared context is just ambiguity.

A high-ICR interaction is not merely brief.

It is dense.

It carries more meaning per token because the surrounding system understands the domain, constraints, policies, tools, and desired state.

That understanding can come from many places:

• Platform profiles.
• Reusable skills.
• Semantic memory.
• Declarative manifests.
• Typed tool interfaces.
• Repository context.
• Organizational conventions.

The point is not to delete words.

The point is to stop repeating what the system should already know.

Compression Without Visibility Is Opacity

There is another trap.

High compression can become dangerous when it hides too much.

If I say:

Deploy the connector.

And the system creates users, roles, policies, network rules, secrets, and tasks behind the scenes, I may get a great experience.

Until something breaks.

Then I need to know what was created, in what order, with what constraints, which step failed, what can be retried safely, and what state was left behind.

This is why ICR must be paired with transparency.

The system should absorb complexity, but not erase visibility.

High ICR without transparency is magic.

High ICR with transparency is engineering.

That distinction matters even more when money enters the picture.

Token-efficient systems that are impossible to debug only move the cost elsewhere.

They save tokens on the happy path and burn engineering hours during failure.

From Prompt Optimization to Intent Optimization

Prompt optimization asks:

How do I phrase this better?

Intent optimization asks:

Why does the user need to phrase this much at all?

That is the deeper shift.

The future of AI systems will not be defined only by larger context windows.

Larger context windows help, but they can also hide bad architecture.

If the system can accept more context, we may simply dump more context into it: more documents, more history, more instructions, more examples, more logs, more everything.

But stuffing the context window is not the same as understanding intent.

A bigger backpack does not make you organized.

It just lets you carry more mess.

The better direction is often not:

Give the model more tokens.

It is:

Require fewer tokens to preserve the same intent.

That is intent optimization.

What ICR Starts to Reveal

Once you look at AI systems through ICR, a few things become obvious.

A verbose prompt is not always rich intent.

Sometimes it is a workaround for missing abstraction.

A clarification loop is not always intelligence.

Sometimes it is the system externalizing its uncertainty back onto the user.

A large context window is not always better architecture.

Sometimes it is a bigger place to store repeated noise.

An agent workflow is not automatically efficient.

Sometimes agents multiply token usage by passing bloated context between themselves.

And a skill is not just automation.

It is a compressed unit of repeatable intent.

That last point matters.

In Intent Driven Development, I argued that developers are not becoming less important because AI can generate code.

They are becoming more important because the bottleneck has moved.

The bottleneck is no longer syntax.

It is clarity of intent.

Token economics reinforces that shift.

A vague developer with AI creates token sprawl.

A precise developer with intent-aware systems creates leverage.

After seeing these patterns, the next question is natural:

Why simulate this instead of using a live LLM?

Why ICR Lab Uses Simulation

ICR Lab v1 uses deterministic simulation instead of live model calls.

That is deliberate.

The goal is not to benchmark OpenAI, Anthropic, Snowflake Cortex, local models, or anything else.

The goal is to isolate the interaction pattern.

If the app used live LLM calls immediately, the conversation would drift into model quality.

Which model is cheaper?
Which model answers better?
Which model counts tokens differently?
Which model follows instructions more reliably?

Those are valid questions.

But they are not the question I wanted to explore.

The question is simpler:

How does interaction architecture change token economics?

Simulation keeps that visible.

Same input.
Same modes.
Same curves.
Same comparison.

No API keys.
No model variance.
No hidden backend.

Live model integrations can come later.

The concept needs to be clear before the instrumentation becomes real.

Closing Thought

The future may not belong to systems that consume the most tokens.

It may belong to systems that need the fewest tokens to understand us correctly.

That does not mean smaller models.

It does not mean shorter prompts.

It does not mean hiding complexity behind magical interfaces.

It means better intent architecture.

Systems that know the context.
Systems that preserve constraints.
Systems that compress repeated patterns.
Systems that expose what they did.
Systems that turn verbose human instruction into transparent machine execution.

That is where ICR and token economics meet.

Not in counting tokens for the sake of counting tokens.

But in asking a better question:

How much useful intent did this system fulfill per token consumed?

That question changes the optimization target.

And once you see it, token cost stops looking like a pricing problem alone.

It starts looking like a design problem.

Try the Demo

I built a small companion app for this post: ICR Lab.

It is open source and runs as a Streamlit app.

The goal is simple:

Enter a task.
Compare interaction modes.
Watch token amplification happen.
Then watch it collapse as intent gets better compressed.

It is not a benchmark.

It is a playground for building intuition.

Because sometimes the best way to understand a systems problem is to make the invisible curve visible.

Related Reading

• ICR Lab GitHub Repo
• ICR Lab Live App
• Infrastructure as Intent: The Field Velocity Blueprint
• The Ghost in the Machine: Why AI Needs the Spirit of UML
• Intent Driven Development: The Shift Developers Can’t Ignore
• Intent Compression Ratio: Measuring the Power of Intent

About the Author

Kamesh Sampath is a Lead Developer Advocate at Snowflake, author, and long-time open-source contributor with 25+ years in enterprise software. He works across data engineering and AI with developer communities, helping practitioners turn modern data platforms into systems that hold up in production.

Through talks, writing, and hands-on demos, Kamesh makes cloud, data, and AI topics easier to understand and apply — grounded in real-world constraints. His sessions mix deep technical detail with practical patterns that developers and data teams can apply right away.

Lately, he’s been speaking about Apache NiFi (Snowflake Openflow), AI (Snowflake Cortex), and PostgreSQL.

He believes technology becomes powerful when it is shared, taught, and built together.

GitHub | LinkedIn | Blog | X

We’ve spent years optimizing execution.

Better APIs.
Better CLIs.
Better automation.

But developers don’t struggle because execution is slow.

They struggle because execution is hidden.

Infrastructure doesn’t fail loudly.
It fails silently — when you get the order or the desired state wrong.

The Problem Isn’t Steps. It’s the Graph

Take a realistic ask:

“I need to deploy a connector with Snowflake Openflow.”

In practice, this often ends with generating a Programmatic Access Token (PAT) to be used by clients like Apache NiPyAPI.

But getting there is not one step.

It’s a graph.

Each node is valid on its own.

But the ordering and desired state are not optional.

A network policy must exist before it’s attached.
An auth policy must reference the correct role.
The PAT must be generated after all constraints are satisfied.

Get this wrong?

No exception.
No obvious failure.

Just a token that doesn’t authenticate.
A connector that doesn’t connect.
And hours spent debugging across identity, network, and auth layers.

Infrastructure is not procedural.

It is relational.

And we’ve been asking developers to execute that relationship manually.

This is also why many AI agent workflows struggle today. We ask agents to operate on systems that still expose too much procedural complexity.

If an agent has to reason through 12 procedural steps to achieve one outcome, the probability of a hallucinated configuration, missed dependency, or ordering error rises quickly.

This is where Intent Compression Ratio(ICR) starts to matter.

High ICR becomes a prerequisite for agentic readiness.

kubectl apply Was a Hint

We’ve seen this pattern before.

kubectl apply didn’t simplify Kubernetes.

It changed the interface.

From:

Step-by-step execution

To intent:

“Here is the desired state.”

And the system:

• Builds the dependency graph
• Determines ordering
• Reconciles toward the desired state

It doesn’t remove complexity.

It absorbs the graph.

But We Never Measured the Absorption

We say:

• Intent-driven
• AI agents
• Skills

But we don’t ask:

How much complexity moved from the developer to the system?

That’s the gap.

Intent Compression Ratio

I’ve started calling this Intent Compression Ratio (ICR):

A way to measure how much execution complexity is collapsed into intent.

To quantify this, we need a way to measure the “work” the system is doing for us.

• Ops : Are the total required operations every distinct state-changing API call e.g. IAM binding, network rule, object creation, policy attachment, or validation step.
• Intent expressions are the semantic goals defined by the user.

A system with an ICR of 1 is just a wrapper.

A system with an ICR of 10 is an architectural partner.

At first glance, this looks like productivity.

It isn’t.

It measures who owns the system’s complexity.

What Actually Gets Compressed

Not just steps.

Cognitive load
The developer no longer tracks ordering or dependencies.

Dependency graph
The system constructs and executes the DAG.

Desired state
The system ensures correctness of what is being built, not just how.

Failure surface
Fewer invalid states. Fewer silent failures.

Higher ICR doesn’t just mean faster.

It means safer and more predictable systems.

ICR in Action

Let’s ground this with a few examples from the Snowflake platform.

Snowflake is a concrete example but Intent Compression Ratio is the general idea. I use Cortex Code as an Agentic Coding tool proof point.

The pattern is broader than any one platform: whenever developers have to coordinate object creation, dependency ordering, policy enforcement, and desired state, there is an opportunity to compress the graph into intent.

Programmatic Access Token

Intent:

“Deploy a Google connector with Openflow.”

Underlying reality:

1. Create service user
2. Create service role
3. Grant role to user
4. Create network rule
5. Create network policy
6. Attach network policy to user
7. Create authentication policy
8. Attach authentication policy to user
9. Configure PAT policy constraints
10. Generate PAT
11. Store PAT securely for client usage
12. Validate token exchange via Apache NiPyAPI

ICR ≈ 12 Ops / 1 intent = 12

Or expressed as intent compression:

1 intent : 12 Ops

This is not step reduction.

This is dependency graph compression.

Network Access

Intent:

“Allow access to googleapis.com.”

Underlying reality:

1. Rule definition
2. Policy creation
3. Attachment
4. Grants

ICR ≈ 4 Ops / 1 intent = 4

Or:

1 intent : 4 Ops

External Volume

Intent:

“Set up an Iceberg table with Snowflake.”

Underlying reality:

1. Storage integration
2. IAM configuration
3. External bindings
4. Volume configuration
5. Access validation
6. Table setup

ICR ≈ 6 Ops / 1 intent = 6

Or:

1 intent : 6 Ops

Where Skills and AI Agents Fit

This becomes critical in an AI-driven world.

AI agents are not good at:

• Long procedural chains
• Fragile ordering
• Implicit dependencies

They are good at:

• Intent
• Bounded abstractions
• Repeatable execution units

This is where skills come in.

A skill is not just automation.

It is a compressed execution graph with guarantees.

Whether implemented via Cortex Code or any other agentic system, a skill:

• Encodes ordering
• Enforces constraints
• Ensures the correct desired state
• Handles sensitive steps, such as token handling, safely

So when an agent executes:

“Deploy a connector with Openflow.”

It doesn’t reason through 12 Ops.

It invokes a single semantic unit.

ICR becomes a proxy for how usable a system is for agents.

The Tradeoff

Not all compression is good.

Too Much Compression: Opaque Systems

You run:

“Deploy my connector.”

It works — until it doesn’t.

Now the connector fails to authenticate.

You don’t know which role was created.
You don’t know which policy was applied.
You don’t know if the network rule exists.
You don’t know whether the PAT was generated under the right constraints.

You try to debug.

But there is no visibility into intermediate steps.
There is no way to fix just one part.
Everything is hidden behind the abstraction.

So you abandon it and go manual.

High ICR.
Zero observability.

That is not effective compression.

That is opacity.

Too Little Compression: Fully Manual Systems

Now take the opposite extreme.

You manually create everything:

• User
• Role
• Grants
• Network rule
• Network policy
• Auth policy
• Token

Everything is visible.

But now you own everything.

You attach the wrong policy.
Or attach it in the wrong order.
Or bind the wrong role.
Or generate the token before the constraints are actually in place.

Nothing errors.

Your PAT is generated.
Your connector is deployed.

But it doesn’t work.

Debugging spans identity, network, and auth layers.

Full control.
Full responsibility.

That is not simplicity.

That is exposed complexity.

Glass Box Compression

The solution to opacity is not to go back to manual scripts.

It is observable intent.

I think of this as Glass Box Compression.

Unlike a black box, where the system performs magic and leaves you clueless when it breaks, a glass box system absorbs complexity while remaining transparent.

Now consider a well-designed skill:

“Deploy a connector with Openflow.”

It executes the full dependency graph.
It enforces ordering.
It validates desired state.
It exposes what it created.

And when something fails, it tells you:

• What failed
• Why it failed
• What was expected
• What can be retried safely

You can inspect the graph.
You can override specific parts.
You can re-run safely.

This is high ICR with visibility.

That is effective compression.

The Shift

We used to optimize for:

• Performance
• Scale
• Reliability

Now we also optimize for:

How much complexity the system can safely absorb.

Because in an intent-driven world:

• Developers express goals
• AI agents invoke abstractions
• Systems execute graphs

The interface moves closer to intent.

The execution moves deeper into the system.

And the value shifts from:

“How many steps can I automate?”

To:

“How much complexity can I safely stop thinking about?”

Closing Thought

The question is no longer:

“What can this system do?”

It is:

“How much of the system can you safely not think about?”

That is the real power of intent.

And that is what Intent Compression Ratio begins to measure.

This brings the series full circle.

In Infrastructure as Intent, I argued that infrastructure is moving from imperative execution toward declarative intent: developers should describe the outcome, not manually orchestrate every step.

In The Ghost in the Machine, I explored why intent still needs structure. We need the “Spirit of UML” — not necessarily UML diagrams themselves, but a way to model relationships, dependencies, and constraints before systems can safely automate them.

In Intent-Driven Development, I looked at the developer shift: from writing every instruction to shaping the intent that AI systems and platforms can execute.

Intent Compression Ratio connects those ideas.

If intent is the interface, and relationships are the hidden graph, then ICR asks:

How much of that graph can the system safely absorb?

That is the bridge from philosophy to measurement.

In the next part of this series, I’ll look at how to build high-ICR skills into custom AI agents.

About the Author

Kamesh Sampath is a Developer Advocate at Snowflake, an author, and a long-time open-source contributor with 25+ years in enterprise software. He works across data engineering and AI with developer communities, helping practitioners turn modern data platforms into systems that hold up in production.

Through talks, writing, and hands-on demos, Kamesh makes cloud, data, and AI topics easier to understand and apply — grounded in real-world constraints. His sessions mix deep technical detail with practical patterns that developers and data teams can apply right away.

Lately, he has been speaking about Apache NiFi, Snowflake Openflow, Snowflake Cortex, and PostgreSQL.

He believes technology becomes powerful when it is shared, taught, and built together.

GitHub | LinkedIn | Blog | X

We spent decades writing code for machines. Now we need to express intent for them.

There’s a quiet anxiety in software development right now.

If AI can write code… where does that leave developers?

The answer is simpler and more uncomfortable than it seems.

Builders are not becoming obsolete.
They are becoming more important.

Because in a world where machines can generate code, the real bottleneck is no longer syntax.

It is clarity of intent.

AI is not replacing developers.
It is amplifying them.

A vague developer with AI produces noise.
A precise developer with AI produces systems.

The leverage has shifted but the responsibility has not.

Developers who can express intent are still in control.

But the interface has changed.

We are no longer just writing code.
We are shaping what systems should do with just pure natural language.

This shift doesn’t come out of nowhere.

My previous article on Infrastructure as Intent, I explored how systems are moving toward declared outcomes over explicit instructions.

In The Spirit of UML, I argued that abstraction is becoming essential again in the age of AI.

In this article, I propose that Intent Driven Development(IDD) is where these ideas meet in practice.

We’re Still Thinking Like Syntaxists

Even with AI tools everywhere, most developers still:

• Write code line by line
• Focus on syntax and structure
• Debug implementation details

AI becomes:

• Faster autocomplete
• Better Stack Overflow

But that misses the point.

The real shift is not AI writing code.
It is:

Developers expressing intent clearly enough that machines can realize it.

From Syntax to Semantics

We were trained to be syntaxists.

We think in functions, APIs, and control flow.

Intent Driven Development requires semanticists:

• Intent
• Constraints
• Context
• Outcomes

Code doesn’t disappear.
But it stops being the starting point.

Intent becomes the source. Code becomes the artifact.

Writing Better Prompts is Writing Better Software

In this world, prompts are not casual inputs.
They are design artifacts.

Weak:

Build a dashboard

Intent-driven:

Build a Streamlit dashboard using Snowflake TPCH data. Show revenue trends over time, allow filtering by region and customer segment, and generate natural language insights. Keep the UI minimal and ensure queries are efficient.

The difference is precision.

A Real Example: From Intent to App (Cortex + Streamlit)

Start with intent that clearly defines the end outcomes:

Build a Streamlit dashboard using Snowflake TPCH data.

• Show revenue trends over time
• Allow filtering by region and customer segment
• Use Cortex to generate natural language insights
• Keep the UI minimal but interactive
• Ensure queries are efficient and reusable

No boilerplate. No step-by-step instructions just clear intents.

Using tools like Cortex Code, this intent can translate into:

• Snowflake queries over TPCH tables
• Aggregations for revenue trends
• A Streamlit UI with filters and charts
• LLM-generated summaries like: “Revenue dipped in Q3 due to decline in APAC region”

You didn’t handwrite everything.
You guided the system with intent.

Where Developers Still Matter

The first version won’t be perfect.

Maybe the query is inefficient.
Maybe the chart is unclear.
Maybe the insights are generic.

You refine the intent:

• Optimize queries for large datasets
• Make insights specific to region trends
• Add anomaly detection

Each iteration improves the system without dropping to low-level code immediately.

But… How Do You Know It’s Good?

You build something like this and think:

“Looks right… but can I trust it?”

This is where most AI-driven apps fail.

Not in building but in knowing whether they’re accurate and efficient.

Adding a Evaluation Layer with TruLens

Intent Driven Development needs evaluation.

Frameworks like TruLens make this practical.

They help measure whether your system aligns with your intent:

• Context Relevance → Is the system finding the right data?
• Groundedness → Are insights tied to actual data?
• Answer Relevance → Is the system completing its goal?
• Plan Quality → Is the agent forming a plan that is optimal for the user’s goal?
• Tool Selection → Is the agent choosing the right tools for each task?
• Plan Adherence → Does the agent follow its own plan, or drift mid-execution?
• Tool Calling → Are tools being called correctly and with the right inputs?
• Logical Consistency → Does the agent maintain coherent reasoning across steps?
• Execution Efficiency → Is the agent solving the task with minimal steps, cost, and latency?

These aren’t just metrics.

They are checks on your intent and the system’s ability to complete your goals.

The New Loop

Intent Driven Development changes the workflow:

1. Define intent
2. Generate system
3. Measure results
4. Refine intent

You’re not just debugging code.

You’re debugging meaning.

Why This Matters Now

• AI can act on intent
• Systems are too complex to handcraft
• The bottleneck is idea → implementation

IDD reduces that gap.

It lets developers operate closer to the problem than the plumbing.

The Developer Shift

The next generation of developers will:

• Express constraints clearly
• Structure intent precisely
• Guide AI systems
• Measure quality continuously

The best developers won’t write the most code.

They’ll express the clearest intent and validate it.

Closing

We taught machines to understand syntax.

Now we need to lead them with intent.

AI hasn’t taken control away from developers.

It has raised the bar for what control means.

The developers who thrive in this shift won’t be the ones who write the most code.

They’ll be the ones who can:

• express intent clearly
• define constraints precisely
• And validate outcomes rigorously

Because in this new model, code is no longer the source of truth.

Intent is.

AI is powerful but it is still just an executor.

The developer remains the authority.

And like any good system, the quality of the output will always depend on the clarity of the input.

It starts with something simple.

The next intent you write.

Resources

If you want to explore Intent Driven Development hands-on, here are a few starting points.

Try the Intent Yourself

Use this prompt with Cortex Code (or a similar tool) and iterate on it:

Intent Prompt:

Build a Streamlit dashboard using Snowflake TPCH data.

Requirements:

• Show revenue trends over time
• Allow filtering by region and customer segment
• Use semantic queries over Snowflake tables
• Generate natural language insights for trends and anomalies
• Ensure all insights are grounded in actual query results
• Keep the UI minimal, fast, and interactive

Constraints:

• Optimize queries for large datasets
• Structure the code for reuse and clarity
• Return both data visualizations and textual summaries

Output:

• Streamlit app code
• Snowflake queries used
• Example insights generated from the data

Start with this, then refine the intent.

That’s the practice: improving the clarity of what you want the system to do.

👉 Check out the TruLens cheatsheet: https://github.com/Snowflake-Labs/sf-cheatsheets/blob/main/rag-evaluation-cheatsheet.md

Tools

• Cortex Code
• Streamlit
• Snowflake
• TruLens

Introduction

AI can generate code at 10x speed. It still doesn’t understand the systems it’s building.

This story is about why we need to revive the spirit of Unified Modelling Language(UML) — not as diagrams, but as living infrastructure.

Back then, the hot trend was Model-Driven Development (MDD) — generating code from models. UML was the flagship: Class Diagrams, Sequence Diagrams, the whole catalog. The belief was simple: if the model was correct, the system would be correct.

We were wrong.

UML didn’t fail because modeling was useless. It failed because of the Sync Tax.

You draw a class diagram. Rational Rose generates clean code. You ship it. QA finds bugs. Hotfixes go in. Production diverges. The model is now stale.

Keeping diagrams and code aligned becomes real work. Expensive work. The tax exceeds the value. So teams stop paying it.

We declared, “The code is the documentation,” and moved on.

Starting to sound familiar?

If you ask me, we are back in the MDD era. We’ve just rebranded it as AI-Driven Development.

The diagram is gone. The model remains. Only now, instead of Sync Tax, we pay the Hallucination Tax.

AI agents generate code at unprecedented speed. Ask, “Generate a Snowflake pipeline to ingest customer events,” and it produces SQL, tasks, maybe even a clean dbt model in seconds.

But the real question is: Does it understand the system it is operating in?

Specifically, when you give it that prompt, does it see the whole picture?

• Your warehouse sizing and cost guardrails?
• Your RBAC model and masking policies?
• Which schema is production-certified versus experimental?
• Which data is PII and which roles are allowed to see it?

Then there’s the Context Gap — the ritual of re-explaining your infrastructure to an agent with limited working memory. Every new chat resets shared understanding. Yes, there are hacks. But none of them feel native.

The spirit of modeling was never about diagrams. It was about shared context.

AI has the muscle. It lacks shared context.

So here’s the twist: UML and AI are two halves of a broken whole.

UML was too rigid for humans to maintain. AI is too fluid for systems to trust.

To solve the Field Velocity problem, we don’t go back to 2005.

We make AI pay the Sync Tax for us.

And the only way to do that is to encode intent as an enforceable structure.

The Living Model: From “Pretty Picture” to “Policy”

In my earlier blog on Infrastructure-as-Intent, I wrote about how human expertise and experience can be carved into agentic Skills(In Part 1, we defined Skills as the codified expertise and human experience that turn operators into mentors for AI agents.) , turning each of us from operators into mentors.

That shift forces us to stop authoring static models and start inferring living ones.

The truth of your system already exists — scattered across scripts, RBAC definitions, masking policies, and CI/CD pipelines.

So why does AI still hallucinate?

Because of the Context Gap — the absence of shared context.

Here is how we revive the UML spirit: by turning Skills into the rules of engagement for our AI agents.

The Guardrail Effect

When we practiced UML, we followed the structure. First system architecture. Then sequence diagrams. Then class diagrams. Step by step, we built a shared understanding.

In the AI era, shared understanding becomes shared constraints.

Instead of executing a prompt literally, the AI must query the Skills.

If a Skill declares, “All customer data requires masking primitive X,” that declaration becomes an enforceable structure. The agent generates within it, not around it. The constraint is applied at the point of creation.

That is the spirit of UML — reborn as an executable policy.

Policy is what machines can execute. Intent is what they can reason about.

Semantic Reasoning

Shared constraints unlock something UML never could: semantic reasoning.

UML provided syntactic structure. The reasoning lived in our heads.

AI can reason — but only if we give it structure.

Ask, “If I change this stream’s schema, who breaks?”

The AI uses the Skill graph to simulate impact and produce a blast-radius report before a single ALTER TABLE runs.

That is the difference between generation and understanding.

This is where intent stops being documentation and starts becoming infrastructure.

Closing the Context Gap

In my earlier post, I introduced the idea of a Shared Manifest — a persistent memory layer for AI agents. Not as a long prompt, but as structured Skills.

With Skills, the agent carries Architectural Memory.

It knows your certified schemas. Your cost guardrails. Your intent. Because it operates inside the same world model as you.

The Living Model is not documentation. It is infrastructure.

This is Infrastructure-as-Intent in action.

We use the rigidity of the model to keep AI honest.

We use the fluidity of AI to pay the Sync Tax — bringing the spirit of UML back as living infrastructure.

A Call to Builders

We don’t need a return to 2005-era static diagrams. We need an Architectural Memory for our systems.

Agentic AI tools have already defined the schema. Don’t panic about learning another DSL 🤣. It’s not one.

It’s a disciplined Markdown. Clear English. Proper structure. The same rigor you apply when writing a design doc.

Except now, that document is executable.

It’s not documentation for humans alone. It is codified wisdom for agents.

The era of blind velocity is over.

It’s time to give the muscle a brain.

Visuals note: The conceptual diagrams in this post were generated by Google Gemini.

About the Author

Kamesh Sampath is a Lead Developer Advocate at Snowflake, author, and long-time open-source contributor with 25+ years in enterprise software. He works across data engineering and AI with developer communities, helping practitioners turn modern data platforms into systems that hold up in production.

Through talks, writing, and hands-on demos, Kamesh makes cloud, data, and AI topics easier to understand and apply — grounded in real-world constraints. His sessions mix deep technical detail with practical patterns that developers and data teams can apply right away.

Lately, he’s been speaking about Apache NiFi (Snowflake Openflow), AI (Snowflake Cortex), and PostgreSQL.

He believes technology becomes powerful when it is shared, taught, and built together.

GitHub | LinkedIn | Blog | X

We ’ve spent years building automation pipelines to deploy infrastructure. We call it Infrastructure-as-Code (IaC) — whether that’s Terraform, Ansible, or even simple CLI scripts. It saves us from nightmares when we need consistency between where we develop and where we deploy. Development == Production.

But when Snowflake launched Cortex Code — an AI coding agent designed for the Data Cloud — it sparked a new thought for me: why can’t we take IaC automation into its next phase? Instead of declaring how to build everything step by step, what if we simply declared the outcome we want?

This idea wasn’t born out of nowhere. It came from my own attempts to modernize my scripts — snow-utils — a collection of utilities I built to repeatedly set up Snowflake environments for demos. They started as a way to eliminate the rinse-and-repeat plumbing work.

The scripts solved the plumbing problem. But they introduced a different one: remembering the right combination of parameters every single time.

That frustration led me to a deeper pattern — one that could fundamentally change how we interact with cloud environments, and one that fits naturally into this new AI-assisted era.

The “Field Velocity” Problem

So we found the problem, built a solution, and then refined it further — something that could take us into the next phase. But before we get there, let’s pause and ask: why does this even matter?

The answer is simple and universal: speed.

We invented computers to help us do things faster. But as computer science evolved, the systems we built became increasingly complex. Somewhere along the way, speed started fighting complexity.

That’s what I call the Field Velocity problem.

What is it exactly?

It’s the time it takes to go from a customer asking:

Can we query Snowflake-managed Iceberg tables from DuckDB?

to a working, deployed proof of concept.

If you look closely at the history of Infrastructure-as-Code, you’ll notice something consistent. The bottleneck is rarely the idea. It’s the plumbing.

So our customer’s question will translate into following tasks(plumbing):

Configuring IAM roles – > Editing trust policies – > Setting up buckets. – > Creating network rules – > DDL Statements.

If I put it mathematically, we spend 90% of our time on plumbing and only 10% on the actual value.

So the fix? Invert the ratio.

That inversion is what I call Infrastructure-as-Intent.

Same wine. New bottle. 😉

The Evolution: From Power CLI to Atomic Primitives

I come from a strong Linux and infrastructure background, and I’m a die-hard CLI person. If something is repetitive, my instinct is to script it.

That “Power CLI” reflex is what led me to write snow-utils.

Let me explain with a simple example.

Say I want to connect Snowflake to an AWS S3 bucket. Sounds straightforward. In reality, it looks like this:

• Log into AWS Console → Create S3 bucket
• Create an IAM policy
• Create an IAM role and attach the policy
• Log into Snowflake → Create the EXTERNAL VOLUME
• Run DESCRIBE EXTERNAL VOLUME to hunt for AWS user ARN
• Go back to AWS → Edit the Trust Relationship JSON and paste those values

And yes — to err is human. But one small typo in that trust policy JSON, or grabbing the wrong property from the Snowflake DESCRIBE output, can cost you hours of debugging before everything finally works.

That’s when I decided to wrap this entire chain of interdependent steps into a single command.

Instead of manually wiring everything together, I created a primitive that understood the sequence.

Now, for the same task, it’s simply:

snow-utils extvolume:up

Done.

One clean shot. Goal scored.

Over time, it became deterministic. It worked every time — no manual hunting, no fragile copy-paste, no mysterious “Access Denied” errors.

That’s when I realized something important: these weren’t just scripts anymore. They were atomic primitives.

The Shift: From Operator to Mentor

Once I had the primitives working reliably, I started asking a new question: why can’t this be more intelligent?

Around that time, Snowflake announced Cortex Code. What caught my attention wasn’t just that it was an AI coding agent, but the concept of Skills — giving the agent structured capabilities to orchestrate tools and perform tasks in a more contextual way.

That’s when I saw the transition from Infrastructure-as-Code (IaC) to Infrastructure-as-Intent (IaI) taking shape.

Instead of keeping a mental checklist of parameters, flags, expiration rules, and policy combinations, why couldn’t I just express the intent and let the system handle the wiring?

“Set up a Snowflake External Volume.”
“Create a Snowflake PAT.”

It suddenly felt possible.

But this wasn’t just about making things simpler. It was about changing roles.

With IaC, I was the operator — running commands, supplying parameters, ensuring everything aligned.

With IaI, I started becoming the mentor.

I wasn’t just teaching syntax. I was embedding experience.

Years of small production scars. Lessons from silent authentication failures. The instinct to validate IAM structures before applying them. The habit of forcing a dry run before destructive changes. The discipline of demanding explicit confirmation when something could break environments.

That’s what I call codifying wisdom — capturing operational judgment and passing it into the system so it behaves the way I would.

Let’s not fear AI. It doesn’t invent wisdom on its own — it executes what we mentor it to do.

Intelligence still comes from somewhere.

Let me make this concrete.

Suppose I want to create a Snowflake Programmatic Access Token (PAT). Traditionally, that involves multiple careful steps: creating a service user, applying network restrictions, attaching the correct policies, defining expiration windows, and ensuring everything aligns securely.

Instead of remembering every option and edge case, I wrote a simple markdown file called SKILLS and embedded all the logic behind the PAT primitive into it — including the commands and my opinions about how it should be done safely.

Now all I need to tell Cortex Code is:

“Set me up a PAT that’s valid for 7 days and expires 30 days from now.”

That’s it.

No more remembering long chains of parameters. No wrappers around wrappers. No cognitive overhead.

What emerged was a composable architecture — where the logic (snow-utils) is unbundled from the interface (Cortex Code).

• The primitives remain solid.
• The skills carry the experience.
• The interface interprets intent.

That separation is what turns automation into intent.

The Pedagogical Pattern: Intentional Failure as a Feature

When I spoke about intent in the previous section, I didn’t just mean my intent to run a set of commands or automation scripts. I had another intent in mind as well.

Too many intents? I can already see an “Intent Overflow Exception” coming. 😉

Jokes apart — as a Power CLI user, clear intent helps me execute tasks efficiently. But a mentor does more than execute. A mentor creates situations where you apply your skills across related areas. A mentor shapes understanding.

As a Developer Advocate, I build demos, tutorials, and utilities like snow-utils. I usually share my GitHub repositories with a README explaining how to run the demo you just saw in my session.

That works.

But nothing matches personalized teaching — where the session leader’s experience, empathy, and small nuances surface naturally. The little warnings. The subtle explanations. The “watch this part carefully” moments.

That’s a classic pedagogical pattern.

Let me explain with the Horizontal Iceberg REST Catalog (HIRC) and DuckDB demo.

All you have to say now is:

“Deploy HIRC DuckDB demo.”

But the real power isn’t just in deploying the environment. It’s in how the Skill is structured.

I intentionally embedded a learning flow into it:

• Say what you are going to do
• Do it
• Let it fail
• Explain why it failed
• Fix it
• Show how it succeeds

That pattern is deliberate.

The goal isn’t just automation. It’s understanding.

That’s the human part — and that doesn’t emerge automatically. You have to mentor it into the system.

Now the Skill doesn’t just deploy infrastructure. It delivers a learning experience.

The “Recorder” Bonus: Intelligent & Interactive Reproduction

Since I’m documenting intent as a Skill, the natural next step was to share it.

That’s how we normally share knowledge — notes, blogs like this, repositories, configuration files.

So I asked myself: what if I wanted my product team to reproduce the exact same demo? Or reproduce a tricky bug?

In a classic IaC world, I would share a configuration file. The other person would clone the repo, pull it down, and then start adjusting dozens of environment settings — region values, parameter overrides, warehouse sizes, credentials — before they could even run it.

It works, but it assumes their environment matches mine.

Now think about how humans do this.

If I explain a setup to you, you don’t copy it blindly. You adapt it to your context. You change the region. You adjust the resource sizes. You map it to your naming conventions. You interpret the intent, not just the configuration.

That’s exactly what the IaI pattern enables with shared manifests.

When I share a manifest, I’m not just sharing a file. I’m sharing intent.

That intent can then be materialized in your environment — with adaptation.

Same architectural goal. Different context.

This isn’t blind automation like traditional IaC. It’s careful intent materialization.

And that difference is subtle — but important.

The Future: The Velocity Catalog

What we’ve discussed so far is not just another pattern or a personal experiment. It’s a mental shift we need to make in this AI era.

snow-utils is simply an example of how this shift took shape for me. The same idea can apply to any tool or script.

The real opportunity is this: we need to mentor AI to think and act the way we would — embedding our experience, judgment, and operational discipline into it.

Maybe that’s the next step for us: inject intent into DevOps.

Resources

The HIRC demo is just one example. Here is the 4-minute time-lapse demo that shows how this pattern works:

The same pattern applies across workloads. In the demo repository, I’ve also included the AI Smart Crowd Counter.

All you have to say is:

“Deploy AI Smart Crowd Counter.”

The underlying primitives remain the same. The intent changes.

One moment you’re setting up a data catalog. The next, you’re provisioning a Streamlit app and an image-processing pipeline.

Same intent model. Different outcome.

That’s the leverage.

If you’re experimenting with AI agents in your DevOps workflows, start by stabilizing your primitives. Once they’re solid, you can begin layering Skills on top of them.

And from there, you can build your own Velocity Catalog.

For reference, the underlying orchestration layer used here is Snowflake Cortex Code:

👉 Snowflake Cortex Code CLI Documentation

Visuals note: The conceptual diagrams in this post were generated by Google Gemini.

About the Author

Kamesh Sampath is a Lead Developer Advocate at Snowflake, author, and long-time open-source contributor with 25+ years in enterprise software. He works across data engineering and AI with developer communities, helping practitioners turn modern data platforms into systems that hold up in production.

Through talks, writing, and hands-on demos, Kamesh makes cloud, data, and AI topics easier to understand and apply — grounded in real-world constraints. His sessions mix deep technical detail with practical patterns that developers and data teams can apply right away.

Lately, he’s been speaking about Apache NiFi (Snowflake Openflow), AI (Snowflake Cortex), and PostgreSQL.

He believes technology becomes powerful when it is shared, taught, and built together.

GitHub | LinkedIn | Blog | X

In today’s fast-paced digital landscape, organizations demand immediate access to information locked inside documents. Traditional batch processing pipelines introduce frustrating delays — users upload files but must wait for scheduled jobs before their data becomes searchable. This lag not only slows down business operations but also wastes resources and complicates error handling.

What if you could build a document processing pipeline that reacts instantly to new uploads?

Imagine a system where every PDF, invoice, or contract is detected, parsed, and made searchable within seconds — without manual intervention or complex scheduling.

This article shows you how to architect such an intelligent, real-time pipeline using Snowflake Streams and Cortex AI.

What You’ll Learn

By the end of this guide, you’ll know how to:

• Instantly detect new file uploads with zero polling
• Extract and chunk document content using AI-powered parsing
• Enable semantic search with vector embeddings
• Automate the entire workflow for reliability and scalability

Whether you’re modernizing invoice processing, legal document search, or any workflow that relies on fast document access, this guide will help you move from slow, error-prone batches to a responsive, AI-driven solution.

The Real Problem: When File Processing Takes Forever

Picture this: You’re building a document search system for invoice processing. Your finance team uploads an invoice PDF at 10:15 AM, but they can’t search it until 11:30 AM because your system processes files in hourly batches.

This is the reality for most organizations today:

-- Traditional batch processing (runs every hour)
SELECT * FROM documents 
WHERE upload_time > DATEADD(hour, -1, CURRENT_TIMESTAMP());

The Pain Points That Kill Productivity

1. Massive Latency

• User uploads invoice at 10:15 AM
• Batch job runs at 11:00 AM
• Invoice becomes searchable at 11:30 AM
• User waits 1 hour 15 minutes!

2. Resource Waste

• Batch processing scans everything
• Scans all files in the stage (e.g., 10,000 files)
• Only a few files (e.g., 5) are actually new since the last run
• Still processes all 10,000 files, wasting compute and time

3. Complex Failure Handling When one document fails in a batch of 1000, what happens?

• Skip it and lose the document?
• Retry the entire batch?
• Build complex error recovery logic?

4. Cost Explosion

• AI Processing: Called on all documents, not just new ones
• Compute: Runs large warehouses even for tiny changes
• Developer Time: Complex scheduling and monitoring

Why Streams Change Everything

This is where Snowflake Streams transform file processing. Instead of periodically scanning for changes, Streams automatically detect file uploads the moment they happen:

-- Stream automatically detects new documents
CREATE STREAM document_changes ON STAGE document_storage;

Then when you upload a PDF to the stage say “Invoice 1045.pdf” from then the query like the following will return the inserted document.

-- Only process actual changes
SELECT relative_path, METADATA$ACTION FROM document_changes;
-- Returns: "Invoice 1045.pdf", "INSERT" (instantly when uploaded)

The magic: Zero polling, zero scanning, instant detection.

Our Solution: Real-Time Processing Pipeline

Here’s what we’ll build to solve these problems:

1. Instant Detection: Stream detects file upload immediately
2. AI Extraction: Cortex parses PDF and extracts text
3. Smart Chunking: Breaks content into searchable segments
4. Vector Embeddings: Creates semantic search capability
5. Real-time Search: Document searchable in ~30 seconds

Let’s build it step by step using real invoice PDFs.

Setting Up the Foundation

NOTE:

If you dont have a Free Snowflake trial, get one signed up here . Please make sure you use AWS us-west-2 and Enterpise edition for the code snippets to work as-is

First, create the infrastructure for file processing:

-- Create database and warehouse
CREATE DATABASE IF NOT EXISTS PDF_PROCESSOR;
CREATE WAREHOUSE IF NOT EXISTS FILE_WH WAREHOUSE_SIZE = SMALL;

USE DATABASE PDF_PROCESSOR;
USE WAREHOUSE FILE_WH;
-- Stage for PDF storage with directory tables enabled
CREATE OR REPLACE STAGE pdf_docs
  ENCRYPTION = (TYPE = 'SNOWFLAKE_SSE') 
  DIRECTORY = (ENABLE = true);
-- Streams to detect file changes
CREATE OR REPLACE STREAM docs_inserted ON STAGE pdf_docs;
CREATE OR REPLACE STREAM docs_deleted ON STAGE pdf_docs;

🔑 Key Point: DIRECTORY = (ENABLE = true) automatically tracks file metadata and enables change detection on the stage.

Creating the AI Processing Pipeline

Next, set up tables and the AI processing logic:

-- Table to store processed document chunks
CREATE OR REPLACE TABLE document_chunks (
    file_name VARCHAR,
    chunk_text VARCHAR,
    chunk_number INT,
    processed_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP()
);

-- Stored procedure for AI-powered processing
CREATE OR REPLACE PROCEDURE process_new_documents()
RETURNS VARCHAR
LANGUAGE SQL
AS
$$
BEGIN
    -- Handle deleted documents first
    DELETE FROM document_chunks
    USING docs_deleted
    WHERE document_chunks.file_name = docs_deleted.relative_path
    AND docs_deleted.METADATA$ACTION = 'DELETE';
    -- Process new documents with AI
    CREATE OR REPLACE TEMPORARY TABLE parsed_docs AS
    SELECT
        relative_path,
        SNOWFLAKE.CORTEX.PARSE_DOCUMENT(
            '@pdf_docs',
            relative_path,
            {'mode': 'LAYOUT'}
        ):content AS extracted_text
    FROM docs_inserted
    WHERE METADATA$ACTION = 'INSERT';
    -- Intelligent chunking for better search
    INSERT INTO document_chunks (file_name, chunk_text, chunk_number)
    SELECT
        relative_path,
        chunk.value,
        chunk.index
    FROM parsed_docs,
    LATERAL FLATTEN(
        input => SNOWFLAKE.CORTEX.SPLIT_TEXT_RECURSIVE_CHARACTER(
            extracted_text,
            'markdown',
            1500,  -- chunk size
            200    -- overlap
        )
    ) chunk;
    RETURN 'Processed ' || (SELECT COUNT(DISTINCT file_name) FROM document_chunks) || ' documents';
END;
$$;

This procedure does three critical things:

1. Cleans up deleted documents automatically
2. Extracts text using Cortex AI’s PARSE_DOCUMENT function
3. Creates intelligent chunks optimized for search

Automation with Smart Tasks

Now make it fully automated with intelligent triggering:

-- Serverless Task runs when either stream has data
CREATE OR REPLACE TASK auto_process_docs
    SCHEDULE = '1 MINUTE'
WHEN
    SYSTEM$STREAM_HAS_DATA('docs_inserted') 
    OR SYSTEM$STREAM_HAS_DATA('docs_deleted')
AS
    CALL process_new_documents();

-- Enable the task
ALTER TASK auto_process_docs RESUME;

Why Two Separate Streams?

Streams become empty once their data is consumed in a DML operation. If we used a single stream for both INSERT and DELETE operations, consuming one type of change would empty the entire stream, potentially losing the other type of changes.

By using two streams:

• docs_inserted captures only INSERT operations (new files)
• docs_deleted captures only DELETE operations (removed files)
• Each can be consumed independently without affecting the other

The task triggers for:

• File uploads (docs_inserted has data) → Process new documents
• File deletions (docs_deleted has data) → Clean up old chunks
• Both operations → Handle mixed scenarios

This ensures your pipeline stays perfectly synchronized with the actual files in the stage.

Testing with Real Data

Let’s test with sample invoice PDFs. Use Snowsight Load Files into a Stage feature to upload invoice files form ./scripts/data folder. Say for example upload the following invoices “Invoice 1045.pdf”, “Invoice 2348.pdf” and “Invoice 3943.pdf”.

Check if the stream detected them:

-- See what the stream captured
SELECT 
    relative_path,
    size,
    last_modified,
    METADATA$ACTION
FROM docs_inserted;

You should see ( other columns trimmed for brevity):

RELATIVE_PATH   | SIZE |  LAST_MODIFIED             | METADATA$ACTION
Invoice 1045.pdf|58850|2025-06-20 07:11:40.000 -0700|INSERT
Invoice 2348.pdf|59213|2025-06-20 07:11:40.000 -0700|INSERT
Invoice 3943.pdf|59008|2025-06-20 07:11:40.000 -0700|INSERT

Within 1–2 minutes, the automated task will process these files. View the Task History for the actual status. Check the results:

-- View processed chunks
SELECT 
    file_name,
    COUNT(*) as chunk_count,
    AVG(LENGTH(chunk_text)) as avg_chunk_size
FROM document_chunks
GROUP BY file_name;

Adding Intelligent Search

Now add semantic search capabilities:

-- Create search service with vector embeddings
CREATE OR REPLACE CORTEX SEARCH SERVICE invoice_search_service
    ON chunk_text
    WAREHOUSE = 'FILE_WH'
    TARGET_LAG = '1 MINUTE'
    EMBEDDING_MODEL = 'snowflake-arctic-embed-l-v2.0'
AS (
    SELECT file_name, chunk_text
    FROM document_chunks
);

Once the Search Service has been created, you can find the status of the same via the Search Service Listing in Snowsight.

Query the Search Service via REST API

Get the Snowflake Keypair Authentication JWT keys, generated using Snowflake CLI:

export CORTEX_SEARCH_JWT=$(snow connection generate-jwt)

TIP:

You can also use Snowflake Programmatic Access Token(PAT) over JWT . The repository has a pat.sh that can be used to generate Service User with PAT.

For production applications, use the REST API to search your documents:

# Search for payment-related content
curl --location https://your-account.snowflakecomputing.com/api/v2/databases/PDF_PROCESSOR/schemas/PUBLIC/cortex-search-services/invoice_search_service:query \
    --header 'X-Snowflake-Authorization-Token-Type: KEYPAIR_JWT' \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header "Authorization: Bearer $CORTEX_SEARCH_JWT" \
    --data '{
        "query": "payment terms and due dates",
        "columns": ["chunk_text", "file_name"],
        "limit": 5
    }'

# Search for vendor information
curl --location https://your-account.snowflakecomputing.com/api/v2/databases/PDF_PROCESSOR/schemas/PUBLIC/cortex-search-services/invoice_search_service:query \
    --header 'X-Snowflake-Authorization-Token-Type: KEYPAIR_JWT' \
    --header 'Content-Type: application/json' \
    --header 'Accept: application/json' \
    --header "Authorization: Bearer $CORTEX_SEARCH_JWT" \
    --data '{
        "query": "supplier vendor company name",
        "columns": ["chunk_text", "file_name"],
        "limit": 5
    }'

Alternative: SQL-based Testing

For quick testing within Snowflake:

-- query the document chunks
SELECT PARSE_JSON(
  SNOWFLAKE.CORTEX.SEARCH_PREVIEW(
      'invoice_search_service',
      '{
         "query": "payment terms and due dates",
         "columns":[
            "file_name",
            "chunk_text"
         ],
         "filter": {"@eq": {"file_name": "Invoice 2348.pdf"} },
         "limit":5
      }'
  )
)['results'] as results;

Advanced AI Features

Extract structured data using Cortex AISQL functions:

-- Extract key invoice information
SELECT 
    file_name,
    SNOWFLAKE.CORTEX.COMPLETE(
        'claude-4-sonnet',
        CONCAT('What is the invoice number and total amount? Use the context between <context>.Format as markdown.', chunk_text,'</context>')
    ) as invoice_and_total
FROM document_chunks
LIMIT 10;

Production Optimizations

For production workloads, consider these enhancements:

Smart Warehouse Sizing

-- Different warehouses for different loads
CREATE WAREHOUSE FILE_WH_SMALL WAREHOUSE_SIZE = 'SMALL' AUTO_SUSPEND = 60;
CREATE WAREHOUSE FILE_WH_LARGE WAREHOUSE_SIZE = 'LARGE' AUTO_SUSPEND = 300;

Error Handling

-- Enhanced procedure with error handling
CREATE OR REPLACE PROCEDURE process_new_documents_safe()
RETURNS VARCHAR
LANGUAGE SQL
AS
$$
DECLARE
    error_count INT DEFAULT 0;
    success_count INT DEFAULT 0;
BEGIN
    FOR doc IN (SELECT relative_path FROM docs_inserted WHERE METADATA$ACTION = 'INSERT') DO
        BEGIN
            -- Process individual document
            INSERT INTO document_chunks (file_name, chunk_text)
            SELECT doc.relative_path, 'Processed: ' || doc.relative_path;
            SET success_count = success_count + 1;
        EXCEPTION
            WHEN OTHER THEN
                INSERT INTO processing_errors (file_name, error_message, error_time)
                VALUES (doc.relative_path, SQLERRM, CURRENT_TIMESTAMP());
                SET error_count = error_count + 1;
        END;
    END FOR;
    
    RETURN 'Success: ' || success_count || ', Errors: ' || error_count;
END;
$$;

From Problem to Solution

What We Started With

Remember our original problems:

• 1 hour 15 minutes from upload to searchable
• Processing 10,000 files when only 5 are new
• Complex batch failure recovery
• High costs from redundant processing

What We Built

A real-time file processing pipeline using Snowflake Streams + Cortex AI that:

1. Detects uploads instantly with Stream-based change tracking
2. Processes only new files using METADATA$ACTION filtering
3. Extracts text intelligently with Cortex PARSE_DOCUMENT
4. Creates semantic search using vector embeddings
5. Handles errors gracefully with automatic retry logic
6. Scales automatically based on actual workload

What We Achieved

Before:

• Upload PDF at 10:15:23 AM
• Wait for batch job at 11:00 AM
• Document searchable at 11:30 AM
• Total time: 1 hour 15 minutes

After:

• Upload PDF at 10:15:23 AM
• Stream detects at 10:15:23 AM (instant)
• AI processing at 10:15:25 AM (2 seconds later)
• Searchable at 10:15:55 AM (30 seconds total)

The Real Business Impact

For most organizations, moving from batch to real-time processing delivers:

💰 Significant Time Savings
Drastically reduces the delay between file upload and searchability

📉 Lower Operational Costs
Cuts down on unnecessary compute and AI processing

⚡ Improved User Experience
Enables instant access to newly uploaded documents

🚀 Higher Developer Productivity
Eliminates the need for complex batch job management and error handling

Key Technical Innovations

Streams for Instant Detection

CREATE STREAM docs_inserted ON STAGE pdf_docs;
-- Zero polling, instant file upload detection

AI-Powered Processing

PARSE_DOCUMENT('@pdf_docs', file_path, {'mode': 'LAYOUT'})
-- Understands complex PDF layouts, tables, and structures

Smart Automation

CREATE TASK auto_process_docs
WHEN SYSTEM$STREAM_HAS_DATA('docs_inserted') 
    OR SYSTEM$STREAM_HAS_DATA('docs_deleted')
-- Triggers for both file additions AND deletions

Why This Matters

This isn’t just about faster file processing. It’s about fundamentally changing how we build data pipelines:

✅ Event-driven architecture instead of scheduled batches
✅ AI-first processing that understands content semantically
✅ Cost optimization through precise change detection
✅ Real-time user experiences in data applications
✅ Self-healing systems that handle failures automatically

The combination of Snowflake Streams for change detection and Cortex AI for intelligent processing creates a new paradigm for document workflows that’s both more efficient and more capable than traditional approaches.

Try It Yourself

Ready to transform your file processing from batch delays to real-time intelligence?

The complete code is available in this GitHub repository. You can:

1. Clone the repo and run the setup scripts
2. Upload your own PDFs to test the pipeline
3. Customize the AI prompts for your document types
4. Scale to production with the optimization patterns

Start building with Snowflake Streams and Cortex AI today.

Further Reading

• Introduction to Streams in Snowflake
• Snowflake Cortex AI SQL Reference
• Snowflake Search Service

If you found this guide helpful, give it a clap 👏 and follow me for more practical guides on real-time data engineering and AI-driven automation.

About the Author
Kamesh Sampath is Lead Developer Advocate at Snowflake and a veteran technology evangelist with over 20 years of experience in cloud data platforms, AI, and developer productivity. He has helped organizations modernize their data and AI workflows using Snowflake, Kubernetes, and serverless technologies, and has been an active open source contributor for more than a decade.

Connect with Kamesh:
🔗 LinkedIn
💻 GitHub
🐦 Twitter

Snowflake’s Programmatic Access Tokens (PAT) offer a modern, secure way to automate workflows and integrate with CI/CD pipelines — without relying on passwords or key pairs. In this post, you’ll learn how to enable PAT authentication in Snowflake, configure your environment, and use PATs with the Snowflake CLI for seamless DevOps automation.

Why Use Programmatic Access Tokens (PAT) in Snowflake?

• Security: PATs are scoped, time-limited, and can be revoked at any time.
• Automation: Ideal for CI/CD, GitHub Actions, and local scripts.
• Flexibility: Easily restrict PATs to specific roles and network policies.

Prerequisites

• A Snowflake Trial Account (if you don’t have one)
• A Snowflake user with ACCOUNTADMIN or equivalent privileges
• Snowflake CLI (v3.9.0 or higher)
• jq (for JSON parsing)
• Github CLI ( for interacting with Github)

Sources

The complete demo sources is available at https://github.com/kameshsampath/snowflake-pat-demo

Step 1: Enable PAT Authentication Policy in Snowflake

First, ensure your Snowflake environment supports PATs and set up an authentication policy with the desired settings. Here’s an example SQL snippet:

Set the following variables:

# your Snowflake account user
export SNOWFLAKE_USER="<your snowflake account user>"
# this we will update once we have the PAT and setup local connection
export SNOWFLAKE_DEFAULT_CONNECTION_NAME="< your snowflake connection name>"
export MY_PAT_DEMO_ROLE="< the role to use to restrict PAT>"
# if your are using KeyPair Auth with Snowflake then set if your key is encrypted
# export PRIVATE_KEY_PASSPHRASE="your private keypassword"    

Let us prepare the Snowflake environment for the Demo

#!/usr/bin/env sh

set -euo pipefail

snow sql --stdin <<EOF
-- create demo database to hold all our demo objects
create or replace database my_demo_db;
use database my_demo_db;
-- schema to hold all the policies
create or replace schema policies;
-- schema to hold all network rules
create or replace schema networks;
-- Create or alter the authentication policy
create or alter authentication policy my_demo_db.policies.demos_auth_policy
  authentication_methods = ('PASSWORD', 'OAUTH', 'KEYPAIR', 'PROGRAMMATIC_ACCESS_TOKEN')
  pat_policy = (
    default_expiry_in_days=7,
    max_expiry_in_days=90,
    network_policy_evaluation = ENFORCED_NOT_REQUIRED
  );
EOF

What does this do?

• Allows multiple authentication methods, including PATs.
• Sets PAT default expiry to 7 days, max to 90 days.
• Enforces network policy evaluation for added security.

Run the following script to create those resources:

curl -Ls \
  https://raw.githubusercontent.com/kameshsampath/snowflake-pat-demo/refs/heads/main/prepare.sh \
  | sh

Step 2: Generate a Personal Access Token (PAT)

You can create a PAT for your Snowflake user and restrict it to a specific role. Here’s how to do it using the Snowflake CLI and jq:

export SNOWFLAKE_PASSWORD=$(snow sql \
  --query "ALTER USER IF EXISTS $SNOWFLAKE_USER ADD PAT my_demo_pat \
   ROLE_RESTRICTION = $MY_PAT_DEMO_ROLE" --format=json \
   | jq -r '.[] | .token_secret')

• This command creates a PAT named my_demo_pat for your user, restricted to the $MY_PAT_DEMO_ROLE.
• The token is stored in the SNOWFLAKE_PASSWORD environment variable for use with the CLI.

Verify if the PAT has been created successfully

snow sql -q "show pats for user $SNOWFLAKE_USER"

Step 3: Configure the Snowflake CLI to Use PAT

To ensure our PAT works let us reconfigure the Snowflake CLI with PAT. To ensure we dont corrupt existing configuration we will setup a new Snowflake configuration home that the CLI can use.

1. Test your connection and export details:

snow connection test --format=json > "$PWD/connection.json"
export SNOWFLAKE_ACCOUNT=$(jq -r '.Account' "$PWD/connection.json")
export SNOWFLAKE_USER=$(jq -r '.User' "$PWD/connection.json")
export SNOWFLAKE_HOST=$(jq -r '.Host' "$PWD/connection.json")

2. Verify PAT using Temporary Connection:

snow sql --temporary-connection --account="${SNOWFLAKE_ACCOUNT}" \
  --host="${SNOWFLAKE_HOST}" \
  --user="${SNOWFLAKE_USER}" \
  --query "select current_user(), current_role(), current_database(), current_schema()"

The connection fails with an error “Invalid connection configuration. 250001 (08001): None: Failed to connect to DB: <your host>:443. Fail : Network policy is required.” It indicates that the Snowflake user used as part of the connection does not have any “network_policy”associated.

4. Create and Set Network Access Policy on User

In the following example, we create a network policy that allows access from your local IP address only.

Let us first build CIDRs list, for an example we will use GitHub Actions CIDRs combined with the your local machine public IP .

#!/usr/bin/env sh

set -euo pipefail 

# Get GitHub Actions IP ranges only IPV4
GH_CIDRS=$(curl -s https://api.github.com/meta | jq -r '.actions | map(select(contains(":") | not)) | map("'\''" + . + "'\''") | join(",")')

# Get local IP and add /32 suffix
LOCAL_IP="$(dig +short myip.opendns.com @resolver1.opendns.com)/32"

# Combine GitHub CIDRs and local IP
CIDR_VALUE_LIST="${GH_CIDRS},'${LOCAL_IP}'"

snow sql --stdin <<EOF
use role $MY_PAT_DEMO_ROLE;
-- create a new network rule that allows all CIDR (for demo only)
create network rule if not exists my_demo_db.networks.pat_gh_actions_local_access_rule
mode = ingress
type = ipv4
value_list = ($CIDR_VALUE_LIST)
comment = 'Allow only GitHub Actions and local machine IPv4 addresses';

use role accountadmin;
-- attach the network rule to the network policy
create network policy if not exists GH_ACTIONS_PAT_NETWORK_POLICY
allowed_network_rule_list = ('my_demo_db.networks.at_gh_actions_local_access_rule')
comment = 'Network policy to allow all IPv4 addresses.';

-- set the network policy to the user to allow using pat with snowflake cli
alter user $SNOWFLAKE_USER set network_policy='GH_ACTIONS_PAT_NETWORK_POLICY';
EOF

A special note of thanks to Myles Borins for the tip on getting GitHub Actions CIDRs

Execute the script

curl -Ls 
   https://raw.githubusercontent.com/kameshsampath/snowflake-pat-demo/refs/heads/main/create_network_rule.sh \
   | sh

Now running the same connection test we did earlier, will succeed.

snow sql --temporary-connection --account="${SNOWFLAKE_ACCOUNT}" \
  --host="${SNOWFLAKE_HOST}" \
  --user="${SNOWFLAKE_USER}" \
  --query "select current_user(), current_role(), current_database(), current_schema()"

GitHub Actions Integration

Fork and clone repository to your local machine.

gh repo fork --clone https://github.com/kameshsampath/snowflake-pat-demo.git
cd snowflake-pat-demo

Then, set up the GitHub secrets for your repository:

gh secret set SNOWFLAKE_PASSWORD --body "$SNOWFLAKE_PASSWORD"
gh secret set SNOWFLAKE_ACCOUNT --body "$SNOWFLAKE_ACCOUNT"
gh secret set SNOWFLAKE_USER --body "$SNOWFLAKE_USER"
gh secret set SNOWFLAKE_HOST --body "$SNOWFLAKE_HOST"

Now just do a empty commit and push to trigger the GitHub Actions workflow.

git commit - allow-empty -m "Trigger GitHub Actions"
git push

This will run the workflow defined in `.github/workflows/snowflake-pat-demo.yml`, which will use the PAT to connect to Snowflake and perform operations

Cleanup

If you want to clean up all the resources that were created during this demo,

#!/usr/bin/env sh
snow sql --stdin <<EOF
use role accountadmin;
alter user ${SNOWFLAKE_USER} unset network_policy;
alter user ${SNOWFLAKE_USER} remove pat my_demo_pat;

drop network policy if exists GH_ACTIONS_PAT_NETWORK_POLICY;
use role $MY_PAT_DEMO_ROLE;
drop database if exists my_demo_db;
EOF

gh secret delete SNOWFLAKE_PASSWORD
gh secret delete SNOWFLAKE_ACCOUNT
gh secret delete SNOWFLAKE_USER
gh secret delete SNOWFLAKE_HOST

rm -rf "$PWD/connection.json"

Run the script

curl -Ls \
  https://raw.githubusercontent.com/kameshsampath/snowflake-pat-demo/refs/heads/main/cleanup.sh \
  | sh

Security Best Practices

Authentication

• Use role restrictions with PATs to limit scope
• Set appropriate expiration periods (recommended: 7–30 days)
• Rotate PATs regularly and before expiration

Network Security

• Always use HTTPS for API calls
• Restrict network access to specific IP ranges in production
• Regularly audit and update allowed IP ranges

Credential Management

• Store PATs securely using secret managers
• Never commit PATs to version control
• Use separate PATs for different applications/services

Conclusion

Programmatic Access Tokens make it easy and secure to automate Snowflake workflows, especially in DevOps and CI/CD scenarios. By following the steps above, you can enable, generate, and use PATs with the Snowflake CLI — unlocking powerful automation for your data projects.

Happy automating with Snowflake!

Further Reading

• Snowflake: Personal Access Tokens (PAT)
• Snowflake CLI Authentication Methods
• Snowflake: CREATE AUTHENTICATION POLICY
• Snowflake: Managing Users and Roles
• Snowflake: Network Policies
• Snowflake CLI GitHub Repository
• jq: Command-line JSON processor
• direnv: Environment Variable Management