Skip to content

Add .sql template extension rendering for BigQueryInsertJobOperator #22859

Description

@LennyKLB

Description

Add .sql as value to template_ext in airflow.providers.google.cloud.operators.bigquery.BigQueryInsertJobOperator

Use case/motivation

It would be helpful if Jinja would also read .sql templates in airflow.providers.google.cloud.operators.bigquery.BigQueryInsertJobOperator, especially for its configuration.query.query parameter. It will render that parameter as sql correctly, but only the outer configuration body -- which is based on Bigquery/rest/v2/Job#jobconfiguration -- is considered for extension-based rendering and only json is considered.

I think there is a clear use-case to support templating SQL-only files. I know that BigQueryExecuteQueryOperator has this but it's deprecated in favour of BigQueryInsertJobOperator.

I currently work around this using a really basic override:

class BigQueryTemplateFileOperator(BigQueryInsertJobOperator):
    template_ext = (".json", ".sql")

However, I'd like to avoid having to replicate this operator everywhere in our codebase if this might actually already be implemented soon.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions