NGINX Configs to Reference Remote External Files#1389

Akshay2191 · 2025-11-12T15:50:37Z

Extends the ConfigApply capability to manage NGINX configurations that reference external resources hosted at remote URLs. The agent now handles downloading of these files into the NGINX configuration directory before applying the new configuration.

Checklist

Before creating a PR, run through this checklist and mark each as complete.

I have read the CONTRIBUTING document
I have run make install-tools and have attached any dependency changes to this pull request
If applicable, I have added tests that prove my fix is effective or that my feature works
If applicable, I have checked that any relevant tests pass after adding my changes
If applicable, I have updated any relevant documentation (README.md)
If applicable, I have tested my cross-platform changes on Ubuntu 22, Redhat 8, SUSE 15 and FreeBSD 13

codecov · 2025-11-12T16:03:00Z

Codecov Report

❌ Patch coverage is 80.74324% with 57 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.05%. Comparing base (02f4c36) to head (c74c4dc).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/file/external_file_operator.go	79.67%	19 Missing and 19 partials ⚠️
internal/file/file_manager_service.go	71.79%	9 Missing and 2 partials ⚠️
internal/file/file_plugin.go	0.00%	4 Missing ⚠️
internal/config/config.go	93.75%	2 Missing and 1 partial ⚠️
internal/file/file_service_operator.go	94.44%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1389      +/-   ##
==========================================
- Coverage   85.13%   85.05%   -0.09%     
==========================================
  Files         102      103       +1     
  Lines       13217    13487     +270     
==========================================
+ Hits        11252    11471     +219     
- Misses       1475     1506      +31     
- Partials      490      510      +20

Files with missing lines	Coverage Δ
internal/config/defaults.go	`100.00% <ø> (ø)`
internal/config/flags.go	`100.00% <ø> (ø)`
internal/config/types.go	`87.65% <ø> (ø)`
internal/model/config.go	`90.47% <ø> (ø)`
internal/file/file_service_operator.go	`77.46% <94.44%> (+0.64%)`	⬆️
internal/config/config.go	`87.70% <93.75%> (+0.26%)`	⬆️
internal/file/file_plugin.go	`75.08% <0.00%> (-0.74%)`	⬇️
internal/file/file_manager_service.go	`74.48% <71.79%> (-0.76%)`	⬇️
internal/file/external_file_operator.go	`79.67% <79.67%> (ø)`

... and 1 file with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 02f4c36...c74c4dc. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

internal/config/config.go

internal/config/config_test.go

internal/file/file_manager_service.go

internal/file/file_manager_service_test.go

internal/config/config.go

internal/config/defaults.go

internal/file/file_service_operator.go

internal/file/file_manager_service.go

craigell · 2025-11-25T12:32:04Z

internal/file/file_service_operator_test.go

+		wantErrMsg string
+		wantErr    bool
+	}{
+		{


The following scenarios could increase coverage:

No destination specified

Restricted directory.

Two files to the same destination.

aphralG · 2025-11-26T11:25:00Z

internal/file/file_service_operator_test.go

+	}{
+		{
+			name: "Test 1: success",
+			prepare: func(t *testing.T) (string, string) {


Can this be changed to follow the way other tests are set up and formatted

internal/config/defaults.go

aphralG · 2025-11-26T11:29:04Z

internal/file/file_manager_service_test.go

+}
+
+//nolint:gocognit,revive,govet // cognitive complexity is 22
+func TestFileManagerService_downloadExternalFiles_Cases(t *testing.T) {


Can this test be simplified and changed to match the way other tests are set up and structured

aphralG · 2025-11-26T11:29:59Z

internal/file/file_manager_service_test.go

+	assert.Equal(t, fileName, dstArg, "RenameExternalFile destination argument mismatch")
+}
+
+func TestDownloadFileContent_MaxBytesLimit(t *testing.T) {


Can these tests please be combined

I tried to consolidate these divergent test scenarios (MaxBytes vs. Proxy Errors) but it spiked the cognitive complexity to 23 due the very different validation steps. Therefore I kept them seperate.

internal/file/file_manager_service_test.go

internal/file/file_manager_service.go

internal/config/config.go

# Conflicts: # internal/config/config.go # internal/config/types.go # internal/file/file_manager_service.go

# Conflicts: # internal/file/file_manager_service.go

github-actions · 2026-01-09T17:33:51Z

✅ All required contributors have signed the F5 CLA for this PR. Thank you!
_{Posted by the CLA Assistant Lite bot.}

internal/config/config_test.go

internal/file/file_manager_service_test.go

internal/file/file_manager_service.go

internal/file/external_file_operator.go

internal/config/config.go

internal/file/external_file_operator.go

internal/file/file_manager_service.go

dhurley · 2026-01-19T14:38:18Z

internal/config/config.go

+	for _, domain := range domains {
+		// Validating syntax using the RFC-compliant regex
+		if !domainRegex.MatchString(domain) || domain == "" {
+			slog.Error("domain specified in allowed_domains is invalid", "domain", domain)


Can you remove this log line and have a log message in the resolveExternalDataSource function instead?

dhurley · 2026-01-19T14:40:09Z

internal/config/config.go

+	}
+
+	if err := validateAllowedDomains(externalDataSource.AllowedDomains); err != nil {
+		return nil


Can you add a log line here like this

slog.Error("External data source not configured due to invalid configuration", "error", err)

dhurley · 2026-01-19T14:54:29Z

internal/file/external_file_operator.go

+	permission := fileAction.File.GetFileMeta().GetPermissions()
+	fileName := fileAction.File.GetFileMeta().GetName()
+
+	slog.InfoContext(ctx, "Downloading external file from", "location", location)


Suggested change

slog.InfoContext(ctx, "Downloading external file from", "location", location)

slog.InfoContext(ctx, "Downloading external file", "location", location)

dhurley · 2026-01-19T15:00:09Z

internal/file/external_file_operator.go

+		headers.ETag = resp.Header.Get("ETag")
+		headers.LastModified = resp.Header.Get("Last-Modified")
+	case http.StatusNotModified:
+		slog.DebugContext(ctx, "File content unchanged (304 Not Modified)", "file_name", fileName)


Suggested change

slog.DebugContext(ctx, "File content unchanged (304 Not Modified)", "file_name", fileName)

slog.InfoContext(ctx, "Received 304 status code in response from external data source, file has not been modified", "file_name", fileName)

I think this would be better as info instead for debug

dhurley · 2026-01-20T10:58:18Z

internal/file/file_manager_service.go

 		case model.Delete:
-			slog.InfoContext(ctx, "Deleting file", "file", fileAction.File.GetFileMeta().GetName())
-			if err := os.Remove(fileAction.File.GetFileMeta().GetName()); err != nil && !os.IsNotExist(err) {
+			slog.DebugContext(ctx, "Deleting file", "file", fileMeta.GetName())


Suggested change

slog.DebugContext(ctx, "Deleting file", "file", fileMeta.GetName())

slog.InfoContext(ctx, "Deleting file", "file", fileMeta.GetName())

dhurley · 2026-01-20T11:00:48Z

internal/file/file_plugin.go

 		if rollbackErr != nil {
+			// include both the original apply error and the rollback error so the management plane
+			// receives actionable information about what failed during apply and what failed during rollback
+			combinedErr := fmt.Sprintf("apply error: %s; rollback error: %s", err.Error(), rollbackErr.Error())


Would it be better to use errors.Join function here?

Akshay2191 requested a review from a team as a code owner November 12, 2025 15:50

github-actions bot added the chore Pull requests for routine tasks label Nov 12, 2025

Akshay2191 self-assigned this Nov 19, 2025

aphralG reviewed Nov 20, 2025

View reviewed changes

dhurley reviewed Nov 21, 2025

View reviewed changes

Akshay2191 requested review from aphralG and dhurley November 24, 2025 16:22

craigell reviewed Nov 25, 2025

View reviewed changes

aphralG reviewed Nov 26, 2025

View reviewed changes

spencerugbo reviewed Dec 4, 2025

View reviewed changes

internal/file/file_manager_service.go Outdated Show resolved Hide resolved

internal/config/config.go Show resolved Hide resolved

dhurley approved these changes Dec 10, 2025

View reviewed changes

github-actions bot added dependencies documentation Improvements or additions to documentation labels Dec 18, 2025

Akshay2191 and others added 16 commits December 18, 2025 23:40

third party config support

e055610

# Conflicts: # internal/config/config.go # internal/config/types.go # internal/file/file_manager_service.go

lint fix

daf36e3

Added a generic file timeout

2b878a4

added generic timeout to create connection

c357355

added uT

d1e8a9d

Added UT

7e616fc

fix lint issues

1fd2cc6

Added UT for Proxy URL

ac5bd5e

increasing UT coverage

280ed4f

updated Mock management to test external file config apply

e494337

PR review comments addressed

28328a4

worked on PR comments

d77faea

worked on PR comments

7b59e69

removed duplicate rename function

2281b14

worked on review comment

3843ee2

Ensure only files are in file overview in config apply request (#1395)

480e402

# Conflicts: # internal/file/file_manager_service.go

Akshay2191 added 3 commits December 19, 2025 00:11

linter fix

4837339

code fix for rebase

3225ff3

fixing rebase errors

2e981e0

Akshay2191 force-pushed the external-file-mgmt-integration branch from e50d100 to 2e981e0 Compare December 19, 2025 00:30

Akshay2191 added 2 commits January 6, 2026 15:21

refactored new code andded file validation

c28e679

fixed push error

e4cfe0c

Akshay2191 marked this pull request as draft January 6, 2026 15:26

Fixed UT test failing in git pipeline

5b8b5ab

Akshay2191 changed the title ~~NGINX Configs to Reference Remote External Files~~ Draft : NGINX Configs to Reference Remote External Files Jan 7, 2026

returning error message with code

99042ad

Akshay2191 force-pushed the external-file-mgmt-integration branch from 52e6aa3 to 99042ad Compare January 12, 2026 11:24

returning the config apply error with rollback error

3777ce0

Akshay2191 changed the title ~~Draft : NGINX Configs to Reference Remote External Files~~ NGINX Configs to Reference Remote External Files Jan 15, 2026

Akshay2191 marked this pull request as ready for review January 15, 2026 10:37

aphralG reviewed Jan 15, 2026

View reviewed changes

worked on review comments

13f6d1f

valyria257 reviewed Jan 15, 2026

View reviewed changes

internal/config/config.go Show resolved Hide resolved

internal/config/config.go Outdated Show resolved Hide resolved

internal/file/external_file_operator.go Outdated Show resolved Hide resolved

internal/file/file_manager_service.go Show resolved Hide resolved

Akshay2191 and others added 2 commits January 19, 2026 10:54

Merge branch 'main' into external-file-mgmt-integration

4c4df84

review comment work

ff2ae47

Akshay2191 requested review from aphralG, craigell, dhurley and valyria257 January 19, 2026 16:03

dhurley reviewed Jan 20, 2026

View reviewed changes

valyria257 approved these changes Jan 20, 2026

View reviewed changes

review comments update

c74c4dc

craigell approved these changes Jan 23, 2026

View reviewed changes

Akshay2191 merged commit 9e67328 into main Jan 23, 2026
66 of 70 checks passed

Akshay2191 deleted the external-file-mgmt-integration branch January 23, 2026 12:18

	slog.InfoContext(ctx, "Downloading external file from", "location", location)
	slog.InfoContext(ctx, "Downloading external file", "location", location)

	slog.DebugContext(ctx, "File content unchanged (304 Not Modified)", "file_name", fileName)
	slog.InfoContext(ctx, "Received 304 status code in response from external data source, file has not been modified", "file_name", fileName)

	slog.DebugContext(ctx, "Deleting file", "file", fileMeta.GetName())
	slog.InfoContext(ctx, "Deleting file", "file", fileMeta.GetName())

Conversation

Akshay2191 commented Nov 12, 2025

Checklist

Uh oh!

codecov bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Jan 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

codecov bot commented Nov 12, 2025 •

edited

Loading

github-actions bot commented Jan 9, 2026 •

edited

Loading