Feature: Add data loader reload API for development environments

[berngp]

Pull Request type

• Feature

Changes in this PR

DGS Data Loader Reload Support

Implements programmatic mechanism to force reloading of Data Loaders, to be used during development without application restart. The implementation provides no overhead when DGS Reloading is disabled. When enabled, it wrapps the current implementation, no called DefaultDgsDataLoaderProvider in such a way that a new one is created when the reload is invoked via DgsDataLoaderReloadController. It supports either reloading by the current application context or a new one provided through the force reload api.

Features:

• DgsDataLoaderReloadController public API with statistics tracking
• ReloadableDgsDataLoaderProvider wrapping DefaultDgsDataLoaderProvider, note that the DgsDataLoaderProvider was renamed as DefaultDgsDataLoaderProvider.
• @ConditionalOnDgsReload annotation for development-only activation
• Zero overhead when disabled (default for non-laptop profiles)
• Symmetric functionality with existing schema reload while maintaining safety and not affecting performance characteristics.

Alternatives considered

Spring Cloud RefreshScope

As an alternative we could leverage Spring Cloud's @RefreshScope mechanism versus implementing a custom reload solution. This will require a re-architecture of the current supported for the schema, broader changes to the DGS Spring Boot integration, and a direct dependency on Spring Cloud.

Spring Cloud RefreshScope Overview

Core Characteristics:

• Creates lazy proxies that initialize when methods are called
• Acts as a cache of initialized values that can be invalidated
• Enables dynamic reconfiguration via configuration property changes
• Supports re-binding of @ConfigurationProperties beans
• Triggered via /refresh endpoint or external configuration changes

Lifecycle Management:

• Beans are destroyed and recreated when refresh occurs
• Proxy wrapper remains but target bean instance changes
• Works primarily with configuration-driven changes

Architectural Decision

Aspect	Spring Cloud RefreshScope	DGS Custom Implementation	Decision
Scope	Any Spring bean	Data loaders only	Custom - Precise control needed
Trigger Mechanism	Configuration changes	Programmatic control	Custom - Development-focused
Performance	Proxy overhead always	Zero overhead when disabled	Custom - Better performance
Thread Safety	Framework-managed complexity	Explicit synchronization	Custom - Predictable behavior
Use Case Alignment	External config management	Development hot-reload	Custom - Purpose-built
Dependencies	Spring Cloud ecosystem	Core DGS only	Custom - Minimal dependencies

Using the DgsDataLoaderReloadController

Spring Cloud Example

The current implementation provides an easy way to leverage Spring Cloud's reload mechanism. This could be done as follows...

import org.springframework.cloud.context.scope.refresh.RefreshScopeRefreshedEvent;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;

@Component
public class DgsDataLoaderRefreshLifecycleListener implements ApplicationListener<RefreshScopeRefreshedEvent> {

    private DgsDataLoaderReloadController controller;

    @Override
    public void onApplicationEvent(RefreshScopeRefreshedEvent event) {
        controller.reloadDataLoaders(event.getApplicationContext());
    }
}

or

import org.springframework.cloud.context.scope.refresh.RefreshScopeRefreshedEvent;
import org.springframework.context.event.EventListener;
import org.springframework.stereotype.Component;

@Component
public class RefreshEventHandler {

    private DgsDataLoaderReloadController controller;

    @EventListener
    public void handleRefresh(RefreshScopeRefreshedEvent event) {
        controller.reloadDataLoaders(event.getApplicationContext());
    }
}

[paulbakker]

Thanks for this @berngp. Added a few comments, will test further as well.

[paulbakker]

Why do we need an external signal here instead of always reloading like we do for the schema? (when reload is enabled)

[berngp]

In this case the signal is needed to trigger the post-construct of the DefaultDgsDataLoaderProvider (previously named as DgsDataLoaderProvider). This also allows us to pass a new, or refreshed, application context that will be used to discover the new versions of the Data Loaders.

This also allowed me to minimize the changes to the current DgsDataLoaderProvider in main, just changing its name to DefaultDgsDataLoaderProvider and the interface that it implements. The controller allows us to understand when the DefaultDgsDataLoaderProvider needs to be rebuilt, included the execution of the post-construct.

[paulbakker]

This shows as a new file in the diff, but is this a one-to-one copy of the existing code?

[berngp]

It is, the name changed as well as implementing DgsDataLoaderProvider, but nothing else. I wish the the diff was a bit smarter.

[berngp]

Updated the PR, these are the changes agains the original file

• name change from DgsDataLoaderProvider to DefaultDgsDataLoaderProvider.
• adopt the DgsDataLoaderProvider.
• add the override for the interface APIs.
• make sure that the dataloaders know their names when they are created.

58c58
< class DgsDataLoaderProvider(
---
> class DefaultDgsDataLoaderProvider(
66c66
< ) {
---
> ) : DgsDataLoaderProvider {
80c80
<     fun buildRegistry(): DataLoaderRegistry = buildRegistryWithContextSupplier { null }
---
>     override fun buildRegistry(): DataLoaderRegistry = buildRegistryWithContextSupplier { null }
82c82
<     fun <T> buildRegistryWithContextSupplier(contextSupplier: Supplier<T>): DataLoaderRegistry {
---
>     override fun <T> buildRegistryWithContextSupplier(contextSupplier: Supplier<T>): DataLoaderRegistry {
232c232
<         return DataLoaderFactory.newDataLoader(extendedBatchLoader, options.build())
---
>         return DataLoaderFactory.newDataLoader(dataLoaderName, extendedBatchLoader, options.build())
270c270
<         return DataLoaderFactory.newDataLoader(extendedBatchLoader, options.build())
---
>         return DataLoaderFactory.newDataLoader(dataLoaderName, extendedBatchLoader, options.build())
291c291
<         return DataLoaderFactory.newMappedDataLoader(extendedBatchLoader, options.build())
---
>         return DataLoaderFactory.newMappedDataLoader(dataLoaderName, extendedBatchLoader, options.build())
302c302,310
<                 is BatchLoader<*, *> -> createDataLoader(holder.theLoader, holder.annotation, holder.name, registry, extensionProviders)
---
>                 is BatchLoader<*, *> ->
>                     createDataLoader(
>                         holder.theLoader,
>                         holder.annotation,
>                         holder.name,
>                         registry,
>                         extensionProviders,
>                     )
> 
311a320
> 
319a329
> 
328a339
> 
378c389
<         private val logger: Logger = LoggerFactory.getLogger(DgsDataLoaderProvider::class.java)
---
>         private val logger: Logger = LoggerFactory.getLogger(DefaultDgsDataLoaderProvider::class.java)

[paulbakker]

I'm in favor of the current implementation in the PR, I would like to stay away from Spring Cloud for this.

[berngp]

Thanks Paul for the review, will address the feedback.