Billing is infrastructure. Treat it like one.

Billing is rarely a core product differentiator, but it has strict correctness and reliability requirements. A mishandled webhook or a race condition in subscription renewal can lead to duplicate charges, reconciliation issues, and support escalations.

This is why many teams use platforms like Chargebee. It handles the billing lifecycle — subscriptions, invoicing, revenue recognition, dunning, and usage-based billing — so application services can focus on domain logic.

In practice, a billing platform is only as effective as its SDK for developers. Backend capability is less useful when integration code is difficult to reason about, hard to test, or prone to regressions.

That’s why we rebuilt the Chargebee Java SDK from the ground up.

What was wrong with v3?

v3 worked. It powered thousands of integrations across startups and enterprises. But it carried patterns from an era when Java development looked very different:

Global mutable state. Configuration lived in a static singleton.

// v3: One global config. Hope nobody changes it mid-request.
Environment.configure("acme", "cb_test_xxx");

If you’ve ever debugged a multithreaded issue where two threads were hitting different Chargebee sites using the same global `Environment`… you know the pain. That’s not a billing problem. That’s a “your SDK fights your architecture” problem.

Untyped responses. Every API call returned a generic Result object. You had to *know* which resource to extract.

Flat parameter lists. Nested objects like billing addresses were flattened into method chains like billingAddressCity(), billingAddressState() — functional, but far from idiomatic Java.

v3 was a product of its time. v4 is built for how Java teams work today.

v4: Java-native by design

Immutable, Thread-safe client

The ChargebeeClient is the single entry point. It’s immutable, thread-safe, and built with the fluent builder pattern that every modern Java library uses.

ChargebeeClient client = ChargebeeClient.builder()
  .siteName("acme")
  .apiKey("cb_test_xxx")
  .timeout(30000, 80000)
  .retry(
    RetryConfig.builder()
      .enabled(true)
      .maxRetries(3)
      .baseDelayMs(500)
      .build()
  )
  .build();

Create it once. Inject it everywhere. No global state, and no synchronised workarounds around billing calls.

Type-safe everything

Every API operation now has typed parameter builders and typed response objects. IDE autocomplete becomes reliable, and the compiler catches structural mistakes earlier.

CustomerCreateResponse response = client.customers().create(
CustomerCreateParams.builder()
  .firstName("Ada")
  .lastName("Lovelace")
  .email("ada@example.com")
  .billingAddress(
      CustomerCreateParams.BillingAddressParams.builder()
        .line1("50 Market St")
        .city("San Francisco")
        .state("CA")
        .zip("94105")
        .country("US")
        .build()
       ).build());
Customer customer = response.getCustomer();

No more stringly-typed parameters. No more generic Result objects. If the method compiles, the parameter structure is valid. That’s the kind of confidence you want when you’re processing payments.

Async out of the box

v4 ships with first-class async support. Every service method has an Async variant that returns a CompletableFuture:

CompletableFuture<CustomerCreateResponse> future =
  client.customers().createAsync(
    CustomerCreateParams.builder()
      .firstName("Ada")
      .build()
    );
future.thenAccept(response -> {
  log.info("Created customer: {}", response.getCustomer().getId());
});

If you’re running a high-throughput system — say, processing usage records for a metered billing model — blocking threads on HTTP calls is a luxury you can’t afford. Async support means your billing integration scales with your application, not against it.

Structured exception handling

v3 gave you string error codes and public fields. v4 gives you a proper exception hierarchy with strongly-typed error enums:

try {
  client.customers().create(params);
} catch (InvalidRequestException e) {
  ApiErrorCode errorCode = e.getApiErrorCode();
  if (errorCode instanceof BadRequestApiErrorCode code) {
    if (code == BadRequestApiErrorCode.DUPLICATE_ENTRY) {
    // Handle duplicate - compiler verified this is a real error code
    }
  }
} catch (TransportException e) {
// Network-level errors: DNS failures, timeouts
// v3 didn't distinguish these from API errors. v4 does.
}

Pattern matching on error codes. TransportException for network issues vs APIException for server errors. This is how error handling should work in a billing integration where reliability is non-negotiable.

Billing gets complex quickly

If you’ve ever tried to build billing in-house, you know the drill. It starts with a Payment gateway integration and a subscriptions table. Then someone asks for prorations. Then annual plans. Then usage-based pricing because your AI feature charges per API call. Then tax compliance for the EU expansion. Then dunning logic because credit cards expire and customers forget.

Suddenly your “simple billing module” is 15,000 lines of edge cases with ownership spread across multiple teams.

This is the context where managed billing platforms are useful. Chargebee handles subscription lifecycle, invoicing, tax, dunning, and revenue recognition; the SDK is the integration boundary for Java services.

What’s relevant to v4 specifically is the rise of usage-based billing. More teams are moving to consumption pricing — API calls, tokens, compute hours — which means the integration needs to handle high-volume event ingestion without becoming a bottleneck. v4’s async APIs and thread-safe client directly address that requirement.

Migrating from v3: Core patterns

The migration follows a consistent set of patterns. Here’s the short version:

The patterns are mechanical and consistent. We’ve also published a detailed migration guide with a transformation prompt you can use with an LLM to automate much of the migration.

Getting started

For existing v3 users — update your dependency and follow the migration guide. The type system will guide you; most v3 patterns have a direct v4 equivalent.

<dependency>
  <groupId>com.chargebee</groupId>
  <artifactId>chargebee-java</artifactId>
  <version>4.5.0</version>
</dependency>

For new projects — add the dependency, create a ChargebeeClient, and start building. v4 requires Java 11+ and uses Gson for JSON processing (included transitively).

implementation 'com.chargebee:chargebee-java:4.5.0'

Engineering takeaways

Billing code is long-lived. Once it’s in your codebase, it tends to remain for years. It touches payments, subscriptions, invoicing, and revenue, so it should be clean, type-safe, testable, and built on patterns that age well.

Chargebee Java SDK v4 focuses on one principle: billing integrations should follow the same engineering standards as the rest of your Java codebase.

No more global state. No more untyped responses. No more pretending it’s still Java 6.

For teams on v3, the migration is mostly mechanical and provides clearer APIs, safer concurrency behaviour, and stronger type guarantees.

Reference: Chargebee Java SDK on GitHub

Questions or feedback: Discord or dx@chargebee.com

Inside Chargebee Java SDK v4: A Practical Redesign was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

TL;DR: We rewrote the Chargebee Go SDK from scratch and v4 is now available for testing. Please join our Discord server to keep up with updates and to share your feedback.

Intro

As the Chargebee API evolved over time, the current version of the chargebee-go SDK had started to show its scars. While it is functional and continues to be supported, it suffered from technical debt and design patterns that felt foreign to Go developers. Towards the end of 2025, it became increasingly clear from community feedback that it was time to modernize the library.

We embarked on an experiment to rewrite the SDK from scratch with a singular focus: Developer Experience. We wanted to build a library that felt like it belonged in the Go ecosystem, leveraging the language’s strengths and making it more idiomatic.

Goals

Before writing a single line of code, we established clear goals to address the pain points of the previous version, gathered from existing and potential customers via various channels like GitHub, Discord, Support teams, etc. We categorized them broadly, and the following goals started to emerge:

Fix package layout

The Problem: In the previous version, related entities were often scattered across multiple sub-packages. To work with a particular resource, one typically had to import the resources, enums, actions, and global enums separately. This led to having multiple import aliases and a cluttered import block at the top of every source file. It also made it difficult to rely on the IDE to suggest which symbols to import. This seemed to confuse LLMs and coding agents as they did not have a predictable way of looking up the required symbols without the entire SDK taking up the context window.

The Solution: We flattened the package layout. The chargebee package is your primary entry point and all resources are now available directly in the root package This structure is significantly more user-friendly, reducing the cognitive load of remembering where specific models or enums live.

/// Before (v3)
import (
    "github.com/chargebee/chargebee-go/v3"
    subscriptionAction "github.com/chargebee/chargebee-go/v3/actions/subscription"
    subscriptionEnum "github.com/chargebee/chargebee-go/v3/models/subscription/enum"
    "github.com/chargebee/chargebee-go/v3/models/subscription"
    "github.com/chargebee/chargebee-go/v3/enum"
)


/// Now (v4)
import "github.com/chargebee/chargebee-go/v4"
// Everything is accessible via the main package

No global state

The Problem: The old SDK relied heavily on global methods and shared variables for configuration (like API keys). This made it difficult to manage multiple Chargebee sites within a single application and potentially introduce concurrency bugs.

The Solution: All state is now encapsulated in a Client struct. Configuration options—such as custom HTTP clients, timeouts, and retry logic are applied on a per-client basis. This makes the SDK concurrency-safe and multi-tenant friendly by default.

/// Before (v3)
chargebee.Configure("{site_api_key}", "{site}")


/// Now (v4)
// Initialize a client with specific config
usConfig := &chargebee.ClientConfig{
    SiteName: "{site_us}",
    ApiKey:   "{us_api_key}",
}
usClient := chargebee.NewClient(usConfig)

// You can have a second client for a different region
euClient, err := chargebee.NewClient(&chargebee.ClientConfig{
    SiteName: "{site_eu}",
    ApiKey:   "{eu_api_key}",
})

Strictly typed API responses

The Problem: Previously, developers had to deal with a generic Result object. Retrieving the actual data required nil-checking and type assertions, which defeated the purpose of using a statically typed language like Go.

The Solution: Thanks to Go generics, we now have typed response objects for all requests. This makes is easier for IDEs to offer better suggestions and to ensure static type checks pass during development and when compiling the program. This also removes unnecessary noise when trying to identify which fields are actually available in a response without repeated nil checks.

/// Before (v3)
var result chargebee.Result
result, err = subscriptionAction.Create(&subscription.CreateRequestParams{}).Request()
// result has > 100 fields without a clear indication of which ones are
// actually available for this request
if result.Subscription != nil && result.Customer != nil {
    fmt.Println("%+v", result.Subscription, result.Customer)
}


/// Now (v4)
var res *chargebee.SubscriptionCreateResponse
res, err := client.Subscription.Create(&chargebee.SubscriptionCreateRequest{})
// response is statically typed and contains these exported fields
fmt.Println("%+v", res.Subscription, res.Customer, res.Card, res.Invoice, res.UnbilledCharges)

Request consistency

The Problem: The old Request() and ListRequest() methods were often leaky abstractions that actually dispatched the API call and return the Result. It wasn't immediately clear where and how these were different.

The Solution: The request object now encapsulates all required fields and options necessary for the API call. Requests are no longer dispatched when the .Request() method is called. Instead, each resource action expects the request object, which is automatically dispatched when the method is called. The distinction between a Request() and ListRequest() also has been done away with.

/// Before (v3)
res, err := subscriptionAction.Create(&subscription.CreateRequestParams{
    PlanId: "cbdemo_grow",
}).Request()

res, err := subscriptionAction.List(&subscription.ListRequestParams{
    Limit: chargebee.Int32(5),
}).ListRequest()


/// Now (v4)
res, err := client.Subscription.Create(&chargebee.SubscriptionCreateRequest{
    PlanId: "cbdemo_grow",
})

res, err := client.Subscription.List(&chargebee.SubscriptionListRequest{
    Limit: chargebee.Int32(5),
})

Idiomatic error handling

The Problem: The old SDK had quite a few places where it would panic(). As we know, libraries that panic are generally frowned upon as they can crash the entire application. Developers had to know this limitation and then mitigate the risk by guarding the calling method using recover. Even though this was a known issue, it was not an easy fix without breaking changes to the SDK.

The Solution: Almost all panics have been removed in favor of returning an error. The library now calls panic() only if there is a configuration error and the chargebee.NewClient() cannot instantiate a client with the given site name and API key.

Our Process

It was quite obvious from the get go that we will need to maintain a new major version of the SDK going forward. Hence, we went through a structured delivery process where we tried to address most if not all known issues with the current version. We went through these three distinct phases:

Top-down API design

We started with Readme Driven Development: documenting the current state and coming up with the equivalent “ideal” state. This was just the high level API design — creating a config, creating the client, invoking the service, creating a request object, etc. This included mock implementations of common workflows like creating a customer, checking out a subscription, handling a webhook, etc. We iterated on the syntax until it felt idiomatic and IDE friendly.

This phase was crucial for nailing the ergonomics of the SDK. Quite a few decisions were debated and iterated internally. For example, we decided to do away with the common Go pattern of passing a context.Context as the first parameter to all service methods. Contexts are useful in very limited scenarios and making them a default first parameter looked noisy and added cognitive load. Given the following options, which one would you rather choose?

// Option 1: With context as first param, where it's mostly not used effectively
req := &chargebee.SubscriptionListRequest{}
res, err := client.Subscription.List(context.TODO(), req)

// Option 2: With an optional method to set the context if required
req := &chargebee.SubscriptionListRequest{}
req.SetContext(context.Background())
res, err := client.Subscription.List(req)

Another example of obsessing over the small details: we did not want a New{Resource}Request wrapper to build each request type. This took a bit of a deep dive into how embedded structs work and how best to expose the additional methods on the request object without leaking abstractions.

// Option 1: With an additional constructor for each request type
req := chargebee.NewSubscriptionListRequest(&chargebee.SubscriptionListParams{...})

// Option 2: No additional function call
req := &chargebee.SubscriptionListRequest{...}

A key guiding principle at this stage was: “It’s OK to be opinionated”. Understanding the options in front of us, and choosing what we think is the best way forward helped us focus on the single best solution. Instead of providing two different ways of doing things which adds to complexity and confusion, we choose the most pragmatic way forward.

We also focused on internal consistency of the library. For example, if we are to make a raw request whose response type is unknown, we can call the internal send method with the following signature:

res, err := send[*UnknownResponse](&BlankRequest{}, config)

P.S: We will eventually expose a RawRequest method using the above technique as an escape hatch, where a constructed http.Request can be sent to the API, and raw response made available for processing.

Bottom-up implementation

Once the desired user facing API was finalized, it was time to get to work. All our SDKs are built as a combination of hand crafted code which lay the foundations, and generated code which provide the required models and methods to invoke the API endpoints. Our sdk-generator is responsible for parsing the OpenAPI spec, constructing the objects (resource, input and output schema, enums, etc) and their relationship and rendering the generated code for each target language.

Since our existing SDK had the low level implementation details, we decided to reuse them to save time and effort. This plumbing code isn’t exposed directly to the users and can be refactored without breaking the high level APIs. Hence, the main focus was around implementing the required apiRequest, apiResponse, ClientConfig and Client structs.

The sdk-generator was then updated to flatten the resource hierarchy, and write out the renamed structs and methods. Every resource (e.g. customer) will now have a customer.go file that contains all the model definitions, and a customer_service.go with the resource methods. This predictable layout makes it easy to look for a symbol manually and using LLMs/coding agents.

Testing

No major rewrite is complete without sufficient testing. One of the challenges we faced was a lack of unit tests in key methods that handle request serialization, response parsing, handling custom fields, etc. To refactor with confidence, we focused on writing tests first before breaking it, and then eventually fixing it with the reimplemented methods.

Some manual testing was also required to ensure the correctness of code snippets that were migrated to use the new version. We will continue E2E testing while we are in the beta phase to maximize our chances of running into potential bugs.

Next Steps

Now that the beta is out, we are turning to the community for help with testing and feedback. If you already use the chargebee-go/v3 library, please test out the new version with a particular attention to usability and correctness. We have a handy migration guide to move from v3 to v4.

Bonus round: To improve the readability of long enums (CreditNoteEstimateLineItemDiscountDiscountTypeDocumentLevelDiscou anyone?!), we are looking to do implement pseudo-namespacing in our generated code as shown below. It is analogous to how Go supports underscore separated numbers to improve readability (10_000_000 === 10000000). Let us know if you think this is a good idea!

var CreditNoteEstimateEnum struct {
  LineItemDiscount struct {
    DiscountType struct {
      ItemLevelCoupon       CreditNoteEstimateLineItemDiscountDiscountType
      DocumentLevelCoupon   CreditNoteEstimateLineItemDiscountDiscountType
      PromotionalCredits    CreditNoteEstimateLineItemDiscountDiscountType
      ProratedCredits       CreditNoteEstimateLineItemDiscountDiscountType
      ItemLevelDiscount     CreditNoteEstimateLineItemDiscountDiscountType
      DocumentLevelDiscount CreditNoteEstimateLineItemDiscountDiscountType
    }
  }
}

func init() {
  CreditNoteEstimateEnum.LineItemDiscount.DiscountType.ItemLevelCoupon = CreditNoteEstimateLineItemDiscountDiscountTypeItemLevelCoupon
  CreditNoteEstimateEnum.LineItemDiscount.DiscountType.DocumentLevelCoupon = CreditNoteEstimateLineItemDiscountDiscountTypeDocumentLevelCoupon
  CreditNoteEstimateEnum.LineItemDiscount.DiscountType.PromotionalCredits = CreditNoteEstimateLineItemDiscountDiscountTypePromotionalCredits
  CreditNoteEstimateEnum.LineItemDiscount.DiscountType.ProratedCredits = CreditNoteEstimateLineItemDiscountDiscountTypeProratedCredits
  CreditNoteEstimateEnum.LineItemDiscount.DiscountType.ItemLevelDiscount = CreditNoteEstimateLineItemDiscountDiscountTypeItemLevelDiscount
  CreditNoteEstimateEnum.LineItemDiscount.DiscountType.DocumentLevelDiscount = CreditNoteEstimateLineItemDiscountDiscountTypeDocumentLevelDiscount
}

Code is rarely “done”, so we will continue testing the SDK with different scenarios, and update the documentation so that everything is set for a stable release. The more feedback we receive, the quicker we can fix any gaps before it’s ready for production use.

As always, you can join our Discord server or send us an email at dx@chargebee.com to share your feedback.

How we rewrote the Chargebee Go SDK v4 from scratch was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

During a press and analyst briefing in March 2015, one of the iconic moments in the software industry, Microsoft CEO Satya Nadella presented a slide that read, “Microsoft loves Linux.” He also stated that he wasn’t interested in fighting old battles, especially when Linux had become a vital part of modern business technology. “If you don’t jump on the new,” he said, “you don’t survive.” Microsoft long held the belief that Windows OS was central to its ecosystem and viewed Linux as a rival. Thus, resisting the broader industry shift toward open-source adoption.

The story at Chargebee wasn’t all that different from that of Microsoft. We traditionally considered the product catalog and the selling rules to be tightly coupled with the billing system, making it difficult to envision any other source of truth for catalog data. This perspective positioned us more as competitors to CPQ systems, rather than collaborators.

In Spring 2025, Chargebee launched a suite of CPQ products, which included:

• a native CPQ offering for customers who can have their quoting, pricing and guided selling within Chargebee Billing System.
• Chargebee CPQ in a CRM platform of your choice
• Bring Your Own CPQ.

With this launch, Chargebee also lets merchants bring any CPQ of their choice and integrate it with Chargebee Billing. In this post, we will cover the technical details that made this possible. With a composable architecture, we’ve now decoupled catalog workflows from billing workflows, enabling seamless integration with any CPQ system. This transforms us from a standalone solution into a more interoperable platform for any CPQ to integrate.

But before diving into those details, let’s first understand the importance of the product catalog in a billing system.

Product Catalog Data Model

A product catalog defines the SKUs that a business wants to sell and also includes the associated price points. A product is critical to the functioning of a billing system, serving as the configuration for price points and selling rules used to create a sales order within the billing system. For a billing system, a product catalog acts as a template created by the merchant to generate specific subscriptions for customers.

Chargebee’s product catalog data model for billing is inherently a flat data model. This allows the billing system to invoice flexibly and structure items in a way that favors invoicing and accounting systems. The items in the catalog are essentially converted into a one-to-one mapping with invoice line items. Therefore, the data model for the product catalog favors a flat structure that enables better invoicing.

Below is a representation of the flat data model used in Chargebee Billing. The items and their prices across different frequencies and currencies are a flat list of prices for each of the items.

A flat model leads to catalog bloat and creates management overhead in handling the flat list of item prices. It also included some built-in selling rules applied when a subscription was created. This tight coupling between the selling rules and the billing system had to be decoupled to enable a truly flexible integration of any CPQ with the billing system.

To solve the Catalog bloat issue in Product Catalog 1.0, Chargebee introduced Product Catalog 2.0 to enable better management of the product catalog. This model brought structure to overall catalog management. Item prices could now be grouped under items — each item acting as a container for prices of the same product with different selling frequencies and currencies. These items are further grouped into item families, allowing similar products sold together to be organized under the same item family.

Product Catalog 2.0 introduces a more structured and flexible system. It organizes catalog items more clearly and separates item configuration and selling rules from pricing. This means you can now manage how items are sold independently of how they’re priced, giving you greater control and customization over selling options.

Additionally, Product Catalog 2.0 introduced differential pricing capabilities to set bundled prices for products when sold together or with specific plans. Product Catalog 2.0 also enables the configuration of selling rules for plans, add-ons, and any one-time charges. The collective nature of these selling rules gives CPQ-like capabilities to the product catalog.

The first major evolutionary step was to decouple the product catalog. The realization was that the catalog needed to exist as an independent, standalone service with its own API. This seemingly simple architectural change had profound implications. By decoupling the product catalog, it could now serve as the single source of truth for multiple systems. This was the key that unlocked the door for a more sophisticated sales and billing motion. The most immediate and impactful integration was with CPQ (Configure, Price, Quote) systems.

Sales teams live in their CRM and CPQ tools. They need to create complex quotes, apply discounts, and structure deals without ever touching the billing system. With a decoupled catalog, the CPQ system could now pull products, pricing, and rules directly from the catalog API. This ensured that the deal structured by the sales team was perfectly aligned with what the billing system could actually execute. The result? A seamless handover from sales to billing, eliminating manual data entry and costly errors.

Introducing the SalesOrder Interface

Decoupling the product catalog and integrating a CPQ was just the beginning. This decoupled structure dramatically reduces complexity and makes it significantly faster to launch in new markets or experiment with pricing strategies. The true transformation is the move to a fully composable, headless architecture. In this model, the core billing engine becomes a powerful, orchestrating hub, connecting to a variety of specialized, best-of-breed services via API.

To create a truly fully composable interface between the sales and billing processes, we introduced a crucial intermediary: the SalesOrders interface to separate the quoting process from the billing process. The Sales Order interface establishes a clear communication channel between the CPQ/Catalog system and the Billing system.

The Sales Order interface captures all the necessary information required for the billing system to initiate invoicing workflows. There are two ways a Sales Order can be created:

• Sales-Led workflows: Once a quote is created and accepted in the CPQ system, the CPQ initiates the workflow to create the Sales Order.
• Checkout workflows: Upon cart payment, the checkout page generates the Sales Order object, which then triggers the downstream billing workflows.

The Sales Order interface clearly delineates the boundary between the Product Catalog and Selling System on one side, and the Billing System on the other. This decoupling allows the Billing System to integrate with any product catalog that includes more complex selling rules — allowing merchants to bring their own CPQ when integrating with Chargebee Billing.

In revenue recognition (RevRec), the Sales Order initiates the revenue recognition process by creating a revenue schedule in accordance with accounting standards. It includes performance obligations, allocates revenue, sets up deferred revenue, and automates expense workflows — ensuring accurate revenue recognition and compliance with ASC 606 and IFRS 15.

Ensuring Backward Compatibility

A key consideration in decoupling the Product Catalog from the Billing system was to ensure that existing customer workflows and assumptions remained unaffected. Hyrum’s Law played a critical role in guiding how we maintained consistency, not only in the APIs but also in the UI. This meant that all API test automation for legacy workflows had to continue functioning exactly as expected. Our robust automation suite was instrumental in enabling this decoupling while preserving backward compatibility. It ensured that any changes introduced did not break existing functionality, providing a safety net for continued reliability.

To further safeguard backward compatibility, we proactively mitigated any risks to existing users, ensuring there was no downtime or disruption to their billing workflows. As a result, customers migrating between CPQ systems on the Chargebee Billing continue to experience consistent and reliable billing and invoicing.

Summary:

Chargebee’s architectural journey has evolved from a tightly coupled monolithic system to a modern, composable, and decoupled ecosystem built for flexibility and scale. This transformation was driven by the growing diversity in merchant needs across industries, each requiring nuanced control over catalog configuration, pricing, and quoting.

Today, Chargebee empowers organizations with the freedom to choose the CPQ solution that best fits their sales workflow — whether native to Chargebee or integrated with best-in-class platforms like Salesforce, HubSpot, Conga, or DealHub. By decoupling the product catalog from the billing engine and introducing the Sales Order interface, we’ve made CPQ interoperability seamless. It is a fundamental shift in how Revenue Growth Management can operate at scale. CPQ workflows are now a native part of the Chargebee DNA, ensuring alignment between sales intent and billing execution without friction or compromise.

A composable architecture unlocks extensible CPQ capabilities with Chargebee Billing.

Chargebee CPQ : Evolution from Product Catalog to CPQ was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Chargebee Node.js SDK has been around for more than a decade. It nets 100K combined weekly downloads. But it just hasn’t kept pace with the fast-moving JavaScript ecosystem.

New JavaScript runtimes have appeared in recent years. All built around goals of delivering speed, well-optimized APIs, and a more complete DX. Goals that we’ve also made central for this new release.

Then serving two separate SDKs — one for Node.js, one for TypeScript — has only bred confusion. Especially as both those SDKs relied on some outdated dependencies, inconsistent TypeScript support, and verbose workflows.

So when we asked ourselves: What should a great SDK look like, today?

The answer couldn’t be a patch or a tweak.

It had to be a rewrite.

The big change: One unified SDK

This new version:

• Supports both ESM and CJS module systems
• Runs smoothly on Bun, Deno, Cloudflare Workers, and other modern runtimes
• Leverages native features such as fetch and Promise

This also means that the chargebee-typescript package is getting deprecated. But anyone using it will have ample time (and support from us) to migrate to the new version.

Let’s look at just some of the quality-of-life improvements that v3 delivers.

What’s new?

Idiomatic, modern JavaScript

The new SDK just works. Here’s how:

• Stronger TypeScript inference that reduces errors and guesswork
• Function parameters now use camelCase instead of snake_case
• Native async/await is supported for cleaner syntax
• The ability to create multiple client instances for more complex use cases

Less boilerplate, more flow

This update significantly tackles verbosity, improving readability across the board. For example:

• You no longer need to append .request() to every API call.
• Efficient client setup — functions like .configure() and .updateTimeoutMillis() are now consolidated into simpler configuration parameters.

(Much) better typings

Version 3 also has great TypeScript coverage that helps with writing correct code faster and catching issues much earlier in the development cycle.

For example: custom fields in Chargebee always begin with cf_

With the new SDK, if you try to name the custom field something else, TypeScript will point it out with that beautiful, signature squiggly line 😄

The type inference for response is far better as well.

The screenshot below shows a code sample for creating a customer in Chargebee in the old version of the SDK. If you look closer, it returns a generic customer object.

Compare this with the same sample in the new SDK version below:

What’s next?

We plan to improve the SDK experience further with more contextual documentation and the ability to use your own HTTP client for added flexibility. We’re not stopping there. This release sets a high standard for the other SDK updates we’re starting to work on.

If you’re a developer building with Chargebee, there are more exciting things coming your way!

A note on how we build:
We auto-generate client libraries in seven different programming languages using a custom code generator (curious about how that works? Leave a comment, we’d love to share 😉) based on our OpenAPI spec.

If you have any feedback/comments/questions, please feel free to reach out to dx[at]chargebee[dot]com.

— Chargebee DX Team

Chargebee’s Node.js SDK Gets a (Long-Due) Makeover was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

• Estimating taxes applicable on an invoice.
• Calculating country-specific taxes.
• Creating tax profiles for distinct product or service groups that fall under varying tax rates and compliance measures.
• Managing tax exemptions for specific customers.
• Validating customer shipping addresses both for accurate tax calculation and product delivery purposes.
• Submitting invoice and credit note documents for precise tax reconciliation.

With businesses spanning across borders, reconciling the numerous global tax rules with recurring invoicing becomes a challenging task. To manage this complexity, we’ve refined Chargebee’s tax system through three stages of innovation. Let’s walk through them:

Stage 1: In-House Solutions

The foundation of our tax system began as an in-house module within our core billing software. Merchants today still have the option of this built-in capability, packed with features like multi-region support, tax exemption management, and detailed tax reports.

Let’s peek into the architecture of this initial system. Merchants connect with Chargebee Billing either through our user interface or APIs. For clarity, the diagrams here focus on the API route:

Going further into the Chargebee container, we reveal the dedicated tax module, which handles all tax-specific tasks:

Over time, we noticed many merchants opting for third-party tax services. While our in-house solution was comprehensive, it became essential to cater to our merchants’ diverse requirements.

Stage 2: Integrating with Third-Party Tax Providers

To address the growing needs, we expanded our horizon and integrated with third-party tax providers like Avalara and TaxJar. The diagram below illustrates this:

The following diagram displays the new internal components that were added to Chargebee Billing as part of the integration: the client modules for Avalara and TaxJar.

Yet, these third-party integrations brought along their own set of challenges, especially in managing integration complexities. Given our vast merchant base, it became evident that a single engineering team building these integrations wouldn’t scale.

Stage 3: The Dawn of Tax SPI

The problem was clear: How do we enable merchants, tax service providers, or independent software vendors (ISVs) to build integrations without being wholly dependent on Chargebee?

Enter the Tax SPI.

We introduced an API contract that sets the expectations for how our Billing module would interact with a tax provider. Any tax service provider keen on integrating with Chargebee can now align with our SPI, creating an adapter, compliant with the SPI’s specifications.

Case in point, the Anrok Adapter was the first to be crafted using the Tax SPI. Merchants can now effortlessly integrate with Anrok via the Chargebee Marketplace. The addition of Anrok’s self-made adapter into our Marketplace stands as a testament to the SPI’s success. Following this, we developed an adapter for another tax provider, Vertex, which is currently in early access.

With the Tax SPI, we’re not just forging a path for streamlined integrations; we’re championing flexibility. While we continue to build and host integrations for strategic tax providers like Vertex, we also empower other tax providers and ISVs to self-serve, allowing them to build and host their own integrations with Chargebee.

All future tax service integrations will adopt this design.

In the diagram below, we show how the Tax SPI restructures Chargebee Billing’s internals, eliminating the need for a dedicated client library for each tax provider. Instead, the same SPI client library is used to connect to all new tax service provider integrations via their respective adapters.

Implementing the Tax SPI

For vendors implementing tax service adapters using our SPI, the process can be summed up in a few steps:

1. Sharing & Implementation: We share the Tax SPI and a configuration template with the third-party vendor. Subsequently, the vendor implements their adapter app in alignment with the SPI. Once this is achieved, they register their adapter app on the Chargebee Marketplace, complete with configuration details, ranging from identity configurations to supported capabilities.
2. Validation & Integration: The configuration undergoes rigorous validation before being loaded into Chargebee. Once the adapter is implemented, we run pre-written Postman tests to evaluate the adapter, sharing any discrepancies for rectification. Furthermore, we have an integration suite assessing end-to-end workflows.
3. Dynamic Integration: Any modifications to the vendor configuration are dynamically reflected on the vendor app’s screen within the Chargebee Billing app. This eliminates the need for additional development work on our end.
4. Merchant Onboarding: After ironing out any issues from the regression and integration suite, we onboard merchants with this new tax vendor integration via the Marketplace.

Keen to delve deeper? Check out Chargebee’s Tax SPI documentation.

Technical review, edits, and diagrams

John Machan, Staff Technical Writer, Chargebee

Streamlining Tax Integrations with Tax SPI was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

With this implementation, our security workflow does not fail even if the tools detected any vulnerabilities in the scans. This was a founding stone to our shift left initiative

Next, to drive the adoption of our security workflow to individual production touching repositories in Chargebee, we decided to build a security policy engine using OPA to control the data produced by open-source tools to act as a gatekeeper so we can effectively control the success or failure of security workflow along with managing exceptions.

This is how the current security workflow looks like

The policy engine evaluates data produced by the open-source tools that we use

• SAST -> SemGrep
• SCA -> OWASP Dependency Checker
• Secret Scanning -> GitLeaks
• SBOM -> CycloneDx

All these tools produce JSON formatted report which is combined together using JQ to feed them to the policy engine

Security Policy Engine

The security policy engine starts with default.rego which checks if the overall count of violations is Zero or not.

package cb.devsecops.policy

default allow = false

allow {
  count(violations) == 0
}

Violations are defined separately for each tool we use in the security pipeline. Let’s consider an example of SemGrep policy implementation. For SemGrep we have sast.rego file created in the policy engine where it initially checks for all the issues against a configured severity for instance if we decide to fail SemGrep only for ERROR categorized issues we can define this severity in the configuration file and the same does for other tools like SCA we can define to fail only on CRITICAL or HIGH severity issues.

This is what a sample SAST policy looks like:

package cb.devsecops.policy

import future.keywords.every
import future.keywords.in

violations[{"message": msg, "code": code}] {
  issue = input.semgrep.results[_]
  issue.extra.severity in data.config.semgrep.fail_on_severities
  not semgrep_exempted(issue)

  msg := "SAST result have issues with WARNING or ERROR"
  code := "sast_fail"
}

semgrep_exempted(issue) {
  exceptions := data.config.exceptions.semgrep

  exceptions[_].attributes.fingerprint = issue.extra.fingerprint

  # Support glob so that we can use patterns like 'javascript.**'
  glob.match(exceptions[_].attributes.check_id, ["."], issue.check_id)
  glob.match(exceptions[_].attributes.path, ["/"], issue.path)
}

Once the initial check for severity is completed it moves toward exception management. Producing false positive issues with security scans is a prevalent scenario.

To deal with this situation we have defined a way to add exceptions for each tool. So the next check is to see if the flagged issue is configured as an exception or not.

For example, if the policy engine catches a HIGH severity issue and it’s made as an exception, the security workflow won’t fail. And if the issue is failing on severity and not configured in exception our security workflow will fail. Let’s consider an example of exception management in the policy engine

{
    "exceptions": {
        "semgrep": [
            {
                "reason": "This is a example exception criteria",
                "attributes": {
                    "check_id": "example-check-id",
                    "fingerprint": "example-fingerprint",
                    "path": "/target/example/path/file.ts"
                }
            }
        ],
        "gitleaks": [
            {
                "reason": "This is a example exception criteria",
                "attributes": {
                    "StartLine": 21,
                    "EndLine": 21,
                    "File":"/target/example/path/file.ts",
                    "Commit":"<commitSHA>"
                }
            }
        ],
        "odc": [

        ]
    }
}

Once the policy engine is integrated with the security workflow this is how it looks in action

Closing note

With this tooling, at Chargebee we intend to drive policy as code adoption and encourage security policies to be codified in the pipeline.

Possibilities of adopting Policy as Code are limitless. Any change in security policies will mean we need to change only the policy code instead of the rest of the system. It also ensures the separation of concerns between tools developers, integrators, and policymakers.

For comments or feedback, you can get in touch with me over Twitter 😀

If you are interested in our work and want to solve complex problems in SaaS products, platform & cloud infrastructure engineering — we are hiring!

Building Policy Gate for DevSecOps using Open Policy Agent was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

The objective of this initiative is to provide centralized visibility of the overall security posture of various production touching components within the organization. This is a stepping stone for establishing a shared security responsibility culture by providing continuous and automated visibility.

Overall Architecture

Self-serve Security Solutions

All of our security solutions are deployed as independent containers on AWS ECR so they can be pulled directly from ECR to integrate into any workflows or can be used locally by developers.

We use the following open-source tools for security scanning

• SAST → SemGrep
• SCA → OWASP Dependency Checker
• Secret Scanning → Gitleaks
• SBOM → CycloneDx

To drive self-service and ease of usability & adoption we wrote a custom wrapper on top of these tools. So our users need not worry about the underlying implementation. This gives us additional control of pushing custom rules any time and have the pipeline apply the rule on all repositories.

For example, the SAST solution asks for a few user inputs and then applies appropriate scan profiles and custom rules based on the input. This makes it easier for our users to use it without worrying about the backend implementation, like what custom rules to use, which SemGrep profiles to use, etc. It also helps us to replace the underlying tool without disturbing the existing implementation.

GitHub Repository Architecture

To achieve continuous security scanning without depending on any team we decided to

• Create an AppSec repository that has a main reusable workflow that can be used to scan any repository we wanted to scan using the standard configuration defined in security-workflow.yml
• For example, if we want to scan a dummyrepository then we create a YML file with the repo name which uses our reusable workflow and on a nightly basis
• As it triggers the security-workflow.yml file that clones the target repository which is dummy.yml and it starts scanning and archiving the report to the S3 bucket.

Security Data Visualization

The data ingestion process is complete once it is archived in the S3 bucket. The next phase is to visualize the data stored in the S3 bucket.

To solve the visualization in a near real-time manner, our Lambda function triggers on S3 PUT event and checks the relevant data, and sends it to the security dashboard.

DefectDojo

DefectDojo has inbuilt parser support for SemGrep, Gitleaks, and ODC. So all the results from these tools go to the DefectDojo dashboard via the lambda function

DependencyTrack

In the DependencyTrack, the results from the software bill of material (SBOM) are stored. DependencyTrack has inbuilt functionality to detect vulnerabilities in different components used in the application and hence it can be used to identify

• List of the components/ libraries used in the application (Inventory management)
• Vulnerabilities in used components/libraries
• Organizational license policy violation detection

Closing note

As a result, our security workflows can be integrated within GitHub Actions and can be used for PR level scanning as well and that is what is driving our shift left initiative.

For comments or feedback, you can get in touch with me over Twitter 😀

If you are interested in our work and want to solve complex problems in SaaS products, platform & cloud infrastructure engineering — we are hiring!

Building AppSec Pipeline for Continuous Visibility was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introduction

GitHub Actions is a very useful tool for implementing developer workflows such as CI/CD (Continuous Integration and Continuous Delivery) pipelines for your application. By default, GitHub Actions jobs are run in the cloud, on machines that are hosted and managed by GitHub.

Self-hosted Runners

However, sometimes, you may want to run your GitHub Actions jobs in your own machine. One reason could be that GitHub-hosted machines do not have the minimum hardware resources required to run your app. For such cases, GitHub gives you the option to use Self-hosted runners. As the name suggests, self-hosted runners are machines which are hosted by you and are capable of running GitHub Actions. However, it requires provisioning and configuration of virtual machine instances to set up these self-hosted runners (Problem A).

Kubernetes Cluster

Containerising apps and deploying them inside Kubernetes cluster is also very common these days. If you already have a Kubernetes cluster, it makes more sense to run self-hosted runners on top of it. Also, self-hosted runners should be able to automatically scale up/down based on the demand for running tests. For example, if two different developers push code to the same branch few seconds apart, the second developer should not have to wait for the first developer’s workflows to finish execution before second developer’s workflows start. Rather, a second runner should be scaled up automatically to serve the second developer. Also, this second runner should scale down automatically when there are no further workflow jobs to run. Having this ability to automatically scale up and scale down based on demand, will enable you to run workflows in parallel in a cost-efficient manner. (Problem B).

Spot Instances

Some cloud providers like AWS offer spot instances. Amazon EC2 Spot instances are spare compute capacity in the AWS cloud available to you at steep discounts (upto 90%) compared to On-Demand prices. You can use Spot Instances for various stateless, fault-tolerant, or flexible applications. However, the catch is that AWS can take back this instance anytime by giving a notification, two minutes before it actually does that. This is called Spot Interruption. Using Spot Instances reduces cost significantly, but you need a way to manage this spot interruption. (Problem C).

In this article, the intent is to provide you a solution to run autoscaled self-hosted runners inside a Kubernetes cluster using spot instances. But before that, let us compare the costs incurred in running GitHub-hosted runners and self-hosted runners on AWS EKS.

Cost Comparison

Let’s say that we need to execute 20 jobs per day and each job takes 1 hour to run. GitHub-hosted runners have the following hardware specifications for a Linux based machine:

• 2-core CPU
• 7 GB of RAM
• 14 GB of SSD space

We will use a t3.large instance for calculating cost for Self-hosted runner. It has the following specifications:

• 2-core CPU
• 8 GB of RAM

GitHub-hosted Runners

Cost of running Linux based GitHub-hosted runner — $0.48/hr

Total time for running all jobs each day — 20 * 1= 20 hrs / day

Total cost per month — 0.48 * 20 * 30 = $288 / month

Self-hosted Runners

Cost of running 1 EKS cluster — $73/month

Cost of running 1 t3.large spot instance with 14 GB storage ~ $23.40 / month

Total cost ~ 73 + 23.40 = $96.40 / month

Clearly, for this example, self-hosted runners are less expensive as compared to GitHub-hosted runners. GitHub-hosted runners are priced based on the time duration for which the job is actually running. On the other hand, self-hosted runners on AWS EKS, are priced based on the time duration for which the host machine is running. This has the following interesting consequence. If we use self-hosted runner, we might be able to run more number of jobs at the same cost, depending on the job length and job hardware requirements. For example, let’s say a certain job requires 0.5 core CPU and 2 Gi RAM. This means that one t3.large will be able to run at-least three such jobs in parallel, without any extra cost. However, GitHub-hosted runner will charge us for these extra two jobs.

Autoscaling

Autoscaling the runners based on demand, is important for reducing costs. Autoscaling will work at two levels.

Node Level Auto-scaling

Nodes will be auto-scaled based on the resource requirements of the pods. This will help us in reducing costs as idle nodes would be automatically scaled down and new nodes will only be created if there is a need for them. It can be implemented using Kubernetes Cluster Autoscaler or Karpenter Open-Source Project.

Pod Level Auto-scaling

Pods will be autoscaled to run on different nodes. If there are more jobs queued by GitHub Actions, more pods will be created to run the jobs. As jobs get complete and queue gets empty, these pods will be scaled down automatically. It is implemented using HorizontalRunnerAutoscaler resource of action-runner-controller.

Architecture

We will use actions-runner-controller (ARC) with AWS EKS to solve problems A and B. ARC is an open-source project that operates and manages self-hosted runners for GitHub Actions on your Kubernetes cluster.

The architecture comprises a single Kubernetes cluster with two namespaces — one for the controller and other for the runners. AWS ECR contains the custom images that will be used to create the runners. Using custom images is useful if your application or jobs need some dependencies to run.

Also, note that there are 4 types of pods:

• Runner — this pod will actually listen to GitHub for any pending jobs and subsequently run them
• RunnerDeployment — this helps us in managing sets of runners so that we do not have to manage them individually
• HorizontalRunnerAutoscaler — this helps the RunnerDeployment to scale up/down Runner pods. Autoscaling can be driven from a webhook event or pull based metrics, based on how the HorizontalRunnerAutoscaler is configured.
• ActionRunnerController — this is responsible for registering the runner with GitHub and managing other tasks, needed for everything to run smoothly.

We will also need to store certain secrets in Kubernetes Secrets Manager. By default, it is unencrypted. We will use AWS Key Management Service (KMS) keys to provide envelope encryption of Kubernetes secrets.

Handling Spot Interruption

AWS only notifies you 2 minutes before it terminates spot instances. This might be not enough for gracefully terminating runners that are running moderately big jobs (that can easily take minutes to complete). Hence, there is not much that you can do to handle the jobs if they get interrupted due to spot interruption. The best that can be done is to create a script which can restart any job that was interrupted due to spot interruption.

However, if we follow the following practices while setting up the cluster, we can minimize Spot Interruption to a great extent.

• Regularly scale down the nodes where the runner pods are scheduled when there are no jobs running.
• Configure the EKS node group with as many different instance types as we can, preferably from different markets (instance classes, availability-zones)
• Set the node group to use “capacity-optimized allocation strategy” to ensure that the most-likely-to-survive instance type is picked every time there’s a scale-up. This is enabled by default if we use managed node group to create the cluster.

Let’s get started with the implementation.

STEP 1: Install utilities

Visit this link for detailed instructions for installing the utilities.

A — Install and configure AWS CLI

B — Install kubectl Utility

C — Install eksctl Utility

STEP 2: Set Up AWS EKS Cluster

A — Create AWS KMS key

1 — Execute the following to create it

aws kms create-key --description "ActionRunnerKey" --region us-east-1

2 — To view the key,

aws kms list-keys

Copy the keyARN of the key that you just created.

B — Create a cluster of spot instances

1 — Copy the following configuration in a file called cluster_config.yaml . Use the keyARN that you copied in the previous step. You can also find it using AWS Console.

2 — Execute the following command to create the cluster

eksctl create cluster -f cluster_config.yaml

STEP 3: Set Up Action-Runner-Controller

Before you set-up ARC, if you want to autoscale nodes, you can use Kubernetes Cluster Autoscaler or Karpenter. To know more, you can see Autoscaling in AWS EKS.

A — Install cert-manager.

Follow the steps mentioned in this link for installing it using Kubectl.

B — Install the custom resource definitions and actions-runner-controller.

It can be done using kubectl or helm. This will create actions-runner-system namespace in your Kubernetes and deploy the required resources.

1 — Download the yaml file using the following command

curl -L -o actions-runner-controller.yaml https://github.com/actions-runner-controller/actions-runner-controller/releases/download/v0.21.0/actions-runner-controller.yaml

2— Now, deploy this. With kubectl, it can be done with the following command

kubectl create -f actions-runner-controller.yaml

C — Set Up Authentication with GitHub API.

You can use PAT based authentication or GitHub App based authentication to authenticate the runners with GitHub. For the purpose of this tutorial, we will use PAT based authentication.

1 — Log-in to a GitHub account that has admin privileges for the repository, and create a personal access token with the appropriate scopes listed below:

Required Scopes for Repository Runners

• repo (Full control)

Required Scopes for Organization Runners

• repo (Full control)
• admin:org (Full control)
• admin:public_key (read:public_key)
• admin:repo_hook (read:repo_hook)
• admin:org_hook (Full control)
• notifications (Full control)
• workflow (Full control)

Required Scopes for Enterprise Runners

• admin:enterprise (manage_runners:enterprise)

For the purpose of this tutorial, we will deploy the self-hosted runner at the repository level.

2 — Once you have created the appropriate token, deploy it as a secret to your Kubernetes cluster that you are going to deploy the solution on:

kubectl create secret generic controller-manager \
    -n actions-runner-system \
    --from-literal=github_token=${GITHUB_TOKEN}

D — Deploy Runners on EKS

1 — To create the runner in custom namespace first create custom namespace using command: kubectl create namespace action-runner-runners

2 — To launch an autoscaled self-hosted runner, you need to create a manifest file that includes the RunnerDeployment resource. In this file, we have mentioned ubuntu:latest as the image that will be used to create the runners. However, you can also use a custom image.

3 — Apply the created manifest file to your Kubernetes in the specified namespace: kubectl --namespace action-runner-runners apply -f runner-deployment.yaml

4 — Then create another manifest file that includes the HorizontalRunnerAutoscaler resource as follows.

We are using PercentageRunnersBusy metric to autoscale the pods. However, there are additional metrics, offered by ARC, to autoscale the pods. Visit this link to know more.

5 — Apply the created manifest file to your Kubernetes cluster.

kubectl --namespace action-runner-runners apply -f horizontal-runner-autoscaler.yaml

6 — The runner you created must now be registered to your repository. To check, open your repository on GitHub. Go to Settings -> Actions -> Runners. It must list a runner with a `self-hosted` tag.

7 — Configure GitHub actions workflows to use self-hosted runner

To specify a self-hosted runner for your GitHub Actions job, configure runs-on field in your workflow file to contain the labels that you mentioned in the RunnerDeployment resource file. Note that GitHub attaches the label self-hosted to self-hosted runners by default

runs-on: [self-hosted, large]

Now, GitHub Actions should run on your self-hosted runner in AWS EKS.

Tear Down

You now know how to setup self-hosted runners using AWS EKS.

To free up all the AWS resources, you can use the following command:

eksctl delete cluster --name my-cluster --region us-east-1

To de-register the runner with your GitHub Repository, open your repository on GitHub. Go to Settings -> Actions -> Runners. You can now delete the runner to de-register it.

For comments or feedback, you can get in touch with me over LinkedIn.

This was an internship project, mentored by Priya Sebastian.

If you are interesting in our work and want to solve complex problems in SaaS products, platform & cloud infrastructure engineering — we are hiring!

Save cost by running GitHub Actions on Spot Instances inside an autoscaled EKS Cluster was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Chargebee, we have been using CodeQL for a while to solve security problems related to finding all variants of a given vulnerability. The same approach can however be used to solve an important engineering problem — Technical Debt reduction and dead code elimination.

Technical Debt Accumulation

Any moderately complex piece of software is an aggregation of code that we write and the libraries that we adopt from open sources in our build. It’s a continuous choice between building from scratch or adopting open source. In any case, evolution of our code base increases the technical debt —

1. Deprecated or unused lines of code
2. Libraries included in build but no longer used
3. Dependency on legacy or unmaintained libraries

Add to the complexity if you are using a legacy build system that uses unmanaged jars sprayed across your Git repositories.

Challenges in Dead Code Elimination

Build tools like Gradle or Maven provide out of box support for identifying dependencies. However, older ant based build systems cannot use such feature readily. Even so, modern build tools will not be capable of detecting unused code blocks or dependencies, especially in case of transitive dependencies.

Using CodeQL to Identify Unused External Libraries

CodeQL is the code analysis platform used by security researchers to automate variant analysis.

We are looking at adopting CodeQL for identifying different variants of a vulnerability, found internally or reported by external security vendors. As part of evaluation, we wrote context specific CodeQL classes modelling our controller class (Java) that can be used to write queries for common vulnerabilities.

We internally ran an experiment to leverage CodeQL to identify unused libraries in a sample application. The general idea is given below

1. Build a CodeQL database for a sample application — This represents a Control Flow Graph (CFG) for us to query upon
2. Write a CodeQL query to identify all cross package MethodCall i.e. caller is defined in com.example.sampleApp and callee is NOT in the same package. To reduce false positive, we filter out the java.* package as well.
3. Create a sorted list of all GAV based on existing jar manifests
4. Any library (jar) for which we do not have at least one cross package Method Call is potentially unused and can be removed.

An example control flow graph (CFG) is given below that visualises the idea where we need to capture method calls across packages.

An example CodeQL query for [2] would look like this

This approach can also be used to identify unused block of code, including class, method or a package with minimal customisation of the above query.

Challenges and Constraints

The approach presented in this post works in general cases but fails to handle dynamically resolved or transitive dependencies. For example, consider external-lib-1 is dependent on external-lib-2. Our approach above will not consider this case. We did not attempt to solve this problem as we believe an application should only manage its immediate dependencies and let the build tool take care of transitive dependencies. Controlling external dependencies, including transitive dependencies, for security or quality gate requirements can be implemented using private repository manager and not really within the scope of this problem.

For comments or feedback, you can get in touch with me over Twitter 😀

If you are interesting in our work and want to solve complex problems in SaaS products, platform & cloud infrastructure engineering — we are hiring!

Solving Engineering Problems using Security Tools — Technical Debt Elimination using CodeQL was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.

We use Mirage at Chargebee and it is the one deed that we did and in return it has filled two of our needs 😇. Our experience with Mirage is quite exciting and we can’t wait to share it here! Let’s go already!

What is Mirage? 🤔

Mirage helps frontend developers to configure a mock server for the backend APIs and the server is centralised for both development and the test suite.

It’s just not a plain old API mocking tool, but has all the batteries included that helps to build complete frontend features even if our APIs doesn’t exist. Let’s say you wanna build up a product prototype rapidly fast with minimal overhead, that’s completely possible with Mirage.

Life before Mirage at Chargebee 😅

Before implementing Mirage, we had quite an inflexible approach where for testing, we were copy-pasting chunks of static fixtures as return values for jest mock functions here and there and testing over them.

This approach was just covering a set of cases (mostly covering the happy path of the users) for us and handling dynamic scenarios was a headache where we were forced to do a lot of work.

For the development of the features without the backend dependency, we were using axios interceptors for mocking the APIs. We were just routing the network calls made in the application to a local node server that handled the endpoints.

This obviously was not living alongside the rest of the application and forced us to switch contexts. Also here again we were tracing the happy path mostly and were not dealing with the network states and latency.

What makes Mirage a better approach? 😎

Mirage runs alongside the rest of the application, no new server processes or terminal windows are needed. It has an in-memory database that ensures referential integrity between different data models. Additionally, there are concepts like,

Factory

Factory layer helps to organize the data-creation logic. Mock server can be quickly put into a different state during development and as well during testing. During development, the in-memory database can be seeded via factories on seeds hook to load some initial data.

Serializer

Serializing layer hooks into the state where route handlers process data models to return a response. It is responsible for the format of the data returned. There are some built-in serializers included and also an option to customize the default behavior.

Woo-hoo! Mock server can be centralized 😇

This is one of the very useful features where the mock server can be shared between the actual development workflow and the test suite. With factories, various dynamic scenarios of a component can be tested.

Enough of theory? Let’s get practical! 🥳

This is the sample application we’ll be using to get a gist of Mirage’s capabilities. It displays random quotes from Game of Thrones on every load. It also includes a route with a dynamic segment that denotes the number of quotes requested (for ex: `/10` would query 10 quotes via the API and render the results)

The application is hosted here.

In the below playground, you can find the main.js where we have the application’s routes config. We are also spinning up the Mirage server for development environment. Now, Mirage can intercept all the network requests triggered from the application and respond with mock response.

The server config of the Mirage can be found here,

We’ve set up the quotes factory. This can come in quite handy to test various dynamic scenarios which we will see later. You can find the routes intercepted under routes method.

If we are spinning up Mirage server on environments like development , usually we will be passing through(instead of mocking) all the APIs to the actual server. Mirage exposes passthrough method on the server instance for this exact functionality.

We can also avoid passing through during active development, for ex: while the API is not yet ready. We can just construct our factories according to the spec of the API and let the mirage server intercept all the requests by enabling routes on development environment too.

Once everything is fine and the API is ready, we could just disable routes on the development environment and pass through them, Now the requests will be passing through the network layers hitting the actual servers. Things should just work!

There shouldn’t be any rework required if we’ve setup our Mirage models exactly mirroring the actual backend. Definitely some can have questions like,

Is this all worth it?

Are we over engineering doing UI without backend?

Instead can we have mock data just as data properties in UI components?

The real power comes when the Mirage mock server is also shared by the test environment. Just have a look at this component test file in the below playground,

The above file handles unit tests for the card component that renders the card view. See how quickly we are able to put the API response into different states and scenarios without copy pasting random mocks here and there! This is where the factories come into play.

To accomplish this, you can see that we’ve done nothing complex. We’ve just setup our quote factory to create required mock dynamically and added a route to intercept in a express style syntax! That’s it! Mirage also has batteries included to handle complex data relationships.

This kinda flexibility can’t be just achieved using data properties on UI components that hold mock data.

Some cons of using just data props for mocking can be,

• Integrity with backend can’t be ensured
• Data fetching, various network states and persistence can be very complex to handle
• There is no straightforward way to dynamically generate responses.
• It will only include user’s happy path.

On the other hand Mirage ticks all these boxes for us and we are efficiently able to handle all our cases here!

Do we have some alternatives? 🤤

Yes. MSW is a similar tool. It uses service worker under the hood to intercept network requests whereas Mirage just mocks out XMLHttpRequests and fetch API.

This is an advantage of MSW over Mirage as the requests are shown in the Network tab of the DevTools — they’re real HTTP requests from the perspective of the browser. This gives a great DX while doing actual development.

MSW’s mock server can also be used in test environment. The route handler gives a request and lets us write a response (interceptor functionality) but nothing more than that.

Mirage on the other hand provides a similar express style route handler API and also brings along an in-memory DB, support for relationships and factories that make creating different data scenarios easy.

We stuck with Mirage here as it came with batteries included for mocking in a way that faithfully reproduces the behavior of the production API that in turn helped us to test various dynamic scenarios and not just the happy path!

So that’s it. This is how beneficial Mirage is for us and we can’t wait to hear your experience with it in the comments. Also, shoot up anything you’ve, let’s keep the discussion on! 👋

Making API mocking easy with Mirage! was originally published in Engineering @ Chargebee on Medium, where people are continuing the conversation by highlighting and responding to this story.