A release that adds an API server, support for submitting compression jobs from the UI, improved performance for concurrent compression jobs, and other improvements.

This release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json: Add a RESTful API server with support for searching compressed logs. (#1534, #1575, #1590, #1612, #1630, #1636, #1663, #1684, #1685, #1695, #1709)
• clp-json/clp-text: Add support for submitting local filesystem compression jobs from the UI. (#1294, #1705, #1707)
• clp-json/clp-text: Add a JSON schema definition for the package’s configuration file. (#1654, #1671)

Bug fixes & improvements

• clp-json/clp-text: Allow concurrent compression job processing by processing batches of compression tasks per job. (#1637)
• webui: Visual and functional improvements to the UI for querying archives through Presto. (#1578, #1569, #1631, #1638)
• KV-IR: Performance improvements for compressing KV-IR files into clp-json/clp-s archives. (#1468, #1544, #1561, #1607)
• clp-json/clp-text: Configuration renaming for consistency:
  • Rename .yml files to .yaml . (#1617)
    • Breaking: This is a rename of CLP’s config files (etc/credentials.yml to etc/credentials.yaml and etc/clp-config.yml to etc/clp-config.yaml).
  • Rename user to username; Update default user to clp-user. (#1610)
    • Breaking: This is a change to CLP’s credential file format (etc/credentials.yaml).
• clp-json/clp-text: Add support for multiple database user credentials. (#1655, #1718)
  • Breaking: CLP’s credentials file now requires root user credentials for the database.

View the full changelog for more details.

Thanks to @20001020ycx, @Bill-hbrhbr, @davemarco, @davidlion, @Eden-D-Zhang, @gibber9809, @hoophalab, @junhaoliao, @kirkrodrigues, @LinZhihao-723, @quinntaylormitchell, and @sitaowang1998 for their contributions.

A release that adds an MCP server, a guided query UI for Presto, expands S3 compression options, migrates CLP’s orchestration to use Docker Compose, and much more.

This release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json: Add an MCP server to facilitate integration with AI. (#1368, #1388, #1401, #1436, #1454, #1456, #1457, #1481, #1489, #1491, #1494, #1515)
• webui: Add a guided experience for constructing filters in CLP’s UI for Presto. (#1276, #1293, #1300, #1304, #1314, #1315, #1332, #1361, #1374, #1376, #1393, #1398, #1421, #1450, #1459, #1465, #1482, #1490, #1511, #1521, #1543)
• clp-json: Add basic support for compressing unstructured text logs. (#1434, #1460, #1505)
• clp-json: Add support for compressing multiple object URLs from S3. (#1407, #1476, #1510)
  • Breaking: sbin/compress.sh no longer supports compressing S3 URIs; that functionality has been moved into a new script, sbin/compress-from-s3.sh.
• clp-json/clp-text: Migrate package orchestration to Docker Compose. (#1178, #1334, #1335, #1357, #1389, #1394, #1501, #1502, #1512, #1518, #1522, #1525, #1528, #1532, #1545, #1555, #1559, #1562, #1563, #1567, #1572, #1577)
  • Breaking:
    • CLP now requires Docker Compose but no longer requires Python.
    • The default storage and query engines has been changed from clp to clp-s.
    • sbin/start-clp.sh and sbin/stop-clp.sh no longer take component names or targets as input. The multi-host deployment guide has been updated accordingly to use Docker Compose directly.
    • Component logs are no longer stored in host-specific directories (e.g., logs will be stored in var/log/query_scheduler rather than var/log/<host>/query_scheduler).
    • The --num-workers argument has been replaced by environment variables.
    • Much of the contents of the package have been moved into the package’s container image (e.g., ./bin no longer exists).
  • This also fixes #1220 where the compression_scheduler’s log level config was set to the query_scheduler’s log level config.

Bug fixes & improvements

• webui: Display full dataset names in the dataset selector popup. (#1400)
• webui: Provide improved calculation of time range for queries. (#1336)
• webui: Show query speed in search status. (#1429)
• clp-json/clp-s: Fix incorrect rounding when comparing of float literals with integer values (fixes #1375). (#1369)
• clp-json/clp-text: Write compression task failure errors to a log, and store the log path (instead of the errors) in the job’s status (fixes #716). (#1425)
• clp-json: Expand ~ in aws_config_directory config (fixes #1257). (#1258)
• clp-json/clp-text: Ensure at least one worker is created even on single-core machines (fixes #1509).

View the full changelog for more details.

Thanks to @20001020ycx, @All-less, @AVMatthews, @Bill-hbrhbr, @davemarco, @Eden-D-Zhang, @gibber9809, @hoophalab, @junhaoliao, @kirkrodrigues, @LinZhihao-723, @quinntaylormitchell, @rishikeshdevsot, @sitaowang1998, and @wraymo for their contributions.

A release that adds support for retaining the format of floating-point numbers from JSON logs, support for CLP’s UI to work with Presto, and some other improvements and bug fixes.

This release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json: Support for retaining the format of floating-point numbers. (#1176, #1321)
• webui: Support for CLP’s UI to work with Presto. (#1095, #1108, #1127, #1150, #1167, #1168, #1175, #1179, #1187, #1188, #1191, #1198, #1206, #1209, #1224, #1236, #1244, #1248, #1250, #1251, #1265, #1280, #1285, #1306, #1310)
  • NOTE: This adds the configuration key presto to etc/clp-config.yml.
• clp-json: Support for compressing Zstandard-compressed JSONL files as well as CLP KV-IR files. (#1312)

Bug fixes & improvements

• clp-json/clp-text: Add support for gracefully shutting down the compression scheduler and compression workers (resolves #1037). (#1169, #1323)
• core: Unescape variable strings before dictionary lookup in EncodedVariableInterpreter::encode_and_search_dictionary (fixes #590). (#1270)
• clp-s: Handle pure wildcards and unexpected literal types correctly in EvaluateTimestampIndex (fixes #1096). (#1277)
• kv-ir: Add support for getting the number of log events read from the deserializer. (#1282)
• kv-ir: Add support for duplicate columns in projections. (#1245)
• clp-json/clp-text: Consider all non-loopback IPv4s when selecting listen address to receive search results (fixes #1316). (#1317)
• clp-json/clp-text: Fail job and report failure to user for compression jobs that encounter at least one invalid input path (fixes #308). (#1125)
• clp-json: Warn the user if they do not use the --timestamp-key flag when compressing with the clp-s storage engine. (#1283)
• webui: Expose rate limit configuration (fixes #1019); Increase rate limit to 1000 req/min (fixes #1020). (#1234)
  • NOTE: This adds the configuration key webui.rate_limit to etc/clp-config.yml.

View the full changelog for more details.

Thanks to @anlowee, @AVMatthews, @Bill-hbrhbr, @davemarco, @gibber9809, @haiqi96, @hoophalab, @junhaoliao, @kirkrodrigues, @LinZhihao-723, and @quinntaylormitchell for their contributions.

A release that includes support for configuring retention periods for archives, deleting datasets entirely, and some other features and bug fixes.

This release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json/clp-text: Add support for configuring retention periods for archives and search results. (#1035, #1181, #1205, #1231)
• clp-json: Add dataset-manager tool to support listing datasets, and deleting them entirely. (#1144, #1215, #1225)

Bug fixes & improvements

• clp-json/clp-text: Fix scheduler freezes by updating celery to 5.5.3 with the redis extra (replacing the direct redis dependency) (fixes #1059). (#1213)
• clp-json/clp-text: Remove dependency on native libraries for scripts run on the host (fixes #895, #1185). (#1105, #1197)
• core-clp: Preserve escaped ?-wildcards in queries (fixes #243). (#1070)
• core: Unify clp-s and clp’s unstructured text parsing and search code. (#1101, #1103, #1112, #1138, #1143, #1163)
  • Breaking: This is a change to the archive format used by clp-json (and the core clp-s binary).
• webui: Delete old search results after 60min by default. (#1231)
  • Breaking: Previously, the results of the last query executed before refreshing the page would be retained indefinitely.
• webui: Update query time range on any change in the UI to prevent query submission with out-of-date time range. (#1171)
• webui: Focus/refocus query input box for ease of use. (#1160)
• clp-json/clp-text: Only mount stream_output_dir when stream storage type is FS, to allow running the webui on a different node when using S3 storage. (#1129)
• clp-json/clp-text: Explicitly convert enum to integer to ensure accurate conversions when using the mysql Python library. (#1133)
• webui: Cast BIGINT values as UNSIGNED to fix MySQL-specific type errors in dashboard stats queries (fixes #1137). (#1136)
• clp-json/clp-text: Mark incomplete jobs as failed when schedulers restart. (#1208)
• clp-json/clp-text: Add verbose logging option to archive_manager and simplify output on errors. (#1173)
• clp-json/clp-text: Add query_engine option to clp-config.yml to support starting only compression and UI components when using the Presto query engine. (#1095)
  • Breaking: This is a change to clp-json/clp-text’s config file format.
• core: Replace YAML config with CLI args and env vars for metadata DB. (#1148)
  • Breaking: This is a change to how the core clp/clp-s binaries can be configured to use a MySQL-based database (the change is transparent to clp-json/clp-text users).
• core: Address CVE-2024-3094 and CVE-2025-31115 for xz/lzma dependency (fixes #1093). (#1094)

View the full changelog for more details.

Thanks to @anlowee, @Bill-hbrhbr, @davemarco, @davidlion, @gibber9809, @haiqi96, @hoophalab, @jackluo923, @junhaoliao, @kirkrodrigues, @LinZhihao-723, @quinntaylormitchell, @SharafMohamed, and @wraymo for their contributions.

A release that includes support for organizing compressed logs into datasets, a revamped UI, support for a new IR format designed for structured log events, and more advanced metadata storage within clp-json archives. This release also includes some other features and bug fixes.

The CLP release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json: Add support for organizing compressed logs into datasets, where each dataset can be searched separately. (#831, #839, #864, #868, #923, #1023, #1029, #1036, #1050, #1091)
• clp-json/clp-text: Replace the UI with a revamped version that has better performance while being more extensible and maintainable. (#645, #647, #791, #797, #804, #807, #824, #830, #848, #866, #875, #880, #891, #892, #899, #910, #912, #913, #915, #931, #933, #936, #937, #939, #941, #949, #950, #954, #956, #957, #958, #959, #962, #967, #972, #973, #974, #988, #990, #991, #993, #995, #997, #999, #1000, #1001, #1003, #1007, #1013, #1015, #1016, #1042, #1053, #1061, #1063, #1064, #1074, #1078, #1079, #1083)
• core: Add support for a new CLP intermediate representation (IR) stream format which is designed for compressing and searching structured (e.g., JSON) log events. (#411, #446, #502, #507, #511, #549, #653, #677, #695, #706, #726, #728, #731, #733, #769, #833, #856, #859, #860, #863, #873, #882, #888, #890, #909, #914)
• clp-s: Add support for storing metadata about ranges of log events (e.g., the name of the log file to which they were printed) in an archive. (#847, #889, #908, #929, #1005, #1026, #1043)

Bug fixes & improvements

• clp-s: Improve compression ratio by delta-encoding the log-order column. (#1021)
• clp-json/clp-text: Add support for configuring the compression level. (#774)
• clp-json: Add support for more AWS authentication methods (e.g., EC2 instance metadata, environment variables, AWS profiles, etc.). (#743, #788, #852)
• clp-json/clp-text: Don't remap output paths when mounting them into CLP's execution container (fixes #960). (#998)
• core: Try to exhaust Zstd's internal buffers when they might contain unconsumed data. (fixes #976). (#977)
• clp-json/clp-text: Use job duration for final compression speed summary. (#823)
• core: Update DictionaryReader::get_entry_matching_value to handle case-insensitive searches (fixes #648). (#690)
• webui: Improve tracking of compression jobs submitted in quick succession (fixes #667). (#679)

View the full changelog for more details.

Thanks to @aestriplex, @anlowee, @AVMatthews, @Bill-hbrhbr, @davemarco, @davidlion, @Eden-D-Zhang, @gibber9809, @haiqi96, @Henry8192, @hoophalab, @jackluo923, @junhaoliao, @kirkrodrigues, @LinZhihao-723, @quinntaylormitchell, @SharafMohamed, @sitaowang1998, and @wraymo for their contributions.

A release that adds support for using clp-json to both compress logs from object storage and store archives on object storage (docs). This release also includes some other features and bug fixes.

NOTE: We are bumping the minor version due to a breaking format change in clp-text/clp-json’s jobs table format and in clp-json/clp-s’ archive format. If this affects you, reach out and we can help with version migration.

The CLP release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• clp-json: Add support for compressing logs from S3. (#651)
• clp-json: Add support for storing, decompressing, and searching archives from S3. (#634, #674, #683)
• clp-json: Add support for viewing logs from S3. (#662, #673, #678)
• clp-s: Add support for compressing logs from S3. (#639)
• clp-s: Add support for writing single-file archives. (#563)
• clp-s: Add support for reading and searching single-file archives, including from S3. (#656)
• Add BoundedReader to prevent out-of-bound reads in segmented input streams. (#624)

Bug fixes & improvements

• clp-json: Add option to output search results as raw logs. (#641)
• clp-package: Unify the metadata schema for JSON and IR streams. (#620)
• clp-package: Enable replica set for the MongoDB results cache and configure it when starting the package. (#632)
• clp: Advance to the next message when a message has an out-of-range timestamp when searching archives (fixes #659). (#660)
• clp-s: Unescape string values during ingestion and fix support for search using escape sequences. (#622)
• clp-s: Improve error reporting for directory-creation failure during compression. (#671, #684)
• clp-s: Rename tables section to use segment numbering scheme. (#666)
• Handle 0-byte reads when BufferReader's underlying buffer is fully consumed. (#687)
• Add missing libcurl4 dependency to clp-core and package execution containers. (#670)
• Disable file system translation in checkinstall during dependency installation (fixes #642). (#644)

View the full changelog for more details.

Thanks to @AVMatthews, @Bill-hbrhbr, @Eden-D-Zhang, @LinZhihao-723, @gibber9809, @haiqi96, @jackluo923, @junhaoliao, and @kirkrodrigues for their contributions.

A release that adds support for viewing JSON search results in context by opening the archive that contains them using the log viewer. This release also includes some features that improve usability.

The CLP release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• Support for viewing JSON search results in context by opening the archive that contains them using the log viewer. (#569, #584, #596, #600, #615)
  • Note that the log viewer doesn’t open the archive directly but rather CLP decompresses the archive into chunks of JSONL files that the log viewer opens. In a future release, these chunks will be IR files to lower resource usage.
• clp-json: Support for querying fields whose keys contain periods by escaping them with a backslash. (#560, #617)
• Support for deleting archives that are entirely within a time range. (#594)

Bug fixes & improvements

• Homebrew path detection for mariadb-connector-c to fix macOS build failures. (#582)

View the full changelog for more details.

Thanks to @AVMatthews, @Bill-hbrhbr, @LinZhihao-723, @anlowee, @gibber9809, @haiqi96, @junhaoliao, @kirkrodrigues, and @wraymo for their contributions.

A release that uses the rewritten log viewer and includes new features for clp-s, bug fixes, and improvements.

NOTE: We are bumping the minor version due to a breaking format change in clp-json/clp-s’ archive format. If this affects you, reach out and we can help with version migration.

The CLP release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

New features

• An upgrade to the rewritten log viewer. (#562, #565)
  • The rewrite is based around the clp-ffi-js library (which uses WebAssembly) and improves performance for switching pages by 2.77-4.67x, and log-level filtering by 1.64-4.84x.
  • The rewrite also includes a UI refresh, more fine-grained log-level filtering, support for exporting the decompressed logs, and support for opening larger log files.
  • NOTE: The rewrite also includes support for formatting and viewing JSON logs which will be integrated with CLP in the next release.
• Support for building on CentOS Stream 9. (#521)
  • NOTE: We have dropped support for building on CentOS 7.4.
• clp-s:
  • Support for projecting a subset of columns during search. (#510)
    • Support for projection from the package will be integrated in a future release.
  • Packing of small tables into a single compression stream which can improve compression ratio as well as compression and search performance. (#466)

Bug fixes & improvements

• Fixes to allow CLP’s core to build with shared linking again. (#526)
• Upgrades for vulnerable dependencies. (#536)

View the full changelog for more details.

Thanks to @anlowee, @Henry8192, @LinZhihao-723, @diy1, @gibber9809, @junhaoliao, @kirkrodrigues, and @zechenericduan for their contributions.

A bug-fix release to improve error reporting, improve the UI, and fix some macOS-specific issues.

The CLP release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

Bug fixes & improvements

• clp-s: Report exactly where parsing error occurs when parsing JSON (fixes #514). (#503)
• webui: Improve UI of links for viewing search results in context. (#515)
• core: Add support for retrieving CURL error messages, handle unexpected CURL return code on macOS, and log such codes in tests (fixes #519). (#517)
• core: Mark spdlog specialization functions as const (fixes #524). (#525)

View the full changelog for more details.

Thanks to @AVMatthews, @Bill-hbrhbr, @gibber9809, @haiqi96, @junhaoliao, @kirkrodrigues, and @LinZhihao-723 for their contributions.

A release with support for viewing text search results in context by opening the file that contains them using using the log viewer. In a future release, we will also add support for viewing JSON search results in context. This release also includes a handful of bug fixes.

The CLP release includes two tars:

• clp-json for compressing and searching JSON logs
• clp-text for compressing and searching unstructured text logs

Docs

The docs for this release are available here.

Features/Changes

Support for viewing text search results in context by opening the file that contains them using the log viewer. (#420, #424, #435, #437, #440, #442, #449, #451, #452, #458, #460, #468, #470, #472, #474, #476, #481, #485, #489, #490)

Bug-fixes

• clp-package: Add support for running package as root (fixes #500). (#464)
• clp-s: Ensure the option to print archive stats remains set when splitting archives. (#462)
• clp-s: Correctly report uncompressed size of archives when splitting archives (fixes #469). (#463)
• core-clp: Add class to encapsulate libcurl's global resource management to fix a race condition. (#461)
• core-clp: Add a more complete UTF-8 validation function to filter out binary data. (#477)
• core-clp: Fix ArchiveMetadata deserialization by deserializing previously missed field. (#488)
• webui: Optimize Meteor bundle size (fixes #498). (#499)

View the full changelog for more details.

Thanks to @Bill-hbrhbr, @gibber9809, @haiqi96, @Henry8192, @junhaoliao, @kirkrodrigues, @LinZhihao-723, and @wraymo for their contributions.