Terrier DataLog

Terrier DataLog is a modern, fast and fully cross-platform CLI application, developed in C# (.NET 8), designed to manage logs, monitoring and data persistence in different environments — including Windows, Linux and (in the future) macOS.

Created with a focus on simplicity, efficiency and extensibility, Terrier DataLog offers a complete flow to start, stop and monitor background data collection processes, using an internal server with file-based communication (PID), in addition to supporting user-friendly configurations via YAML.

Runs natively on:

• Windows (x64)
• Linux (x64, Debian-based)

Distributed as a self-contained single-file application with no external dependencies.

Terrier DataLog includes an internal background server that can be started and stopped directly through CLI commands:

terrier start
terrier stop
terrier status

Processes are fully isolated through:

• User-local runtime directory
• PID files
• Dedicated logging system

The terrier configure command creates and manages a YAML configuration file containing:

• SQLite database path
• Endpoints
• Tokens
• Storage directories

Uses sqlite-net-pcl as the local database engine — efficient, lightweight, and ideal for desktop or server environments.

• Install via .deb on Linux (Debian/Ubuntu based)
• Install via .exe or extracted folder on Windows
• Global terrier command available after installation

Terrier DataLog is ideal for:

• Systems requiring continuous data collection (data logging)
• Monitoring applications for sensors, equipment, or processes
• Automation workflows that write into SQLite
• Internal company tools needing a lightweight daemon
• Developers wanting to embed a simple background “mini server”

• .NET 8 (C#)
• Self-contained single-file CLI
• SQLite via sqlite-net-pcl
• YAML for configuration
• Background process with PID control
• .deb packaging for Linux distribution

Download the official Windows x64 installer from the release page:

👉 https://github.com/micilini/TerrierDataLog/releases/tag/windows-x64-1.0.0

Run the installer and follow the setup steps.
After installation, the terrier command will be available globally in any terminal through the system PATH.

Download the official Debian/Ubuntu .deb package:

👉 https://github.com/micilini/TerrierDataLog/releases/tag/debian-ubuntu-x64

Install the package using your system's package manager (e.g., double-click or dpkg -i).
Once installed, the terrier command becomes globally available in any terminal via the system PATH.

• Use Ctrl+C to stop the tail.
• Combine commands with --json for integration with external tools or UI interfaces.

Before running TerrierDataLog for the first time, you must generate the initial configuration file.
This is done using:

terrier configure

The command will guide you interactively and create a config.yml inside your user-local data directory.
Each prompt includes a default value — just press ENTER to accept defaults.

Below is a detailed explanation of each field requested during the configuration process.

When the configuration completes, TerrierDataLog will show:

• The path where config.yml was saved
• The generated server token
• A note suggesting you run:

terrier status

This confirms that the configuration file is valid and readable.

Below is an example of a generated configuration file after running terrier configure:

server:
  httpListen: 127.0.0.1:4925
  udpListen: 127.0.0.1:5514
  token: 539074ea9378f29197d2187773d0fe28
  limits:
    maxBodyKb: 1024
    bulkMaxItems: 100

storage:
  sqlitePath: C:\Users\William Lima\AppData\Local\TerrierDataLog\terrier.db
  fileRotations:
    enabled: true
    dir: C:\Users\William Lima\AppData\Local\TerrierDataLog\archive
    period: daily
    compress: true
    dbMaxSizeMb: 3
  retention:
    maxDays: 14
    maxSizeMb: 5120

After completing the configuration step, you can launch the Terrier background server using:

terrier start

This command starts the internal runtime process responsible for handling log ingestion, health checks, archiving, and SQLite persistence.
Once running, Terrier exposes both HTTP and (optional) UDP endpoints defined in your config.yml.

You can verify if the service is active using:

terrier status

If the HTTP listener is active, Terrier will accept incoming log requests at the configured address and port.

All HTTP requests must include the following headers:

Example:

Accept: application/json
X-Terrier-Token: 539074ea9378f29197d2187773d0fe28

Below are the available endpoints.

Checks if the Terrier background process is alive and responding.

curl -X GET http://127.0.0.1:4925/healthz \
  -H "Accept: application/json" \
  -H "X-Terrier-Token: 539074ea9378f29197d2187773d0fe28"

This endpoint accepts a single JSON document representing one structured log.

{
  "app_id": "orchestrator",
  "level": "info",
  "title": "hello",
  "message": "world",
  "kv": { "foo": "bar" }
}

curl -X POST http://127.0.0.1:4925/logs \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "X-Terrier-Token: 539074ea9378f29197d2187773d0fe28" \
  -d '{
        "app_id": "orchestrator",
        "level": "info",
        "title": "hello",
        "message": "world",
        "kv": { "foo": "bar" }
      }'

Used for batch ingestion. Accepts an array of log objects.

[
  {
    "app_id": "orchestrator",
    "level": "info",
    "title": "ok-1",
    "message": "m1"
  },
  {
    "app_id": "orchestrator",
    "level": "error",
    "title": "oops",
    "details": "..."
  },
  {
    "app_id": "",
    "level": "warn",
    "title": "missing-app"
  } // invalid: app_id cannot be empty
]

curl -X POST http://127.0.0.1:4925/logs/bulk \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "X-Terrier-Token: 539074ea9378f29197d2187773d0fe28" \
  -d '[ 
        { "app_id": "orchestrator", "level": "info", "title": "ok-1", "message": "m1" },
        { "app_id": "orchestrator", "level": "error", "title": "oops", "details": "..." },
        { "app_id": "", "level": "warn", "title": "missing-app" }
      ]'

UDP is recommended for lightweight, non-critical, fast-fire log messages.

$endpointIp = "127.0.0.1"
$endpointPort = 5514

# Payload JSON
$payload = @{
    app_id  = "udp-tester"
    env     = "dev"
    level   = "info"
    title   = "UDP test"
    message = "Hello from UDP via PS1!"
} | ConvertTo-Json -Depth 3

# Convert to bytes
$bytes = [System.Text.Encoding]::UTF8.GetBytes($payload)

# Send via UDP
$client = New-Object System.Net.Sockets.UdpClient
$client.Send($bytes, $bytes.Length, $endpointIp, $endpointPort) | Out-Null
$client.Close()

Write-Host "UDP sent successfully!"

You can use socat or netcat (nc):

echo '{"app_id":"udp-tester","level":"info","title":"linux-test","message":"Hello from Linux UDP"}' \
  | nc -w0 -u 127.0.0.1 5514

echo '{"app_id":"udp-client","level":"info","title":"socat-test"}' \
  | socat - UDP:127.0.0.1:5514

UDP messages do not require headers or authentication tokens.

After sending logs, you can monitor incoming data using:

terrier tail

This displays real-time aggregated data from SQLite and the log buffer.

terrier tail allows you to watch incoming logs in real time, directly from the TerrierDataLog SQLite pipeline.
It continuously follows new entries as they are written, similar to tail -f, but enriched with structured fields such as timestamps, log levels, app identifiers, and message metadata.

Usage is straightforward:

terrier tail

You can also enable full-application streaming or adjust the refresh interval:

terrier tail --allApps
terrier tail --interval 5

Press Ctrl+C at any time to exit the live stream.

This command is ideal for debugging, monitoring services, inspecting background workers, or validating real-time log ingestion during development.

Want to help improve TerrierDataLog? You're welcome to contribute!

If you'd like to add new features, improve existing functionality, or submit translations or documentation updates, simply create a Pull Request (PR).
Bug fixes, enhancements, and quality-of-life improvements are also highly appreciated.

Feel free to open Issues as well if you have ideas, suggestions, or questions about future development.

TerrierDataLog is open-source and released under the MIT License.
You are free to use, modify, and distribute this project as long as you retain the original license notice.