Tutorial on PIGSTY

Getting Started

Mon, 01 Jan 0001 00:00:00 +0000

Here is a simple getting started tutorial to help you experience the core capabilities of the PIG package manager.

Short Version

curl -fsSL https://repo.pigsty.io/pig | bash # Install PIG from Cloudflare
pig repo set # One-time setup for Linux, Pigsty + PGDG repos (overwrites!)
pig install -v 18 -y pg18 pg_duckdb vector # Install PG 18 kernel, pg_duckdb, pgvector extensions...

Installation

You can install pig with the following command:

China Mainland:

curl -fsSL https://repo.pigsty.cc/pig | bash

Global (Cloudflare CDN):

Installation

Mon, 01 Jan 0001 00:00:00 +0000

Source: https://patroni.readthedocs.io/en/latest/installation.html

Pre-requirements for Mac OS

To install requirements on a Mac, run the following:

brew install postgresql etcd haproxy libyaml python

Psycopg

Starting from psycopg2-2.8 the binary version of psycopg2 will no longer be installed by default. Installing it from the source code requires C compiler and postgres+python dev packages. Since in the python world it is not possible to specify dependency as psycopg2 OR psycopg2-binary you will have to decide how to install it.

User Guide (Debian/Ubuntu)

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

This user guide is intended to be followed sequentially from beginning to end — each section depends on the last. For example, the Restore section relies on setup that is performed in the Quick Start section. Once pgBackRest is up and running then skipping around is possible but following the user guide in order is recommended the first time through.

Although the examples in this guide are targeted at Debian/Ubuntu and PostgreSQL 16, it should be fairly easy to apply the examples to any Unix distribution and PostgreSQL version. The only OS-specific commands are those to create, start, stop, and drop PostgreSQL clusters. The pgBackRest commands will be the same on any Unix system though the location of the executable may vary. While pgBackRest strives to operate consistently across versions of PostgreSQL, there are subtle differences between versions of PostgreSQL that may show up in this guide when illustrating certain examples, e.g. PostgreSQL path/file names and settings.

User Guide (RHEL)

Mon, 01 Jan 0001 00:00:00 +0000

Introduction

Although the examples in this guide are targeted at RHEL and PostgreSQL 13, it should be fairly easy to apply the examples to any Unix distribution and PostgreSQL version. The only OS-specific commands are those to create, start, stop, and drop PostgreSQL clusters. The pgBackRest commands will be the same on any Unix system though the location of the executable may vary. While pgBackRest strives to operate consistently across versions of PostgreSQL, there are subtle differences between versions of PostgreSQL that may show up in this guide when illustrating certain examples, e.g. PostgreSQL path/file names and settings.

PgBouncer compilation and installation

Mon, 01 Jan 0001 00:00:00 +0000

Source: https://www.pgbouncer.org/install.html

Building

PgBouncer depends on few things to get compiled:

GNU Make 3.81+
Libevent 2.0+
pkg-config
OpenSSL 1.0.1+ for TLS support
(optional) c-ares as alternative to Libevent’s evdns
(optional) LDAP libraries
(optional) PAM libraries

When dependencies are installed just run:

$ ./configure --prefix=/usr/local
$ make
$ make install

If you are building from Git, or are building for Windows, please see separate build instructions below.

DNS lookup support

PgBouncer does host name lookups at connect time instead of just once at configuration load time. This requires an asynchronous DNS implementation. The following table shows supported backends and their probing order:

About

Mon, 01 Jan 0001 00:00:00 +0000

Inventory

Mon, 01 Jan 0001 00:00:00 +0000

Every Pigsty deployment corresponds to an Inventory that describes key properties of the infrastructure and database clusters.

Configuration File

Pigsty uses Ansible YAML configuration format by default, with a single YAML configuration file pigsty.yml as the inventory.

~/pigsty
 ^---- pigsty.yml # <---- Default configuration file

You can directly edit this configuration file to customize your deployment, or use the configure wizard script provided by Pigsty to automatically generate an appropriate configuration file.

Configuration Structure

The inventory uses standard Ansible YAML configuration format, consisting of two parts: global parameters (all.vars) and multiple groups (all.children).

Get Started

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty uses a scalable architecture design, suitable for both large-scale production environments and single-node development/demo environments. This guide focuses on the latter.

If you intend to learn about Pigsty, you can start with the Quick Start single-node deployment. A Linux virtual machine with 1C/2G is sufficient to run Pigsty.

You can use a Linux MiniPC, free/discounted virtual machines provided by cloud providers, Windows WSL, or create a virtual machine on your own laptop for Pigsty deployment. Pigsty provides out-of-the-box Vagrant templates and Terraform templates to help you provision Linux VMs with one click locally or in the cloud.

Single-Node Installation

Mon, 01 Jan 0001 00:00:00 +0000

This is the Pigsty single-node install guide. For multi-node HA prod deployment, refer to the Deployment docs.

Pigsty single-node installation consists of three steps: Install, Configure, and Deploy.

Summary

Prepare a node with compatible OS, and run as an admin user with nopass ssh and sudo:

curl -fsSL https://repo.pigsty.io/get | bash;

curl -fsSL https://repo.pigsty.cc/get | bash;

This command runs the install script, downloads and extracts Pigsty source to your home directory and installs dependencies. Then complete Configure and Deploy:

Docker Deployment

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty is designed for native Linux, but can also run in Linux containers with systemd. If you don’t have native Linux (e.g., macOS or Windows), use Docker to spin up a local single-node Pigsty for testing.

Quick Start

Enter the docker/ dir in Pigsty source and launch with one command:

cd ~/pigsty/docker
make launch # Start container + generate config + deploy

After deployment, access services:

Service	URL / Command	Credentials
SSH	`ssh root@localhost -p 2222`	Password: `pigsty`
Web Portal	http://localhost:8080	-
Grafana	http://localhost:8080/ui	`admin` / `grafana_admin_password`
PostgreSQL	`psql 'postgres://dbuser_dba:<pg_admin_password>@localhost:5432/postgres'`	`pg_admin_password`

make launch runs ./configure -g internally to generate random passwords. You can check them with:

Web Interface

Mon, 01 Jan 0001 00:00:00 +0000

After single-node installation, you’ll have the INFRA module installed on the current node, which includes an out-of-the-box Nginx web server.

The default server configuration provides a WebUI graphical interface for displaying monitoring dashboards and unified proxy access to other component web interfaces.

Access

You can access this graphical interface by entering the deployment node’s IP address in your browser. By default, Nginx serves on standard ports 80/443.

Direct IP Access	Domain (HTTP)	Domain (HTTPS)	Demo
`http://10.10.10.10`	`http://i.pigsty`	`https://i.pigsty`	`https://demo.pigsty.io`

Getting Started with PostgreSQL

Mon, 01 Jan 0001 00:00:00 +0000

PostgreSQL (abbreviated as PG) is the world’s most advanced and popular open-source relational database. Use it to store and retrieve multi-modal data.

This guide is for developers with basic Linux CLI experience but not very familiar with PostgreSQL, helping you quickly get started with PG in Pigsty.

We assume you’re a personal user deploying in the default single-node mode. For prod multi-node HA cluster access, refer to Prod Service Access.

Basics

In the default single-node installation template, you’ll create a PostgreSQL database cluster named pg-meta on the current node, with only one primary instance.

Customize Pigsty with Configuration

Mon, 01 Jan 0001 00:00:00 +0000

Besides using the configuration wizard to auto-generate configs, you can write Pigsty config files from scratch. This tutorial guides you through building a complex inventory step by step.

If you define everything in the inventory upfront, a single deploy.yml playbook run completes all deployment—but it hides the details.

This doc breaks down all modules and playbooks, showing how to incrementally build from a simple config to a complete deployment.

Minimal Configuration

The simplest valid config only defines the admin_ip variable—the IP address of the node where Pigsty is installed (admin node):

Run Playbooks with Ansible

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty uses Ansible to manage clusters, a very popular large-scale/batch/automation ops tool in the SRE community.

Ansible can use declarative approach for server configuration management. All module deployments are implemented through a series of idempotent Ansible playbooks.

For example, in single-node deployment, you’ll use the deploy.yml playbook. Pigsty has more built-in playbooks, you can choose to use as needed.

Understanding Ansible basics helps with better use of Pigsty, but this is not required, especially for single-node deployment.

Offline Installation

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty installs from Internet upstream by default, but some envs are isolated from the Internet. To address this, Pigsty supports offline installation using offline packages. Think of them as Linux-native Docker images.

Overview

Offline packages bundle all required RPM/DEB packages and dependencies; they are snapshots of the local APT/YUM repo after a normal installation.

In serious prod deployments, we strongly recommend using offline packages. They ensure all future nodes have consistent software versions with the existing env, and avoid online installation failures caused by upstream changes (quite common!), guaranteeing you can run it independently forever.

Slim Installation

Mon, 01 Jan 0001 00:00:00 +0000

If you only want HA PostgreSQL database cluster itself without monitoring, infra, etc., consider Slim Installation.

Slim installation has no INFRA module, no monitoring, no local repo—just ETCD and PGSQL and partial NODE functionality.

Slim installation is suitable for:

Only needing PostgreSQL database itself, no observability infra required.
Extremely resource-constrained envs unwilling to bear infra overhead (~0.2 vCPU / 500MB on single node).
Already having external monitoring system, wanting to use your own unified monitoring framework.
Not needing the Grafana visualization dashboard component.

Limitations of slim installation:

No INFRA module, cannot use WebUI and local software repo features.
Offline Install is limited to single-node mode; multi-node slim install can only be done online.

Overview

To use slim installation, you need to:

Security Tips

Mon, 01 Jan 0001 00:00:00 +0000

For Demo/Dev single-node deployments, Pigsty’s default config is secure enough as long as you change default passwords.

If your deployment is exposed to Internet or office network, consider adding firewall rules to restrict port access and source IPs for enhanced security.

Additionally, we recommend protecting Pigsty’s critical files (config files and CA private key) from unauthorized access and backing them up regularly.

For enterprise prod envs with strict security requirements, refer to the Deployment - Security Hardening documentation for advanced configuration.

Deployment

Mon, 01 Jan 0001 00:00:00 +0000

Unlike Getting Started, production Pigsty deployments require more Architecture Planning and Preparation.

This chapter helps you understand the complete deployment process and provides best practices for production environments.

Before deploying to production, we recommend testing in Pigsty’s Sandbox to fully understand the workflow. Use Vagrant to create a local 4-node sandbox, or leverage Terraform to provision larger simulation environments in the cloud.

For production, you typically need at least three nodes for high availability. You should understand Pigsty’s core Concepts and common administration procedures, including Configuration, Ansible Playbooks, and Security Hardening for enterprise compliance.

Install Pigsty for Production

Mon, 01 Jan 0001 00:00:00 +0000

This is the Pigsty production multi-node deployment guide. For single-node Demo/Dev setups, see Getting Started.

Summary

Prepare nodes with SSH access following your architecture plan, install a compatible Linux OS, then execute with an admin user having passwordless ssh and sudo:

curl -fsSL https://repo.pigsty.io/get | bash; # International
curl -fsSL https://repo.pigsty.cc/get | bash; # Backup Mirror

This runs the install script, downloading and extracting Pigsty source to your home directory with dependencies installed. Complete configuration and deployment to finish.

Before running deploy.yml for deployment, review and edit the configuration inventory: pigsty.yml.

Prepare Resources for Serious Deployment

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty runs on nodes (physical machines or VMs). This document covers the planning and preparation required for deployment.

Node

Pigsty currently runs on Linux kernel with x86_64 / aarch64 architecture. A “node” refers to an SSH accessible resource that provides a bare Linux OS environment. It can be a physical machine, virtual machine, or a systemd-enabled container equipped with systemd, sudo, and sshd.

Deploying Pigsty requires at least 1 node. You can prepare more and deploy everything in one pass via playbooks, or add nodes later. The minimum spec requirement is 1C1G, but at least 1C2G is recommended. Higher is better—no upper limit. Parameters are auto-tuned based on available resources.

Planning Architecture and Nodes

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty uses a modular architecture. You can combine modules like building blocks and express your intent through declarative configuration.

Common Patterns

Here are common deployment patterns for reference. Customize based on your requirements:

Pattern	INFRA	ETCD	PGSQL	MINIO	Description
Single-node (`meta`)	1	1	1		Single-node deployment default
Slim deploy (`slim`)		1	1		Database only, no monitoring infra
Infra-only (`infra`)	1				Monitoring infrastructure only
Rich deploy (`rich`)	1	1	1	1	Single-node + object storage + local repo with all extensions

Multi-node Pattern	INFRA	ETCD	PGSQL	MINIO	Description
Two-node (`dual`)	1	1	2		Semi-HA, tolerates specific node failure
Three-node (`trio`)	3	3	3		Standard HA, tolerates any one failure
Four-node (`full`)	1	1	1+3		Demo setup, single INFRA/ETCD
Production (`simu`)	2	3	n	n	2 INFRA, 3 ETCD
Large-scale (custom)	3	5	n	n	3 INFRA, 5 ETCD

Your architecture choice depends on reliability requirements and available resources. Serious production deployments require at least 3 nodes for HA configuration. With only 2 nodes, use Semi-HA configuration.

Setup Admin User and Privileges

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty requires an OS admin user with passwordless SSH and Sudo privileges on all managed nodes.

This user must be able to SSH to all managed nodes and execute sudo commands on them.

User

Typically use names like dba or admin, avoiding root and postgres:

Using root for deployment is possible but not a production best practice.
Using postgres (pg_dbsu) as admin user is strictly prohibited.

Passwordless

The passwordless requirement is optional if you can accept entering a password for every ssh and sudo command.

Sandbox

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty provides a standard 4-node sandbox environment for learning, testing, and feature demonstration.

The sandbox uses fixed IP addresses and predefined identity identifiers, making it easy to reproduce various demo use cases.

Description

The default sandbox environment consists of 4 nodes, using the ha/full.yml configuration template.

ID	IP Address	Node	PostgreSQL	INFRA	ETCD	MINIO
1	`10.10.10.10`	`meta`	`pg-meta-1`	`infra-1`	`etcd-1`	`minio-1`
2	`10.10.10.11`	`node-1`	`pg-test-1`
3	`10.10.10.12`	`node-2`	`pg-test-2`
4	`10.10.10.13`	`node-3`	`pg-test-3`

The sandbox configuration can be summarized as the following config:

Vagrant

Mon, 01 Jan 0001 00:00:00 +0000

Vagrant is a popular local virtualization tool that creates local virtual machines in a declarative manner.

Pigsty requires a Linux environment to run. You can use Vagrant to easily create Linux virtual machines locally for testing.

Quick Start

Install Dependencies

First, ensure you have Vagrant and a virtual machine provider (such as VirtualBox or libvirt) installed on your system.

On macOS, you can use Homebrew for one-click installation:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
brew install vagrant virtualbox ansible

VirtualBox requires reboot after installation

After installing VirtualBox, you need to restart your system and allow its kernel extensions in System Preferences.

Terraform

Mon, 01 Jan 0001 00:00:00 +0000

Terraform is a popular “Infrastructure as Code” tool that you can use to create virtual machines on public clouds with one click.

Pigsty provides Terraform templates for Alibaba Cloud, AWS, and Tencent Cloud as examples.

Quick Start

Install Terraform

On macOS, you can use Homebrew to install Terraform:

brew install terraform

For other platforms, refer to the Terraform Official Installation Guide.

Initialize and Apply

Enter the Terraform directory, select a template, initialize provider plugins, and apply the configuration:

Security

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty’s default configuration is sufficient to cover the security needs of most scenarios.

Pigsty already provides out-of-the-box authentication and access control models that are secure enough for most scenarios.

If you want to further harden system security, here are some recommendations:

Confidentiality

Important Files

Protect your pigsty.yml configuration file or CMDB

The pigsty.yml configuration file usually contains highly sensitive confidential information. You should ensure its security.
Strictly control access permissions to admin nodes, limiting access to DBAs or Infra administrators only.
Strictly control access permissions to the pigsty.yml configuration file repository (if you manage it with git)

Protect your CA private key and other certificates, these files are very important.

Tutorials

Mon, 01 Jan 0001 00:00:00 +0000

This section provides step-by-step tutorials for common PostgreSQL tasks and scenarios.

Citus Cluster: Deploy and manage Citus distributed clusters
Disaster Drill: Emergency recovery when 2 of 3 nodes fail
PG VIP: Configure L2 VIP for PostgreSQL clusters

Deploy HA Citus Cluster

Sat, 11 Jan 2025 00:00:00 +0000

Citus is a PostgreSQL extension that transforms PostgreSQL into a distributed database, enabling horizontal scaling across multiple nodes to handle large amounts of data and queries.

Patroni v3.0+ provides native high-availability support for Citus, simplifying the setup of Citus clusters. Pigsty also provides native support for this.

Note: Citus 13.x supports PostgreSQL 18, 17, 16, 15, and 14. Pigsty extension repo provides Citus ARM64 packages.

Quick Start

Mon, 01 Jan 0001 00:00:00 +0000

Using extensions in Pigsty requires four steps: Download, Install, Config, and Create.

Download: Download extension packages to the local repository (Pigsty has already downloaded mainstream extensions by default)
Install: Install extension packages on cluster nodes
Config: Some extensions need to be preloaded or configured with parameters
Create: Execute CREATE EXTENSION in the database to create the extension

Declarative Configuration

Declare extensions in the Pigsty configuration manifest, and they will be automatically installed and created during cluster initialization:

Install

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty uses the operating system’s package manager (yum/apt) to install extension packages.

Two parameters are used to specify extensions to install:

Parameter	Purpose	Default Behavior
`pg_packages`	Global common packages	Ensure present (no upgrade)
`pg_extensions`	Cluster-specific extensions	Install latest version

pg_packages is typically used to specify base components needed by all clusters (PostgreSQL kernel, Patroni, pgBouncer, etc.) and essential extensions.

pg_extensions is used to specify extensions needed by specific clusters.

Tutorial on PIGSTY

Getting Started

Short Version

Installation

Installation

Pre-requirements for Mac OS

Psycopg

User Guide (Debian/Ubuntu)

Introduction

User Guide (RHEL)

Introduction

PgBouncer compilation and installation

Building

DNS lookup support

About

Inventory

Configuration File

Configuration Structure

Get Started

Single-Node Installation

Summary

Docker Deployment

Quick Start

Web Interface

Access

Getting Started with PostgreSQL

Basics

Customize Pigsty with Configuration

Minimal Configuration

Run Playbooks with Ansible

Offline Installation

Overview

Slim Installation

Overview

Security Tips

Deployment

Install Pigsty for Production

Summary

Prepare Resources for Serious Deployment

Node

Planning Architecture and Nodes

Common Patterns

Setup Admin User and Privileges

User

Passwordless

Sandbox

Description

Vagrant

Quick Start

Install Dependencies

Terraform

Quick Start

Install Terraform

Initialize and Apply

Security

Confidentiality

Important Files

Tutorials

Deploy HA Citus Cluster

Quick Start

Declarative Configuration

Install

Related Parameters