[{"content":"Links:\nPlugin: https://community.obsidian.md/plugins/sqlite-db-viewer GitHub: https://github.com/bakanovski/obsidian-sqlite-db-viewer. Uses WASM SQLite injection directly in the plugin, so it works completely locally and without installing external dependencies.\nNotes have two rendering modes:\nVia code blocks with a keyword Via internal links (![[]]) You can import existing Markdown tables into an SQLite database.\nFull documentation is available on GitHub.\nUsing the same DB with sqlite3 tools:\n","permalink":"https://debops.ru/posts/obsidian_sqlite_plugin/","summary":"New Obsidian plugin to manage SQLite DB inside you notes","title":"Obsidian: SQLite DB viewer release"},{"content":"AmneziaWG + Xray + WARP WARP is a free VPN from Cloudflare that allows you to hide your public IP (for bypassing Gemini blocking or for security purposes). You can operate in SOCKS5 mode, proxying only selected traffic. Xray is a smart routing core that directs traffic to the SOCKS5 WARP proxy according to specified rules (in the case of our DNS). For Debian-based OS, update packages and install basic utilities:\nsudo apt update \u0026amp;\u0026amp; sudo apt upgrade -y sudo apt install -y curl gnupg lsb-release Install WARP:\ncurl -fsSL https://pkg.cloudflareclient.com/pubkey.gpg | sudo gpg --yes --dearmor -o /usr/share/keyrings/cloudflare-warp-archive-keyring.gpg echo \u0026#34;deb [arch=amd64 signed-by=/usr/share/keyrings/cloudflare-warp-archive-keyring.gpg] https://pkg.cloudflareclient.com/ $(lsb_release -cs) main\u0026#34; | sudo tee /etc/apt/sources.list.d/cloudflare-client.list sudo apt update \u0026amp;\u0026amp; sudo apt install cloudflare-warp -y Register with WARP and run SOCKS5 proxy:\nwarp-cli registration new warp-cli mode proxy warp-cli proxy port 40000 warp-cli connect # Check functionality, should return an IP address from Cloudflare curl --socks5 127.0.0.1:40000 ifconfig.me Install Xray:\nbash -c \u0026#34;$(curl -L https://github.com/XTLS/Xray-install/raw/main/install-release.sh)\u0026#34; @ install Replace the file /usr/local/etc/xray/config.json with the contents:\n{ \u0026#34;log\u0026#34;: { \u0026#34;loglevel\u0026#34;: \u0026#34;warning\u0026#34; }, \u0026#34;inbounds\u0026#34;: [ { \u0026#34;tag\u0026#34;: \u0026#34;transparent\u0026#34;, \u0026#34;port\u0026#34;: 12345, // Port where AmneziaWG will forward traffic \u0026#34;protocol\u0026#34;: \u0026#34;dokodemo-door\u0026#34;, \u0026#34;settings\u0026#34;: { \u0026#34;network\u0026#34;: \u0026#34;tcp\u0026#34;, // UDP will be intercepted by iptables \u0026#34;followRedirect\u0026#34;: true }, \u0026#34;sniffing\u0026#34;: { \u0026#34;enabled\u0026#34;: true, \u0026#34;destOverride\u0026#34;: [\u0026#34;http\u0026#34;, \u0026#34;tls\u0026#34;] } } ], \u0026#34;outbounds\u0026#34;: [ { \u0026#34;tag\u0026#34;: \u0026#34;direct\u0026#34;, \u0026#34;protocol\u0026#34;: \u0026#34;freedom\u0026#34;, \u0026#34;settings\u0026#34;: {} }, { \u0026#34;tag\u0026#34;: \u0026#34;warp-socks\u0026#34;, \u0026#34;protocol\u0026#34;: \u0026#34;socks\u0026#34;, \u0026#34;settings\u0026#34;: { \u0026#34;domainStrategy\u0026#34;: \u0026#34;UseIPv4\u0026#34;, \u0026#34;servers\u0026#34;: [ { \u0026#34;address\u0026#34;: \u0026#34;127.0.0.1\u0026#34;, // WARP SOCKS5 proxy address and port \u0026#34;port\u0026#34;: 40000 } ] } } ], \u0026#34;routing\u0026#34;: { \u0026#34;domainStrategy\u0026#34;: \u0026#34;AsIs\u0026#34;, \u0026#34;rules\u0026#34;: [ { \u0026#34;type\u0026#34;: \u0026#34;field\u0026#34;, \u0026#34;domain\u0026#34;: [ \u0026#34;geosite:google\u0026#34;, \u0026#34;geosite:google-gemini\u0026#34;, \u0026#34;domain:2ip.io\u0026#34;, // To check functionality ], \u0026#34;outboundTag\u0026#34;: \u0026#34;warp-socks\u0026#34; }, { \u0026#34;type\u0026#34;: \u0026#34;field\u0026#34;, \u0026#34;network\u0026#34;: \u0026#34;tcp,udp\u0026#34;, \u0026#34;outboundTag\u0026#34;: \u0026#34;direct\u0026#34; // Everything else is direct } ] } } Restart the service and add it to startup:\nsudo systemctl restart xray sudo systemctl enable xray Add additional iptables rules:\n# Forward all AmneziaWG traffic to port 12345 sudo iptables -t nat -I PREROUTING 1 -i amn0 -p tcp -j REDIRECT --to-ports 12345 # Disable UDP for HTTPS (QUIC) for amn0 (SOCKS5 only supports TCP) sudo iptables -I FORWARD -i amn0 -p udp --dport 443 -j DROP # Block port 12345 on the interface with a white IP address (you can find it with the command ip -c -br a) sudo iptables -I INPUT -i ens1 -p tcp --dport 12345 -j DROP To make iptables rules persist across reboots:\nsudo apt install iptables-persistent # During installation, you will be prompted to save the current rules, but you can do so with a separate command. sudo netfilter-persistent save You can check if your Geosite DNS is working by going to 2ip.io; the site should display Cloudflare\u0026rsquo;s IP address.\nTo install the latest Geosite DNS:\nsudo curl -L -o /usr/local/share/xray/geosite.dat https://github.com/Loyalsoldier/v2ray-rules-dat/releases/latest/download/geosite.dat AmneziaWG + WARP For a simpler configuration, you can use the script https://github.com/isultanov99/amnezia-warp-host-routing. This will route all AmneziaWG traffic through WARP.\nsudo apt install git git clone https://github.com/isultanov99/amnezia-wg-warp-host-routing/blob/master/deploy_amnezia_warp_host.sh cd amnezia-wg-warp-host-routing/ chmod +x deploy_amnezia_warp_host.sh # Run script ./deploy_amnezia_warp_host.sh # Functionality check sudo systemctl status amnezia-warp-routing@v2.service ","permalink":"https://debops.ru/posts/linux_awg_warp/","summary":"Learn how to configure AmneziaWG with WARP and Xray","title":"Linux: AmneziaWG + Xray + WARP configuration"},{"content":"I posted freeipa_rpc crate to work with FreeIPA JSON-RPC in Rust. For now it contains a minimal set of operations, I will add more as needed.\nTesting To deploy FreeIPA locally:\nmkdir ipa42 docker run --name freeipa_server -ti \\ --privileged \\ -p 443:443 \\ -h ipa.example.test \\ -v $(pwd)/ipa42:/data:Z \\ -e PASSWORD=\u0026#39;Astra1234\u0026#39; \\ freeipa/freeipa-server:fedora-42 \\ ipa-server-install -U -r EXAMPLE.TEST --no-ntp SERVER_IP=$(docker inspect -f \u0026#39;{{ .NetworkSettings.IPAddress }}\u0026#39; freeipa_server) echo \u0026#34;$SERVER_IP ipa.example.test\u0026#34; | sudo tee -a /etc/hosts To run tests:\nIPA_BASE_URL=\u0026#34;https://ipa.example.test/ipa/\u0026#34; IPA_USER=\u0026#34;admin\u0026#34; IPA_PASSWORD=\u0026#34;Astra1234\u0026#34; IPA_INSECURE=1 pre-commit run -a ","permalink":"https://debops.ru/posts/crates_freeipa_rpc/","summary":"New SDK for FreeIPA JSON-RPC in Rust","title":"Crates: new freeipa_rpc library SDK for FreeIPA JSON-RPC"},{"content":"I posted ansible_module crate to write Ansible modules in Rust.\nYou can use cargo doc for viewing docs or see the examples section.\nTesting I provided rewrited ansible.builtin.slurp module rewritten in Rust in the examples section of the project, so let\u0026rsquo;s try it!\ngit clone https://github.com/bakanovskii/ansible_module_rs.git cd ansible_module_rs/ cargo build --examples --r cd examples/ Test small file Firstly let\u0026rsquo;s try to compare on a small file:\nANSIBLE_LIBRARY=../target/release/examples/ ansible-playbook test_slurp.yml So Rust version resulted in 0.03s, while Python resulted in 0.21s Not bad!\nTest big file Now let\u0026rsquo;s try to compare on a big file:\nANSIBLE_LIBRARY=../target/release/examples/ ansible-playbook test_slurp_big.yml Now Rust version resulted in 16.87s, while Python resulted in 23.41s Why not that big of a difference? Python uses C bindings for base64 encoding/decoding\n","permalink":"https://debops.ru/posts/crates_ansible_module/","summary":"New framework for Ansible modules in Rust","title":"Crates: new ansible_module library for Ansible"},{"content":"According to ansible docs, starting from Ansible 2.2 we can use binary modules which means any compiled language can be used. (source: https://docs.ansible.com/ansible/latest/dev_guide/developing_program_flow_modules.html#binary-modules)\nLet\u0026rsquo;s write a basic echo module in both Python and Rust and compare them.\nPython module Writing modules in python is pretty easy since there are a lot of builtin classes from the Ansible itself:\n#!/usr/bin/python # coding: utf-8 from __future__ import absolute_import, annotations, division, print_function __metaclass__ = type DOCUMENTATION = r\u0026#34;\u0026#34;\u0026#34; --- module: py_echo author: abakanovskii short_description: Echo some string description: - This module echoes strings. options: msg: description: - Message to echo. type: str required: true \u0026#34;\u0026#34;\u0026#34; EXAMPLES = r\u0026#34;\u0026#34;\u0026#34; - name: Echo 123 py_echo: msg: 123 \u0026#34;\u0026#34;\u0026#34; RETURN = r\u0026#34;\u0026#34;\u0026#34; msg: description: Message returned: always type: string sample: 123 \u0026#34;\u0026#34;\u0026#34; from ansible.module_utils.basic import AnsibleModule def main(): arg_spec = dict( msg=dict(type=\u0026#39;str\u0026#39;, required=True,), ) module = AnsibleModule( supports_check_mode=True, argument_spec=arg_spec, ) module.exit_json(msg=module.params[\u0026#39;msg\u0026#39;]) if __name__ == \u0026#39;__main__\u0026#39;: main() The module is pretty simple: it takes msg as a required argument and returns it.\nRust module In Rust we must implement required structs and functions, but thankfully there are an example from Ansible itself. Module reads from file first because ansible runs modules with a path to file, which contains all arguments in a JSON format.\nCreate project Let\u0026rsquo;s create a project and write code in a src/main.rs:\ncargo new ansible_mod Code:\nuse serde::{Deserialize, Serialize}; use serde_json::Error; use std::{ env, process, fs::File, io::Read, }; #[derive(Serialize, Deserialize)] struct ModuleArgs { msg: String, } #[derive(Clone, Serialize, Deserialize)] struct Response { msg: String, changed: bool, failed: bool, } fn exit_json(response_body: Response) { return_response(response_body) } fn fail_json(response_body: Response) { let failed_response: \u0026amp;mut Response = \u0026amp;mut response_body.clone(); failed_response.failed = true; return_response(failed_response.clone()) } fn return_response(resp: Response) { println!(\u0026#34;{}\u0026#34;, serde_json::to_string(\u0026amp;resp).unwrap()); process::exit(resp.failed as i32); } fn read_file_contents(file_name: \u0026amp;str) -\u0026gt; Result\u0026lt;String, Box\u0026lt;std::io::Error\u0026gt;\u0026gt; { let mut json_string: String = String::new(); File::open(file_name)?.read_to_string(\u0026amp;mut json_string)?; Ok(json_string) } fn parse_module_args(json_input: String) -\u0026gt; Result\u0026lt;ModuleArgs, Error\u0026gt; { Ok( ModuleArgs::from( serde_json::from_str( json_input.as_str() )? ) ) } fn main() { let args: Vec\u0026lt;String\u0026gt; = env::args().collect(); let program: \u0026amp;String = \u0026amp;args[0]; let input_file_name: \u0026amp;str = match args.len() { 2 =\u0026gt; \u0026amp;args[1], _ =\u0026gt; { eprintln!(\u0026#34;Module \u0026#39;{program}\u0026#39; expects exactly one argument!\u0026#34;); fail_json(Response { msg: \u0026#34;No module arguments file provided\u0026#34;.to_string(), changed: false, failed: true, }); \u0026#34;\u0026#34; } }; let json_input: String = read_file_contents(input_file_name).map_err(|err| { eprintln!(\u0026#34;Could not read file \u0026#39;{input_file_name}\u0026#39;: {err}\u0026#34;); fail_json(Response { msg: format!(\u0026#34;Could not read input JSON file \u0026#39;{input_file_name}\u0026#39;: {err}\u0026#34;), changed: false, failed: true, }) }).unwrap(); let module_args: ModuleArgs = parse_module_args(json_input).map_err(|err| { eprintln!(\u0026#34;Error during parsing JSON module arguments: {err}\u0026#34;); fail_json(Response { msg: format!(\u0026#34;Malformed input JSON module arguments: {err}\u0026#34;), changed: false, failed: true, }) }).unwrap(); exit_json(Response { msg: format!(\u0026#34;{}\u0026#34;, \u0026amp;module_args.msg), changed: false, failed: false, }); } Build Build it:\ncargo build -r cp target/release/ansible_mod . Test and compare To run modules locally without using collections or roles we must specify the ANSIBLE_LIBRARY environment varible for each command. To keep it simple we use pwd result as a value:\nANSIBLE_LIBRARY=$(pwd) ansible localhost -m py_echo -a msg=123 ANSIBLE_LIBRARY=$(pwd) ansible localhost -m ansible_mod -a msg=123 Result: Now let\u0026rsquo;s compare performance of these two with a hyperfine: Obviously the Rust module is faster and the more complex the module get, the more complex the module becomes, the more noticeable the difference in speed will be. (Ansible itself takes a lot of time while connecting, executing and etc)\nBut as I mentioned earlier, writing modules in Python is much easier, due to the abundance of built-in methods and classes for their operation. In the Python code I wrote a little documentation for the module which I can view with ansible-doc:\nANSIBLE_LIBRARY=$(pwd) ansible-doc -t module py_echo.py But I cannot do the same with a compiled Rust module. Of course, you can create a stub file ansible_mod.py with only the documentation variables, but there are other tools that are written only for python (for example, ansible-lint)\nConclusion For heavy operations, you can rewrite/write a module in a compiled language, but rewriting all modules for the sake of minor speedup does not make much sense, for a number of reasons:\nIn Ansible, Python has a developed ecosystem and a large number of examples Python modules can be changed on the fly (the user can find an error in the code and check it himself) Distributing modules in Python is much easier and safer If the goal is to significantly speed up the overall execution time of a playbook, then the bottleneck will ultimately be in Ansible itself, not in its individual modules.\nThere was an attempt to rewrite Ansible in Rust by the creator himself, but the project was soon abandoned as it did not find much demand in the community. (see: https://github.com/jetporch/jetporch)\n","permalink":"https://debops.ru/posts/ansible_rust_module/","summary":"Learn how to write a basic module for ansible in Rust","title":"Ansible: writing module with Rust"},{"content":"Definition This article discusses the different ways to obtain a Kerberos TGT in Python and compares them.\nPreparation For testing, we will deploy MIT Kerberos in Docker (you can deploy FreeIPA, but it will be redundant)\nThe simplest Dockerfile:\nFROM debian ARG DEBIAN_FRONTEND=noninteractive RUN apt-get update \u0026amp;\u0026amp; apt-get install -y krb5-kdc krb5-admin-server RUN kdb5_util create -s -P admin123 -r EXAMPLE.ORG COPY krb5.conf /etc/krb5.conf COPY kadm5.acl /etc/krb5kdc/kadm5.acl RUN service krb5-kdc start || true RUN service krb5-admin-server start || true RUN kadmin.local -q \u0026#34;add_principal -pw admin123 admin/admin@EXAMPLE.ORG\u0026#34; RUN kadmin.local -q \u0026#34;add_principal -pw admin123 admin@EXAMPLE.ORG\u0026#34; EXPOSE 88 ENTRYPOINT service krb5-kdc start \u0026amp;\u0026amp; service krb5-admin-server start \u0026amp;\u0026amp; /bin/bash Before building, let\u0026rsquo;s create files from COPY instructions:\nkadm5.acl:\n# This file Is the access control list for krb5 administration. # When this file is edited run service krb5-admin-server restart to activate # One common way to set up Kerberos administration is to allow any principal # ending in /admin is given full administrative rights. # To enable this, uncomment the following line: */admin * krb5.conf\n[libdefaults] default_realm = EXAMPLE.ORG dns_lookup_realm = false dns_lookup_kdc = false dns_canonicalize_hostname = false [realms] EXAMPLE.ORG = { kdc = kdc.example.org admin_server = kdc.example.org } [domain_realm] .example.org = EXAMPLE.ORG example.org = EXAMPLE.ORG The dns_* options are needed to avoid wasting time on DNS when getting a ticket\nBuilding the image:\ndocker build . -t my_kerberos Run in terminal:\ndocker run -h kdc.example.org -it --rm my_kerberos bash After starting the container, be sure to add an entry to the /etc/hosts file with the container hostname:\n# You can find the IP using the commands below, or find it yourself and add it to the /etc/hosts file id=$(docker container ls --all --filter=ancestor=\u0026#34;my_kerberos\u0026#34; --format \u0026#34;{{.ID}}\u0026#34;) ip=$(docker inspect -f \u0026#39;{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}\u0026#39; $id) echo \u0026#34;$ip kdc.example.org\u0026#34; \u0026gt;\u0026gt; /etc/hosts Let\u0026rsquo;s try to get a ticket from another terminal:\necho admin123 | KRB5_CONFIG=krb5.conf kinit admin We check that the ticket has been received:\nklist Practice After setting up the test environment, you can start checking\nMethod #1: Subprocess The simplest method, the only advantage over gssapi is the built-in support for multithreading (in gevent for sure, I don\u0026rsquo;t know about other libraries)\nLet\u0026rsquo;s write a Python script:\n#!/usr/bin/python3 import time import subprocess import os os.environ[\u0026#34;KRB5_CONFIG\u0026#34;] = \u0026#34;./krb5.conf\u0026#34; start = time.perf_counter() for _ in range(1): subprocess.run([\u0026#39;kinit\u0026#39;, \u0026#34;admin\u0026#34;], input=b\u0026#39;admin123\u0026#39;, stdout=subprocess.DEVNULL) end = (time.perf_counter() - start) * 1000 print(\u0026#34;The time of execution of above program is :\u0026#34;, end, \u0026#34;ms\u0026#34;) Result of execution for for _ in range(1): and for _ in range(100): We get the speed of receiving a ticket on average 8-9 ms (~10 ms for 1 receipt, 8-9 ms for 100 receipts)\nСпособ №2: python-gssapi The recommended method uses a call to functions from the C language, which makes the code for obtaining a ticket thread-blocking (in gevent you can bypass it using pool.spawn())\nTo fully imitate the behavior of kinit, we will add saving the ticket to a file, but this step can be skipped, since the ticket can be used from a variable\nWriting a Python script:\nimport time import gssapi import os os.environ[\u0026#34;KRB5_CONFIG\u0026#34;] = \u0026#34;./krb5.conf\u0026#34; start = time.perf_counter() name = gssapi.Name(\u0026#34;admin@EXAMPLE.ORG\u0026#34;, gssapi.NameType.kerberos_principal) for _ in range(1): creds = gssapi.raw.acquire_cred_with_password(name, b\u0026#34;admin123\u0026#34;) res = gssapi.Credentials(creds.creds) res.store(store={\u0026#34;ccache\u0026#34;: \u0026#34;FILE:/tmp/test_krb5cc\u0026#34;}, overwrite=True) end = (time.perf_counter() - start) * 1000 print(\u0026#34;The time of execution of above program is :\u0026#34;, end, \u0026#34;ms\u0026#34;) Result of execution for for _ in range(1): and for _ in range(100): We get the speed of receiving a ticket on average 5-7 ms (~7 ms for 1 receipt, 5-6 ms for 100 receipts)\nMethod #3: minikerberos A little-known library written in pure Python, but has a large set of functions and classes for working with Kerberos, including for pentesting\nHas a built-in ability to work with asyncio, but can work with others, since it is written in pure Python\nCons: obviously lower speed, as well as inconvenient kerberos_url format (for example, you can\u0026rsquo;t use admin/admin as a user because of formatting)\nWriting a Python script:\n#!/usr/bin/python3 import os import time from minikerberos.common.factory import KerberosClientFactory os.environ[\u0026#34;KRB5_CONFIG\u0026#34;] = \u0026#34;./krb5.conf\u0026#34; start = time.perf_counter() kerberos_url = \u0026#34;kerberos+password://EXAMPLE.ORG\\\\admin:admin123@172.20.0.2\u0026#34; for _ in range(1): cu = KerberosClientFactory.from_url(kerberos_url) client = cu.get_client_blocking() client.get_TGT(with_pac=None) client.ccache.to_file(\u0026#39;/tmp/test_krb5cc\u0026#39;) end = (time.perf_counter() - start) * 1000 print(\u0026#34;The time of execution of above program is :\u0026#34;, end, \u0026#34;ms\u0026#34;) Result of execution for for _ in range(1): and for _ in range(100): We get the speed of receiving a ticket on average 27-33 ms (~30-33 ms for 1 receipt, 27-28 ms for 100 receipts)\nOther libraries The well-known impacket library has functions for working with Kerberos, but I couldn\u0026rsquo;t get them to work\nFor working with Kerberos in HTTP, there is a requests_gssapi library\n","permalink":"https://debops.ru/posts/python_kerberos_tgt/","summary":"Comparison of different ways to get a Kerberos ticket in Python","title":"Python: Obtaining a Kerberos Ticket (TGT)"},{"content":"The simplest way to configure the /etc/hosts file for all hosts in one play is:\n--- - name: Make sure all hosts are reachable via FQDN hosts: all become: true gather_facts: true tasks: - name: Generate /etc/hosts file ansible.builtin.template: src: hosts.j2 dest: /etc/hosts owner: root group: root mode: \u0026#34;0644\u0026#34; tags: - preconfigure Content of hosts.j2:\n{{ ansible_managed | comment }} 127.0.0.1 localhost localhost.localdomain ::1 localhost ip6-localhost ip6-loopback ff02::1 ip6-allnodes ff02::2 ip6-allrouters {% for item in play_hosts %} {% set short_name = hostvars[item][\u0026#39;ansible_facts\u0026#39;][\u0026#39;fqdn\u0026#39;].split(\u0026#39;.\u0026#39;) %} {% set fqdn_name = hostvars[item][\u0026#39;ansible_facts\u0026#39;][\u0026#39;fqdn\u0026#39;] %} {{ hostvars[item][\u0026#39;ansible_host\u0026#39;] }} {{ fqdn_name }} {{ short_name[0] }} {% endfor %} You can read about ansible_managed here\n","permalink":"https://debops.ru/posts/ansible_generate_hosts/","summary":"Learn how to create right /etc/hosts file for all hosts in Ansible","title":"Ansible: how to create right /etc/hosts file for all hosts"},{"content":"Definition This option allows you to run a small init system in the container in order to get rid of zombie processes. Let me remind you that in the container, by default, the parent process is not systemd/init, but the value from ENTRYPOINT/CMD\nWhere to use? For example, in Ansible Execution-Environment — a container for running a playbook or task in AWX, when starting this container there is a high probability that during the execution of the playbook a bunch of zombie processes will appear and as a result a fork error may appear with the playbook/task stopping (see the picture related)\nhttps://github.com/ansible/ansible/issues/49270 https://github.com/operator-framework/operator-sdk/issues/1551 Docker Docker uses tini:\nsudo apt install tini docker run -it --init busybox sh / # ps PID USER TIME COMMAND 1 root 0:00 /sbin/docker-init -- sh 7 root 0:00 sh 9 root 0:00 ps / # exit Podman Podman uses catatonit, but also has a --init-path option to specify the path to a different init system:\nsudo apt install catatonit podman run -it --init busybox sh / # ps PID USER TIME COMMAND 1 root 0:00 /run/podman-init -- sh 7 root 0:00 sh 8 root 0:00 ps / # exit # You can run it with tini instead of catatonit podman run -it --init --init-path /usr/bin/tini busybox sh Entrypoint In fact, --init simply substitutes the init system binary into the ENTRYPOINT instruction (at the very beginning), for adding to the image via Dockerfile/Containerfile:\nFROM busybox:latest ENTRYPOINT [\u0026#34;/usr/bin/tini\u0026#34;] ","permalink":"https://debops.ru/posts/podman_docker_init/","summary":"Learn how to use \u0026ndash;init option in Podman/Docker","title":"Podman,Docker: --init option"},{"content":"Definition args: is a directive that allows you to \u0026ldquo;unpack\u0026rdquo; arguments with values ​​into a module, most often we pass only the value of previously predefined arguments (something like **kwargs in Python)\nWhat does this mean? I\u0026rsquo;ll show you in practice:\n- name: Create file ansible.builtin.command: touch some_file args: chdir: /tmp/ creates: /tmp/some_file Completely identical to the task:\n- name: Create file ansible.builtin.command: cmd: touch some_file chdir: /tmp/ creates: /tmp/some_file But in the first option we pass the command as a string and there is simply no place to add chdir and creates\nI will give a few more useful examples:\nargs: и loop: Using args: and loop: (or with_X) you can make tasks more compact, for example:\n# Without args: - name: Remove Qemu-NAT line and add DNS server to resolv.conf ansible.builtin.lineinfile: path: /etc/resolv.conf state: \u0026#34;{{ item.state }}\u0026#34; line: \u0026#34;{{ item.line }}\u0026#34; loop: - { state: absent, line: nameserver 10.0.2.3 } - { state: present, line: nameserver 192.0.2.1 } # With args: - name: Remove Qemu-NAT line and add DNS server to resolv.conf ansible.builtin.lineinfile: path: /etc/resolv.conf args: \u0026#34;{{ item }}\u0026#34; loop: - { state: absent, line: nameserver 10.0.2.3 } - { state: present, line: nameserver 192.0.2.1 } args: и default(omit) You can make different sets of arguments for each dictionary element similar to the behavior of default(omit)::\n# Without args: - name: Install deb pkgs ansible.builtin.apt: deb: http://192.0.2.1/repo/somepackage-x86_64.deb state: present - name: Install pkgs on arm host ansible.builtin.apt: name: - firefox - chromium - libreoffice update_cache: true state: present install_recommends: false # With args: - name: Install pkgs ansible.builtin.apt: state: present args: \u0026#34;{{ item }}\u0026#34; loop: - deb: http://192.0.2.1/repo/somepackage-x86_64.deb - name: - firefox - chromium - libreoffice update_cache: true install_recommends: false Security However, it is worth considering that using variables in this case (if you are writing a role or collection) may be unsafe, because it will be easy to override arguments and values ​​(https://docs.ansible.com/ansible/devel/reference_appendices/faq.html#when-is-it-unsafe-to-bulk-set-task-arguments-from-a-variable)\n","permalink":"https://debops.ru/posts/ansible_args/","summary":"Learn how to use args in Ansible","title":"Ansible: args"},{"content":"Definition Let\u0026rsquo;s say we have a static inventory with 10 hosts, but we know for sure that some of these hosts will be unavailable\nOf course, we can use a dynamic inventory and do ansible.builtin.meta: refresh_inventory during playbook execution\nBut this will be true only when running via CLI (ansible-playbook/ansible-navigator/ansible), AWX/AAC does not support ansible.builtin.meta:refresh_inventory (see https://github.com/ansible/awx/issues/7276) and also does not support passing extra_vars to dynamic inventory (see https://github.com/ansible/awx/issues/13004)\nIn simpler cases, the ability to use jinja directly in the hosts: field of the play will help, for example:\n- name: Install FreeIPA hosts: \u0026#34;{{ \u0026#39;freeipa_hosts\u0026#39; if deploy_freeipa else \u0026#39;none\u0026#39; }}\u0026#34; become: true tasks: ... That is, if the value of deploy_freeipa is true at the global level, then the task will be executed, and if not, then it will be skipped\nHowever, it is often convenient to use the all group to execute tasks on all hosts from the inventory, which is impossible in our case (we will access non-existent hosts and get the UNREACHABLE error), as an option - use ignore_unreachable: true, but then you will have to spend time waiting in each play task\nWithin one play you can do this:\n- name: Configure DNS on all hosts hosts: all gather_facts: false become: true tasks: - name: Remove unused hosts block: - name: Remove unused hosts ansible.builtin.wait_for_connection: timeout: 30 rescue: # The task below is needed so that the result of the playbook execution is not considered FAILED - name: Clean errors from them ansible.builtin.meta: clear_host_errors - name: End play for these hosts ansible.builtin.meta: end_host ... In this case, unavailable hosts will simply be removed from this play, but what if there are several such plays?\nThen you can create a group of unavailable hosts and use host patterns in subsequent plays (link leads to a post)\n- name: Configure DNS on all hosts hosts: all gather_facts: false become: true tasks: - name: Remove unused hosts block: - name: Remove unused hosts ansible.builtin.wait_for_connection: timeout: 30 rescue: - name: Create host group for failed host ansible.builtin.group_by: key: failed_hosts # The task below is needed so that the result of the playbook execution is not considered FAILED - name: Clean errors from them ansible.builtin.meta: clear_host_errors - name: End play for these hosts ansible.builtin.meta: end_host ... - name: Configure HOSTNAME on all hosts hosts: all:!failed_hosts gather_facts: false become: true tasks: ... ","permalink":"https://debops.ru/posts/ansible_remove_hosts/","summary":"Learn how to remove some hosts from all future plays in playbook runtime in Ansible","title":"Ansible: how to remove hosts from all plays in a playbook runtime"},{"content":"Definition In Ansible lookup plugins are Ansible-specific extensions to the Jinja2 templating engine that are used to get some information on the node that is the controller (usually your PC) Examples of lookup plugins:\ncommunity.hashi_vault.vault_kv2_get — to get the value of a key from Hashicorp Vault ansible.builtin.file — to read a file ansible.builtin.env — the value of an environment variable Here is the full list of lookup plugins from ansible-galaxy\nusing delegate_to will not allow the lookup plugin to be executed on a managed node, it can only be executed on a controller node\nLet me give you a very simple example: let\u0026rsquo;s display the contents of the environment variable $HOME on my computer:\n--- - hosts: localhost become: false gather_facts: false tasks: - name: Display $HOME ansible.builtin.debug: msg: \u0026#34;{{ lookup(\u0026#39;env\u0026#39;, \u0026#39;HOME\u0026#39;) }}\u0026#34; Result:\nTASK [Display $HOME] ok: [localhost] =\u0026gt; msg: /home/alexander Interpolation However, the logic of lookup differs from the usual logic of modules, because their definition occurs at the moment of interpolation:\n--- - hosts: localhost become: false gather_facts: false vars: current_time: \u0026#34;{{ lookup(\u0026#39;pipe\u0026#39;,\u0026#39;date +%H:%M:%S\u0026#39;) }}\u0026#34; tasks: - name: Display time 1/2 ansible.builtin.debug: msg: Current time {{ current_time }} - name: Pause for 3 seconds ansible.builtin.pause: seconds: 3 - name: Display time 2/2 ansible.builtin.debug: msg: Current time is {{ current_time }} In the example above, it may seem that both tasks will output the same result, but as I said, the result of the work will be obtained at the moment of interpolation:\nTASK [Display current time 1/2] ok: [localhost] =\u0026gt; msg: Current time 21:21:44 TASK [Pause for 3 seconds] Pausing for 3 seconds (ctrl+C then \u0026#39;C\u0026#39; = continue early, ctrl+C then \u0026#39;A\u0026#39; = abort) ok: [localhost] TASK [Display current time 2/2] ok: [localhost] =\u0026gt; msg: Current time is 21:21:47 until: Another interesting example of working together with the until: directive (let me remind you that until: allows you to restart the module until you get the desired result)\nLet\u0026rsquo;s assume that we need to generate a UUID that starts with zero and we wrote a simple play like this:\n--- - hosts: localhost become: false gather_facts: false tasks: - name: Generate random UUID ansible.builtin.set_fact: random_uuid: \u0026#34;{{ lookup(\u0026#39;pipe\u0026#39;, \u0026#39;uuidgen\u0026#39;) }}\u0026#34; until: random_uuid.startswith(\u0026#39;0\u0026#39;) delay: 1 retries: 20 At first glance, everything looks correct: we get the random_uuid value from the lookup plugin and then check if it starts with zero, but in fact the random_uuid value will be obtained once during the first interpolation of the plugin and then until: will check the same value\nHow to bypass this limitation? The is no way, you can\u0026rsquo;t combine until:+lookup and \u0026ldquo;catch\u0026rdquo; the resulting value, the lookup value is generated once before the module itself is executed, the best thing to do in this case is to select the right plugin (or write it yourself)\nYou can achieve correct operation of the until:+lookup condition by substituting the lookup plugin directly into the condition, but you still won\u0026rsquo;t be able to get the desired value, since the set_fact module will store the value it received at the time of the call (see picril)\n--- - hosts: localhost become: false gather_facts: false tasks: - name: Generate random UUID vars: _random_uuid: \u0026#34;{{ lookup(\u0026#39;pipe\u0026#39;, \u0026#39;uuidgen\u0026#39;) }}\u0026#34; ansible.builtin.set_fact: random_uuid: \u0026#34;{{ _random_uuid }}\u0026#34; until: _random_uuid.startswith(\u0026#39;0\u0026#39;) delay: 1 retries: 20 ","permalink":"https://debops.ru/posts/ansible_lookup_plugins/","summary":"Learn how lookup plugins work with jinja2 in Ansible","title":"Ansible: lookup plugins and jinja2"},{"content":"Definition The run_once: true directive allows you to run the task once, regardless of the number of managed hosts\nWhy might this be necessary? For example:\nIf a cluster is configured and changes need to be made only on the master node If you need to make an API request only once If you need to generate or get a token run_once and register Interestingly, the result of executing the module via the register: directive will be passed to all other hosts within the play, for example, the following play will execute the date command on one of the hosts (don\u0026rsquo;t do this in real playbooks) and then output the contents of the result of this command on all hosts of this play:\n- name: A simple example hosts: all tasks: - name: Run this ask only on one of the hosts ansible.builtin.command: date run_once: true register: host_kernel - name: Show it on all hosts ansible.builtin.debug: var: host_kernel run_once and set_fact Unlike register:, set_fact: is a module, and therefore a variable from set_fact will only belong to the host that was selected for run_once:, but I will remind you that set_fact can be passed between different plays (more details in the post about set_fact):\n- name: A simple example hosts: all tasks: - name: Run this ask only on one of the hosts ansible.builtin.set_fact: sample_variable: Played on {{ ansible_host }} run_once: true # Эта таска упадет с ошибкой на всех других хостах - name: Try to show it on all hosts ansible.builtin.debug: var: sample_variable run_once and handlers With handlers, run_once works differently than with register:, in this example the handler will be called only once, on the host that was selected for run_once (you can read more about handlers in the corresponding post):\n- name: A simple example hosts: all handlers: - name: Say hello ansible.builtin.debug: msg: Hello from {{ ansible_host }} tasks: - name: Run this ask only on one of the hosts ansible.builtin.command: /bin/true run_once: true notify: Say hello run_once and delegate_to Using run_once and delegate_to you can explicitly specify on which host in the group the task should be executed:\n- name: A simple example hosts: all tasks: - name: Run this ask only on localhost ansible.builtin.command: /bin/true run_once: true delegate_to: localhost ","permalink":"https://debops.ru/posts/ansible_run_once/","summary":"Learn how to use run_once in Ansible","title":"Ansible: run_once"},{"content":"Definition In most OS running the systemd init subsystem, all legacy power management programs are symbolic links to systemctl subcommands to preserve backward compatibility.\nsystemctl reboot — reboot systemctl poweroff/systemctl shutdown — shutdown systemctl halt — shuts down the system, stopping all services and processes and unmounting the file system, but leaves the computer on\nPS: systemctl knows what it was started from by using argv[0], which is a standard construct in C, here is a snippet of systemctl source code (reduced tabs for readability):\nint systemctl_dispatch_parse_argv(int argc, char *argv[]) { assert(argc \u0026gt;= 0); assert(argv); if (invoked_as(argv, \u0026#34;halt\u0026#34;)) { arg_action = ACTION_HALT; return halt_parse_argv(argc, argv); } else if (invoked_as(argv, \u0026#34;poweroff\u0026#34;)) { arg_action = ACTION_POWEROFF; return halt_parse_argv(argc, argv); } else if (invoked_as(argv, \u0026#34;reboot\u0026#34;)) { arg_action = ACTION_REBOOT; return halt_parse_argv(argc, argv); } else if (invoked_as(argv, \u0026#34;shutdown\u0026#34;)) { arg_action = ACTION_POWEROFF; return shutdown_parse_argv(argc, argv); ","permalink":"https://debops.ru/posts/linux_power_supply/","summary":"Learn about managing power supply using systemctl in Linux","title":"Linux: power supply using systemctl"},{"content":"Definition Terraform/OpenTofu is a declarative tool, and therefore when describing resources it is very important to understand the sequence of their creation, for this purpose hidden dependencies are automatically built between resources, by the way, performing a traversal of the dependency tree, Terraform/Tofu tries to parallelize the creation of resources whenever possible, which leads to a fairly efficient application of changes\nFor example, if we deployed the code below, Terraform/Tofu would already know to first create a resource with a virtual network, and only after that it would add the address pool to that network:\nresource \u0026#34;opennebula_virtual_network\u0026#34; \u0026#34;ceph-virtnet\u0026#34; { name = \u0026#34;ceph-intnet\u0026#34; group = \u0026#34;oneadmin\u0026#34; permissions = \u0026#34;600\u0026#34; bridge = \u0026#34;vnet0\u0026#34; gateway = \u0026#34;192.0.2.1\u0026#34; dns = \u0026#34;192.0.2.1 8.8.8.8\u0026#34; network_mask = \u0026#34;255.255.255.0\u0026#34; } resource \u0026#34;opennebula_virtual_network_address_range\u0026#34; \u0026#34;ceph\u0026#34; { virtual_network_id = opennebula_virtual_network.ceph-virtnet.id ar_type = \u0026#34;IP4\u0026#34; size = 250 ip4 = \u0026#34;192.0.2.2\u0026#34; } Usually you don\u0026rsquo;t need to interfere with dependency building, but there are rare cases where you need to define explicit dependencies to manually determine the order of resource execution using the depends_on meta-argument, for example:\nresource \u0026#34;opennebula_virtual_network\u0026#34; \u0026#34;ceph-virtnet\u0026#34; { name = \u0026#34;ceph-intnet\u0026#34; group = \u0026#34;oneadmin\u0026#34; permissions = \u0026#34;600\u0026#34; bridge = \u0026#34;vnet0\u0026#34; gateway = \u0026#34;192.0.2.1\u0026#34; dns = \u0026#34;192.0.2.1 8.8.8.8\u0026#34; network_mask = \u0026#34;255.255.255.0\u0026#34; } resource \u0026#34;opennebula_virtual_network_address_range\u0026#34; \u0026#34;ceph\u0026#34; { virtual_network_id = opennebula_virtual_network.ceph-virtnet.id ar_type = \u0026#34;IP4\u0026#34; size = 250 ip4 = \u0026#34;192.0.2.2\u0026#34; depends_on = [ opennebula_virtual_network.ceph-virtnet ] } Practice If desired, the dependency graph can be printed using the graph command:\ntofu graph terraform graph The graph is output in a graph description language called DOT, which can be easily converted to an image using, for example, the GraphvizOnline application, for this you just need to:\nRun tofu graph/terraform graph Copy the output of the command Paste the content into GraphvizOnline and view the image Or you can use the dot utility and generate an image with one command, execute it and view the graph.png file:\ntofu graph -type=plan | dot -Tpng \u0026gt;graph.png ","permalink":"https://debops.ru/posts/terraform_dependencies/","summary":"Learn about to dependencies in Terraform/Tofu","title":"Terraform,Tofu: hidden and explicit dependencies"},{"content":"Definition To test expressions in Terraform/Tofu there is an interactive console, which can be called via tofu console/terraform console A very convenient thing for checking what the expression will give at the output without resorting to output and test runs\nAn example for counting the number of nodes from a network of the format \u0026lt;IP\u0026gt;:\u0026lt;MASK\u0026gt;:\ntofu console \u0026gt; pow(2, 32 - split(\u0026#34;/\u0026#34;, \u0026#34;172.16.0.0/20\u0026#34;)[1]) - 2 4094 \u0026gt;pow(2, 32 - split(\u0026#34;/\u0026#34;, \u0026#34;10.0.0.0/8\u0026#34;)[1]) - 2 16777214 You can pass a variable with -var 'NAME=VALUE', a file with variables with -var-file=FILENAME or, if the file has the extension \u0026lt;name\u0026gt;.auto.tfvars, they will be automatically included in the console (as with any other Terraform/Tofu commands):\nIt is important to note that the variables passed must be defined in the corresponding .tf files\nPractice The easiest way to use it is in a directory with all Terraform/Tofu files, to fully simulate working in .tf files:\n# Let\u0026#39;s look at the value of the first variable \u0026gt; var.one_digit \u0026#34;5\u0026#34; # Let\u0026#39;s look at the value of the second variable \u0026gt; var.two_digit \u0026#34;50\u0026#34; # Find greatest \u0026gt; max(var.one_digit, var.two_digit) 50 # Find sum \u0026gt; sum([var.one_digit, var.two_digit]) 55 # The function below appends a value to a string, preserving the leading zero for two-digit values ​​(00, 01, 02, .., 99) # Try for a single digit number \u0026gt; \u0026#34;yay-10${var.one_digit \u0026gt;= 0 \u0026amp;\u0026amp; var.one_digit \u0026lt; 10 ? join(\u0026#34;\u0026#34;,[\u0026#34;0\u0026#34;,tostring(var.one_digit)]) : var.one_digit}\u0026#34; \u0026#34;yay-1005\u0026#34; # Try for a double digit number \u0026gt; \u0026#34;yay-10${var.two_digit \u0026gt;= 0 \u0026amp;\u0026amp; var.two_digit \u0026lt; 10 ? join(\u0026#34;\u0026#34;,[\u0026#34;0\u0026#34;,tostring(var.two_digit)]) : var.two_digit}\u0026#34; \u0026#34;yay-1050\u0026#34; # Read the contents of a text file (uses a relative path to files by default) \u0026gt; file(text) \u0026#34;Hello!\u0026#34; You can even pass it via a pipe to stdin, for example:\necho \u0026#34;max(1, 2, 3)\u0026#34; | tofu console 3 # Don\u0026#39;t forget to escape special characters with a backslash echo \u0026#34;base64encode(\\\u0026#34;Hello World\\\u0026#34;)\u0026#34; | tofu console \u0026#34;SGVsbG8gV29ybGQ=\u0026#34; ","permalink":"https://debops.ru/posts/terraform_console/","summary":"Learn how to use console in Terraform/Tofu","title":"Terraform,Tofu: console"},{"content":"Definition There are 5 main modules for managing file contents:\nansible.builtin.lineinfile — manages lines in a file ansible.builtin.blockinfile — manages a block of lines in a file (separates them from both sides) ansible.builtin.template — replaces a Jinja2 file with a template ansible.builtin.copy — replaces a file with its contents ansible.builtin.replace — replaces all lines that match a regular expression Let\u0026rsquo;s look at the advantages and disadvantages of each of them.\nansible.builtin.lineinfile Pros: Allows flexibility and precision in creating or removing the lines you want in the right place Cons: If you change line: after adding it, when you run it again, Ansible will not be able to track the previous line and will leave it (ending up with 2 lines)\nansible.builtin.blockinfile Pros: Allows you to conveniently track the beginning and end of changes that Ansible has made using a marker (the lines for marking can be changed via the marker_begin and marker_end parameters), and you don’t have to worry about previously added lines, since Ansible will work exclusively within the block Cons: Doesn\u0026rsquo;t allow adding anything separately from the block, when restarting with other data, replaces the existing block with a new one\nansible.builtin.template Pros: Allows flexible configuration of the configuration file using the entire arsenal of Jinja2 templates, also can create a marker similar to the ansible.builtin.blockinfile module, more details in the post about ansible_managed Cons: By default, completely replaces the file (force: true), not intended to change existing files\nansible.builtin.copy Pros: Doesn\u0026rsquo;t require creating a Jinja2 template, intuitive Cons: By default, completely replaces the file (force: true), interpolation can lead to unpredictable results\nansible.builtin.replace Pros: Allows you to replace several lines in one file at once Cons: You can accidentally change lines that you don\u0026rsquo;t need to change, and it also can\u0026rsquo;t add the necessary lines pointwise\nBest-practice When and which to use:\nIf you need to replace a line/lines in an existing file, it is better to use ansible.builtin.lineinfile If you need to add a block with lines to an existing file with the ability to mark changes, it is better to use ansible.builtin.blockinfile If you need to replace a file with some simple content, it is better to use ansible.builtin.copy If you need to dynamically create a file and/or modify it, it is better to use ansible.builtin.template If you need to replace all lines according to some regular expression, it is better to use ansible.builtin.replace ","permalink":"https://debops.ru/posts/ansible_managing_files/","summary":"Learn how to manage files content in Ansible","title":"Ansible: managing files content"},{"content":"Definition A handler is a type of task in Ansible that is intended to be executed after being called by other tasks\nThe main differences between a task and a handler:\nHandlers are executed at the very end of a group of tasks and can be called up to 3 times (see the previous post about pre_tasks/post_tasks), while the order of execution will correspond to the order of recording Handlers are called by the notify: \u0026lt;handler_name\u0026gt; directive and are executed only if the task result has the CHANGED status The handler will be executed once, even if it was called several times According to the official documentation, handlers should be used to restart or reboot services, but their scope of application does not end there; in general, the use of a handler looks like this:\ntasks: - name: Configure network ansible.builtin.template: dest: /etc/network/interfaces src: networking.j2 owner: root group: root mode: \u0026#34;0644\u0026#34; backup: true notify: Restart networking handlers: - name: Restart networking ansible.builtin.systemd: name: networking state: restarted Practice Logically, the task below is equivalent to the task above, but using when: \u0026lt;some_var\u0026gt;.changed is very bad ansible practice:\ntasks: - name: Configure network ansible.builtin.template: dest: /etc/network/interfaces src: networking.j2 owner: root group: root mode: \u0026#34;0644\u0026#34; backup: true register: network_config - name: Restart networking ansible.builtin.systemd: name: networking state: restarted when: network_config.changed But that\u0026rsquo;s not all, handlers can do much more, for example:\nA handler can call other handlers or even a group of tasks via include_tasks or import_tasks:\ntasks: - name: Always changed ansible.builtin.command: /bin/true notify: - First handler handlers: - name: First handler ansible.builtin.command: /bin/true notify: Inlcude tasks - name: Inlcude tasks ansible.builtin.include_tasks: tasks/main. You can change the name to call the handler by using a Jinja2 pattern in the handler name, or by using the listen: directive, but only static strings can be used for listen::\nvars: service: apache2 tasks: - name: Always changed ansible.builtin.command: /bin/true notify: - Restart {{ service }} - Reboot handlers: - name: Restart {{ service }} ansible.builtin.systemd: name: \u0026#34;{{ service }}\u0026#34; state: restarted - name: Reboot task ansible.builtin.reboot: listen: Reboot Using the ansible.builtin.meta:flush_handlers module, you can execute all previously called handlers and reset their counter, allowing you to call them again:\ntasks: - name: Configure network ansible.builtin.template: dest: /etc/network/interfaces src: networking.j2 owner: root group: root mode: \u0026#34;0644\u0026#34; backup: true notify: Restart networking - name: Restart right away ansible.builtin.meta: flush_handlers - name: Notify again ansible.builtin.command: /bin/true notify: Restart networking handlers: - name: Restart networking ansible.builtin.systemd: name: networking state: restarted Since roles is simply \u0026ldquo;included\u0026rdquo; in tasks, it is possible to call handlers from a role (an example can also be found in the previous post), and for convenience, it is possible to use the role_name/FQCN_role : handler name construction, for example:\n- name: Always change ansible.builtin.command: /bin/true notify: \u0026#39;k3s.orchestration.raspberrypi : Reboot Pi\u0026#39; Based on point 3, the only condition for calling the handler is the CHANGED status, and, as a consequence, you can define the change condition yourself by changed_when: \u0026lt;condition\u0026gt;:\ntasks: - name: Run sample task ansible.builtin.command: /bin/true changed_when: false notify: Say hello # Will not be called due to \u0026#39;changed_when: false\u0026#39; handlers: - name: Say hello ansible.builtin.debug: msg: Hello guys ","permalink":"https://debops.ru/posts/ansible_handlers/","summary":"Learn how to use handlers in Ansible","title":"Ansible: handlers"},{"content":"Definition A sparse file is a file in which sequences of zero bytes are replaced with information about these sequences (a list of holes)\nA hole is a sequence of zero bytes inside a file that is not written to disk. Information about holes (offset from the beginning of the file in bytes and the number of bytes) is stored in the FS metadata\nOne of the most common use cases can be found in raw virtual machine images, since unlike qcow2, raw images do not contain any meta information to save disk space\nHowever, the virtual size of such a file can exceed the maximum capacity on the FS where it is located\nPractice Let\u0026rsquo;s try to create such a file in different ways:\nqemu-img create -f raw gigantic_image.img 10T truncate -s 100G big_file dd of=sparse-file bs=10G seek=10 count=0 If we look at the size of these files in the usual way using the ls utility, we will see only the virtual size of the files:\nls -lh total 4.0K -rw-r--r-- 1 alexander alexander 100G Sep 18 21:10 big_file -rw-r--r-- 1 alexander alexander 10T Sep 18 21:10 gigantic_image.img -rw-r--r-- 1 alexander alexander 100G Sep 18 21:10 sparse-file To see the actual size that these files occupy on the disk, you can use any of the following commands:\nls -s — the -s, --size flag will display the real size of the files on the left (before the discretionary rights) du -h \u0026lt;file\u0026gt; — the -h flag is needed for human-readable output of file sizes, du can recognize such files without additional flags qemu-img info \u0026lt;image\u0026gt; — for images (also works with regular files) stat \u0026lt;file\u0026gt; — although it shows the virtual size, the number of blocks will correspond to the real one The list of commands does not end there, there is no point in listing all possible methods\nYou can also convert a regular file into a sparse one:\ndd if=/dev/zero of=sparse_file bs=50M count=5 ls -lhs total 250M 250M -rw-r--r-- 1 alexander alexander 250M Sep 18 21:26 sparse_file fallocate -d sparse_file ls -lhs total 0 0 -rw-r--r-- 1 alexander alexander 250M Sep 18 21:27 sparse_file When copying via cp, you can pass the --sparse=always flag, which will create a sparse version of a file from a non-sparse file\ndd if=/dev/zero of=sparse_file bs=50M count=5 cp --sparse=always sparse_file sparse_copy ls -lsh total 251M 0 -rw-r--r-- 1 alexander alexander 250M Sep 18 21:32 sparse_copy 251M -rw-r--r-- 1 alexander alexander 250M Sep 18 21:32 sparse_file However, in addition to the obvious advantages in the form of saving disk space, which is especially useful in virtual machines, there are also a number of significant disadvantages:\nfile fragmentation when frequently writing data into holes copying a sparse file with a program that does not support this type of file may create a file with no size compression filling systems with such files may lead to unexpected errors ","permalink":"https://debops.ru/posts/linux_sparse_file/","summary":"Learn about sparse files in Linux","title":"Linux: sparse file"},{"content":"Definition Host patterns allow you to specify exactly which nodes require tasks to be performed\nPatterns with example:\nall or * — all hosts stage:dev — union/logical OR stage:\u0026amp;dev — intersection/logical AND stage:!dev — exclusion *.example — wildcard dev[5:10] — host range (counting starts from zero) ~web\\d+\\.example — regular expressions Instead of the : separator, you can use , which is preferable when working with IPv6 networks or ranges\nThe processing order is as follows:\n: and , \u0026amp; ! You can use combinations of patterns:\nhosts: dev:staging:\u0026amp;database:!queque (all hosts from the dev and staging groups if they are also in the database group, but excluding hosts from the queque group) You can also use variables to pass groups via -e, --extra-vars, for example: hosts: webservers:!{{ excluded }}:\u0026amp;{{ required }}\nYou can use it in playbooks and ad-hoc commands, but when using a playbook file, you need to use the -l, --limit flag:\nExamples for ansible-playbook and ansible binaries (don\u0026rsquo;t forget to escape patterns with special bash characters with quotes)\nansible-playbook -l hosts main.yml ansible-playbook -l \u0026#39;staging:\u0026amp;database\u0026#39; main.yml ansible-playbook -l \u0026#39;staging,\u0026amp;database\u0026#39; main.yml ansible all:localhost -m setup ansible all,localhost -m setup ansible \u0026#39;staging:\u0026amp;database\u0026#39; -m reboot ansible \u0026#39;staging,\u0026amp;database\u0026#39; -m reboot Playbook example:\n- name: Play this on all hosts except db group hosts: all:!db tasks: - name: Change hostname ansible.builtin.file: use: systemd name: \u0026#34;{{ inventory_hostname }}\u0026#34; - name: Play this on all host that are also in a backup group but not in a web one hosts: all:\u0026amp;backup:!web tasks: - name: Add host record to /etc/hosts ansible.builtin.lineinfile: path: /etc/hosts line: \u0026#34;{{ ansible_host }} {{ inventory_hostname }}\u0026#34; state: present insertbefore: BOF ","permalink":"https://debops.ru/posts/ansible_host_patterns/","summary":"Learn about host patterns in Ansible","title":"Ansible: host patterns"},{"content":"Definition podman allows you to enter another USER and MNT namespace without starting a container, to see what\u0026rsquo;s going on without going beyond the normal process (by the way, this is just a convenient add-on to the regular unshare command)\nPreparation Let\u0026rsquo;s check the behavior in another USER namespace:\n# See UID mapping podman unshare cat /proc/self/uid_map # Will show nobody podman unshare ls -ld / # Checi the USER/MNT namespace booth on the host and in an isolated namespace ls -l /proc/self/ns/user /proc/self/ns/mnt podman unshare ls -l /proc/self/ns/user /proc/self/ns/mnt Let\u0026rsquo;s check the behavior in another MNT namespace:\necho hello \u0026gt; /tmp/testfile mount --bind /tmp/testfile /etc/shadow mount: /etc/shadow: must be superuser to use mount. podman unshare bash -c \u0026#34;mount -o bind /tmp/testfile /etc/shadow; cat /etc/shadow\u0026#34; hello Practice Now let\u0026rsquo;s move on to viewing the contents of the image using podman mount\nIn general, the command for viewing looks like this: podman mount \u0026lt;URL/Image\u0026gt; (you can use the image name or path to it for podman pull)\nHowever podman mount requires either elevated privileges or running in an isolated USER namespace, and here the podman unshare command comes to the rescue, which creates a dedicated USER and MNT namespace\nIn addition, you will need the podman image mount command, which mounts the FS from the image to the host FS and displays the full path to it on the host\nLet\u0026rsquo;s try:\n# Enter another namespace podman unshare # Write path to a variable mnt=$(podman image mount \u0026lt;URL/image\u0026gt;) # Now we can view image filesystem without running ls $mnt/var/ cat $mnt/etc/shadow # To exit podman image unmount \u0026lt;URL/image\u0026gt; exit ","permalink":"https://debops.ru/posts/podman_image_discovery/","summary":"Learn how to view image content without creating container it in Podman","title":"Podman: how to view image content without creating container"},{"content":"Definition Let\u0026rsquo;s start with the fact that the variable obtained using set_fact will exist until the completion of the playbook From this we get - this variable remains \u0026ldquo;fixed\u0026rdquo; to the host or group of hosts on which it was obtained\nMethod №1: Getting fact directly For example, I showed that you can write it into a separate variable, or use it directly (fact_1 and fact_2)\n- name: Set fact on localhost hosts: localhost tasks: - name: Set fact ansible.builtin.set_fact: localhost_variable: Hello! - hosts: debian-host vars: shared_variable: \u0026#34;{{ hostvars[\u0026#39;localhost\u0026#39;][\u0026#39;localhost_variable\u0026#39;] }}\u0026#34; tasks: - name: Show this variable ansible.builtin.debug: msg: \u0026#34;{{ shared_variable }}\u0026#34; - name: Set fact for this host ansible.builtin.set_fact: fact_1: \u0026#34;{{ shared_variable }}\u0026#34; fact_2: \u0026#34;{{ hostvars[\u0026#39;localhost\u0026#39;][\u0026#39;localhost_variable\u0026#39;] }}\u0026#34; Method №2: Using group all and run_once: true In the first task we use the parameter run_once: true to execute the first task only once and then we execute the task on all hosts, which will write the value for each of the hosts (if we simply generate on each host separately, then each of them will have a random value)\n- name: Play this on all hosts hosts: all tasks: - name: Generate password and token ansible.builtin.set_fact: _password: \u0026#34;{{ lookup(\u0026#39;ansible.builtin.password\u0026#39;, \u0026#39;/dev/null\u0026#39;, chars=[\u0026#39;ascii_letters\u0026#39;, \u0026#39;digits\u0026#39;], length=10) }}\u0026#34; _token: \u0026#34;{{ lookup(\u0026#39;community.general.random_string\u0026#39;, length=32) }}\u0026#34; run_once: true - name: Make these values shared ansible.builtin.set_fact: password: \u0026#34;{{ _some_password }}\u0026#34; token: \u0026#34;{{ _some_token }}\u0026#34; Method №3: Using delegate_to It is important to use delegate_facts: true because when changing the host via delegate_to, the inventory_hostname variable is preserved and therefore all facts will be assigned to the original host (in the example, this host is localhost)\nlocalhost is not in the all group, so if you need to determine the fact on it, you need to combine the host lists (for example, \u0026quot;{{ groups['all'] + ['localhost'] }}\u0026quot;):\n- name: Play this locally hosts: localhost tasks: - name: Generate password and token ansible.builtin.set_fact: _password: \u0026#34;{{ lookup(\u0026#39;ansible.builtin.password\u0026#39;, \u0026#39;/dev/null\u0026#39;, chars=[\u0026#39;ascii_letters\u0026#39;, \u0026#39;digits\u0026#39;], length=10) }}\u0026#34; _token: \u0026#34;{{ lookup(\u0026#39;community.general.random_string\u0026#39;, length=32) }}\u0026#34; - name: Make these values shared ansible.builtin.set_fact: password: \u0026#34;{{ lookup(\u0026#39;ansible.builtin.password\u0026#39;, \u0026#39;/dev/null\u0026#39;, chars=[\u0026#39;ascii_letters\u0026#39;, \u0026#39;digits\u0026#39;], length=10) }}\u0026#34; token: \u0026#34;{{ lookup(\u0026#39;community.general.random_string\u0026#39;, length=32) }}\u0026#34; loop: \u0026#34;{{ groups[\u0026#39;all\u0026#39;] + [\u0026#39;localhost\u0026#39;] }}\u0026#34; delegate_to: \u0026#34;{{ item }}\u0026#34; delegate_facts: true ","permalink":"https://debops.ru/posts/ansible_share_fact/","summary":"Learn how to share facts from set_fact module in Ansible","title":"Ansible: how to share facts from set_fact to all hosts"},{"content":" raw — sends a low-level command directly via ssh command — sends a command to the operating system shell — sends a command to the operating system shell raw should be used on a remote host to install python, or to control active network hardware without the ability to install python\nshell should be used if shell capabilities are needed (environment variables, anonymous pipes, redirecting output streams, etc.)\ncommand should be used in all other cases, since using shell is slower than command, and raw makes no sense if python is already installed.\nEach of these modules is not idempotent and their use is only permissible if there is no alternative in the form of another module\nBut if you still decide to use these modules, then you should follow ansible best-practice:\nFor the shell and command modules, it is necessary to define changed_when: \u0026lt;statement\u0026gt; a condition that will determine when the module changed the state of the infrastructure (CHANGED), and when the change has already been made (OK) For the shell module, when using anonymous pipes (pipe, |), use set -o pipefail beforehand to handle errors correctly Example:\n- name: Get Kerberos ticket ansible.builtin.shell: cmd: | set -o pipefail echo {{ password }} | kinit admin changed_when: false ","permalink":"https://debops.ru/posts/ansible_raw_command_shell/","summary":"Learn how to use raw, command and shell in Ansible","title":"Ansible: raw vs command shell"},{"content":"Definition Why would you need ansible_managed in Ansible?\nansible_managed is a special variable created exclusively for the ansible.builtin.template and ansible.windows.win_template modules, which allows you to leave a special identifying mark in the generated templates\nAccording to ansible best-practice, every Jinja2 template should have such a mark at the beginning of the template so that it can be easily determined which files were created or modified using Ansible\nThere are additional subvariables for ansible_managed:\n{file} — full path to template (on controller host) {host} — hostname of controller {uid} — username on controller host strftime timestamp — %Y-%m-%d %H:%M:%S (man strftime 3) Note that using any of the above variables (especially timestamp) may break idempotency!\nIt is recommended to use {{ ansible_managed | comment }} instead of # {{ ansible_managed }}, since the comment filter can automatically match the comment to the file type and the number of comment characters when processing multi-line, for example with the following configuration:\n[defaults] ansible_managed = This file is managed by Ansible.%n template: {file} date: %Y-%m-%d %H:%M:%S user: {uid} host: {host} Practice The # {{ ansible_managed }} option will add (the values ​​are taken as an example):\n# This file is managed by Ansible. template: tmpl.j2 date: 2024-01-01 01:01:01 user: admin host: localhost At the same time, the {{ ansible_managed | comment }} option will add:\n# # This file is managed by Ansible. # # template: tmpl.j2 # date: 2024-01-01 01:01:01 # user: admin # host: localhost # Well, since Ansible allows you to extend its functionality using plugins, we can define any lookup plugin and substitute its value in ansible_managed. For example, this is how you can add the latest project commit to the template (example was taken from here https://jpmens.net/2020/09/29/using-ansible-managed/):\n[defaults] ansible_managed = \u0026#34;$Ansible {{{{ template_path|basename + lookup(\u0026#39;pipe\u0026#39;, \u0026#39;git log --format=\u0026#34;,%%h %%ad %%ae\u0026#34; -1 --date=format:\u0026#34;%%Y/%%m/%%d %%H:%%M\u0026#34; \u0026#39; + template_fullpath|quote)|default(\u0026#34;,UNCOMMITED\u0026#34;, True) }}}}$\u0026#34; ","permalink":"https://debops.ru/posts/ansible_managed/","summary":"Learn about ansible_managed in Ansible","title":"Ansible: ansible_managed"},{"content":"Definition Ansible uses different inventory plugins to manage data transferred from files or scripts and convert it to JSON format\nIn general there are 3 types of inventory:\nStatic — plain file Directory — directory with files Dynamic — executable file At the same time if you can use second (Directory) type with first (Static) type and third (Dynamic) type simultaneously, and the resulting inventory will be a collection of all inventories from this directory.\nSupported formats for Static inventory are limited by inventory plugins, which you could write by yourself, however in ansible.builtin (Ansible core) you can only use:\nyaml - ansible.builtin.yaml json - ansible.builtin.yaml (let me remind you that correctly written JSON is also YAML) ini - ansible.builtin.ini toml - ansible.builtin.toml Most often used are yaml and ini, but according to ansible best-practice you shoud use yaml format All types of plugins for inventory can be found here (https://docs.ansible.com/ansible/latest/collections/index_inventory.html)\nPractice As an example, I will demonstrate the same static inventory in different formats:\ninventory.ini:\n[awx_root] awx ansible_user=admin ansible_host=192.0.2.10 inventory.toml:\n[all.vars] ansible_user = \u0026#34;admin\u0026#34; [awx_root.hosts] awx = { ansible_host = \u0026#34;192.0.2.10\u0026#34; } inventory.yml:\n--- all: vars: ansible_user: admin hosts: awx: ansible_host: 192.0.2.10 children: awx_root: hosts: awx: inventory.json:\n{ \u0026#34;all\u0026#34;: { \u0026#34;vars\u0026#34;: { \u0026#34;ansible_user\u0026#34;: \u0026#34;admin\u0026#34; }, \u0026#34;hosts\u0026#34;: { \u0026#34;awx\u0026#34;: { \u0026#34;ansible_host\u0026#34;: \u0026#34;192.0.2.10\u0026#34; } }, \u0026#34;children\u0026#34;: { \u0026#34;awx_root\u0026#34;: { \u0026#34;hosts\u0026#34;: { \u0026#34;awx\u0026#34;: null } } } } } Using the command ansible-inventory -i \u0026lt;имя_файла\u0026gt; --list you can check all 4 files and see that the resulting JSON response will be the same (you can add the --yaml flag for convenient output in yaml format)\nA more detailed description with examples for dynamic inventories will be in a separate article.\n","permalink":"https://debops.ru/posts/ansible_inventory/","summary":"Learn about inventory types in Ansible","title":"Ansible: inventory types"},{"content":"Definition Why do we need pre_tasks, tasks, roles, post_tasks in Ansible?\nIn general, the order of execution will always be as follows:\n1) pre_tasks -\u0026gt; 2) roles -\u0026gt; 3) tasks -\u0026gt; 4) post_tasks Main differences:\npre_tasks and post_tasks are needed to be used together with the roles directive to execute tasks before or after a role roles and tasks share the same space and should not be used in the same play according to ansible best-practice However, there is another important feature when working with handlers — within one play it can be called up to 3 times:\nin a pre_tasks in a roles/tasks (when used together, handler will be called at the end of tasks once) in a post_tasks Also, it is important to note here that in pre_tasks and post_tasks you can call handlers from a role and it, according to the rule above, can be executed up to 3 times\nPractice Let\u0026rsquo;s write a simple playbook, role and config with the following structure:\n├── ansible.cfg ├── roles │ └── client │ ├── handlers │ │ └── main.yml │ └── tasks │ └── main.yml └── test.yml Content of ansible.cfg:\n[defaults] roles_path = roles stdout_callback = yaml Content of roles/client/handlers/main.yml:\n--- - name: Print role message ansible.builtin.debug: msg: Role handler executed succesfully Content of roles/client/tasks/main.yml:\n--- - name: Send role message ansible.builtin.command: /bin/true notify: - Print message - Print role message Content of test.yml\n--- - name: Testing handlers hosts: localhost gather_facts: false handlers: - name: Print message ansible.builtin.debug: msg: Handler executed succesfully pre_tasks: - name: Echo pre tasks ansible.builtin.command: /bin/true notify: - Print message - Print role message roles: - client tasks: - name: Echo tasks ansible.builtin.command: /bin/true notify: - Print message - Print role message post_tasks: - name: Post tasks ansible.builtin.command: /bin/true notify: - Print message - Print role message Result of running ansible-playbook test.yml: ","permalink":"https://debops.ru/posts/ansible_pre_post_tasks/","summary":"Learn about pre_tasks, tasks, roles and post_tasks in Ansible","title":"Ansible: pre_tasks, tasks, roles, post_tasks"}]