- Python 100%
| demo | ||
| src | ||
| .gitignore | ||
| masync | ||
| README.md | ||
Table of Contents
Synopsis
masync is a command line tool that allows you to mirror and keep synchronised an arbitraries number of folders (destinations) with one or more sources.
Sources can be local or remote. SFTP is used for keep in sync remote sources.
A sync (task) consists of a destination folder, always local, and a source folder. Each task managed by masync is identified by a numeric id
These are the goals of this tool in the developer's intentions:
- Selfhost and privacy: encourage persistence and redundancy of your data on your own servers (with your keys)
- Multiple task: you can manage multiple synchronised folders with their remote/local destination using a single command from the console
- No install: there is no need to install any server-side software; you just need to have access to your server via SSH.
- 2 ways Synchronisation: you can synchronise from destination to source and from source to destination, integrating the changes made on the source.
- Conflict management you will not lose any file versions or changes. The tool try to help you which version to keep
Prerequisites - server side
The foundamental prerequisite is that you must have configured SSH access on the servers where the remote destinations are located. Then, in order you should have followed these steps:
install the ssh daemon on your favorite linux distribution (on the server)
Using the package manager of your distribution:
sudo apt install openssh-server
sudo dnf install openssh-server
sudo xbps-install -Su openssh
generate a key pair with ssh-keygen, in the local computer
ssh-keygen -f ~/<nome_file> -t rsa -b 2048
add the public key to the authorised keys on server side.
Enter on your server and run
cat <file_name> >> /home/{user}/.ssh/authorized_keys
- {user} must have 700 on home/{user}.ssh on local machine
- {user} must have 600 on
/home/{user}/.ssh/authorized_keysin the server
edit the local ssh config so that you can access with ssh to the server
Add these lines on your ssh config (usually located in ~/.ssh/config)
Host {hostname}
IdentityFile {path_to_private_key}
User {username}
Port {port}
Hostname {hostip}
by replacing:
- {hostname}: the name you use for connecting with ssh
- {path_to_private_key}: usually under ~/.ssh
- {username}: username used at server side
- {port}: port of ssh daemon
- {hostip}: the ip of host
Prerequisites - local
My advice is to keep the system clean and create a virtualenv for running masync
In this virtual environment you must install paramiko python package.
It is used to establish a connection with remote sources
pip install paramiko
Installation - local
Clone the repository and link the file masync in your ~/.local/bin/
cd ~
git clone https://codeberg.org/notanamber/Masync/
ln -s ~/Masync/src/masync ~/bin/masync
make sure in the .bashrc there is something like this
`export PATH=~/bin/:$PATH`
to make everything work properly you need to provide the tool with the paths of
known_hosts, the private key and the ssh configuration.
Put in your home directory the file masync.conf with a content similar to this
user@linux:~$ cat ~/masync.conf
HOSTS_KEYS_PATH = /home/{user}/.ssh/known_hosts
PKEY_PATH = /home/{user}/.ssh/notanamber_rsa
SSH_CONFIG_PATH = /home/{user}/.ssh/config
Using the command
masync has several subcommands (that you can view using masync -h). Their use will be explained within a typical workflow.
init
Initially the list of your task will be empty, then
you will want to mirror a remote source folder on your local computer (the destination). You will use the init subcommand in a similar way to this:
masync init /home/{user}/<local_folder>/ <remoteuser>@<server>:/home/{remote_user}/<remote_folder>/
Basically, you will specify a folder on the local host as the first parameter and a remote folder as second parameter
The second parameter must be in the form of user and host as the structure of an SSH command that connects to a remote server as a specific user
and can then execute a command or set the environment on that host
list
It shows the list of all synched local folders with respective remote folder and some other usefull information.
After the initial mirror the execution of list command will show the sync task just added
masync list
You can view more useful informations about each sync task managed:
- the integer id assigned to each sync.
- the local folder you are keeping in sync and the remote folder.
- last time of action done
- last action
In most subcommands you must refer the task sync with the integer id you can read in the output of this command
sync
Usually you will use the sync command after modifying the local tree: This will keep the local tree in sync with the source tree.
The only parameter required is the integer id of sync task provided from masync list command.
The program attempts to integrate locally the changes that have been made on the remote destination and then sends the changes made locally.
masync sync -s 2
Conflicts management
Conflicts may arise during synchronisation: suppose you have changed locally a file and you are trying to push it in the destination folder
but the version you changed not is the same present in the destination folder at the synchronization.
This leads to a conflict that you can resolve choosing which version of the file (if the local version or the source version) to keep
To check conflicts generated by a specific task sync, use the subcommand show-conflitcs. As usual the first parameter must be the id of task sync, for example
masync show-conflicts -s 2
When a conflict is generated by a text file, a viewable diff is saved in the .masy/diff folder, reproducing the original path of node.
If you want to mantain the local version of file you can edit the diff directly in .masy/diff folder, then resolve the conflict with the option keep-destination, for example:
masync resolve -s 2 -c 1 -X keep-destination
in any case they exist 2 possibilities for the strategy: keep-source or keep-destination. You can specify it with the flag -X