Synoindex Watcher is a media index updater for Synology DiskStations based on inotify and synoindex. It watches the media-folders of your DiskStation an updates the media index every time a file or directory inside the media-folders is created, deleted or changed. It is written in Python and licensed as open-source under the GPL version 3.
The original version was written by Mark Houghton, who published it in his "codesourcery" blog. However, most of the original code has been replaced or rewritten by now.
This Python package is currently in beta-stage. All planned features are implemented, it is working and used on a number of devices. The only thing that's missing are some automated testing-routines, so there could still be undiscovered bugs.
- Lightweight No need to install extra Synology packages
- Intelligent Executes only if a file has actually been added/deleted/renamed
- Agnostic Python 2 and Python 3 compatible (minimum tested version is 2.7)
Synoindex Watcher cannot be installed via Synology's Package Center. You have to log in via SSH and use the terminal and install the Synoindex Watcher module manually. This will also install its dependency inotifyrecursive and also configparser, if you use Python <= 3.5.
It is recommended to install the Synindex Watcher module in a Python virtual environment:
$ sudo python -m venv /usr/local/var/venv/synoindexwatcher
$ sudo /usr/local/var/venv/synoindexwatcher/bin/python -m pip install --upgrade synoindexwatcher
It is recommended to use the Python package manager pip. Synology DSM 6 does not include pip by default, so you have to install it manually before installing the Synoindex Watcher module:
$ wget https://bootstrap.pypa.io/get-pip.py -qO - | sudo python
$ sudo python -m pip install --upgrade synoindexwatcher
You can use the following commands to upgrade Synoindex Watcher from an older version:
- DSM 7.x:
$ sudo /usr/local/var/venv/synoindexwatcher/bin/python -m pip install --upgrade synoindexwatcher - DSM 6.x:
$ sudo python -m pip install --upgrade synoindexwatcher
You can start Synoindex Watcher with one of the following commands:
- DSM 7.x:
$ sudo /usr/local/var/venv/synoindexwatcher/bin/python -m synoindexwatcher - DSM 6.x:
$ sudo python -m synoindexwatcher
Synoindex Watcher will watch the directories /volume1/music, /volume1/photo and /volume1/video by default. You can change this, as well as some other things, by adding some command-line arguments or using a configuration-file. You can also use an init-script to start Synoindex Watcher on boot.
For the sake of simplicity I will use the DSM 6.x command in the rest of the documentation.
The default behaviour of Synoindex Watcher can be changed with various command-line arguments:
-
path [path]: By appending one or more paths to the command-line you can define which directories shall be watched by synoindexwatcher. For examplepython -m synoindexwatcher /home/me/Musicwill tell Synoindex Watcher to watch only the directory /home/me/Music. -
--blacklist=regex: Define a regular-expression for a global blacklist. For examplepython -m synoindexwatcher --blacklist="foo|bar"will tell Synoindex Watcher ignore files and directories withfooorbarin their name. The default regular-expression is^\.|^\@|\.tmp$, which means that files and directories are ignored if they start with.or@or end with.tmp. The blacklist is applied after the whitelist. -
--whitelist=regex: Define a regular-expression for a global whitelist. For examplepython -m synoindexwatcher --whitelist="foo|bar"will tell Synoindex Watcher to watch only files and directories withfooorbarin their name. The default regular-expression is empty, which means that all files and directories, which are not blacklisted, are added to the media-index. The whitelist is applied before the blacklist. -
--logfile=file: Write log-messages to the specified file. For examplepython -m synoindexwatcher --logfile=/home/me/watcher.logwill tell Synoindex Watcher to write into the file /home/me/watcher.log. By default Synoindex Watcher will write to the terminal, if it is attached to one, or to /var/log/synoindexwatcher.log otherwise. -
--loglevel=value: Synoindex Watcher logs errors, warnings and informational messages by default. You can change this by setting the log-level to eitherDEBUG,INFO,WARNINGorERROR. For examplepython -m synoindexwatcher --loglevel=DEBUGwill also log (a lot of) debugging messages along with errors, warnings and infos. -
--config=file: Get the default-configuration from a certain file. For examplepython -m synoindexwatcher --config=/etc/synoindexwatcher.confwill tell Synoindex Watcher to use the values in /etc/synoindexwatcher.conf as its default-values. Any additional command-line arguments will override the values read from the configuration-file. -
--rebuild-index: Add all allowed files and directories in the watched directories and exit afterwards. Use it, if your media-index lacks entries for existing files. Read the related FAQ-entry, if you want to remove non-existing files from the media-index. -
--generate-init: Generate an init-script, write it to the standard output and exit afterwards. Any additional command-line arguments will be integrated into the generated script. See the start on boot section above for further details. -
--generate-config: Generate a configuration-file, write it to the standard output and exit afterwards. Any additional command-line arguments will be integrated into the generated configuration. See the configuration-file section below for further details. -
--help: Write a short online help to the standard output and exit afterwards.
The default behaviour of Synoindex Watcher can also be changed via a configuration-file instead of command-line arguments. Use the following command to create a configuration file:
$ python -m synoindexwatcher --generate-config | sudo tee /usr/local/etc/synoindexwatcher.conf
The generated file is split into several sections: The section [GLOBAL] may contain default-values for some command-line arguments, while each of the other sections (e.g. [/volume1/music]) represents a directory that shall be watched. The directory-sections contain no values so far.
You have to explicitly tell Synoindex Watcher to use a configuration-file by calling it like this:
$ python -m synoindexwatcher --config=/usr/local/etc/synoindexwatcher.conf
Keep in mind, that you can use additional command-line arguments to override the values from the configuration-file.
Use the following commands to create an init-script that starts Synoindex Watcher in the background when your DiskStation boots:
$ python -m synoindexwatcher --generate-init | sudo tee /usr/local/etc/rc.d/S99synoindexwatcher.sh
$ sudo chmod a+x /usr/local/etc/rc.d/S99synoindexwatcher.sh
Please note that any messages are written into the file /var/log/synoindexwatcher.log, if Synoindex Watcher is running as a background process. You can use the --logfile parameter to write the output to another file.
Also other command-line arguments will be integrated into the init-script. The following line will generate an init-script that tells Synoindex Watcher to watch only the directory /home/me/Music and to log messages to /home/me/watcher.log:
$ python -m synoindexwatcher --generate-init --logfile=/home/me/watcher.log /home/me/Music
It is recommended to use a configuration file along with the init script, because it allows you to change the behaviour of Synoindex Watcher without the need to regenerate the init-script after every change. Assuming you have created a configuration-file with the commands in the previous section, you can use the following line to generate an init-script that recognizes this file:
$ python -m synoindexwatcher --generate-init --config=/usr/local/etc/synoindexwatcher.conf
This actually does not mean that you run out of disk space, but rather that your system does not allow enough inode-watchers to watch all your media files. The message is quite confusing, though.
To fix this temporarily you could simply type echo 204800 > /proc/sys/fs/inotify/max_user_watches as root. The maximum number of inode-watchers in the user-space would be 204800 afterwards, which should be enough for most use-cases. Unfortunately this fixes the problem only until the next reboot.
To set the maximum value during boot, the line sysctl -w fs.inotify.max_user_watches=204800 has to be added to an init-script. The simplest way would be to add the command to the start-section of the init-script for Synonindex Watcher, but you could also use a dedicated init-script.
To get rid of entries for non-existing files in the media-index, you have to clear the whole media-index and repopulate it afterwards. Since this requires to modify the database directly with SQL-commands, it is strongly recommended to create a backup before executing the following commands that clear the media-index tables:
$ sudo synoservice --hard-stop synoindexd
$ psql mediaserver -tAc "SELECT string_agg(tablename, ',') from pg_catalog.pg_tables WHERE tableowner = 'MediaIndex'"
$ psql mediaserver -c "TRUNCATE `psql mediaserver -tAc "SELECT string_agg(tablename, ',') from pg_catalog.pg_tables WHERE tableowner = 'MediaIndex'"` RESTART IDENTITY"
$ sudo synoservice --start synoindexd"
Afterwards you can use Synoindex Watcher to repopulate the media-index:
$ python -m synoindexwatcher --rebuild-index
Make sure to add additional arguments like --config or the paths you want to watch.
Copyright (c) 2019-2022 Torben Haase <https://letorbi.com>