This project provides scripts and a windows exe which allows you to easily manage and schedule backups to your content on ArcGIS Online.
NOTE: This project is still in active development. The code should work but no garantees are provided.
These instructions will give you a copy of the project to get it up and running on your local machine for development and testing purposes. See deployment for notes on deploying the project on a live system.
Requirements for the software and other tools to build, test and push
A step by step series of examples that tell you how to get a development environment running. It is recomended to setup a virtual environment for running this project, especially if you plan to build it to and exe or equilivant file.
# Setup environment
python -m venv venv
# Activate environment
venv\Scripts\activate.batThis step can be handled by the requirements.txt file, however it will install the full ArcGIS API for Python which contains much more than is needed for this app. It is being managed manually here to reduce the size of files compiled for the window executable.
pip install arcgis --no-deps
# Upgrade
python -m pip install --upgrade pip
#Install requirements
pip install -r requirements.txtThe project is provided as a set of python scripts. These can be built using pyinstaller (see below) but are not provided as a part of the project.
To setup an environment for scheduled backups, first build a config file using backup_cfg_mgr (see below). Once setup, setup a schedule in your operating system to run the scripts according to a schedule. The schedule should be no longer than your shortest hour delay period. For example, if one of your backups are requested hourly, run your script hourly. Multiple config files can be specified and passed to backup_mgr
Backups are committed to either a local folder or a managed local git repo. If git is enabled, using your preferred git client i.e. Github Desktop, you can access previous versions and updates to data. The script does not manage git folder size, so you will want to keep an eye on the size of your repo and perhaps clean it up periodically to flush old backups.
This tool manages backs up and ArcGIS Online item to a local folder including definitions, data and features in the requested format
optional arguments:
-h, --help show this help message and exit
-c [CONFIG], --config [CONFIG]
Config item defining items to backup
-v Verbose, also logs debug messages
-q Do not log script progress to file
NOTE: The GUI works, but is slow. It may appear to be locked up whilst connecting to an AGOL/Portal instance with a significant number of items. Patience is a virtue.
ANOTHER NOTE: Config files are stored with passwords as plain text. You should store config files in your profile and not share them with others. Encryption may become a feature of this script but it is not within its current scope.
This tool manages backs up and ArcGIS Online item to a local folder including definitions, data and features in the requested format
optional arguments:
-h, --help show this help message and exit
-c CONFIG [CONFIG ...], --config CONFIG [CONFIG ...]
Config item defining items to backup
-v Verbose, also logs debug messages
-r Reset items by ignoring timestamps
-q Do not log script progress to file
This tool backs up and ArcGIS Online item to a local folder including its definition, data and features in the requested fmt
positional arguments:
portal AGOL portal
username AGOL username
password AGOL password
itemid AGOL item id
outputdir Output directory
{csv,shp,geojson,fgdb,gpkg,kml,spkg,vtpk,xls,none}
fmt to export features to (if features exist)
optional arguments:
-h, --help show this help message and exit
-o [{item,data,metadata,thumbnail,url,sharing,appinfo,related,service,resources,comments,all} ...]
Options for export
-s Skip unmodified items (works when backing up to the same location as last time)
-v Verbose, also logs debug messages
-q Do not log script progress to file
See Backup Items Options for more detail on export options.
This tool backs up and ArcGIS Online admin item to a JSON file. It has three modes, defined by the component you wish to export.
components:
self,user,group component to export
positional arguments:
portal AGOL Portal
username AGOL Username
password AGOL Password
outputdir Output directory
optional arguments:
-h, --help show this help message and exit
-o Options for export (component type specific)*
-v Verbose, also logs debug messages
-q Do not log script progress to file
*See Backup Admin Options for more detail on export options.
To build a standalone apps, compile with pyinstaller. The below should build the four apps into executables in the 'dist' folder. These executables will work on the system upon which is was built.
pyinstaller --noconfirm --onefile --console --icon "img/backup_mgr.ico" --add-data "certifi;certifi" "backup_mgr.py"
pyinstaller --noconfirm --onefile --console --icon "img/backup_items.ico" --add-data "certifi;certifi" "backup_items.py"
pyinstaller --noconfirm --onefile --console --icon "img/backup_admin.ico" --add-data "certifi;certifi" "backup_admin.py"
pyinstaller --noconfirm --onefile --console --icon "img/backup_cfg_mgr.ico" --add-data "certifi;certifi" "backup_cfg_mgr.py"
NOTE: The resulting executables are not garanteed to work within an enteprise systems.
- Paul Skeen - Provided main code base
- Billie Thompson - Provided README Template
Inspired by the lack of a descent solution from ESRI for backing up AGOL content and the work carried out by SEMCOG