snapshots module

class snapshots.FileInfoDict

Bases: dict

A dict that maps a path (as bytes) to a tuple (int, bytes, bytes).

class snapshots.GenericNonSnapshot(date, cfg)

Bases: SID

property displayID

Snapshot ID in a user-readable format: YYYY-MM-DD HH:MM:SS

Returns:

formatted sID

Return type:

str

property displayName

Combination of displayID, name and error indicator (if any)

Returns:

name

Return type:

str

property tag

Snapshot ID’s tag

Returns:

tag (last three digits)

Return type:

str

property withoutTag

Snapshot ID without tag

Returns:

YYYYMMDD-HHMMSS

Return type:

str

class snapshots.NewSnapshot(cfg)

Bases: GenericNonSnapshot

Snapshot ID object for ‘new_snapshot’ folder

Parameters:

cfg (config.Config) – current config

NEWSNAPSHOT = 'new_snapshot'

SAVETOCONTINUE = 'save_to_continue'

property hasChanges

Check if there where changes in previous sessions.

Returns:

True if there where changes

Return type:

bool

property name

Name of this snapshot

Returns:

name of this snapshot

Return type:

str

property saveToContinue

Check if ‘save_to_continue’ flag is set

Parameters:

enable (bool) – set or remove flag

Returns:

True if flag is set

Return type:

bool

class snapshots.RootSnapshot(cfg)

Bases: GenericNonSnapshot

Snapshot ID for the filesystem root folder (‘/’)

Parameters:

cfg (config.Config) – current config

property name

Name of this snapshot

Returns:

name of this snapshot

Return type:

str

path(*path, use_mode=[])

Current path of this snapshot automatically altered for remote/encrypted version of this path

Parameters:

• *path (str) – one or more folder/files to join at the end of the path.

• use_mode (list) – list of modes that should alter this path. If the current mode is in this list, the path will automatically altered for the remote/encrypted version of this path.

Returns:

full snapshot path

Return type:

str

class snapshots.SID(date, cfg)

Bases: object

Snapshot ID object used to gather all information for a snapshot

See Snapshots to understand the difference.

Parameters:

• date (str, datetime.date or datetime.datetime) – used for creating this snapshot. str must be in snapshot ID format (e.g 20151218-173512-123)

• cfg (config.Config) – current config

Raises:

• ValueError – if date is str instance and doesn’t match the snapshot ID format (20151218-173512-123 or 20151218-173512)

• TypeError – if date is not str, datetime.date or datetime.datetime type

FAILED = 'failed'

FILEINFO = 'fileinfo.bz2'

INFO = 'info'

LOG = 'takesnapshot.log.bz2'

NAME = 'name'

property displayID

Snapshot ID in a user-readable format: YYYY-MM-DD HH:MM:SS

Returns:

formatted sID

Return type:

str

property displayName

Combination of displayID, name and error indicator (if any)

Returns:

name

Return type:

str

exists()

True if the snapshot folder and the “backup” folder inside exist

Returns:

True if exists

Return type:

bool

property failed

This snapshot has failed

Parameters:

enable (bool) – set or remove flag

Returns:

True if flag is set

Return type:

bool

property fileInfo

Load/save “fileinfo.bz2”

Parameters:

d (FileInfoDict) – dict of: {path: (permission, user, group)}

Returns:

dict of: {path: (permission, user, group)}

Return type:

FileInfoDict

property info

Load/save “info” file which contains additional information about this snapshot (using configfile.ConfigFile)

Parameters:

i (configfile.ConfigFile) – info that should be saved.

Returns:

snapshots information

Return type:

configfile.ConfigFile

isExistingPathInsideSnapshotFolder(path)

True if path is a file inside this snapshot

Parameters:

path (str) – path from local filesystem (no snapshot path)

Returns:

True if file exists

Return type:

bool

property lastChecked

Date when snapshot has finished last time. This can be the end of creation of this snapshot or the last time when this snapshot was checked against source without changes.

Returns:

date and time of last check (YYYY-MM-DD HH:MM:SS)

Return type:

str

log(mode=None, decode=None)

Load log from “takesnapshot.log.bz2”

Parameters:

• mode (int) – Mode used for filtering. Take a look at snapshotlog.LogFilter

• decode (encfstools.Decode) – instance used for decoding lines or None

Yields:

str – filtered and decoded log lines

makeDirs(*path)

Create snapshot directory

Parameters:

*path (str) – one or more folder/files to join at the end of the path

Returns:

True if successful

Return type:

bool

makeWritable()

Make the snapshot path writable so we can change files inside

Returns:

True if successful

Return type:

bool

property name

Name of this snapshot

Parameters:

name (str) – new name of the snapshot

Returns:

name of this snapshot

Return type:

str

path(*path, use_mode=[])

Current path of this snapshot automatically altered for remote/encrypted version of this path

Parameters:

• *path (str) – one or more folder/files to join at the end of the path.

• use_mode (list) – list of modes that should alter this path. If the current mode is in this list, the path will automatically be altered for the remote/encrypted version of this path.

Returns:

full snapshot path

Return type:

str

pathBackup(*path, **kwargs)

‘backup’ folder inside snapshots path

Parameters:

• *path (str) – one or more folder/files to join at the end of the path.

• use_mode (list) – list of modes that should alter this path. If the current mode is in this list, the path will automatically be altered for the remote/encrypted version of this path.

Returns:

full snapshot path

Return type:

str

setLastChecked()

Set info files atime to current time to indicate this snapshot was checked against source without changes right now.

setLog(log)

Write log to “takesnapshot.log.bz2”

Parameters:

log – full snapshot log

split()

Split self.sid into a tuple of int’s with Year, Month, Day, Hour, Minute, Second

Returns:

tuple of 6 int

Return type:

tuple

property tag

Snapshot ID’s tag

Returns:

tag (last three digits)

Return type:

str

property withoutTag

Snapshot ID without tag

Returns:

YYYYMMDD-HHMMSS

Return type:

str

class snapshots.Snapshots(cfg=None)

Bases: object

Collection of take-snapshot and restore commands.

BUHTZ 2022-10-09: In my understanding this is the representation of a snapshot in the “application layer”. This seems to be the difference to the class SID which represents a snapshot in the “data layer”.

BUHTZ 2024-02-23: Not sure but it seems to be one concret snapshot and not a collection of snapshots. In this case the class name is missleading because it is in plural form.

Parameters:

cfg (config.Config) – current config

GLOBAL_FLOCK = '/tmp/backintime.lock'

SNAPSHOT_VERSION = 3

backup(force=False)

Wrapper for takeSnapshot() which will prepare and clean up things for the main takeSnapshot() method.

This will check that no other snapshots are running at the same time, there is nothing prohibiting a new snapshot (e.g. on battery) and the profile is configured correctly. This will also mount and unmount remote destinations.

Parameters:

force (bool) – Force taking a new snapshot even if the profile is not scheduled or the machine is running on battery.

Returns:

True if there was an error.

Return type:

bool

backupConfig(sid)

Backup the config file to the snapshot and to the backup root if backup is encrypted.

Parameters:

sid (SID) – snapshot in which the config should be stored

backupInfo(sid)

Save infos about the snapshot into the ‘info’ file.

Parameters:

sid (SID) – snapshot that should get an info file

backupPermissions(sid)

Save permissions (owner, group, read-, write- and executable) for all files in Snapshot sid into snapshots fileInfoDict.

Parameters:

sid (SID) – snapshot that should be scanned

Returns:

Return code of rsync.

Return type:

int

backupPermissionsCallback(line, user_data)

Rsync callback for Snapshots.backupPermissions().

Parameters:

• line (bytes) – output from rsync command

• user_data (tuple) – two item tuple of (FileInfoDict, encfstools.Decode)

backupSuffix()

Get suffix for backup files.

Returns:

backup suffix in form of ‘.backup.YYYYMMDD’

Return type:

str

busy()

clearIdCache()

Reset the cache for UIDs and GIDs.

clearNameCache()

Reset the cache for user and group names.

clearTakeSnapshotMessage()

Delete message and progress file

collectPermission(fileinfo, path)

Collect permission infos about path and store them into fileinfo.

Parameters:

• fileinfo (FileInfoDict) – dict of: {path: (permission, user, group)} Using sideefect on changing dict item will change original dict, too.

• path (bytes) – full path to file or folder

createLastSnapshotSymlink(sid)

Create symlink ‘last_snapshot’ to snapshot sid

Parameters:

sid (SID) – snapshot that should be linked.

Returns:

True if successful

Return type:

bool

decMonth(date)

First day of previous month of date with respect on previous years. So if date is January this will return 1st of December previous year.

Parameters:

date (datetime.date) – old date that should be decreased

Returns:

1st day of previous month

Return type:

datetime.date

deletePath(sid, path)

Delete path and all files and folder inside in snapshot sid.

Parameters:

• sid (SID) – snapshot ID in which path should be deleted

• path (str) – path to delete

filter(base_sid, base_path, snapshotsList, list_diff_only=False, flag_deep_check=False, list_equal_to='')

Filter snapshots from snapshotsList based on whether base_path file is included and optional if the snapshot is unique or equal to list_equal_to.

Parameters:

• base_sid (SID) – snapshot ID that contained the original file base_path

• base_path (str) – path to file on root filesystem.

• snapshotsList (list) – List of SID objects that should be filtered

• list_diff_only (bool) – if True only return unique snapshots. Which means if a file is exactly the same in different snapshots only the first snapshot will be listed

• flag_deep_check (bool) – use md5sum to check uniqueness of files. More accurate but slow

• list_equal_to (str) – full path to file. If not empty only return snapshots which have exactly the same file as this file

Returns:

filtered list of SID objects

Return type:

list

filterRsyncProgress(line)

Filter rsync’s stdout for progress information and store them in ‘~/.local/share/backintime/worker<N>.progress’ file.

Parameters:

line (str) – stdout line from rsync

Returns:

line if it had no progress infos. None if

line was a progress

Return type:

str

flockExclusive()

Block backup() from other profiles or users and run them serialized

flockRelease()

Release lock so other snapshots can continue

freeSpace(now)

Remove old snapshots on based on different rules (only if enabled). First rule is to remove snapshots older than X years. Next will call smartRemove() to remove snapshots based on configurable intervals. Third rule is to remove the oldest snapshot until there is enough free space. Last rule will remove the oldest snapshot until there are enough free inodes.

‘last_snapshot’ symlink will be fixed when done.

Parameters:

now (datetime.datetime) – date and time when takeSnapshot was started

gid(name, callback=None, backup=None)

Get the Group identifier (GID) for the group in name. name->gid will be cached to speed up subsequent requests.

Parameters:

• name (str, bytes) – groupname to search for

• callback (method) – callable which will handle a given message

• backup (int) – GID which will be used if the groupname is unknown on this machine

Returns:

GID of the group in name or -1 if not found

Return type:

int

groupName(gid)

Get the groupname for the given gid. gid->name will be cached to speed up subsequent requests.

Parameters:

gid (int) – Group identifier (GID) to search for

Returns:

name of the Group with GID gid or ‘.’ if not found

Return type:

str

incMonth(date)

First day of next month of date with respect on new years. So if date is December this will return 1st of January next year.

Parameters:

date (datetime.date) – old date that should be increased

Returns:

1st day of next month

Return type:

datetime.date

makeDirs(path)

Wrapper for tools.makeDirs(). Create directories path recursive and return success. If not successful send error-message to log.

Parameters:

path (str) – fullpath to directories that should be created

Returns:

True if successful

Return type:

bool

pid()

remove(sid)

Remove snapshot sid.

BUHTZ 2022-10-11: From my understanding rsync is used here to sync the directory of a concrete snapshot (sid) against an empty temporary directory. In the consequence the sid directory is empty but not deleted. To delete that directory simple rm call (via shutil package) is used to delete the directory. No need to do this via SSH because the directory is temporary mounted.

It is not clear for me why it is done that way. Why not simply “rm” the directory when it is mounted instead of using rsync in a previous step?! But I won’t change it yet.

Parameters:

sid (SID) – snapshot to remove

Returns:

True if succeeded otherwise False.

Return type:

(bool)

restore(sid, paths, callback=None, restore_to='', delete=False, backup=True, only_new=False)

Restore one or more files from snapshot sid to either original or a different destination. Restore is done with rsync. If available permissions will be restored from fileinfo.bz2.

Parameters:

• sid (SID) – snapshot from whom to restore

• paths (list, tuple or str) – single path (str) or multiple paths (list, tuple) that should be restored. For every path this will run a new rsync process. Permissions will be restored for all paths in one run

• callback (method) – callable instance which will handle messages

• restore_to (str) – full path to restore to. If empty restore to original destination

• delete (bool) – delete newer files which are not in the snapshot

• backup (bool) – create backup files (*.backup.YYYYMMDD) before changing or deleting local files.

• only_new (bool) – Only restore files which do not exist or are newer than those in destination. Using rsync --update option.

restoreCallback(callback, ok, msg)

Format messages thrown by restore depending on whether they where successful or failed.

Parameters:

• callback (method) – callable instance which will handle the message

• ok (bool) – True if the logged action was successful or False if it failed

• msg (str) – message that should be send to callback

restorePermission(key_path, path, fileInfoDict, callback=None)

Restore permissions (owner, group and mode). If permissions are already identical with the new ones just skip. Otherwise try to ‘chown’ to new owner and new group. If that fails (most probably because we are not running as root and normal user has no rights to change ownership of files) try to at least ‘chgrp’ to the new group. Finally ‘chmod’ the new mode.

Parameters:

• key_path (bytes) – original path during backup. Same as in fileInfoDict.

• path (bytes) – current path of file that should be changed.

• fileInfoDict (FileInfoDict) – FileInfoDict

rsyncCallback(line, params)

Parse rsync’s stdout, send it to takeSnapshotMessage and takeSnapshotLog. Also check if there has been changes or errors in current rsync.

Parameters:

• line (str) – stdout line from rsync

• params (list) – list of two bool ‘[error, changes]’. Uses a side effect by changing list items here to change the original list of the caller, too (lists are passed as reference in Python). If rsync reported an error params[0] will be set to True. If rsync reported a changed file params[1] will be set to True

rsyncExclude(excludeFolders=None)

Format exclude list for rsync

Parameters:

excludeFolders (list) – list of folders to exclude

Returns:

rsync exclude options

Return type:

OrderedSet

rsyncInclude(includeFolders=None)

Format include list for rsync. Returns a tuple of two include strings. First string need to come before exclude, second after exclude.

Parameters:

includeFolders (list) – folders to include. list of tuples (item, int) where int is 0 if item is a folder or 1 if item is a file

Returns:

two item tuple of

(OrderedSet('include1 opions'), OrderedSet('include2 options'))

Return type:

tuple

rsyncRemotePath(path, use_mode=['ssh', 'ssh_encfs'], quote='"')

Format the destination string for rsync depending on which profile is used.

Parameters:

• path (str) – destination path

• use_mode (list) – list of modes in which the result should change to user@host:path instead of just path

• quote (str) – use this to quote the path

Returns:

quoted path like ‘”/foo”’

or if the current mode is using ssh and current mode is in use_mode a combination of user, host and path like ‘’user@host:”/foo”’’

Return type:

str

rsyncSuffix(includeFolders=None, excludeFolders=None)

Create suffixes for rsync.

Parameters:

• includeFolders (list) – folders to include. list of tuples (item, int) Where int is 0 if item is a folder or 1 if item is a file

• excludeFolders (list) – list of folders to exclude

Returns:

rsync include and exclude options

Return type:

list

setTakeSnapshotMessage(type_id, message, timeout=-1)

Update the status message of the active snapshot creation job

Write the status message into a message file to allow async processing for the GUI, plug-ins (like user-callback) and desktop notifications.

Parameters:

• type_id – Simplified severity level of the status message: 0: INFO 1: ERROR other values: defaults to INFO (may change in the future)

• message – status message string

• timeout – Requested maximum processing duration in plug-ins. Default: -1 (no limit) Called plug-ins must try to keep the timeout itself (it is not enforced by the PluginManager. In fact currently all known plug-ins do ignore the timeout value!

smartRemove(del_snapshots, log=None)

Remove multiple snapshots either with Snapshots.remove() or in background on the remote host if mode is ssh or ssh_encfs and smart-remove in background is activated.

Parameters:

• del_snapshots (list) – list of SID that should be removed

• log (method) – callable method that will handle progress log

smartRemoveKeepAll(snapshots, min_date, max_date)

Return all snapshots between min_date and max_date.

Parameters:

• snapshots (list) – full list of SID objects

• min_date (datetime.date) – minimum date for snapshots to keep

• max_date (datetime.date) – maximum date for snapshots to keep

Returns:

set of snapshots that should be kept

Return type:

set

smartRemoveKeepFirst(snapshots, min_date, max_date, keep_healthy=False)

Return only the first snapshot between min_date and max_date.

Parameters:

• snapshots (list) – full list of SID objects

• min_date (datetime.date) – minimum date for snapshots to keep

• max_date (datetime.date) – maximum date for snapshots to keep

• keep_healthy (bool) – return the first healthy snapshot (not marked as failed) instead of the first at all. If all snapshots failed this will again return the very first snapshot

Returns:

set of snapshots that should be kept

Return type:

set

smartRemoveList(now_full, keep_all, keep_one_per_day, keep_one_per_week, keep_one_per_month)

Get a list of old snapshots that should be removed based on configurable intervals.

Parameters:

• now_full (datetime.datetime) – date and time when takeSnapshot was started

• keep_all (int) – keep all snapshots for the last keep_all days

• keep_one_per_day (int) – keep one snapshot per day for the last keep_one_per_day days

• keep_one_per_week (int) – keep one snapshot per week for the last keep_one_per_week weeks

• keep_one_per_month (int) – keep one snapshot per month for the last keep_one_per_month months

Returns:

snapshots that should be removed

Return type:

list

statFreeSpaceLocal(path)

Get free space on filesystem containing path in MiB using os.statvfs(). Depending on remote SFTP server this might fail on sshfs mounted shares.

Parameters:

path (str) – full path

Returns:

int free space in MiB

statFreeSpaceSsh()

Get free space on remote filesystem in MiB. This will call df on remote host and parse its output.

Returns:

int free space in MiB

takeSnapshot(sid, now, include_folders)

This is the main backup routine.

It will take a new snapshot and store permissions of included files and folders into fileinfo.bz2.

Parameters:

• sid (SID) – snapshot ID which the new snapshot should get

• now (datetime.datetime) – date and time when this snapshot was started

• include_folders (list) – folders to include. list of tuples (item, int) where int is 0 if item is a folder or 1 if item is a file

Returns:

list of two bool (ret_val, ret_error) where

ret_val is True if a new snapshot has been created and ret_error is True if there was an error during taking the snapshot

Return type:

list

takeSnapshotMessage()

Get the current message from the message file.

Returns:

The message type and its string or None. See

setTakeSnapshotMessage() for further details.

Return type:

(tuple(int, str))

Dev note (buhtz):

Too many try..excepts in here.

uid(name, callback=None, backup=None)

Get the User identifier (UID) for the user in name. name->uid will be cached to speed up subsequent requests.

Parameters:

• name (str, bytes) – username to search for

• callback (method) – callable which will handle a given message

• backup (int) – UID which will be used if the username is unknown on this machine

Returns:

UID of the user in name or -1 if not found

Return type:

int

userName(uid)

Get the username for the given uid. uid->name will be cached to speed up subsequent requests.

Parameters:

uid (int) – User identifier (UID) to search for

Returns:

name of the user with UID uid or ‘-’ if not found

Return type:

str

snapshots.iterSnapshots(cfg, includeNewSnapshot=False)

Iterate over snapshots in current snapshot path. Use this in a ‘for’ loop for faster processing than list object

Parameters:

• cfg (config.Config) – current config

• includeNewSnapshot (bool) – include a NewSnapshot instance if ‘new_snapshot’ folder is available.

Yields:

SID – snapshot IDs

snapshots.lastSnapshot(cfg)

Most recent snapshot.

Parameters:

cfg (config.Config) – current config (config.Config instance)

Returns:

most recent snapshot ID

Return type:

SID

snapshots.listSnapshots(cfg, includeNewSnapshot=False, reverse=True)

List of snapshots in current snapshot path.

Parameters:

• cfg (config.Config) – current config (config.Config instance)

• includeNewSnapshot (bool) – include a NewSnapshot instance if ‘new_snapshot’ folder is available

• reverse (bool) – sort reverse

Returns:

list of SID objects

Return type:

list