Jtdylktuy

I am using the ideas laid out in this article to create incremental versioned backups of my data. I basically sync the data to a current folder in my backup destination and then create a date folder with hard links to the current folder. I end up with this:

$ ls

...

2019-01-01_10-00-01

2019-01-02_10-00-01

...

2019-02-15_10-00-01

...

current

It works great. If I ever need to do a full restore from a specific date, I can just restore everything from that date's folder.

But if you're looking for previous versions of a specific file, you have to go through each of the date folder's to find what you want. I want to create another folder that has a running total of all files, each time they were changed. A combined view if you will.

I came up with this, and it works, but I am wondering if there is a more elegant, standard way to do this.

#!/bin/bash



NOW=$(/bin/date +%Y-%m-%d_%H-%M-%S)



# the data that needs to be backed up

SOURCES=("/path/to/source 1" "/path/to/source 2")



# where it is going

DESTINATION="/path/to/backup"



# make sure the destination exists

mkdir -p "$DESTINATION"



# make sure there is a place to put the current data

mkdir -p "$DESTINATION/current"



# make sure there is a place to put the "combined" data

mkdir -p "$DESTINATION/combined"



# sync the data

rsync -v -a --delete "${SOURCES[@]}" "$DESTINATION/current"



# check if files were backed up

# any file with only one link is either new, and needs to have a hard link version

# or it wasn't fully backed up previously and needs a hard link version

if [[ $(find "$DESTINATION/current" -type f -links 1 | wc -l) -ne 0 ]] ; then

    # make a date folder backup using hard links

    cp -al "$DESTINATION/current" "$DESTINATION/$NOW"



    # make a combined view

    #  - find all files with 2 links

    #    - one link is to the file in the $DESTINATION/current

    #    - the other link is to the file in $DESTINATION/$NOW

    # - there should never be any files with only 1 hard link since the previous command

    #   is sure to have created a second link

    # - any files with more than 2 links were, hopefully, already covered during a previous iteration

    cd "$DESTINATION/current" && find * -type f -links 2 -print0 | while IFS= read -r -d $'' filePath

    do

        fileName="$(basename "$filePath")"

        fileFolder="$(dirname "$filePath")"



        # where the file will live in the combined folder

        # need to mirror the folder structure

        destinationFolder="$DESTINATION/combined/$fileFolder"

        mkdir -p "$destinationFolder"



        # make a hard link to it

        cp -al "$filePath" "$destinationFolder/$fileName.$NOW"

    done

fi

The code does work. After a few iterations, this is what it creates:

Files in the current folder (this is a "live" copy of the source data):

backup/current/source 1/001

backup/current/source 1/002

backup/current/source 1/003

backup/current/source 1/file 100

backup/current/source 1/folder/004

backup/current/source 2/006

Files in the date specific folders (note files from the first backup have files that aren't in the second because they were deleted):

backup/2019-01-15_23-08-02/source 1/001

backup/2019-01-15_23-08-02/source 1/002

backup/2019-01-15_23-08-02/source 1/003

backup/2019-01-15_23-08-02/source 1/file 100

backup/2019-01-15_23-08-02/source 1/folder/004

backup/2019-01-15_23-08-02/source 1/folder/005

backup/2019-01-15_23-08-02/source 2/006

backup/2019-01-15_23-08-02/source 2/007



backup/2019-01-15_23-09-00/source 1/001

backup/2019-01-15_23-09-00/source 1/002

backup/2019-01-15_23-09-00/source 1/003

backup/2019-01-15_23-09-00/source 1/file 100

backup/2019-01-15_23-09-00/source 1/folder/004

backup/2019-01-15_23-09-00/source 2/006

And these are the files in the combined view:

backup/combined/source 1/001.2019-01-15_23-08-02

backup/combined/source 1/002.2019-01-15_23-08-02

backup/combined/source 1/003.2019-01-15_23-08-02

backup/combined/source 1/003.2019-01-15_23-09-00

backup/combined/source 1/file 100.2019-01-15_23-08-02

backup/combined/source 1/folder/004.2019-01-15_23-08-02

backup/combined/source 1/folder/004.2019-01-15_23-09-00

backup/combined/source 1/folder/005.2019-01-15_23-08-02

backup/combined/source 2/006.2019-01-15_23-08-02

backup/combined/source 2/006.2019-01-15_23-09-00

backup/combined/source 2/007.2019-01-15_23-08-02

This way, if I need to find a previous version of source 1/folder/004, I just need to go to it's matching folder in backup/combined/ (backup/combined/source 1/folder) and all the 004 files are there, with a date/time stamp appended.

Is there a better, more elegant way to do this?