Clone Files to Cloud Storage

Links

• http://wiki.linuxquestions.org/wiki/Rsync_with_Google_Drive
• https://rclone.org/drive/
• https://developers.google.com/identity/protocols/OAuth2ServiceAccount

Install

wget https://rclone.org/install.sh
chmod +x install.sh
sudo ./install.sh

Configure Google Drive Client Access

Configure the account :

• Go to the Developer Console: https://console.developers.google.com/apis/dashboard
• Login if needed, makes sure you have the corrrect account
• Create a project and select it
• Select "ENABLE APIS AND SERVICES" and enable the "Google Drive API"

Configure rclone

Note:

• Token renewal seems to be done by rclone automatically. Still
• For automatic configuration you will need a browser in a desktop session, for info on headless configuration go here:
  • https://rclone.org/remote_setup/

rclone config
	n                   # New remote
	name> gdrive1       # Any short name
	Storage Type> drive # Google Drive
	client_id> 
	client_secret> 
	scope> drive        # Full access all files, excluding Application Data Folder
	root_folder_id> 
	service_account_file>
	Auto config> y      #your browser will open, login with your gmail account and allow access
	Team drives> n	
	is this OK?> y

Sync from local to Google Drive, name 'gdrive1', folder 'rclone':

rclone sync -v ~/test4.txt "gdrive1:/rclone/"

Sync from Google Drive to local:

rclone sync -v  "gdrive1:/rclone/test4.txt" ~/rclone/

Configure Google Drive Service Account

Note:

• The service account and the main account do not share access to Google Drive files.
• You can not use the Webinterface to access the service account Google Drive files. However, a service account is the preferred method for machine-to-machine interaction.
• To be able to use 'drive-impersonate' you need a "G Suite domain", for this you will need to use a domain that you 'own'. There are various ways to prove you 'own' a domain.

Create a service account :

• Go to the Developer Console: https://console.developers.google.com/apis/dashboard
• Login if needed, makes sure you have the correct account
• Create a project and select it
• Select "ENABLE APIS AND SERVICES" and enable the "Google Drive API"
• Use the " Create Credentials" button to create a "Service account key". Pick a "Service account name". "Role" can be empty.
• Select "Key type JSON" and save the file (rclone.json) for use with rclone config

Configure rclone

rclone config
	n                   # New remote
	name> gdrive1       # Any short name
	Storage Type> drive # Google Drive
	client_id> 
	client_secret> 
	scope> drive        # Full access all files, excluding Application Data Folder
	root_folder_id> 
	service_account_file> /home/pi/Scripts/rclone.json
	Edit advanced config?> n
	Configure this as a team drive?> n
	is this OK?> y

List contents:

rclone lsf -v "gdrive1:"

Sync:

rclone sync -v /home/pi/Documents "gdrive1:/rclone/"

List of rclone commands:

• https://rclone.org/commands/

Configure rclone encryption drive

Define an encrypted target, in this example the configured remote 'gdrive1' is used as storage in subfolder 'crypt'.

Note:

• more info on https://rclone.org/crypt/
• the pass-phrase / salt in this case is more like another password, pick a strong password and you don't need the pass-phrase

rclone config
	e/n/d/r/c/s/q> n
	name> gdrive2
	Storage> crypt
	remote> gdrive1:/crypt
	filename_encryption> 2
	directory_name_encryption> 1
	type password y/g/n> y
	seed y/g/n> n
	advanced config y/n> n

Sync from local to Google Drive, name 'gdrive2':

rclone sync -v /home/pi/Documents "gdrive2:/"

List contents (encrypted):

rclone lsf -v "gdrive:/crypt"

List contents (decrypted):

rclone lsf -v "gdrive2:"

Other commands

Used space / free space / trashed:

rclone about gdrive1:

Clean trash / bin. Note that this is done in background, it may take some time to finish:

rclone cleanup gdrive1:

Five most recently modified files with size and total number of files:

rclone tree --sort-modtime --human gdrive1:/ | tail -5