Clone Files to Cloud Storage

Links

• http://wiki.linuxquestions.org/wiki/Rsync_with_Google_Drive
• https://rclone.org/drive/
• https://developers.google.com/identity/protocols/OAuth2ServiceAccount

Install

sudo apt install rclone

Or from the source:

wget https://rclone.org/install.sh
chmod +x install.sh
sudo ./install.sh

Configure Google Drive Client Access

Configure the account :

• Go to the Developer Console: https://console.developers.google.com/apis/dashboard
• Login if needed, make sure you have the correct account
• Create a project and select it
• Select "ENABLE API'S AND SERVICES" and enable the "Google Drive API"

Configure rclone

Note:

• Token renewal seems to be managed by rclone.
• For automatic configuration you will need a browser in a desktop session, for info on headless configuration go here:
  • https://rclone.org/remote_setup/

rclone config
	n                   # New remote
	name> gdrive1       # Any short name
	Storage Type> drive # Google Drive
	client_id> 
	client_secret> 
	scope> drive        # Full access all files, excluding Application Data Folder
	root_folder_id> 
	service_account_file>
	Auto config> y      # Your browser will open, login with your gmail account and allow access
	Team drives> n	
	is this OK?> y

If you don't have a browser (for example via ssh) take the same steps except for:

	Auto config> n

Then:

1. Copy and past the URL to a browser in an other window
2. Login to your google account
3. Allow access to rclone
4. Copy and paste the returned code back and continue from there with "Team drives>"

Sync from local to Google Drive, name 'gdrive1', folder 'rclone':

rclone sync -v ~/test4.txt "gdrive1:/rclone/"

Sync from Google Drive to local:

rclone sync -v  "gdrive1:/rclone/test4.txt" ~/rclone/

Configure Google Drive Service Account

Note:

• The service account and the main account do not share access to Google Drive files.
• You can not use the Webinterface to access the service account Google Drive files. However, a service account is the preferred method for machine-to-machine interaction.
• To be able to use 'drive-impersonate' you need a "G Suite domain", for this you will need to use a domain that you 'own'. There are various ways to prove you 'own' a domain.

Create a service account :

• Go to the Developer Console: https://console.developers.google.com/apis/dashboard
• Login if needed, makes sure you have the correct account
• Create a project and select it
• Select "ENABLE APIS AND SERVICES" and enable the "Google Drive API"
• Use the " Create Credentials" button to create a "Service account key". Pick a "Service account name". "Role" can be empty.
• Select "Key type JSON" and save the file (rclone.json) for use with rclone config

Configure rclone

rclone config
	n                   # New remote
	name> gdrive1       # Any short name
	Storage Type> drive # Google Drive
	client_id> 
	client_secret> 
	scope> drive        # Full access all files, excluding Application Data Folder
	root_folder_id> 
	service_account_file> /home/pi/Scripts/rclone.json
	Edit advanced config?> n
	Configure this as a team drive?> n
	is this OK?> y

List contents:

rclone lsf -v "gdrive1:"

Sync:

rclone sync -v /home/pi/Documents "gdrive1:/rclone/"

Configure rclone encryption drive

Define an encrypted target, in this example the configured remote 'gdrive1' is used as storage in subfolder 'crypt'.

Note:

• more info on https://rclone.org/crypt/
• the pass-phrase / salt in this case is more like another password, pick a strong password and you don't need the pass-phrase

rclone config
	e/n/d/r/c/s/q> n
	name> gdrive2
	Storage> crypt
	remote> gdrive1:/crypt
	filename_encryption> 2
	directory_name_encryption> 1
	type password y/g/n> y
	salt y/g/n> n
	advanced config y/n> n

Sync from local to Google Drive, name 'gdrive2':

rclone sync -v /home/pi/Documents "gdrive2:/"

List contents (encrypted):

rclone lsf -v "gdrive1:/crypt"

List contents (decrypted):

rclone lsf -v "gdrive2:"

Other commands

Note:

• Commands seem to be instantly applied, however, server-side it may take some time to complete. For example, when you 'cleanup' and then check the used space with 'about' repeatedly, you may see 'Trashed' going down to 0 in steps.

Used space / free space / trashed:

rclone about gdrive1:

Clean trash / bin.

rclone cleanup gdrive1:

Five most recently modified files with size and total number of files:

rclone tree --sort-modtime --human gdrive1:/ | tail -5

List of rclone commands:

• https://rclone.org/commands/

See also

• Enlarge disk partition image
• rcloneExplorer (Android)
• RcloneBrowser (Windows, Linux)
  • Download RcloneBrowser : https://github.com/mmozeiko/RcloneBrowser/releases
  • Download Rclone: https://rclone.org/downloads/