Clone Files to Cloud Storage: Difference between revisions

From WickyWiki
← Older editNewer edit →
mNo edit summary
mNo edit summary
Line 20: Line 20:
= Configure =
= Configure =


== Google account and Google Drive API ==
== Google Drive Client Access ==


[[TODO: Google account and Google Drive API]]
Configure the account :
* Go to the Developer Console: https://console.developers.google.com/apis/dashboard
* Login if needed, makes sure you have the corrrect account
* Create a project and select it
* Select "ENABLE APIS AND SERVICES" and enable the "Google Drive API"


== Google Drive Client Access ==
=== Configure rclone ===


This will grant access for a couple of hours.
Note:
* This will partly be repeated after the access token has expired.


<syntaxhighlight lang=bash>
<syntaxhighlight lang=bash>
rclone config
rclone config
n/s/q> n        # new
n                   # New remote
name> remote    # 'remote' is an example name, pick what you want
name> gdrive1      # Any short name
Storage> drive   # type=Google Drive
Storage Type> drive # Google Drive
client_id>      
client_id>  
client_secret>  
client_secret>  
scope> 1        # Select your scope, 1 = full
scope> drive        # Full access all files, excluding Application Data Folder
root_folder_id>
root_folder_id>  
svc acc file>  
service_account_file>
Auto config> y   #login with your gmail account and allow access
Auto config> y     #your browser will open, login with your gmail account and allow access
Team drives> n
Team drives> n
is this OK?> y
</syntaxhighlight>
</syntaxhighlight>


From local to Google Drive, name 'remote', folder 'rclone'
Sync from local to Google Drive, name 'gdrive1', folder 'rclone':


<syntaxhighlight lang=bash>
<syntaxhighlight lang=bash>
rclone sync -v ~/test4.txt "[remote]:/rclone/"
rclone sync -v ~/test4.txt "gdrive1:/rclone/"
</syntaxhighlight>
</syntaxhighlight>


From Google Drive to local
Synx from Google Drive to local:


<syntaxhighlight lang=bash>
<syntaxhighlight lang=bash>
rclone sync -v  "remote:/rclone/test4.txt" ~/rclone/
rclone sync -v  "gdrive1:/rclone/test4.txt" ~/rclone/
</syntaxhighlight>
</syntaxhighlight>


== Google Drive Service Account ==
== Google Drive Service Account ==


* https://developers.google.com/drive/api/v2/reference/permissions/insert
Note:
* The service account and the main account do not share access to Google Drive files.
* You can not use the Webinterface to access the service account Google Drive files. However, a service account is still the preferred method if you want to machine-to-machine interaction.
* To be able to use 'drive-impersonate' you need a "G Suite domain", for this you will need to use a domain that you 'own'. There are various ways to prove you 'own' a domain.


We will need a json file
Create a service account :
* Go to the Developer Console: https://console.developers.google.com/apis/dashboard
* Login if needed, makes sure you have the corrrect account
* Create a project and select it
* Select "ENABLE APIS AND SERVICES" and enable the "Google Drive API"
* Use the " Create Credentials" button to create a "Service account key". Pick a "Service account name". "Role" can be empty.
* Select "Key type JSON" and save the file for use with rclone config


[[TODO Accesibility of files - can't see the files of the other account in Google Drive]]
=== Configure rclone ===


Create a service account :
<syntaxhighlight lang=bash>
* To create a service account and obtain its credentials, go to the Google Developer Console. [https://console.developers.google.com/apis/dashboard
rclone config
* You must have a project - create one if you don’t.
n                  # New remote
* Use the “Create Credentials” button. Fill in “Service account name” with something that identifies your client. “Role” can be empty.
name> gdrive        # Any short name
* select “Key type JSON”.
Storage Type> drive # Google Drive
* If you ever need to remove access, press the “Delete service account key” button.
client_id>
* save the json file for use with rclone config
client_secret>
scope> drive        # Full access all files, excluding Application Data Folder
root_folder_id>
service_account_file> /home/pi/Scripts/rclone-4682b69ab73b.json
Edit advanced config?> n
Configure this as a team drive?> n
is this OK?> y
</syntaxhighlight>
 
List contents:
 
<syntaxhighlight lang=bash>
rclone lsf -v "gdrive:"
</syntaxhighlight>
 
Sync: 
 
<syntaxhighlight lang=bash>
rclone sync -v /home/pi/Documents "gdrive:/rclone/"
</syntaxhighlight>


rclone config
List of rclone commands:
  ...
* https://rclone.org/commands/
  service_account_file> ~/Uploader-75551fa3f8b6.json
  ...


== Encrypted ==
== Encryption ==


* https://rclone.org/crypt/
* https://rclone.org/crypt/


Define an encrypted target, the configured remote in the previous paragraph will be used as storage.
Define an encrypted target, in this axample the configured remote gdrive1 is used as storage.


<syntaxhighlight lang=bash>
<syntaxhighlight lang=bash>
rclone config
rclone config
Current remotes:
Name                Type
====                ====
remote              drive
e/n/d/r/c/s/q> n
e/n/d/r/c/s/q> n
name> cryptremote
name> gdrive2
Storage> crypt
Storage> crypt
remote> remote:/rclone/encrypted
remote> gdrive:/rclone2
filename_encryption> 2
filename_encryption> 2
directory_name_encryption> 1
directory_name_encryption> 1
Line 99: Line 126:
</syntaxhighlight>
</syntaxhighlight>


From local to Google Drive, name 'cryptremote'
Sync from local to Google Drive, name 'gdrive2':
<syntaxhighlight lang=bash>
<syntaxhighlight lang=bash>
rclone sync -v ~/test4.txt "cryptremote:/"
rclone sync -v /home/pi/Documents "gdrive2:/"
</syntaxhighlight>
 
List contents (encrypted):
 
<syntaxhighlight lang=bash>
rclone lsf -v "gdrive:/rclone2"
</syntaxhighlight>
 
List contents (decrypted):
 
<syntaxhighlight lang=bash>
rclone lsf -v "gdrive2:"
</syntaxhighlight>
</syntaxhighlight>

Revision as of 20:08, 19 September 2018


Links

Install

wget https://rclone.org/install.sh
chmod +x install.sh
sudo ./install.sh

Configure

Google Drive Client Access

Configure the account :

Configure rclone

Note:

  • This will partly be repeated after the access token has expired.
rclone config
	n                   # New remote
	name> gdrive1       # Any short name
	Storage Type> drive # Google Drive
	client_id> 
	client_secret> 
	scope> drive        # Full access all files, excluding Application Data Folder
	root_folder_id> 
	service_account_file>
	Auto config> y      #your browser will open, login with your gmail account and allow access
	Team drives> n	
	is this OK?> y

Sync from local to Google Drive, name 'gdrive1', folder 'rclone': 

rclone sync -v ~/test4.txt "gdrive1:/rclone/"

Synx from Google Drive to local: 

rclone sync -v  "gdrive1:/rclone/test4.txt" ~/rclone/

Google Drive Service Account

Note:

  • The service account and the main account do not share access to Google Drive files.
  • You can not use the Webinterface to access the service account Google Drive files. However, a service account is still the preferred method if you want to machine-to-machine interaction.
  • To be able to use 'drive-impersonate' you need a "G Suite domain", for this you will need to use a domain that you 'own'. There are various ways to prove you 'own' a domain.

Create a service account :

  • Go to the Developer Console: https://console.developers.google.com/apis/dashboard
  • Login if needed, makes sure you have the corrrect account
  • Create a project and select it
  • Select "ENABLE APIS AND SERVICES" and enable the "Google Drive API"
  • Use the " Create Credentials" button to create a "Service account key". Pick a "Service account name". "Role" can be empty.
  • Select "Key type JSON" and save the file for use with rclone config

Configure rclone

rclone config
	n                   # New remote
	name> gdrive        # Any short name
	Storage Type> drive # Google Drive
	client_id> 
	client_secret> 
	scope> drive        # Full access all files, excluding Application Data Folder
	root_folder_id> 
	service_account_file> /home/pi/Scripts/rclone-4682b69ab73b.json
	Edit advanced config?> n
	Configure this as a team drive?> n
	is this OK?> y

List contents: 

rclone lsf -v "gdrive:"

Sync: 

rclone sync -v /home/pi/Documents "gdrive:/rclone/"

List of rclone commands:

Encryption

Define an encrypted target, in this axample the configured remote gdrive1 is used as storage. 

rclone config
	e/n/d/r/c/s/q> n
	name> gdrive2
	Storage> crypt
	remote> gdrive:/rclone2
	filename_encryption> 2
	directory_name_encryption> 1
	type password y/g/n> y
	seed y/g/n> n
	advanced config y/n> n

Sync from local to Google Drive, name 'gdrive2': 

rclone sync -v /home/pi/Documents "gdrive2:/"

List contents (encrypted): 

rclone lsf -v "gdrive:/rclone2"

List contents (decrypted): 

rclone lsf -v "gdrive2:"
Retrieved from "https://wjv.duckdns.org/mediawiki/index.php?title=Clone_Files_to_Cloud_Storage&oldid=2641"