Sharepoint support?

sevimo · 14 July 2022 06:20

Alright, so I tried to use Sharepoint sites as storage, and so far I failed to achieve success. There was a mention of this some time ago on the forums, but it seems that it didn’t go anywhere. So existing OneDrive backend supports personal and business OneDrive, but not Sharepoint? It probably makes sense to support it as Sharepoint seems to be the end game for business OneDrive as far as Microsoft is concerned (OD is seens as personal file storage, anything team/organization is supposed to go Sharepoint). I also don’t think that you can get more than 5TB on OneDrive, and it is not pooled storage unlike GDrive. Sharepoint sites can be provisioned up to 25TB (that’s how they add storage on unlimited enterprise plans).

I thought that the easiest way to add support for that without touching any of the front end stuff would be supporting Sharepoint links within OD4B. This effectively mounts Sharepoint library(ies) somewhere in regular OD4B tree. This works seamlessly from web interface (e.g. it is just another folder). RClone also supports it mostly seamlessly (it doesn’t show SP links in ls commands, but if you use explicit path it can list and use all the files in the mounted SP library straight from OD backend).

I believe this will need some work to support SP links in OD storage as API is different, but all of this can be done on the OD backend. Don’t think it would be necessary to touch any of the OD authorization/token stuff, perhaps add some permissions specific to Sharepoint.

P.S. I probably can do it running webdav backend over rclone serve, but… you know

gchen · 18 July 2022 17:13

According to this page: Operations using SharePoint REST v2 (Microsoft Graph) endpoints | Microsoft Docs, SharePoint and OneDrive Business should use the same API endpoints. So maybe it will just work if you follow the steps to setup it up as a OneDrive Business backend? I don’t have a Sharepoint subscription to test it out.

sevimo · 18 July 2022 17:39

No, unfortunately it doesn’t work. Even with linking SP libraries into OD tree, API results for regular OD folder nodes and SP library nodes are different, and cannot find any content under SP mounts. Though I believe it should be relatively straightforward to augment existing OD backend.

If you don’t have O365 subscription (I believe any subscription includes some SharePoint), it is possible to get developer trial account for free. You can see the process here: How to Create a SharePoint Online Free Trial for Developers? - SharePoint Diary

gchen · 21 July 2022 02:48

Creating a shortcut to the Sharepoint document library in OneDrive doesn’t work, because when you list an OneDrive directory, shortcuts are never returned.

However, if you really want to make it work, you can change the url in this line:

github.com

gilbertchen/duplicacy/blob/54952cef26563d376e4a273e05973a0038e811bc/src/duplicacy_oneclient.go#L74

    
      
          	client := &OneDriveClient{
          		HTTPClient: http.DefaultClient,
          		TokenFile:  tokenFile,
          		Token:      token,
          		TokenLock:  &sync.Mutex{},
          		IsBusiness: isBusiness,
          	}
          
          
	if isBusiness {
          		client.RefreshTokenURL = "https://duplicacy.com/odb_refresh"
          		client.APIURL = "https://graph.microsoft.com/v1.0/me"
          	} else {
          		client.RefreshTokenURL = "https://duplicacy.com/one_refresh"
          		client.APIURL = "https://api.onedrive.com/v1.0"
          	}
          
          
	client.RefreshToken(false)
          
          
	return client, nil
          }

to

client.APIURL = "https://graph.microsoft.com/v1.0/sites/root"

And then build your own CLI. I’ve tested it successfully without other changes.

sevimo · 21 July 2022 20:48

This doesn’t quite work, for several reasons. Minor thing is using duplicacy.com token will return 403 errors as I am pretty sure some of the permissions are missing. I think by default asks for Files.ReadWrite and offline_access only. With expanding permissions to Files.Read , Files.ReadWrite , Files.Read.All , Files.ReadWrite.All , offline_access , User.Read and Sites.Read.All modified CLI can access some things

The bigger problem is that unlike regular OneDrive of which there can be only one per user, SharePoint is not quite like that. A single tenant may have multiple sites+subsites, each may have multiple document libraries. Document libraries correspond to individual drives and drive API can be used from that point onwards.

So APIURL at https://graph.microsoft.com/v1.0/sites/root means we can only access default SharePoint site, any other sites are not accessible. More so, because OD client hardcodes /drive/root in addition to APIURL, this means we can only access default document library (usually Shared Documents) on default Sharepoint site. Everything else is not accessible.

You can see top level sites like this: https://graph.microsoft.com/v1.0/sites?search=*. Individual document libraries and corresponding drive ids can be accessed by https://graph.microsoft.com/v1.0/sites/root:/sites/SITENAME:/drives. This may or may not be relevant for , depending if you want to see folder structure in the GUI for selecting the storage folder. What is important is that once we know id of the target SharePoint document library, the rest of the operations can be done in exactly the same fashion as with regular OneDrive (exact same API).

So basically, https://graph.microsoft.com/v1.0/me/drives or https://graph.microsoft.com/v1.0/sites/root:/sites/SITENAME:/drives can get drive id for regular OneDrive or a document library in SharePoint (id would be something that starts with b!). Then APIURL can be set like this:

client.APIURL = "https://graph.microsoft.com/v1.0/drives/b!xxxxxxxxxxxxxxxx"

And everything else should work automagically for both SharePoint libraries and OneDrive (for business) with all references to /drive/root replaced with just /root and default APIURLs replaced with https://graph.microsoft.com/v1.0/me/drive and https://api.onedrive.com/v1.0/drive.

The easiest way imho to bring in decent support to SharePoint would be to make the change above (e.g. removing hardcoded /drive/ prefix and moving it to suffix in APIURL) and then allow to override drive ids on per-storage basis, e.g. via enviornment variables again. Something like DUPLICACY_storagename_ODB_DRIVE_ID=b!xxxxxxxxxxxxxx which then will go into APIURL as described above. Ideally all this would be packaged into regular settings/preferences, but environment variable overrides should work with rather minimal impact.

sevimo · 23 July 2022 03:44

And while we’re at it, I submitted another pull request which adds support for SharePoint document libraries: https://github.com/gilbertchen/duplicacy/pull/633. This makes some changes to the ODB backend. I used the same format as for Team Drives in GCD, so odb://DRIVEID@path/to/storage, where DRIVEID is in the format “b!xxxxxxx”. If not using @ in the odb:// storage specification the behaviour should be exactly the same as before, so there are no changes to existing setups.

No to minimal changes needed for the GUI as pathspec can go into the existing field (filtering may need to be adjusted).

This pull request is built on top of custom_odb_creds PR - not because they have dependencies on each other (they don’t), but because they modify the same files so merge would have been non-trivial.

Hopefully it can be merged into the main codebase.