Ask HN: What storage protocol for Bring-Your-Own-Storage applications

1 pointsbeingflo3y ago5 comments

I'm currently working on a tiny task management web application for my own needs. To minimize the implementation work I came up with a concept that I think could suit a number of applications like this as well:

- No user signup (no backend at all)

- Local first (via localstorage)

- Optional bring-you-own-storage functionality for sync and backup

The last point would allow the user to provide, say, an S3 endpoint and api key to allow the application to synchronize its state to a remote data store. By supplying the same credentials to the application on another device, one would have cross-device sync effectively for free. To avoid exposing data to the storage provider I'm also implementing client-side encryption.

Now the only remaining question is what storage protocol to use. I've found S3 client libraries to be severely lacking for browser environments (the AWS SDK is a mess and seems to rely on nodejs shims to work, not to mention its gargantuan size).

What are other easy to use and widely available storage protocols? WebDAV? A git repo? Curious to hear what else is out there!

5 comments

ev13y ago

Is this strictly for techies that would possess AWS keys in the first place (see Dropbox HN launch post, just set up your own FTP server, etc) or eventually intended for normal people?

If you are implementing client side encryption, something like "log in with dropbox" oauth, GDrive, etc or similar to just serialize an encrypted snapshot into a known folder would be the most usable for non techies based on previous stuff I've worked on. Asking people to set up restricted IAM roles and keys to write to a single bucket for a todo list is honestly very high up there on the ridiculous list, as much as I would totally do that.

beingfloOP3y ago

It's only intended for techies (and friends thereof I suppose). It's really a tiny project that I've set out to finish in 16 hours as a challenge. So far the progress is great but those S3 libraries are throwing a wrench in my schedule. Because of that tight time budget I think oauth would be out of scope as I've never done anything like it.

Getting into this I was under the impression that uploading a file to S3 was as simple as chucking the api key into the Autorization header of a post and off we go. That's the type of ease of use I'm looking for. Turns out its not quite as trivial with a whole signature process etc.

ev13y ago

I haven't had an issue with it, you're basically just calling a function.

https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/ja...

1 more reply

tgflynn3y ago

If it's the AWS SDK that bothers you can't you just use the S3 REST API (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html) directly ?

j / k navigate · click thread line to collapse

Ask HN: What storage protocol for Bring-Your-Own-Storage applications

1 pointsbeingflo3y ago5 comments

- No user signup (no backend at all)

- Local first (via localstorage)

- Optional bring-you-own-storage functionality for sync and backup

What are other easy to use and widely available storage protocols? WebDAV? A git repo? Curious to hear what else is out there!

5 comments

ev13y ago

Is this strictly for techies that would possess AWS keys in the first place (see Dropbox HN launch post, just set up your own FTP server, etc) or eventually intended for normal people?

beingfloOP3y ago

ev13y ago

I haven't had an issue with it, you're basically just calling a function.

https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/ja...

1 more reply

tgflynn3y ago

If it's the AWS SDK that bothers you can't you just use the S3 REST API (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html) directly ?

j / k navigate · click thread line to collapse