Bug in S3 service?

[Samuel Tai]

According to /usr/local/lib/python3.7/site-packages/middlewared/plugins/s3.py, both the access key and the secret key can only contain alphanumeric characters. However, S3 allows / and + in the secret key, and Cloud credentials accepts these characters for S3. Shouldn't the S3 service match the behavior of the actual AWS S3 service?

 

[vmikalinis]

Having the exact same problem. I made another key and it also has a + in it. Can't get past it.

 

[Zeff]

Same problem.

 

[vmikalinis]

Is there a file we can manually edit to add the information to get it working? I'm not sure where the GUI puts it.

 

[danb35]

Shouldn't the S3 service match the behavior of the actual AWS S3 service?

Why should it? This is its own service, with its own credentials. You wouldn't ever authenticate to your S3 instance with a set of AWS credentials, would you? Nor would you use these credentials to authenticate to AWS. From what I can tell (admittedly not much; TFM is 100% not helpful here), the access key and secret key you enter here act just as a username/password pair for clients of your system to authenticate. So however you generate those credentials (which FreeNAS really should do for you, or at least have the ability to do for you), use those same credentials in your client system(s). Or am I missing something?

 

[danb35]

Having the exact same problem.

What is the problem, exactly?

 

[Zeff]

What is the problem, exactly?

The problem, exactly, is the succinct description provided in the first post by @Samuel Tai

 

[vmikalinis]

I see what's going on here, does the freenas s3 service provide access to your own s3 mimic service hosted on Freenas or does it connect to amazon in some way?

 

[danb35]

The problem, exactly, is the succinct description provided in the first post by @Samuel Tai

Well, no, his first post doesn't in the least describe why he (or you) think this is a problem; it assumes knowledge of the workflow that all three of you are apparently using for whatever reason. But his post on my thread from today clarifies quite a bit as to why it's a problem, though it leads to the question of why on earth he (and presumably you) are doing what you're doing. So you and he are using a third-party service to generate access credentials for a service running on your FreeNAS box, and those credentials don't work. And this is a problem with FreeNAS? Granted the docs could be a lot better on this, but I really don't understand how you (any of you) came up with a workflow of "use AWS to generate credentials for something that will never touch AWS". Why would you want AWS (or any other third party) to have access credentials for a service they aren't running or accessing?

It seems that a much simpler workflow would be to use any method you want (a password manager is nice and easy, or you can mess around with incantations involving openssl or /dev/urandom) to generate alphanumeric strings of the desired length. Copy and paste them into the service configuration. All done.

I'm definitely open to hearing why this isn't a good idea--I'm a complete n00b with S3--but this seems simpler, easier (those aren't the same), and more secure than having AWS generate your credentials for you.

does the FreeNAS s3 service provide access to your own s3 mimic service hosted on FreeNAS

Yes.

does it connect to amazon in some way?

No. If you want to deal with outside S3-compatible storage, you'd use the Cloud Sync feature.

Now, Minio (which is what FreeNAS uses to provide this service) does have the ability to connect to remote S3-compatible storage, so if you want to mess around with the CLI, that's another way you can use outside S3-compatible storage. But that's all at the CLI, so the middleware's limitations aren't relevant.

 

[Samuel Tai]

In my case, it was strictly curiosity, but I can see a workflow where the desire is to mimic S3 as closely as possible for a proof of concept. However, the middleware input validator for the S3 service restricts both the access key and the secret key to alphanumeric characters only, which is more restrictive than the actual AWS service.

 

[danb35]

the middleware input validator for the S3 service restricts both the access key and the secret key to alphanumeric characters only

...which is kind of interesting, because Minio has no such restriction. IMO, this isn't a "bug" so much as an enhancement request, but no reason not to file a ticket if you feel otherwise. But I think the solution to the problem noted in this thread is "don't use AWS to generate your authentication keys."