sObject - Access Using S3tools
Publisher: Psychz Networks, April 10,2024- Introduction
- Install S3tools
- Create .s3cfg File
- Create Bucket
- Upload file to Bucket
- Allow Public Access
Introduction
Object storage solutions have become increasingly prevalent due to their scalability and flexibility. Object Storages are widely adopted because they offers seamless integration with various applications and platforms. To interact with sObject which is S3-compatible storage systems, tools like s3cmd provide a convenient command-line interface.
This guide will walk you through setting up and using s3cmd on a Linux server to manage object storage.
Important Note: While creating the sObject, the system will display the following details
Access Key: DRTYHBVJGBGCHBNJ
Secret Access Key: UTGjuhtfgyTHJGHhuuhgghhtYUIJHGfyghhbhjgfHGGH
Endpoint URL: https://sobject.uswest1.psychz.net
You must copy and save the data in a secure place, as you will need this information while creating the bucket. After 60 minutes, this information will disappear from the dashboard.
Install S3tools
You can use tools like s3cmd or any S3-compatible client to interact with Psychz's sObject. Let us begin with installing s3cmd on a Linux server.
Debian/ Ubuntu
Import S3tools signing key:
# wget -O- -q http://s3tools.org/repo/deb-all/stable/s3tools.key | apt-key add -Add the repo to sources.list:
# wget -O/etc/apt/sources.list.d/s3tools.list http://s3tools.org/repo/deb-all/stable/s3tools.listRefresh package cache and install the newest s3cmd:
# apt-get update && apt get install s3cmdCheck s3cmd version
# s3cmd --versionOutput
Before using s3cmd, you must configure your credentials (access and secret access keys) for authentication. Create a configuration file (.s3cfg in your home directory) or use environment variables.
AlmaLinux/ Rocky Linux/ RHEL
First, make sure your system has the necessary tools and Python installed:
# yum install epel-release -y # yum install python3 -yNow, install the s3cmd package
# yum install s3cmd -yTo check the version of s3cmd installed on your system, use the following command:
# s3cmd --versionOutput
Create .s3cfg File
Use your client's command to create a .s3cfg (s3 Configuration File).
# s3cmd --configureBy the running the above command the system will prompt you to input required details in the following sequence
Important: Use the content highlighted in the output below. The rest of the fields are not mandatory and can be left blank as shown below. The Access Key and Secret Key mentioned below are dummy and for representation purposes only. Please use your own Access Key and Secret Key when configuring the .s3cfg file.
Output:
Refer to user manual for detailed description of all options.
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key : ASDFGHRTYU2345ERTYUDFG
Secret Key : 34ERTYGHCVBRTY5678TYUBNDFGHB123DFG
Default Region : us-west-zg-1
Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.
S3 Endpoint : sobject.uswest1.psychz.net
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used if the target S3 system supports dns based buckets.
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: sobject.uswest1.psychz.net/%(bucket)
Encryption password is used to protect your files from reading by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program [/usr/bin/gpg]:
When using secure HTTPS protocol all communication with Amazon S3 servers is protected from 3rd party eavesdropping. This method is slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [Yes]: Yes
On some networks all internet access must go through a HTTP proxy. Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:
Once you complete the above steps, the system will ask you to verify the details
Access Key: ASDFGHRTYU2345ERTYUDFG
Secret Key: 34ERTYGHCVBRTY5678TYUBNDFGHB123DFG
Default Region: us-west-zg-1
S3 Endpoint: sobject.uswest1.psychz.net
DNS-style bucket+hostname:port template for accessing a bucket: sobject.uswest1.psychz.net/%(bucket)
Encryption password:
Path to GPG program: /usr/bin/gpg
Use HTTPS protocol: TrueHTTP Proxy server name:
HTTP Proxy server port: 0
To continue further press Y followed by enter. To modify details you filled in press N and enter. Press Y and enter again to save the settings.
Please wait, attempting to list all buckets...
Success. Your access key and secret key worked fine :-)
Now verifying that encryption works...
Not configured. Never mind.
Save settings? [y/N] y
Configuration saved to '/root/.s3cfg'
Creating a Bucket
We will now use s3cmd command to create a bucket.
# s3cmd mb s3://bucket-nameNote: type s3cmd -h to access all the options and commands
Output
Note: To list all your previously created buckets, you can use s3cmd ls
Upload the File to a Bucket
Use the following command to upload the file to an existing bucket:
# s3cmd put /home/user/documents/myfile.txt s3://bucket-name/
This command will upload myfile.txt from your local system folder (/home/user/documents/) to the S3 bucket named 'bucket-name'.
Now verify the file upload
# s3cmd ls s3://bucket-name/
Allow Public Access to a Specific File
If you want the file to be publicly accessible, you can set permissions using:
# s3cmd setacl s3://bucket-name/myfile.txt --acl-publicThis will apply the public-read ACL to just that object.
This file/folder can now be accessed using the following virtual bucket hostname: <bucket-name>.sobject.uswest1.psychz.net.
Congratulations! You have successfully created your first bucket. You can now start transferring files to your storage space.