Amazon S3
Amazon Simple Storage Service (S3) is a highly scalable object storage service and one of the most popular cloud storage solutions. In Blober, S3 paths use this format:
bucket-name/path/to/file.extCapabilities
Section titled “Capabilities”- ✅ Browse buckets and objects
- ✅ Upload files (multipart for large files)
- ✅ Download files
- ✅ Delete objects
- ✅ Copy/move within S3 and across providers
- ✅ Storage class selection
Prerequisites
Section titled “Prerequisites”- An AWS account (create one)
- An S3 bucket (or permission to list buckets)
- IAM user or role with S3 permissions
- Access key ID and secret access key
Required Credentials
Section titled “Required Credentials”Access Key ID
Section titled “Access Key ID”- Option key:
accessKeyId - Format: 20 uppercase alphanumeric characters
- Example:
AKIAIOSFODNN7EXAMPLE
Secret Access Key
Section titled “Secret Access Key”- Option key:
secretAccessKey - Format: 40 characters
- Example:
wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Optional Settings
Section titled “Optional Settings”Storage Class
Section titled “Storage Class”- Option key:
storageClass - Default:
STANDARD
| Storage Class | Use Case | Retrieval |
|---|---|---|
STANDARD | Frequently accessed data | Immediate |
INTELLIGENT_TIERING | Unknown/changing access patterns | Immediate |
STANDARD_IA | Infrequent access, rapid retrieval | Immediate |
ONEZONE_IA | Infrequent, non-critical data | Immediate |
GLACIER_IR | Archive with instant retrieval | Immediate |
GLACIER | Long-term archive | Minutes to hours |
DEEP_ARCHIVE | Lowest cost archive | 12-48 hours |
Setup (AWS Console)
Section titled “Setup (AWS Console)”1. Create an S3 Bucket
Section titled “1. Create an S3 Bucket”- Go to S3 Console
- Click “Create bucket”
- Enter a bucket name (must be globally unique)
- Choose your preferred AWS Region
- Configure settings as needed
- Click “Create bucket”
2. Create an IAM User
Section titled “2. Create an IAM User”- Go to IAM Console
- Click Users => Create user
- Enter a username (e.g.,
blober-s3-user) - Click Next
- Select Attach policies directly
- For quick setup: Attach
AmazonS3FullAccess- For production: Create a custom policy (see below)
- Click through to Create user
3. Generate Access Keys
Section titled “3. Generate Access Keys”- Click on your new user
- Go to Security credentials tab
- Under Access keys, click Create access key
- Select Application running outside AWS
- Save the Access Key ID and Secret Access Key (shown only once!)
4. Configure in Blober
Section titled “4. Configure in Blober”- In Blober, go to Workflows => New Workflow
- Select Amazon S3 as source or destination
- Enter your Access Key ID and Secret Access Key
- Test by browsing your buckets
Troubleshooting
Section titled “Troubleshooting””Access Denied” or “403 Forbidden”
Section titled “”Access Denied” or “403 Forbidden””- Double-check your Access Key ID and Secret Access Key
- Verify the IAM user has the correct S3 permissions
- Ensure the bucket policy doesn’t block your IAM user
Buckets not showing up
Section titled “Buckets not showing up”- Your IAM user needs
s3:ListAllMyBucketspermission - If using a restricted policy, add
ListBucketfor specific buckets
Slow transfers
Section titled “Slow transfers”- Choose an S3 region geographically close to you
- Large files are automatically uploaded in parts for reliability
- Check your network speed - S3 performance depends on your connection
Best Practices
Section titled “Best Practices”Security
Section titled “Security”- Create a dedicated IAM user for Blober - avoid using root credentials
- Use the minimum permissions needed (don’t use
AmazonS3FullAccessin production) - Rotate access keys periodically
- Enable MFA on your AWS account
Cost Management
Section titled “Cost Management”- Use Intelligent Tiering for data with unpredictable access patterns
- Move old backups to Glacier or Deep Archive for significant savings
- Enable S3 Lifecycle rules to automate tier transitions
- Monitor costs in the AWS Billing Console