Update: 4th January 2025 - Using Profiles in the terminal window
We’ve seen how using AWS CLI improves the speed of the upload process dramatically but what happens if you want to download files.
In this section we will look at the commands required to download a single file or the complete contents of an S3 bucket.
Note: this 4 part tutorial requires you have administration access to an AWS Account
Part 1: Install takes you through Download, Installing and Testing the AWS CLI Application
Part 2: Configure shows you how to create an Identity Access Management (IAM) user, collect configuration data and configure the CLI.
Part 3: Upload shows how to upload your local files to your S3 Bucket/s and highlights some other useful Command line operations.
Part 4; Download shows how to download a single file or a complete folder from your S3 Bucket/s.
Configure the CLI and review the Buckets
1. Open the Terminal app if it’s not already open
2. Enter the command ‘aws configure’ or your AWS_PROFILE if AWS is not already showing it’s configured (figure #1)
Figure #1: Configure CLI
Note: after you enter a Command line always select the ‘return’ key to implement it. (figure #2)
3. Lets start by looking at a list of all the S3 Buckets we have created.
AWS Command: aws s3 ls
4. Enter the command ‘aws s3 ls’ and hit return (figure #3)
Figure #3: Show all buckets
Note: You can see that I have 2 S3 buckets s3360panos and s3threesixtypanos (figure #4)
Figure #4: List of all our S3 buckets
View the contents of our Target Bucket and Folder
AWS Command: aws s3 ls s3://your-bucket-name
1. To see the contents of a particular bucket, enter the command ‘aws s3 ls s3://s3360panos‘ (or the name of your bucket) and hit return (figure #5)
Figure #5: Contents of the bucket s3360panos
2. This command lists the complete contents of the s3360panos bucket (figure #6)
Figure #6: List the bucket contents
AWS Command: aws s3 ls s3://your-bucket-name/path/to/your/folder/
3. To review all of the files in a specific folder ie Glass_Jungle enter the command ‘aws s3 ls s3://s3360panos/Glass_Jungle/‘ and hit return (figure #7)
Figure #7: List the contents of a specific folder
4. This command lists the complete contents of the Glass_Jungle folder (figure #8)
Figure #8: Contents of the folder Glass_Jungle
Download a single file from a folder
AWS Command: aws s3 cp s3://your-bucket-name/path/to/your/file /path/to/local/destination
1. To download a single file (in this example tour.html) enter the command ‘aws s3 ls s3://s3360panos/Glass_Jungle/tour.html /path/to/local/destination’
Note: the first part of the command is the location of the file on AWS, the second part is the destination of the file on your local system (figure #9)
Figure #9: Download a single file - path to the file
Note: a little trick to set the destination
2. Drag the destination folder onto the Terminal window (figure #10)
Figure #10: Drag the destination folder onto the Terminal window
3. When you release the mouse the folders path will be shown (figure #11)
Figure #11: Local path for the download
4. Hit return to start the download (figure #12)
Figure #12: The file tour.html has been downloaded to the local folder
5. The file is now available in the local folder (figure #13)
Figure #13: Single file downloaded to the local folder
Download all the files from a Folder
Note: To download all the files in a Folder we don’t need to identify the individual files just the Folder but add ‘--recursive’ to the end of the command.
Note: Make sure you have 2 ‘-’ before the word recursive, sometimes it may type out as ‘—’ and it will not work.
AWS Command: aws s3 cp s3://your-bucket-name/path/to/your/folder /path/to/local/destination --recursive
1. Enter the command ‘aws s3 cp s3://s3360panos/Glass_Jungle/ /Users/tonyredhead/Desktop/Glass\ Jungle --recursive’ and hit return (figure #14)
Figure #14: Downloading the contents of a folder to a local file
Note: the terminal will list all of the files as it downloads them (figure #15)
Figure #15: Copy command with --exclude “.DS_Store”
5. The files are now available in the local folder (figure #16)
Figure #16: All of the files in the Folder have been downloaded
This brings us to the end of Part 4: Download and the end of the tutorial.
Quite a journey and we have only briefly touched on just a few of the numerous commands that can be used with AWS CLI and S3.
For more check out Gaurav Gupta’s AWS S3 CLI Commands page.
Summary
"Hey there! 👋 If my tutorials have made your tech life a little easier (or a lot less confusing), consider buying me a virtual coffee! ☕️ Retirement means more time to help you, but sadly, less income for snacks. Click the Donation button 🙌 and thanks for the love!"
