401 Unauthorized Error with S3 get_object on EC2 Instance

[niharikaa0418]

Describe the bug

I am encountering a 401 Unauthorized error when attempting to download an S3 object from an EC2 instance using the aws-sdk-ruby gem. I have verified that the IAM role attached to the instance has the necessary s3:GetObject permission.

*** LOCAL GEMS ***

aws-sdk-core (3.220.1)
aws-sdk-kms (1.99.0)
aws-sdk-s3 (1.182.0)
aws-sdk-secretsmanager (1.113.0)

ruby-version: 3.2.2
chef-version:18.4.12 which is using embedded ruby version (3.1.2)
OS: Amazon Linux 2023

RestClient::Unauthorized: 401 Unauthorized
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/rest-client-2.1.0/lib/restclient/abstract_response.rb:249:in exception_with_response' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/rest-client-2.1.0/lib/restclient/abstract_response.rb:129:in return!'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/rest-client-2.1.0/lib/restclient/request.rb:836:in process_result' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/rest-client-2.1.0/lib/restclient/request.rb:743:in block in transmit'
/opt/chef/embedded/lib/ruby/3.1.0/net/http.rb:966:in start' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/rest-client-2.1.0/lib/restclient/request.rb:727:in transmit'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/rest-client-2.1.0/lib/restclient/request.rb:163:in execute' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/rest-client-2.1.0/lib/restclient/request.rb:63:in execute'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/rest-client-2.1.0/lib/restclient.rb:66:in get' /var/cache/chef/cookbooks/s3_file/providers/default.rb:24:in block in class_from_file'
(eval):2:in block in action_create' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/provider.rb:304:in instance_eval'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/provider.rb:304:in compile_and_converge_action' (eval):2:in action_create'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/provider.rb:245:in run_action' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource.rb:601:in block in run_action'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource.rb:628:in with_umask' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource.rb:600:in run_action'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/runner.rb:74:in run_action' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/runner.rb:108:in block in run_all_actions'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/runner.rb:108:in each' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/runner.rb:108:in run_all_actions'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/runner.rb:132:in block in converge' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource_collection/resource_list.rb:96:in block in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource_collection/stepable_iterator.rb:114:in call_iterator_block' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource_collection/stepable_iterator.rb:85:in step'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource_collection/stepable_iterator.rb:103:in iterate' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource_collection/stepable_iterator.rb:54:in each_with_index'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/resource_collection/resource_list.rb:94:in execute_each_resource' /opt/chef/embedded/lib/ruby/3.1.0/forwardable.rb:238:in execute_each_resource'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/runner.rb:130:in converge' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/client.rb:869:in block in converge'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/client.rb:864:in catch' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/client.rb:864:in converge'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/client.rb:888:in converge_and_save' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/client.rb:298:in run'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/application.rb:305:in run_with_graceful_exit_option' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/application.rb:281:in block in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/local_mode.rb:42:in with_server_connectivity' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/application.rb:264:in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/application/base.rb:354:in run_application' /opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-18.4.12/lib/chef/application.rb:67:in run'
/opt/chef/embedded/lib/ruby/gems/3.1.0/gems/chef-bin-18.4.12/bin/chef-client:25:in <top (required)>' /usr/bin/chef-client:183:in load'

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The S3 object should be downloaded successfully.

Current Behavior

s3_file "/etc/chef/#{node['encrypted_data_bag_key_file']}" do
remote_path "#{node['encrypted_data_bag_key_path']}/#{node['encrypted_data_bag_key_file']}"
bucket "#{node['packages']['bucket']}"
end

This code fails with 401 Unauthorized error

Reproduction Steps

s3_file "/etc/chef/#{node['encrypted_data_bag_key_file']}" do
remote_path "#{node['encrypted_data_bag_key_path']}/#{node['encrypted_data_bag_key_file']}"
bucket "#{node['packages']['bucket']}"
end

Possible Solution

No response

Additional Information/Context

No response

Gem name ('aws-sdk', 'aws-sdk-resources' or service gems like 'aws-sdk-s3') and its version

aws-sdk-s3 (1.182.0)

Environment details (Version of Ruby, OS environment)

ruby-version: 3.2.2, OS: Amazon Linux 2023

[mullermp]

The stack trace does not use aws-sdk-s3 at all. The error is a generic s3 service error. Ensure the credentials you are using are the ones you expect.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.