Merged
Conversation
Add new table to query Kafka topics on MSK clusters via the ListTopics and DescribeTopic APIs. Upgrade kafka SDK from v1.31.2 to v1.49.1 to access the topic management APIs.
When Steampipe fans out a WHERE cluster_arn = ? query to all configured connections, each connection attempts kafka-cluster:Connect on the cluster regardless of account ownership. This causes 403 AccessDeniedException errors from connections whose account does not own the cluster, because kafka-cluster:* requires a resource-based policy for cross-account access. Parse the owning account from the cluster ARN and skip early if the current connection's account does not match. This mirrors how other resource types naturally avoid cross-account errors by only returning resources they own.
…chemas Replace required cluster_arn key column with ParentHydrate pattern, enumerating topics across all provisioned MSK clusters automatically. Removes the cross-account STS identity workaround.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
aws_msk_topictable to query Kafka topics on MSK clustersaws-sdk-go-v2/service/kafkafrom v1.31.2 to v1.49.1 to access topic management APIsTest plan
make installsteampipe query ".inspect aws_msk_topic"to verify table registrationselect topic_name, partition_count from aws_msk_topic where cluster_arn = '<arn>'select * from aws_msk_topic where cluster_arn = '<arn>' and topic_name = '<name>'