Skip to content

Commit 25a5773

Browse files
kmcquadekmcquade
authored
Fix elasticache issue and bump version (#224)
* Fix elasticache issue and bump version * Fix tests due to AWS changes
1 parent c27ad7f commit 25a5773

File tree

103 files changed

+4036
-1491
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

103 files changed

+4036
-1491
lines changed

‎CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
11
# Changelog
22

3+
## 0.8.7 (2020-09-06)
4+
* Fixes elasticache query issue #223
5+
36
## 0.8.6 (2020-09-05)
47
* Fix issue with ARN matching (#215)
58
* Fixed issue where query command was not leveraging local database (#220)

‎docs/contributing/iam-database.md

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -67,8 +67,6 @@ Keys pages per-service to the `policy_sentry/shared/data/docs` folder.
6767

6868
- The HTML files will be stored in
6969
`policy_sentry/shared/data/docs/list_*.html`
70-
- It also add a file titled `policy_sentry/shared/data/links.yml` as
71-
well.
7270
- It also builds a JSON file that serves as the IAM data source to include as part of the PyPi
7371
package.
7472

@@ -82,8 +80,7 @@ python3 ./utils/download_docs.py
8280
```
8381

8482
This downloads the Actions, Resources, and Condition Keys pages
85-
per-service to the `policy_sentry/shared/data/docs` folder. It also add
86-
a file titled `policy_sentry/shared/data/links.yml` as well.
83+
per-service to the `policy_sentry/shared/data/docs` folder.
8784

8885
When a user runs `policy_sentry initialize`, these files are copied over
8986
to the config folder (`~/.policy_sentry/`).

‎examples/yml/crud-with-override.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ permissions-management:
44
- arn:aws:s3:::example-org-s3-access-logs
55
wildcard-only:
66
single-actions:
7-
- secretsmanager:CreateSecret
7+
- ram:getresourcepolicies
88
skip-resource-constraints:
99
- ssm:GetParameter
1010
- ssm:GetParameters

‎examples/yml/crud-with-wildcard-service-level.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ wildcard-only:
66
single-actions:
77
- ram:enablesharingwithawsorganization
88
- ram:getresourcepolicies
9-
- secretsmanager:createsecret
109
service-read:
1110
- ecr # GetAuthorizationToken
1211
- s3 # GetAccessPoint, GetAccountPublicAccessBlock, ListAccessPoints

‎examples/yml/crud-with-wildcard.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,5 +6,4 @@ wildcard-only:
66
single-actions:
77
- ram:enablesharingwithawsorganization
88
- ram:getresourcepolicies
9-
- secretsmanager:createsecret
109
- secretsmanager:putsecretvalue # attempting bypass

‎policy_sentry/bin/cli.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"""
33
Policy Sentry is a tool for generating least-privilege IAM Policies.
44
"""
5-
__version__ = "0.8.6"
5+
__version__ = "0.8.7"
66
import click
77
from policy_sentry import command
88

‎policy_sentry/command/initialize.py

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -93,13 +93,11 @@ def initialize(access_level_overrides_file=None, fetch=False, build=False):
9393
shutil.copy(BUNDLED_DATASTORE_FILE_PATH, database_path)
9494

9595
# --fetch: wget the AWS IAM Actions, Resources and Condition Keys pages and store them locally.
96-
# if --build and --fetch are both supplied, just do --fetch
9796
if fetch:
9897
# `wget` the html docs to the local directory
9998
update_html_docs_directory(LOCAL_HTML_DIRECTORY_PATH)
100-
create_database(CONFIG_DIRECTORY, overrides_file)
10199

102-
# initialize --build
100+
# --build
103101
if build or access_level_overrides_file or fetch:
104102
create_database(CONFIG_DIRECTORY, overrides_file)
105103
print("Created the database!")
@@ -135,7 +133,6 @@ def create_policy_sentry_config_directory():
135133
def create_html_docs_directory():
136134
"""
137135
Copies the HTML files from the pip package over to its own folder in the CONFIG_DIRECTORY.
138-
Also copies over the links.yml file, which is a mapping of services and relevant HTML links in the AWS docs.
139136
Essentially:
140137
mkdir -p ~/.policy_sentry/data/docs
141138
cp -r $MODULE_DIR/policy_sentry/shared/data/docs ~/.policy_sentry/data/docs

‎policy_sentry/command/query.py

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
"""
22
Allow users to use specific pre-compiled queries against the action, arn, and condition tables from command line.
33
"""
4+
import os
45
import json
56
import logging
67
import click
@@ -25,9 +26,11 @@
2526
get_condition_keys_for_service,
2627
get_condition_key_details,
2728
)
29+
from policy_sentry.shared.constants import DATASTORE_FILE_PATH, LOCAL_DATASTORE_FILE_PATH
2830

2931
logger = logging.getLogger(__name__)
3032
click_log.basic_config(logger)
33+
iam_definition_path = DATASTORE_FILE_PATH
3134

3235

3336
@click.group()
@@ -84,7 +87,13 @@ def action_table(name, service, access_level, condition, wildcard_only, fmt):
8487
def query_action_table(
8588
name, service, access_level, condition, wildcard_only, fmt="json"
8689
):
87-
"""Query the Action Table from the Policy Sentry database. Use this one when leveraging Policy Sentry as a library."""
90+
"""Query the Action Table from the Policy Sentry database.
91+
Use this one when leveraging Policy Sentry as a library."""
92+
if os.path.exists(LOCAL_DATASTORE_FILE_PATH):
93+
logger.info(f"Using the Local IAM definition: {LOCAL_DATASTORE_FILE_PATH}. To leverage the bundled definition instead, remove the folder $HOME/.policy_sentry/")
94+
else:
95+
# Otherwise, leverage the datastore inside the python package
96+
logger.debug("Leveraging the bundled IAM Definition.")
8897
# Actions on all services
8998
if service == "all":
9099
all_services = get_all_service_prefixes()
@@ -191,6 +200,11 @@ def arn_table(name, service, list_arn_types, fmt="json"):
191200

192201
def query_arn_table(name, service, list_arn_types, fmt):
193202
"""Query the ARN Table from the Policy Sentry database. Use this one when leveraging Policy Sentry as a library."""
203+
if os.path.exists(LOCAL_DATASTORE_FILE_PATH):
204+
logger.info(f"Using the Local IAM definition: {LOCAL_DATASTORE_FILE_PATH}. To leverage the bundled definition instead, remove the folder $HOME/.policy_sentry/")
205+
else:
206+
# Otherwise, leverage the datastore inside the python package
207+
logger.debug("Leveraging the bundled IAM Definition.")
194208
# Get a list of all RAW ARN formats available through the service.
195209
if name is None and list_arn_types is False:
196210
output = get_raw_arns_for_service(service)
@@ -236,7 +250,13 @@ def condition_table(name, service, fmt):
236250

237251

238252
def query_condition_table(name, service, fmt="json"):
239-
"""Query the condition table from the Policy Sentry database. Use this one when leveraging Policy Sentry as a library."""
253+
"""Query the condition table from the Policy Sentry database.
254+
Use this one when leveraging Policy Sentry as a library."""
255+
if os.path.exists(LOCAL_DATASTORE_FILE_PATH):
256+
logger.info(f"Using the Local IAM definition: {LOCAL_DATASTORE_FILE_PATH}. To leverage the bundled definition instead, remove the folder $HOME/.policy_sentry/")
257+
else:
258+
# Otherwise, leverage the datastore inside the python package
259+
logger.debug("Leveraging the bundled IAM Definition.")
240260
# Get a list of all condition keys available to the service
241261
if name is None:
242262
output = get_condition_keys_for_service(service)

‎policy_sentry/shared/awsdocs.py

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -151,11 +151,14 @@ def create_database(destination_directory, access_level_overrides_file):
151151
schema = []
152152

153153
# for filename in ['list_amazonathena.partial.html']:
154-
for filename in [
155-
f
156-
for f in os.listdir(BUNDLED_HTML_DIRECTORY_PATH)
157-
if os.path.isfile(os.path.join(BUNDLED_HTML_DIRECTORY_PATH, f))
158-
]:
154+
file_list = []
155+
for filename in os.listdir(BUNDLED_HTML_DIRECTORY_PATH):
156+
if os.path.isfile(os.path.join(BUNDLED_HTML_DIRECTORY_PATH, filename)):
157+
if filename not in file_list:
158+
file_list.append(filename)
159+
160+
file_list.sort()
161+
for filename in file_list:
159162
if not filename.startswith("list_"):
160163
continue
161164

@@ -316,9 +319,8 @@ def create_database(destination_directory, access_level_overrides_file):
316319

317320
# Get resource table
318321
for table in tables:
319-
if "<th> Resource Types </th>" not in [
320-
chomp(str(x)) for x in table.find_all("th")
321-
]:
322+
header_cells = [chomp(str(x)) for x in table.find_all("th")]
323+
if "<th> Resource Types </th>" not in header_cells:
322324
continue
323325

324326
rows = table.find_all("tr")

‎policy_sentry/shared/constants.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,8 @@
3030
# Check for the existence of the local datastore first.
3131
if os.path.exists(LOCAL_DATASTORE_FILE_PATH):
3232
# If it exists, leverage that datastore instead of the one bundled with the python package
33-
logger.info(f"The IAM definition at {LOCAL_DATASTORE_FILE_PATH} exists. "
34-
f"Leveraging that IAM definition instead of the definition bundled with the python package. "
35-
f"To leverage the bundled definition, remove the folder $HOME/.policy_sentry/")
33+
logger.info(f"Leveraging the local IAM definition at the path: {LOCAL_DATASTORE_FILE_PATH} "
34+
f"To leverage the bundled definition instead, remove the folder $HOME/.policy_sentry/")
3635
DATASTORE_FILE_PATH = LOCAL_DATASTORE_FILE_PATH
3736
else:
3837
# Otherwise, leverage the datastore inside the python package

‎policy_sentry/shared/data/docs/list_amazonathena.html

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,7 @@ <h2 id="amazonathena-actions-as-permissions">
216216
</p>
217217
<div class="table-container">
218218
<div class="table-contents">
219-
<table id="w468aac34c14c29c75c11b9">
219+
<table id="w468aac33c14c29c75c11b9">
220220
<thead>
221221
<tr>
222222
<th>
@@ -1116,7 +1116,7 @@ <h2 id="amazonathena-resources-for-iam-policies">
11161116
</p>
11171117
<div class="table-container">
11181118
<div class="table-contents">
1119-
<table id="w468aac34c14c29c75c13b5">
1119+
<table id="w468aac33c14c29c75c13b5">
11201120
<thead>
11211121
<tr>
11221122
<th>
@@ -1236,7 +1236,7 @@ <h2 id="amazonathena-policy-keys">
12361236
</p>
12371237
<div class="table-container">
12381238
<div class="table-contents">
1239-
<table id="w468aac34c14c29c75c15b7">
1239+
<table id="w468aac33c14c29c75c15b7">
12401240
<thead>
12411241
<tr>
12421242
<th>

‎policy_sentry/shared/data/docs/list_amazonchime.html

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -784,6 +784,38 @@ <h2 id="amazonchime-actions-as-permissions">
784784
<td>
785785
</td>
786786
</tr>
787+
<tr>
788+
<td>
789+
<a id="amazonchime-CreateMeetingWithAttendees">
790+
</a>
791+
<a href="https://docs.aws.amazon.com/chime/latest/APIReference/API_CreateMeetingWithAttendees.html">
792+
CreateMeetingWithAttendees
793+
</a>
794+
</td>
795+
<td>
796+
Grants permission to create a new Amazon Chime SDK meeting in the specified media
797+
Region, with a set of attendees
798+
</td>
799+
<td>
800+
Write
801+
</td>
802+
<td>
803+
</td>
804+
<td>
805+
<p>
806+
<a href="#amazonchime-aws_RequestTag___TagKey_">
807+
aws:RequestTag/${TagKey}
808+
</a>
809+
</p>
810+
<p>
811+
<a href="#amazonchime-aws_TagKeys">
812+
aws:TagKeys
813+
</a>
814+
</p>
815+
</td>
816+
<td>
817+
</td>
818+
</tr>
787819
<tr>
788820
<td>
789821
<a id="amazonchime-CreatePhoneNumberOrder">

‎policy_sentry/shared/data/docs/list_amazoncloudwatch.html

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,7 @@ <h2 id="amazoncloudwatch-actions-as-permissions">
216216
</p>
217217
<div class="table-container">
218218
<div class="table-contents">
219-
<table id="w468aac34c14c29d158c11b9">
219+
<table id="w468aac33c14c29d158c11b9">
220220
<thead>
221221
<tr>
222222
<th>
@@ -1216,7 +1216,7 @@ <h2 id="amazoncloudwatch-resources-for-iam-policies">
12161216
</p>
12171217
<div class="table-container">
12181218
<div class="table-contents">
1219-
<table id="w468aac34c14c29d158c13b5">
1219+
<table id="w468aac33c14c29d158c13b5">
12201220
<thead>
12211221
<tr>
12221222
<th>
@@ -1364,7 +1364,7 @@ <h2 id="amazoncloudwatch-policy-keys">
13641364
</p>
13651365
<div class="table-container">
13661366
<div class="table-contents">
1367-
<table id="w468aac34c14c29d158c15b7">
1367+
<table id="w468aac33c14c29d158c15b7">
13681368
<thead>
13691369
<tr>
13701370
<th>

‎policy_sentry/shared/data/docs/list_amazoncloudwatchsynthetics.html

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,7 @@ <h2 id="amazoncloudwatchsynthetics-actions-as-permissions">
216216
</p>
217217
<div class="table-container">
218218
<div class="table-contents">
219-
<table id="w468aac34c14c29d170c11b9">
219+
<table id="w468aac33c14c29d170c11b9">
220220
<thead>
221221
<tr>
222222
<th>
@@ -537,7 +537,7 @@ <h2 id="amazoncloudwatchsynthetics-resources-for-iam-policies">
537537
</p>
538538
<div class="table-container">
539539
<div class="table-contents">
540-
<table id="w468aac34c14c29d170c13b5">
540+
<table id="w468aac33c14c29d170c13b5">
541541
<thead>
542542
<tr>
543543
<th>

‎policy_sentry/shared/data/docs/list_amazoncodeguru.html

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,7 @@ <h2 id="amazoncodeguru-actions-as-permissions">
216216
</p>
217217
<div class="table-container">
218218
<div class="table-contents">
219-
<table id="w468aac34c14c29d194c11b9">
219+
<table id="w468aac33c14c29d194c11b9">
220220
<thead>
221221
<tr>
222222
<th>

‎policy_sentry/shared/data/docs/list_amazoncodegurureviewer.html

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,7 @@ <h2 id="amazoncodegurureviewer-actions-as-permissions">
216216
</p>
217217
<div class="table-container">
218218
<div class="table-contents">
219-
<table id="w468aac34c14c29d202c11b9">
219+
<table id="w468aac33c14c29d202c11b9">
220220
<thead>
221221
<tr>
222222
<th>
@@ -576,7 +576,7 @@ <h2 id="amazoncodegurureviewer-resources-for-iam-policies">
576576
</p>
577577
<div class="table-container">
578578
<div class="table-contents">
579-
<table id="w468aac34c14c29d202c13b5">
579+
<table id="w468aac33c14c29d202c13b5">
580580
<thead>
581581
<tr>
582582
<th>
@@ -708,7 +708,7 @@ <h2 id="amazoncodegurureviewer-policy-keys">
708708
</p>
709709
<div class="table-container">
710710
<div class="table-contents">
711-
<table id="w468aac34c14c29d202c15b7">
711+
<table id="w468aac33c14c29d202c15b7">
712712
<thead>
713713
<tr>
714714
<th>

‎policy_sentry/shared/data/docs/list_amazoncomprehend.html

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,7 @@ <h2 id="amazoncomprehend-actions-as-permissions">
216216
</p>
217217
<div class="table-container">
218218
<div class="table-contents">
219-
<table id="w468aac34c14c29d234c11b9">
219+
<table id="w468aac33c14c29d234c11b9">
220220
<thead>
221221
<tr>
222222
<th>
@@ -1670,7 +1670,7 @@ <h2 id="amazoncomprehend-resources-for-iam-policies">
16701670
</p>
16711671
<div class="table-container">
16721672
<div class="table-contents">
1673-
<table id="w468aac34c14c29d234c13b5">
1673+
<table id="w468aac33c14c29d234c13b5">
16741674
<thead>
16751675
<tr>
16761676
<th>
@@ -1856,7 +1856,7 @@ <h2 id="amazoncomprehend-policy-keys">
18561856
</p>
18571857
<div class="table-container">
18581858
<div class="table-contents">
1859-
<table id="w468aac34c14c29d234c15b7">
1859+
<table id="w468aac33c14c29d234c15b7">
18601860
<thead>
18611861
<tr>
18621862
<th>

‎policy_sentry/shared/data/docs/list_amazondynamodb.html

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -216,7 +216,7 @@ <h2 id="amazondynamodb-actions-as-permissions">
216216
</p>
217217
<div class="table-container">
218218
<div class="table-contents">
219-
<table id="w468aac34c14c29d318c11b9">
219+
<table id="w468aac33c14c29d318c11b9">
220220
<thead>
221221
<tr>
222222
<th>
@@ -1850,7 +1850,7 @@ <h2 id="amazondynamodb-resources-for-iam-policies">
18501850
</p>
18511851
<div class="table-container">
18521852
<div class="table-contents">
1853-
<table id="w468aac34c14c29d318c13b5">
1853+
<table id="w468aac33c14c29d318c13b5">
18541854
<thead>
18551855
<tr>
18561856
<th>
@@ -2087,7 +2087,7 @@ <h2 id="amazondynamodb-policy-keys">
20872087
</div>
20882088
<div class="table-container">
20892089
<div class="table-contents">
2090-
<table id="w468aac34c14c29d318c15b9">
2090+
<table id="w468aac33c14c29d318c15b9">
20912091
<thead>
20922092
<tr>
20932093
<th>

0 commit comments

Comments
 (0)