Skip to content

Updates for javabuilder-demo to go live #393

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
May 24, 2023
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .ruby-version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.7.7
2.7.8
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change was needed for the ci/cd pipeline to build correctly for a dev instance, so I think it will be needed for prod as well. We've done similar changes in the past to keep up the the AWS version of ruby.

4 changes: 3 additions & 1 deletion cicd/3-app/javabuilder/config/dev.config.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
"ReservedConcurrentExecutions": "3",
"LimitPerHour": "50",
"LimitPerDay": "150",
"SilenceAlerts": "true"
"SilenceAlerts": "false",
"HighConcurrentExecutionsTopic": "javabuilder-low-urgency",
"HighConcurrentExecutionsAlarmThreshold": 40
},
"Tags" : {
"EnvType" : "development"
Expand Down
9 changes: 6 additions & 3 deletions cicd/3-app/javabuilder/config/production-demo.config.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,14 @@
"BaseDomainName": "code.org",
"SubdomainName": "javabuilder-demo",
"BaseDomainNameHostedZonedID": "Z2LCOI49SCXUGU",
"ProvisionedConcurrentExecutions": "1",
"ReservedConcurrentExecutions": "5",
"ProvisionedConcurrentExecutions": "5",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with 5 here somewhat arbitrarily, lmk if we have another number we want to go with.

"ReservedConcurrentExecutions": "50",
"LimitPerHour": "-1",
"LimitPerDay": "50",
"SilenceAlerts": "false"
"SilenceAlerts": "false",
"HighConcurrentExecutionsTopic": "javabuilder-low-urgency",
"HighConcurrentExecutionsAlarmThreshold": 45

},
"Tags": {
"EnvType": "production"
Expand Down
4 changes: 3 additions & 1 deletion cicd/3-app/javabuilder/config/production.config.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@
"LimitPerHour": "1000",
"LimitPerDay": "-1",
"SilenceAlerts": "false",
"TeacherLimitPerHour": "25000"
"TeacherLimitPerHour": "25000",
"HighConcurrentExecutionsTopic": "CDO-Urgent",
"HighConcurrentExecutionsAlarmThreshold": 400
},
"Tags" : {
"EnvType" : "production"
Expand Down
4 changes: 3 additions & 1 deletion cicd/3-app/javabuilder/config/test.config.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
"ReservedConcurrentExecutions": "25",
"LimitPerHour": "50",
"LimitPerDay": "150",
"SilenceAlerts": "false"
"SilenceAlerts": "false",
"HighConcurrentExecutionsTopic": "javabuilder-low-urgency",
"HighConcurrentExecutionsAlarmThreshold": "20"
},
"Tags" : {
"EnvType" : "test"
Expand Down
22 changes: 16 additions & 6 deletions cicd/3-app/javabuilder/template.yml.erb
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,14 @@ Parameters:
AllowedValues: [true, false]
Description: If alerts should be silenced on this instance
Default: false
HighConcurrentExecutionsTopic:
Type: String
Description: The name of the SNS topic to publish to for a high concurrent executions alarm.
Default: CDO-Urgent
HighConcurrentExecutionsAlarmThreshold:
Type: Number
Description: The threshold for the high concurrent executions alarm.
Default: 400
<%
JAVALAB_APP_TYPES = %w(
Theater
Expand All @@ -63,6 +71,7 @@ Globals:
Tracing: Active
Conditions:
IsDevCondition: !Equals [!Ref BaseDomainName, "dev-code.org"]
IsDemoCondition: !Equals [!Ref SubdomainName, "javabuilder-demo"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you continued the existing pattern here, but we should try to keep this template environment-agnostic, and I would much rather refactor the design to allow us to eliminate the IsDevCondition rather than add a new IsEnvironmentCondition

You're using IsDemoCondition in two places, so I'll make comments directly there.

Copy link
Contributor

@cat5inthecradle cat5inthecradle May 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR to start eliminating IsDevCondition, definitely a non-school-year project ;)
#394

SilenceAlertsCondition: !Or [Condition: IsDevCondition, !Equals [!Ref SilenceAlerts, "true"]]
Resources:
# Note: We can't update the name of a DomainName resource once it has been created because the
Expand Down Expand Up @@ -857,18 +866,19 @@ Resources:
Properties:
AlarmName: !Sub "${SubdomainName}_<%=name.downcase%>_high_concurrent_executions"
AlarmDescription: !Sub |
Alarm if javabuilder usage exceeds 400 concurrent
executions for 10 minutes. Occasional spikes are expected, but
long-running high usage is an indication of an attack. Page the student learning
team for further investigation. See this doc for investigation steps
Alarm if javabuilder usage has high concurrent executions for 10 minutes.
Occasional spikes are expected, but long-running high usage is an indication
of an attack. If this is occuring on the demo environment, this is a non-urgent
issue as we expect occasional periods of high usage. If it is on production,
page the student learning team for further investigation. See this doc for investigation steps
https://docs.google.com/document/d/1bHvV6pvUcwxgZpw0YWBmxFggQL5KqYx9zwolwkZhjU8/edit#bookmark=id.xs1gcuxrw6ze
ActionsEnabled: true
AlarmActions:
- !If [SilenceAlertsCondition, !Ref AWS::NoValue, !Sub "arn:aws:sns:${AWS::Region}:${AWS::AccountId}:CDO-Urgent"]
- !If [SilenceAlertsCondition, !Ref AWS::NoValue, !Sub "arn:aws:sns:${AWS::Region}:${AWS::AccountId}:{HighConcurrentExecutionsTopic}"]]
EvaluationPeriods: 10
DatapointsToAlarm: 10
Period: 60
Threshold: 400
Threshold: !Ref HighConcurrentExecutionsAlarmThreshold
ComparisonOperator: GreaterThanThreshold
TreatMissingData: notBreaching
MetricName: ConcurrentExecutions
Expand Down