Remove unused code that inflated our session size#2412
Merged
Conversation
**Why**: This is the root cause for the KMS errors we experienced in production yesterday. We were storing the OIDC request params in the session for no reason. The more parameters an agency included in their initial OIDC request, the bigger the session size. In the case of USAJOBS, storing this data in the session increased the length of the string that was sent to KMS after 2FA from 2948 to 4128. There were exactly 2 SAML requests during the outage, so they played an insignificant role. This most likely affected every single USAJOBS session, which accounted for 74% of requests during that time. I don't believe TTP was affected. The rest of the requests (4% of total requests) came from 3 other SPs that I haven't looked into yet. Note that this only solves the problem for LOA1 OIDC requests (as they are currently made). SAML requests remain much larger than 4096 bytes. In general, this KMS limit problem can be solved by one or more of these solutions, which we have considered in the past: - Storing the SP requests in the DB instead of the session - Only encrypting the info that needs to be encrypted, as opposed to the entire session
stevegsa
approved these changes
Aug 3, 2018
Contributor
stevegsa
left a comment
There was a problem hiding this comment.
I don't see any references. LGTM
Contributor
|
I'm always wary that someone might be assembling a token on the fly with a to_sym... |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why: This is the root cause for the KMS errors we experienced in
production yesterday. We were storing the OIDC request params in the
session for no reason. The more parameters an agency included in their
initial OIDC request, the bigger the session size. In the case of
USAJOBS, storing this data in the session increased the length of the
string that was sent to KMS after 2FA from 2948 to 4128.
There were exactly 2 SAML requests during the outage, so they played
an insignificant role. This most likely affected every single USAJOBS
session, which accounted for 74% of requests during that time. I don't
believe TTP was affected. The rest of the requests (4% of total
requests) came from 3 other SPs that I haven't looked into yet.
Note that this only solves the problem for LOA1 OIDC requests (as they
are currently made). SAML requests remain much larger than 4096 bytes.
In general, this KMS limit problem can be solved by one or more of these
solutions, which we have considered in the past:
the entire session
Hi! Before submitting your PR for review, and/or before merging it, please
go through the checklists below. These represent the more critical elements
of our code quality guidelines. The rest of the list can be found in
CONTRIBUTING.md
Controllers
authenticated, make sure to add
before_action :confirm_two_factor_authenticatedas the first callback.
Database
Unsafe migrations are implemented over several PRs and over several
deploys to avoid production errors. The strong_migrations gem
will warn you about unsafe migrations and has great step-by-step instructions
for various scenarios.
Indexes were added if necessary. This article provides a good overview
of indexes in Rails.
Verified that the changes don't affect other apps (such as the dashboard)
When relevant, a rake task is created to populate the necessary DB columns
in the various environments right before deploying, taking into account the users
who might not have interacted with this column yet (such as users who have not
set a password yet)
Migrations against existing tables have been tested against a copy of the
production database. See LG-228 Make migrations safer and more resilient #2127 for an example when a migration caused deployment
issues. In that case, all the migration did was add a new column and an index to
the Users table, which might seem innocuous.
Encryption
Routes
state or result in destructive behavior).
Session
user_sessionhelperinstead of the
sessionhelper so the data does not persist beyond the user'ssession.
Testing
and invalid inputs.