Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(dev/core#4462) Afform - Add support for page-level authentication links #30585

Merged
merged 14 commits into from
Sep 20, 2024

Conversation

totten
Copy link
Member

@totten totten commented Jul 1, 2024

Overview

When you create a custom form and enable email-token support, it will generate authenticated hyperlinks with session-level authentication (clicking the link logs you into a session). However, page-level authentication (granting access to one specific page) is also useful.

For more discussion about the differences, see: https://lab.civicrm.org/dev/core/-/issues/4462. (That issue is focused more on email-driven workflows. However, the same mechanism can also be incorporated into cross-site workflows -- where a remote site wants to grant limited access to display a specific form for a specific user.)

Before

  • Sending an email with a token like {afform.myFormUrl} will always generate a URL with a session-level authentication token, e.g.

    https://example.com/civicrm/my-form?_authx=GENERAL_LOGIN_TOKEN&_authxSes=1
    

After

  • There is a setting ("Administer > System Settings > Form Core") which determines whether to prefer page-level authentication or session-level authentication.

  • Sending an email with a token like {afform.myFormUrl} will generate a URL with a token. The token will look like either:

    https://example.com/civicrm/my-form?_authx=GENERAL_LOGIN_TOKEN&_authxSes=1
    

    or

    https://example.com/civicrm/my-form?_aff=LIMITED_PAGE_TOKEN
    

Comments

  • Testing
    • Probably needs more interactive testing. I've been focused mostly on phpunit.
    • This includes some end-to-end tests using both Guzzle and Mink/Chrome.
    • I couldn't get the Mink/Chrome tests to implement "wait-until-condition" behavior. I included several comments about some of the attempted formulations, but none of them seemed to work. (Well, the test might pass - but only have after waiting until the full timeout. So they'd idle longer than necessary) For the moment, it's just using a static wait(2000). Maybe someone with more Mink experience (like @demeritcowboy) can figure a better technique?
  • Approach
    • This only targets Afform.
    • In Afform, all authenticated requests go through REST/APIv4. Civi's REST (when using authx) has support for processing narrow requests (i.e. stateless; no cookie; fake session). Also, you can propagate credentials using a single mechanism (jQuery's $.ajaxSetup()).
    • In other kinds of forms (eg QuickForm), there's a mix of GETs, POST-backs, adhoc AJAX, and redirects. Making a generic mechanism for this would be tougher.

Copy link

civibot bot commented Jul 1, 2024

🤖 Thank you for contributing to CiviCRM! ❤️ We will need to test and review this PR. 👷

Introduction for new contributors...
  • If this is your first PR, an admin will greenlight automated testing with the command ok to test or add to whitelist.
  • A series of tests will automatically run. You can see the results at the bottom of this page (if there are any problems, it will include a link to see what went wrong).
  • A demo site will be built where anyone can try out a version of CiviCRM that includes your changes.
  • If this process needs to be repeated, an admin will issue the command test this please to rerun tests and build a new demo site.
  • Before this PR can be merged, it needs to be reviewed. Please keep in mind that reviewers are volunteers, and their response time can vary from a few hours to a few weeks depending on their availability and their knowledge of this particular part of CiviCRM.
  • A great way to speed up this process is to "trade reviews" with someone - find an open PR that you feel able to review, and leave a comment like "I'm reviewing this now, could you please review mine?" (include a link to yours). You don't have to wait for a response to get started (and you don't have to stop at one!) the more you review, the faster this process goes for everyone 😄
  • To ensure that you are credited properly in the final release notes, please add yourself to contributor-key.yml
  • For more information about contributing, see CONTRIBUTING.md.
Quick links for reviewers...

➡️ Online demo of this PR 🔗

@civibot civibot bot added the master label Jul 1, 2024
@demeritcowboy
Copy link
Contributor

The wait() calls you tried look fine. Might need to step-debug it.

@eileenmcnaughton
Copy link
Contributor

api\v4\Authx\AuthxCredentialTest::testValidation
Undefined variable $e

/home/homer/buildkit/build/build-1/web/sites/all/modules/civicrm/ext/authx/Civi/Authx/Authenticator.php:166
/home/homer/buildkit/build/build-1/web/sites/all/modules/civicrm/ext/authx/Civi/Api4/Action/AuthxCredential/Validate.php:54
/home/homer/buildkit/build/build-1/web/sites/all/modules/civicrm/Civi/Api4/Provider/ActionObjectProvider.php:70
/home/homer/buildkit/build/build-1/web/sites/all/modules/civicrm/Civi/API/Kernel.php:153
/home/homer/buildkit/build/build-1/web/sites/all/modules/civicrm/Civi/Api4/Generic/AbstractAction.php:256
/home/homer/buildkit/build/build-1/web/sites/all/modules/civicrm/ext/authx/tests/phpunit/api/v4/AuthxCredentialTest.php:63
/home/homer/buildkit/extern/phpunit9/phpunit9.phar:2307

@eileenmcnaughton
Copy link
Contributor

Civi\Authx\CustomFlowsTest::testCliPipeUntrustedLogin
Civi\Pipe\JsonRpcMethodException: implode(): Argument #1 ($array) must be of type array, string given

/home/homer/buildkit/build/build-1/web/sites/all/modules/civicrm/Civi/Pipe/BasicPipeClient.php:132
/home/homer/buildkit/build/build-1/web/sites/all/modules/civicrm/ext/authx/tests/phpunit/Civi/Authx/CustomFlowsTest.php:94
/home/homer/buildkit/extern/phpunit9/phpunit9.phar:2307

@ufundo ufundo self-requested a review July 25, 2024 21:39
Copy link
Contributor

@ufundo ufundo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just code comments so far, will r-run now and let you know how I go.

ext/afform/core/Civi/Afform/PageTokenCredential.php Outdated Show resolved Hide resolved
ext/afform/core/Civi/Afform/PageTokenCredential.php Outdated Show resolved Hide resolved

$extraFields = array_diff(array_keys($parsed), $routeInfo['allowFields']);
if (!empty($extraFields)) {
\Civi::log()->warning("Malformed request. Routes matching $regex only support these input fields: " . json_encode($routeInfo['allowFields']));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "unauthorized" => "malformed" and "support" => "permit" might be clearer log message to show it's a permission-y issue

ext/afform/core/Civi/Afform/PageTokenCredential.php Outdated Show resolved Hide resolved
}
}

// Actually, we may not need this? aiming for model where top page-request auth is irrelevant to subrequests...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the idea that the page with the form on it could be public (or authenticated some other way) and the permission checks only happen on the AJAX calls?

sounds neat

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't see at the moment how the token would make it here on the initial request.

It's passed in _aff param and the only place that seems to go is to the onInvoke above?

If we want to use this doesn't it need to be picked up in the same way an _authx param is in Authenticator::on_civi_invoke_auth ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So currently if the form doesn't have anonymous access then I get access denied, even if I have a token.

I could get it working with:

diff --git a/ext/authx/Civi/Authx/Authenticator.php b/ext/authx/Civi/Authx/Authenticator.php
index d961b3a8a6..50cd5104bf 100644
--- a/ext/authx/Civi/Authx/Authenticator.php
+++ b/ext/authx/Civi/Authx/Authenticator.php
@@ -63,6 +63,10 @@ class Authenticator extends AutoService implements HookInterface {
         _authx_redact(['_authx']);
       }
     }
+    if (!empty($params['_aff'])) {
+        $this->auth($e, ['flow' => 'param', 'cred' => $params['_aff'], 'siteKey' => $siteKey]);
+        _authx_redact(['_aff']);
+    }
   }
 
   /**

Which then meant it got picked up here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also tried getting my head around the "independent top page request" model.

If I've understood correctly, I create a form where the form itself is anonymously accessible, but it behaves differently for a specific-user.

So I created an anonymous form to create some entities with Role based access, and then tried submitting with/without the _aff token. Without token - the entities don't get created as the anonymous user doesn't have permission. With the token, it works!

But I don't see the harm of authenticating the top level request using the token. And it's quite weird to have to remember that you have to make the form world-accessible; and maybe sometimes you don't want it to be, and then how do you provide that indepedent authentication?

In essence I think the Form-Based / User-Based access controls are already pretty hard to get one's head around, so seems like splitting the form page and form api authentication adds another dimension of complexity. So maybe to keep bundled for now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ufundo Oh, very good point! The admin-experience (configuring perms) will be more forgiving if the token confers access to top-level page.

In fact, it is probably useful to be able to express the distinction between:

  • "civicrm/abc allows fully anonymous usage; it can be used with or without tokens"
  • "civicrm/abc is not anonymous; but it can be used with tokens"

And there's no particular drawback. Yes, maybe someday we'd want to take a specific/anonymous form and export a static bundle of JS/HTML. But including auth here doesn't actually prevent that. Going down that path would require more/different QA, but the mechanisms can coexist.

I'll play with your patch to confirm (and hopefully include in the next push).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's still probably worthwhile to make this work, but I think I found trade-off regarding masquerade scenarios -- e.g. if you're logged in as contact 100 and click a link with ?_aff=JWT{...cid=200...}, then what happens? Where does it draw the line between the session-level-user and the page-level-user?

Intuitively, I think it plays a bit easier if the Afform JS/HTML content is essentially anonymous -- then we only need to think about the AJAX calls. And you can imagine different AJAX calls running with different identities. (For central page-content, the AJAX calls use the page-token; for peripheral page-content -- eg navbar -- the AJAX calls use your session-token.) Hard to imagine what you do with the top-level page if needs to simultaneously use permissions of both session-level-user and page-level-user.

Still, in either case, I think masquerade is tricky. If we want something nice, we have to put it back in the oven to bake longer...


Of course, "do nothing" is bad option. Right now, it abends and prints 1-line:

HTTP 401 Cannot login. Session already active.

Maybe in the meantime, a minimal improvement would be to show a regular HTML page with a message like:

<p>You are trying to view a page as another user:</p>

<center><a href="the-full-url-with-auth-token">https://example.com/civicrm/my-form</a></center>

<p>To continue as the other user, please right-click the link and open it in a private window.</p>

It's not pretty, but it's more coherent...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This comment originally misplaced on a different subthread. Re-placing in correct context)

I added a WIP-commit for the idea of using _aff token to identity for the main page-view. (It moved a little bit to better respect module-boundaries, but still basically the same thing.) However, this technique is having an undesirable side-effect -- it causes the user-session to be authenticated. I'm not entirely sure why (and getting too sleep to dig into that). Hence the WIP-commit.

Copy link
Contributor

@ufundo ufundo Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

e.g. if you're logged in as contact 100 and click a link with ?_aff=JWT{...cid=200...}, then what happens

As you say, currently you get an error.

To me this seems like a known, pre-existing bug: https://lab.civicrm.org/dev/core/-/issues/4464 (edit 4464 not 4463)

I def think it would be good to fix, but it seems outside of the scope of this PR.

Won't it affect the AJAX requests anyway? So a bit orthogonal to the question of which requests are in/out of the tokens authorising power?

ext/authx/Civi/Authx/Authenticator.php Outdated Show resolved Hide resolved
@aydun
Copy link
Contributor

aydun commented Aug 12, 2024

A bit late to the party here ... but great to see this progressing.

However, a setting at the global level seems rather limiting. On a complex site you may well have some forms that want page-level authentication and some that want session-level.

How about making this a setting on the form instead?

This only runs when you enable debugging for e2e tests (DEBUG=1).
Before: Separate tests for different permutations of token-name/output-media

After: One test checks consistency across token-names/output-media -- asserting
       that they all point to the same URL.

       Another test checks whether that URL behaves corectly.
@totten
Copy link
Member Author

totten commented Sep 6, 2024

Rebased. Test(+functionality) had regressed due some intervening changes on master, but that's updated/fixed now. Included several (tho not all) discussion-points.

@eileenmcnaughton
Copy link
Contributor

@ufundo all good now?

@totten
Copy link
Member Author

totten commented Sep 9, 2024

I believe the open points are:

  • (Re: @ufundo) Authentication for private forms: Basically, the form itself needs to be public at the moment -- but then you can view it on behalf of the authenticated contact. There is a small patch to support private forms (original; revision) which sort-of works, but not properly. Straight-fix requires a deeper dive (maybe 1hr? maybe 4hr? maybe 3 days?). We either take that dive upfront... or accept it (for now) as a "Known Limitation".
  • (Re: @aydun ) Configuration scope: If you have a lot of different forms/workflows/use-cases (or if you've already started with session-level authn and want to incrementally adopt page-level authn), then you need finer controls about when to use session-level vs page-level.
    1. The current global setting enum afform_mail_auth_token is the simplest thing that could possibly work. But it's coarse. (The coarseness is good+bad -- you cannot fine-tune. But it's easy to bulk-convert old=>new behavior.)
    2. If we add an identical field at the form-level, then that's OK. The two can coexist. (The global setting is the default -- and forms can be individually switched.) This can be a follow-up PR.
    3. The other way is to use different tokens with different names. Perhaps admin uses the form's "Expose To" field to enable Message Tokens (Page-Only) and/or Message Tokens (Session-Login). This really gives the best granularity (allowing either/both). Any conversion need to be case-by-case. (In this scenario, this PR isn't ready. Global-setting should be removed; and the new behavior should be attached to new tokens.)
    4. (Aside: I don't think that (ii) or (iii) are so different in terms of total effort. But (ii) is more flexible with ordering of tasks. After kicking it around, I think I lean toward (ii)...)

@eileenmcnaughton
Copy link
Contributor

@totten are the outstanding discussion points requiring resolution as part of this PR - or are they potential follow ups?

@totten
Copy link
Member Author

totten commented Sep 10, 2024

IMHO, they can mostly be viewed as follow-ups. (But I'd allow a little more time for Ben or Aidan to feedback on that.)

@aydun
Copy link
Contributor

aydun commented Sep 10, 2024 via email


// \CRM_Core_Session::useFakeSession();
// $params = ($_SERVER['REQUEST_METHOD'] === 'GET') ? $_GET : $_POST;
// $authenticated = \Civi::service('authx.authenticator')->auth($e, ['flow' => 'param', 'cred' => $params['_aff'], 'siteKey' => NULL]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't you pass useSession => FALSE here to make it stateless for the top level page load?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you would think so. I'm pretty sure I dipped a toe in that direction -- setting the explicit value and/or inspecting the effective value (with debugger). But while inspecting I realized that useSession=>FALSE is already the default.

It's particularly strange because StatelessFlowsTest is passing. (If the problem was on J/WP/SA, then this known quirk could explain the discrepancy. But I'm pretty sure my final testing was on D7 which should be more robust.)

Of course, it is worth double-checking the simple things again.

But this is why I'm expecting it needs a deeper trace of the session-management. (...Like maybe the FakeSession isn't superceding the UF's default session for some reason? Or something in UF-land is forcing the session to start?...)

@ufundo
Copy link
Contributor

ufundo commented Sep 10, 2024

My worry with the top-level-page auth thing is that out there in the world, the current behaviour is so intricate/subtle to the point of being unusable. I think there's a pretty high risk as-is of admins not even trying or getting cut by the subtlety at the first attempt, and never coming back.

Having said that, I think it's functionally separate - so reasonable to split into a follow up PR. Just think it would be good to have that follow up before this makes it into any release.

@ufundo
Copy link
Contributor

ufundo commented Sep 10, 2024

I leant towards II the same as you in terms of the setting @totten

Though that does seem a valid concern in terms of the lifecycle of usage @aydun . I think on the flip side, having lots of settings is also error prone. Attempt to mitigate with a pretty serious health-warning on the global setting "changing this affects ALL your currently configured forms..." ?

In terms of the lifecycle - how does it actually work? Is it for new tokens generated going forward (ie when new emails are sent)? Maybe if you have a tokenised link in an automated email that's going to slip through the net? Or a re-usable block in your email template or something like that.

@ufundo
Copy link
Contributor

ufundo commented Sep 13, 2024

@aydun will you be at the sprint next week? Can we pick this up then?

@aydun
Copy link
Contributor

aydun commented Sep 13, 2024 via email

@totten
Copy link
Member Author

totten commented Sep 20, 2024

We discussed this a fair amount at Ashbourne - e.g. considering various migration paths.

At one point, discussion moved toward using Afform.placement field to define two flags for the two kinds of tokens (session-level/page-level). This led to a lot of consideration about how exactly to present the options (e.g. "Message Tokens (Session-Login)", "Message Tokens (Login)", "Message Tokens (Single Page)", "Message Tokens (Just This Form)", "Message Tokens (Form Only)", etc). It was hard to find consensus on labels, so then we reconsidered how much we really wanted both kinds of functionality in core.

This led to another approach:

  1. In civicrm-core, do a straight-up switch in the behavior (i.e. only include support for page-level tokens).
  2. Move support for a session-level tokens to an extension -- and offer the extension for adoption.

It's gonna take some massaging to the patch-set, and it may help to keep this branch for reference. So I think it's probably better to close this PR -- and submit other PRs(s) for that approach.

@totten totten merged commit 60130c2 into civicrm:master Sep 20, 2024
1 check passed
@totten totten deleted the master-page-token branch September 20, 2024 16:25
totten added a commit to totten/civicrm-core that referenced this pull request Oct 30, 2024
Before+After
------------

* Up through 5.78-stable, `afform_core` included support for generating emails
  with form-links w/login-tokens. However, there was a long-standing
  request from multiple people to support form-links w/page-tokens.

* During 5.79-alpha (civicrm#30585), we introduced page-tokens. We discussed more
  at the sprint, and there was a distinct feeling that page-tokens were more
  desirable (more approprate to more users/less foot-gunny). We agreed that
  login-tokens were hypothetically better for some, but no one in that
  discussion was eager to use them. So we moved login-tokens to a contrib
  extension.

* But during 5.79-beta cycle, we got testing feedback from more people
  who were keen on login-tokens.

* This PR re-introduces login-token support as a core-extension for 5.79-beta,
  which means that:

    * The default/typical mode of operation is based on page-tokens.
    * Using login-tokens is a little bit of work, but not as much.
totten added a commit to totten/civicrm-core that referenced this pull request Oct 30, 2024
Before+After
------------

* Up through 5.78-stable, `afform_core` included support for generating emails
  with form-links w/login-tokens. However, there was a long-standing
  request from multiple people to support form-links w/page-tokens.

* During 5.79-alpha (civicrm#30585), we introduced page-tokens. We discussed more
  at the sprint, and there was a distinct feeling that page-tokens were more
  desirable (more approprate to more users/less foot-gunny). We agreed that
  login-tokens were hypothetically better for some, but no one in that
  discussion was eager to use them. So we moved login-tokens to a contrib
  extension.

* But during 5.79-beta cycle, we got testing feedback from more people
  who were keen on login-tokens.

* This PR re-introduces login-token support as a core-extension for 5.79-beta,
  which means that:

    * The default/typical mode of operation is based on page-tokens.
    * Using login-tokens is a little bit of work, but not as much.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants