Skip to content

Commit

Permalink
Bump qs from 6.5.2 to 6.5.3 (#6684)
Browse files Browse the repository at this point in the history
* Add second (non-admin) default user to "initial data" trigger (#6666)

* add second (non-admin) user to initial data trigger

* rename second user and DRY insertion method

* format

* name default organization team "Default" when inserting initial data (analoguous to general organization creation)

* Add OIDC authentication (#6534)

Co-authored-by: Florian M <[email protected]>
Co-authored-by: Philipp Otto <[email protected]>

* Guard against invalid-mag bucket volume save actions (#6660)

* [WIP] guard against invalid-mag bucket volume save actions

* avoid unnecessary creation of objects in ResolutionInfo instances

* use layer's resolution info when in wkstore adapter

* rename bucketPositionToGlobalAddress to bucketPositionToGlobalAddressOld

* get rid of bucketPositionToGlobalAddressOld

* rename bucketPositionToGlobalAddressNew to bucketPositionToGlobalAddress

* simplify getBucketExtent signature

* fix tests

* fix quick-select tool for scenarios where the color layer is available in the current mag but the volume layer isn't

* Update frontend/javascripts/test/model/binary/layers/wkstore_adapter.spec.ts

* avoid losing error chain during conversion

Co-authored-by: Philipp Otto <[email protected]>
Co-authored-by: Philipp Otto <[email protected]>

* Provide valid JSON schema (#6642)

extra `{}` cause JSON decode error in python standard JSON library and webknossos instance

Co-authored-by: Philipp Otto <[email protected]>

* Improve layout of dashboard

* Swagger annotation for shortLinkByKey (#6682)

* Bump qs from 6.5.2 to 6.5.3

Bumps [qs](https://github.com/ljharb/qs) from 6.5.2 to 6.5.3.
- [Release notes](https://github.com/ljharb/qs/releases)
- [Changelog](https://github.com/ljharb/qs/blob/main/CHANGELOG.md)
- [Commits](ljharb/qs@v6.5.2...v6.5.3)

---
updated-dependencies:
- dependency-name: qs
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Philipp Otto <[email protected]>
Co-authored-by: frcroth <[email protected]>
Co-authored-by: Florian M <[email protected]>
Co-authored-by: Florian M <[email protected]>
Co-authored-by: Philipp Otto <[email protected]>
Co-authored-by: erjel <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
  • Loading branch information
7 people authored Dec 13, 2022
1 parent 1162ddc commit 399078e
Show file tree
Hide file tree
Showing 45 changed files with 533 additions and 144 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.unreleased.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
[Commits](https://github.com/scalableminds/webknossos/compare/22.12.0...HEAD)

### Added
- Added sign in via OIDC. [#6534](https://github.com/scalableminds/webknossos/pull/6534)
- Added a new datasets tab to the dashboard which supports managing datasets in folders. Folders can be organized hierarchically and datasets can be moved into these folders. Selecting a dataset will show dataset details in a sidebar. [#6591](https://github.com/scalableminds/webknossos/pull/6591)

### Changed
Expand Down
2 changes: 2 additions & 0 deletions MIGRATIONS.unreleased.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,6 @@ User-facing changes are documented in the [changelog](CHANGELOG.released.md).
- Bulk task creation now needs the taskTypeId, the task type summary will no longer be accepted. If you have scripts generating CSVs for bulk task creation, they should not output task type summaries. [#6640](https://github.com/scalableminds/webknossos/pull/6640)

### Postgres Evolutions:

- [091-folders.sql](conf/evolutions/091-folders.sql)
- [092-oidc.sql](conf/evolutions/092-oidc.sql)
119 changes: 96 additions & 23 deletions app/controllers/AuthenticationController.scala
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@ import akka.actor.ActorSystem
import com.mohiva.play.silhouette.api.actions.SecuredRequest
import com.mohiva.play.silhouette.api.exceptions.ProviderException
import com.mohiva.play.silhouette.api.services.AuthenticatorResult
import com.mohiva.play.silhouette.api.util.Credentials
import com.mohiva.play.silhouette.api.util.{Credentials, PasswordInfo}
import com.mohiva.play.silhouette.api.{LoginInfo, Silhouette}
import com.mohiva.play.silhouette.impl.providers.CredentialsProvider
import com.scalableminds.util.accesscontext.{AuthorizedAccessContext, DBAccessContext, GlobalAccessContext}
import com.scalableminds.util.tools.JsonHelper.validateJsValue
import com.scalableminds.util.tools.{Fox, FoxImplicits, TextUtils}
import models.analytics.{AnalyticsService, InviteEvent, JoinOrganizationEvent, SignupEvent}
import models.annotation.AnnotationState.Cancelled
Expand All @@ -27,7 +28,7 @@ import play.api.data.Forms.{email, _}
import play.api.data.validation.Constraints._
import play.api.i18n.Messages
import play.api.libs.json._
import play.api.mvc.{Action, AnyContent, PlayBodyParsers}
import play.api.mvc.{Action, AnyContent, Cookie, PlayBodyParsers, Request, Result}
import utils.{ObjectId, WkConf}

import java.net.URLEncoder
Expand Down Expand Up @@ -55,6 +56,7 @@ class AuthenticationController @Inject()(
annotationDAO: AnnotationDAO,
voxelyticsDAO: VoxelyticsDAO,
wkSilhouetteEnvironment: WkSilhouetteEnvironment,
openIdConnectClient: OpenIdConnectClient,
sil: Silhouette[WkEnv])(implicit ec: ExecutionContext, bodyParsers: PlayBodyParsers)
extends Controller
with AuthForms
Expand Down Expand Up @@ -83,7 +85,7 @@ class AuthenticationController @Inject()(
errors ::= Messages("user.lastName.invalid")
""
}
multiUserDAO.findOneByEmail(email)(GlobalAccessContext).toFox.futureBox.flatMap {
multiUserDAO.findOneByEmail(email)(GlobalAccessContext).futureBox.flatMap {
case Full(_) =>
errors ::= Messages("user.email.alreadyInUse")
Fox.successful(BadRequest(Json.obj("messages" -> Json.toJson(errors.map(t => Json.obj("error" -> t))))))
Expand All @@ -98,25 +100,15 @@ class AuthenticationController @Inject()(
inviteBox.toOption,
organizationName)(GlobalAccessContext) ?~> Messages("organization.notFound", signUpData.organization)
autoActivate = inviteBox.toOption.map(_.autoActivate).getOrElse(organization.enableAutoVerify)
user <- userService.insert(organization._id,
email,
firstName,
lastName,
autoActivate,
passwordHasher.hash(signUpData.password)) ?~> "user.creation.failed"
multiUser <- multiUserDAO.findOne(user._multiUser)(GlobalAccessContext)
_ = analyticsService.track(SignupEvent(user, inviteBox.isDefined))
_ <- Fox.runOptional(inviteBox.toOption)(i =>
inviteService.deactivateUsedInvite(i)(GlobalAccessContext))
brainDBResult <- brainTracing.registerIfNeeded(user, signUpData.password).toFox
_ <- createUser(organization,
email,
firstName,
lastName,
autoActivate,
Option(signUpData.password),
inviteBox,
registerBrainDB = true)
} yield {
if (conf.Features.isDemoInstance) {
mailchimpClient.registerUser(user, multiUser, tag = MailchimpTag.RegisteredAsUser)
} else {
Mailer ! Send(defaultMails.newUserMail(user.name, email, brainDBResult, autoActivate))
}
Mailer ! Send(
defaultMails.registerAdminNotifyerMail(user.name, email, brainDBResult, organization, autoActivate))
Ok
}
}
Expand All @@ -126,6 +118,35 @@ class AuthenticationController @Inject()(
)
}

private def createUser(organization: Organization,
email: String,
firstName: String,
lastName: String,
autoActivate: Boolean,
password: Option[String],
inviteBox: Box[Invite] = Empty,
registerBrainDB: Boolean = false)(implicit request: Request[AnyContent]): Fox[User] = {
val passwordInfo: PasswordInfo =
password.map(passwordHasher.hash).getOrElse(userService.getOpenIdConnectPasswordInfo)
for {
user <- userService.insert(organization._id, email, firstName, lastName, autoActivate, passwordInfo) ?~> "user.creation.failed"
multiUser <- multiUserDAO.findOne(user._multiUser)(GlobalAccessContext)
_ = analyticsService.track(SignupEvent(user, inviteBox.isDefined))
_ <- Fox.runIf(inviteBox.isDefined)(Fox.runOptional(inviteBox.toOption)(i =>
inviteService.deactivateUsedInvite(i)(GlobalAccessContext)))
brainDBResult <- Fox.runIf(registerBrainDB)(brainTracing.registerIfNeeded(user, password.getOrElse("")))
_ = if (conf.Features.isDemoInstance) {
mailchimpClient.registerUser(user, multiUser, tag = MailchimpTag.RegisteredAsUser)
} else {
Mailer ! Send(defaultMails.newUserMail(user.name, email, brainDBResult.flatten, autoActivate))
}
_ = Mailer ! Send(
defaultMails.registerAdminNotifyerMail(user.name, email, brainDBResult.flatten, organization, autoActivate))
} yield {
user
}
}

def authenticate: Action[AnyContent] = Action.async { implicit request =>
signInForm.bindFromRequest.fold(
bogusForm => Future.successful(BadRequest(bogusForm.toString)),
Expand Down Expand Up @@ -430,7 +451,7 @@ class AuthenticationController @Inject()(
request.identity match {
case Some(user) =>
// logged in
// Check if the request we recieved was signed using our private sso-key
// Check if the request we received was signed using our private sso-key
if (shaHex(ssoKey, sso) == sig) {
val payload = new String(Base64.decodeBase64(sso))
val values = play.core.parsers.FormUrlEncodedParser.parse(payload)
Expand All @@ -457,6 +478,58 @@ class AuthenticationController @Inject()(
}
}

lazy val absoluteOpenIdConnectCallbackURL = s"${conf.Http.uri}/api/auth/oidc/callback"

def loginViaOpenIdConnect(): Action[AnyContent] = sil.UserAwareAction.async { implicit request =>
openIdConnectClient.getRedirectUrl(absoluteOpenIdConnectCallbackURL).map(url => Ok(Json.obj("redirect_url" -> url)))
}

private def loginUser(loginInfo: LoginInfo)(implicit request: Request[AnyContent]): Future[Result] =
userService.retrieve(loginInfo).flatMap {
case Some(user) if !user.isDeactivated =>
for {
authenticator: CombinedAuthenticator <- combinedAuthenticatorService.create(loginInfo)
value: Cookie <- combinedAuthenticatorService.init(authenticator)
result: AuthenticatorResult <- combinedAuthenticatorService.embed(value, Redirect("/dashboard"))
_ <- multiUserDAO.updateLastLoggedInIdentity(user._multiUser, user._id)(GlobalAccessContext)
_ = userDAO.updateLastActivity(user._id)(GlobalAccessContext)
} yield result
case None =>
Future.successful(BadRequest(Messages("error.noUser")))
case Some(_) => Future.successful(BadRequest(Messages("user.deactivated")))
}

// Is called after user was successfully authenticated
def loginOrSignupViaOidc(oidc: OpenIdConnectClaimSet): Request[AnyContent] => Future[Result] = {
implicit request: Request[AnyContent] =>
userService.userFromMultiUserEmail(oidc.email)(GlobalAccessContext).futureBox.flatMap {
case Full(user) =>
val loginInfo = LoginInfo("credentials", user._id.toString)
loginUser(loginInfo)
case Empty =>
for {
organization: Organization <- organizationService.findOneByInviteByNameOrDefault(None, None)(
GlobalAccessContext)
user <- createUser(organization, oidc.email, oidc.given_name, oidc.family_name, autoActivate = true, None)
// After registering, also login
loginInfo = LoginInfo("credentials", user._id.toString)
loginResult <- loginUser(loginInfo)
} yield loginResult
case _ => Future.successful(InternalServerError)
}
}

def openIdCallback(): Action[AnyContent] = Action.async { implicit request =>
for {
code <- openIdConnectClient.getToken(
absoluteOpenIdConnectCallbackURL,
request.queryString.get("code").flatMap(_.headOption).getOrElse("missing code"),
)
oidc: OpenIdConnectClaimSet <- validateJsValue[OpenIdConnectClaimSet](code).toFox
user_result <- loginOrSignupViaOidc(oidc)(request)
} yield user_result
}

private def shaHex(key: String, valueToDigest: String): String =
new HmacUtils(HmacAlgorithms.HMAC_SHA_256, key).hmacHex(valueToDigest)

Expand All @@ -476,7 +549,7 @@ class AuthenticationController @Inject()(
errors ::= Messages("user.lastName.invalid")
""
}
multiUserDAO.findOneByEmail(email)(GlobalAccessContext).toFox.futureBox.flatMap {
multiUserDAO.findOneByEmail(email)(GlobalAccessContext).futureBox.flatMap {
case Full(_) =>
errors ::= Messages("user.email.alreadyInUse")
Fox.successful(BadRequest(Json.obj("messages" -> Json.toJson(errors.map(t => Json.obj("error" -> t))))))
Expand Down
47 changes: 38 additions & 9 deletions app/controllers/InitialDataController.scala
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ class InitialDataService @Inject()(userService: UserService,
implicit val ctx: GlobalAccessContext.type = GlobalAccessContext

private val defaultUserEmail = conf.WebKnossos.SampleOrganization.User.email
private val defaultUserEmail2 = conf.WebKnossos.SampleOrganization.User.email2
private val defaultUserPassword = conf.WebKnossos.SampleOrganization.User.password
private val defaultUserToken = conf.WebKnossos.SampleOrganization.User.token
private val additionalInformation = """**Sample Organization**
Expand All @@ -75,9 +76,11 @@ Samplecountry
PricingPlan.Custom,
ObjectId.generate)
private val organizationTeam =
Team(organizationTeamId, defaultOrganization._id, defaultOrganization.name, isOrganizationTeam = true)
Team(organizationTeamId, defaultOrganization._id, "Default", isOrganizationTeam = true)
private val userId = ObjectId.generate
private val multiUserId = ObjectId.generate
private val userId2 = ObjectId.generate
private val multiUserId2 = ObjectId.generate
private val defaultMultiUser = MultiUser(
multiUserId,
defaultUserEmail,
Expand All @@ -99,6 +102,27 @@ Samplecountry
isDeactivated = false,
lastTaskTypeId = None
)
private val defaultMultiUser2 = MultiUser(
multiUserId2,
defaultUserEmail2,
userService.createPasswordInfo(defaultUserPassword),
isSuperUser = false,
)
private val defaultUser2 = User(
userId2,
multiUserId2,
defaultOrganization._id,
"Non-Admin",
"User",
System.currentTimeMillis(),
Json.obj(),
userService.createLoginInfo(userId2),
isAdmin = false,
isDatasetManager = false,
isUnlisted = false,
isDeactivated = false,
lastTaskTypeId = None
)
private val defaultPublication = Publication(
ObjectId("5c766bec6c01006c018c7459"),
Some(System.currentTimeMillis()),
Expand All @@ -119,7 +143,8 @@ Samplecountry
_ <- insertRootFolder()
_ <- insertOrganization()
_ <- insertTeams()
_ <- insertDefaultUser()
_ <- insertDefaultUser(defaultUserEmail, defaultMultiUser, defaultUser, true)
_ <- insertDefaultUser(defaultUserEmail2, defaultMultiUser2, defaultUser2, false)
_ <- insertToken()
_ <- insertTaskType()
_ <- insertProject()
Expand All @@ -143,19 +168,23 @@ Samplecountry
case _ => folderDAO.insertAsRoot(Folder(defaultOrganization._rootFolder, folderService.defaultRootName))
}

private def insertDefaultUser(): Fox[Unit] =
private def insertDefaultUser(userEmail: String,
multiUser: MultiUser,
user: User,
isTeamManager: Boolean): Fox[Unit] =
userService
.userFromMultiUserEmail(defaultUserEmail)
.userFromMultiUserEmail(userEmail)
.futureBox
.flatMap {
case Full(_) => Fox.successful(())
case _ =>
for {
_ <- multiUserDAO.insertOne(defaultMultiUser)
_ <- userDAO.insertOne(defaultUser)
_ <- userExperiencesDAO.updateExperiencesForUser(defaultUser, Map("sampleExp" -> 10))
_ <- userTeamRolesDAO.insertTeamMembership(defaultUser._id,
TeamMembership(organizationTeam._id, isTeamManager = true))
_ <- multiUserDAO.insertOne(multiUser)
_ <- userDAO.insertOne(user)
_ <- userExperiencesDAO.updateExperiencesForUser(user, Map("sampleExp" -> 10))
_ <- userTeamRolesDAO.insertTeamMembership(
user._id,
TeamMembership(organizationTeam._id, isTeamManager = isTeamManager))
_ = logger.info("Inserted default user")
} yield ()
}
Expand Down
10 changes: 9 additions & 1 deletion app/controllers/ShortLinkController.scala
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package controllers

import com.mohiva.play.silhouette.api.Silhouette
import com.scalableminds.util.tools.FoxImplicits
import io.swagger.annotations.{Api, ApiOperation, ApiParam}
import models.shortlinks.{ShortLink, ShortLinkDAO}
import oxalis.security.{RandomIDGenerator, WkEnv}
import play.api.libs.json.Json
Expand All @@ -11,12 +12,14 @@ import utils.{ObjectId, WkConf}
import javax.inject.Inject
import scala.concurrent.ExecutionContext

@Api
class ShortLinkController @Inject()(shortLinkDAO: ShortLinkDAO, sil: Silhouette[WkEnv], wkConf: WkConf)(
implicit ec: ExecutionContext,
val bodyParsers: PlayBodyParsers)
extends Controller
with FoxImplicits {

@ApiOperation(hidden = true, value = "")
def create: Action[String] = sil.SecuredAction.async(validateJson[String]) { implicit request =>
val longLink = request.body
val _id = ObjectId.generate
Expand All @@ -28,7 +31,12 @@ class ShortLinkController @Inject()(shortLinkDAO: ShortLinkDAO, sil: Silhouette[
} yield Ok(Json.toJson(inserted))
}

def getByKey(key: String): Action[AnyContent] = Action.async { implicit request =>
@ApiOperation(value = "Information about a short link, including the original long link.",
nickname = "shortLinkByKey")
def getByKey(
@ApiParam(value = "key of the shortLink, this is the short random string identifying the link.",
example = "aU7yv5Aja99T0829")
key: String): Action[AnyContent] = Action.async { implicit request =>
for {
shortLink <- shortLinkDAO.findOneByKey(key)
} yield Ok(Json.toJson(shortLink))
Expand Down
2 changes: 1 addition & 1 deletion app/controllers/TaskController.scala
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ Expects:
file.filename.toLowerCase.endsWith(".nml") || file.filename.toLowerCase.endsWith(".zip"))
_ <- bool2Fox(inputFiles.nonEmpty) ?~> "nml.file.notFound"
jsonString <- body.dataParts.get("formJSON").flatMap(_.headOption) ?~> "format.json.missing"
params <- JsonHelper.parseJsonToFox[NmlTaskParameters](jsonString) ?~> "task.create.failed"
params <- JsonHelper.parseAndValidateJson[NmlTaskParameters](jsonString) ?~> "task.create.failed"
_ <- taskCreationService.assertBatchLimit(inputFiles.length, List(params.taskTypeId))
taskTypeIdValidated <- ObjectId.fromString(params.taskTypeId) ?~> "taskType.id.invalid"
taskType <- taskTypeDAO.findOne(taskTypeIdValidated) ?~> "taskType.notFound" ~> NOT_FOUND
Expand Down
2 changes: 1 addition & 1 deletion app/models/annotation/Annotation.scala
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ class AnnotationDAO @Inject()(sqlClient: SQLClient, annotationLayerDAO: Annotati
for {
state <- AnnotationState.fromString(r.state).toFox
typ <- AnnotationType.fromString(r.typ).toFox
viewconfigurationOpt <- Fox.runOptional(r.viewconfiguration)(JsonHelper.parseJsonToFox[JsObject](_))
viewconfigurationOpt <- Fox.runOptional(r.viewconfiguration)(JsonHelper.parseAndValidateJson[JsObject](_))
visibility <- AnnotationVisibility.fromString(r.visibility).toFox
annotationLayers <- annotationLayerDAO.findAnnotationLayersFor(ObjectId(r._Id))
} yield {
Expand Down
10 changes: 5 additions & 5 deletions app/models/binary/DataSet.scala
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,10 @@ class DataSetDAO @Inject()(sqlClient: SQLClient,
for {
scale <- parseScaleOpt(r.scale)
defaultViewConfigurationOpt <- Fox.runOptional(r.defaultviewconfiguration)(
JsonHelper.parseJsonToFox[DataSetViewConfiguration](_))
JsonHelper.parseAndValidateJson[DataSetViewConfiguration](_))
adminViewConfigurationOpt <- Fox.runOptional(r.adminviewconfiguration)(
JsonHelper.parseJsonToFox[DataSetViewConfiguration](_))
details <- Fox.runOptional(r.details)(JsonHelper.parseJsonToFox[JsObject](_))
JsonHelper.parseAndValidateJson[DataSetViewConfiguration](_))
details <- Fox.runOptional(r.details)(JsonHelper.parseAndValidateJson[JsObject](_))
} yield {
DataSet(
ObjectId(r._Id),
Expand Down Expand Up @@ -452,9 +452,9 @@ class DataSetDataLayerDAO @Inject()(sqlClient: SQLClient, dataSetResolutionsDAO:
resolutions <- Fox.fillOption(standinResolutions)(
dataSetResolutionsDAO.findDataResolutionForLayer(dataSetId, row.name) ?~> "Could not find resolution for layer")
defaultViewConfigurationOpt <- Fox.runOptional(row.defaultviewconfiguration)(
JsonHelper.parseJsonToFox[LayerViewConfiguration](_))
JsonHelper.parseAndValidateJson[LayerViewConfiguration](_))
adminViewConfigurationOpt <- Fox.runOptional(row.adminviewconfiguration)(
JsonHelper.parseJsonToFox[LayerViewConfiguration](_))
JsonHelper.parseAndValidateJson[LayerViewConfiguration](_))
} yield {
category match {
case Category.segmentation =>
Expand Down
2 changes: 1 addition & 1 deletion app/models/binary/explore/RemoteLayerExplorer.scala
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ trait RemoteLayerExplorer extends FoxImplicits {
protected def parseJsonFromPath[T: Reads](path: Path): Fox[T] =
for {
fileAsString <- tryo(new String(Files.readAllBytes(path), StandardCharsets.UTF_8)).toFox ?~> "Failed to read remote file"
parsed <- JsonHelper.parseJsonToFox[T](fileAsString) ?~> "Failed to validate json against data schema"
parsed <- JsonHelper.parseAndValidateJson[T](fileAsString) ?~> "Failed to validate json against data schema"
} yield parsed

protected def looksLikeSegmentationLayer(layerName: String, elementClass: ElementClass.Value): Boolean =
Expand Down
Loading

0 comments on commit 399078e

Please sign in to comment.