-
Notifications
You must be signed in to change notification settings - Fork 9
Configuration
- Directory layout
- Security configuration
- Bootstrap registers
- Base URI
- General configuration
- Logging
- UI templates
All configuration and template information for the registry is stored (by default) in the directory /opt/ldregistry.
Runtime information such as the internal triple store and record of uploaded files are in /var/opt/ldregistry.
Logging information is in /var/log. Most logging information recorded as part of the web app container and so appears in /var/log/tomcat7/catalina.out. There is a separate log file for the velocity template engine which appears in /var/log/ldregistry-velocity.log.
Note that this is a change from the proof-of-concept implementation which used /var/local/registry for everything. That does not follow linux file system layout conventions. Arguably the configuation information should be accessible in /etc/ldregistry, a deployment could use symbolic links to achive this. The use of ldregistry rather than registry is a prefix is to disambiguate against other registry-like software.
When a registry instance first starts up then each area of the configuration directory will be initialized if it does not exist. The only exception to this is that, for technical reasons with the Tomcat context definition, an empty ui/ directory should exist for successful start-up.
The default layout of /opt/ldregistry is summarized below:
| Directory or file | Purpose |
|---|---|
boot/ |
Definitions of bootstrap registers used when starting a registry with no data. |
boot/system |
The bootstrap definition of the system registers. |
config/ |
Configuration and initialization files. |
config/root-register.ttl |
RDF file describing the root register, used during bootstrapping only. |
config/app.conf |
Configuration parameters for the software, defines the set of components that make up the registry and provides configuration parameters for them. |
config/user.ini |
Initialization for the security database, provides OpenIDs and grant information for initial users. Not consulted once the userstore database exists. |
config/oauth.conf |
Configuration of OpenID Connect provider |
proxy-conf.sh |
Script used to trigger the nginx proxy to reconsult configuration information, should be set to be sudo executable by the tomcat user. |
templates/ |
Set of templates defining the built in UI, written in Apache Velocity. |
ui/ |
Resources (CSS, image and javascript files) which should be served from the /ui path of the registry URL. |
The default layout of /var/opt/ldregistry is summarized below:
| Directory or file | Purpose |
|---|---|
index/ |
Lucene free text index for the registry content. |
logstore/ |
File copies of all uploaded registration and update payloads. |
store/ |
The RDF data store (an Apache Jena TDB instance). |
userstore/ |
Database holding security information (registrations and grants), an embedded instance of Apache Derby. |
proxy-registry.conf |
Proxy configuration script generated by the registry. Other scripts with the naming pattern proxy-*.conf may be added here to support additional manual proxy rules. |
Once the registry is running all security information is held in the database in the userstore directory. Configuration of user permissions is then done through the user interface.
If no userstore database exists then it will be initialized by loading the initial user information from config/user.ini. If no such configuration file exists then a null one will be created.
To set up an administrator for a new installation then the bootstrap userstore should be removed and the bootstrap config/user.ini should be edited to define the administrator.
The user.ini file comprises a set of declarations, one per line (lines beginning with # are comment lines).
User registration entries start with the user keyword and take the form:
user id "name"
For example:
user https://me.yahoo.com/a/rb93y8N72dK9HoXHwgNqBKlikg-- "Alice"
Each OpenID or OpenID Connect provider has a different means of obtaining a user-specific OpenID. For example Yahoo! allows creation of an OpenID through http://openid.yahoo.com/ from which a user can determine (or create) an OpenID identity. A variant on this declaration is to specify a password for the user as a final entry on the user line:
user [email protected] "Dave" foobar
In this case the user identity can be any sufficiently unique string, such as an email address, and need not be an OpenID URL (though it can be). N.B. This password in clear text and so should be used for bootstrap only and changed as part of system commissioning.
There is a built in anonymous user with pseudo OpenID of http://localhost/anon. It is convenient to declare that user in the initialization file as well, so as to bind a visible name to that id. For example:
user http://localhost/anon "Any authenticated"
The second type of declaration line is used to grant permissions to a user. These take the form:
openid permission
For example:
https://myopenid Update:/register/item
https://myotheropenid GrantAdmin
In particular the GrantAdmin permission makes the user an administrator.
Thus a minimal user.ini configuration looks something like:
user https://profiles.google.com/1147194443288764760228 "Alice" changeme
user http://localhost/anon "Any authenticated"
https://profiles.google.com/1147194443288764760228 GrantAdmin
To use the OpenID Connect protocol for user authentication, the registry instance must be configured with app-wide OAuth settings, and must specify at least one OAuth provider. Note that you must set the config.passwordLoginOnly property to false in your app.conf to enable this feature.
App-wide settings should be configured in /opt/ldregistry/config/oauth2/oauth.conf. This includes the following properties:
| Name | Meaning | Type | Default |
|---|---|---|---|
| ldregistry.usehttps | Toggle to force redirection to https for requests after successful authentication and back to https after logout when https is not being used by default. | Boolean | false |
| ldregistry.providers | Comma separated list of names of OAuth providers with which users will be able to authenticate. Names should correspond to the name specified in the provider configuration. |
String |
Provider settings should be configured in .properties files contained in the opt/ldregistry/config/oauth2/provider directory.
You can specify any number of providers, but only those listed by the app-wide ldregistry.providers property will be enabled.
Each file corresponds to one provider, and includes the following properties:
| Name | Meaning | Type | Default |
|---|---|---|---|
| name | The uniquely identifying name for the provider. This should correspond to the values of the ldregistry.providers specified by the app-wide configuration. |
String | |
| label | The user friendly display name for the provider. | String | name |
| client.id | The unique key that identifies the registry instance with the provider. | String | |
| client.secret | The secret value which authenticates the registry instance with the provider. | String | |
| auth.endpoint | The full URI of the authorization endpoint exposed by the provider. | String | |
| auth.scope | The OpenID scope which defines what user information will be accessed. The default should not be overridden as long as the provider supports standard OpenID scopes. | String | openid email |
| token.endpoint | The full URI of the access token endpoint exposed by the provider. | String | |
| userInfo.endpoint | The full URI of the user info endpoint exposed by the provider. | String | |
| userInfo.key | The name of the property, on the deserialized user info response, whose value is the unique ID of the authenticated user (this should usually be their OpenID identity). | String | |
| userInfo.name | The name of the property, on the deserialized user info response, whose value is the user friendly display name for the authenticated user. | String | userInfo.key |
The documentation for your specific provider's OAuth2 and OpenID Connect APIs should help you to configure these properties.
If you choose to use an OAuth provider which does not support OpenID, make sure that the userInfo.key value is sufficiently unique, and be aware of the privacy implications of storing personal information such as email addresses as unique IDs.
To use Google OAuth and identities provided by Google Plus then the steps are:
- Open https://console.developers.google.com
- Create a "project"
- Select
APIs and Authand enable the "Google+" API. - Configure the "consent screen", which determines what users will be told about the registry application when granting permission to use their identity
- Under
credentialsuseCreate new Client ID. SelectWeb Applicationand provide the URIs which your registry instance will be based at (including localhost in these can be useful for development purposes). For the redirect URIs then append/system/security/responseoato the base URI for your instance, for examplehttp://location.data.gov.uk/registry/system/security/responseoa - Copy the generated client id and client secret to a
google.propertiesfile under theoauth2/providersdirectory (the file name doesn't matter as long as the extension isproperties).
An example Google+ Credentials configuration:
Client ID for web application
Client ID ...
Email address ...
Client secret ...
Redirect URIs http://localhost:8080/registry/system/security/responseoa
http://environment.data.gov.uk/registry/system/security/responseoa
Javascript Origins http://localhost
http://localhost:8080
http://environment.data.gov.uk
An example Google+ provider configuration file:
name = googleplus
label = Google+
client.id = ## your Client ID
client.secret = ## your Client Secret
auth.endpoint = https://accounts.google.com/o/oauth2/auth
token.endpoint = https://accounts.google.com/o/oauth2/token
userInfo.endpoint = https://www.googleapis.com/oauth2/v3/userinfo
userInfo.key = profile
userInfo.name = name
An example app-wide configuration file:
ldregistry.usehttps = true
ldregistry.providers = googleplus
Note that users may only use Google to authenticate if their account has enabled Google+.
An example GitHub provider configuration file:
name = github
label = GitHub
client.id = ## your client id
client.secret = ## your client secret
auth.endpoint = https://github.com/login/oauth/authorize
token.endpoint = https://github.com/login/oauth/access_token
userInfo.endpoint = https://api.github.com/user
userInfo.key = url
userInfo.name = login
When a registry instance runs for the first time it needs to create an initial RDF description. There are two parts to this bootstrapping - the root register and all other registers (including the system registers and manager-supplied initial data).
The root register is a special case because of its different naming convention and because there is no register into which to add it. The root register is bootstrapped by loading an initial RDF file (/config/root-register.ttl) which must contain both the reg:Register declaration and an associated reg:RegisterItem. Each of these must meet minimum constraints for the versioning model (have an owl:versionInfo value and a version:currentVersion link, which can point to itself). The URI of the root register must match the declared base URI for the registry installation (see below).
Once the root register is in place then the normal API mechanisms for loading register declarations and registered items can function. In particular use of "relative" addresses is then possible so any such bootstrap files can be independent of the registry base URI.
The additional initial registers are loaded from the area boot/. Each directory in the boot area corresponds to a register and should contain a metadata.ttl defining it. It may also contain sub-directories for sub-registers and other *.ttl files defining items to be included in the register.
When a registry first starts up, if the bootstrap files for either of these kinds of register do not exist then a default minimum set will be created. Then if the RDF data store does not exist (store/ and the associated free index in index/) then one will be created and initialized from the bootstrap files.
Thus if you start a blank registry installation that will create both a minimal set of bootstrap files and an initialized store. Normally you will then need to halt the registry, edit the bootstrap files to suit the deployment requirements and delete the initial store/ and index/ areas. Restarting the registry will then rebuild the store based on your customized bootstrap.
The registry will create and serve all data relative to an assumed base URI. This needed not be the same as the URL at which the registry installation is hosted - either because this is a staging deployment or because some external proxy will map requests from the real URI to the hosting system.
The base URI can only be set at the time that a registry instance initially starts up. Once the initial store has been created all data in that store will hold absolute URIs which are based on the configured base URI.
There are two places where the base URI must be configured.
Firstly, the registry software itself is configured from the file config/app.conf, see below. In that file change the following line to give a suitable base URI:
registry.baseURI = http://ukgovld-registry.dnsalias.net/
Secondly, the initial bootstrap RDF for the root register must be consistent with this. This is defined in config/root-register.ttl (see above).
The registry software itself is configured through a single file config/app.conf. The file is used to configure a series of software components which together make up the registry. It is possible to provide alternative implementations of some of these components and configure them through this file. The configuration is a series of blocks, one for each component, which take the form:
component = qualified.name.of.class
component.prop1 = value1
...
component.propN = valueN
In most cases the name of the component is arbitrary. The value may be a string, a long integer, a comma separated array of values or a reference to an earlier component (using the syntax $componentname).
The parameter string {webapp} can be used in configuration values and will be expanded with the location of the running registry application.
In normal use very few of these parameters should need customization, so we will only summarize the key ones.
Note. Prior to release 0.1.1 configuration was through config/services.conf and there are some syntax changes between the two files. These include use of $ for cross referencing components, use of {webapp} instead fo ${webwapp} to reference to files within the war and some differences in the specific java components used. In most cases porting from a prior release to the current release will simply mean taking the sample from src/main/webapp/WEB-INF/app.conf, customising the registry.baseURI setting and deploying this to /opt/ldregistry/config/app.conf.
The registry service itself.
| Key property | Meaning |
|---|---|
| baseURI | The base URI at which the registry is assumed to operate. |
| pageSize | The number of items that should be returned by list views of collections (i.e. registers). |
| log | Area used for recording registration/update payloads. |
| bootSpec | A sequence of possible locations for the bootstrap root register, tried in order. |
| systemBoot | The location of the boot areas for loading system and other registers. |
| store, forwarder, userStore | References to the other required components |
| backupDir | Directory where snapshot backups of the repository dataset will be saved |
| configExtensions | Optional reference to a set of general configuration options used to customize UI behaviour |
The template rendering system used the generate the default UI.
| Key property | Meaning |
|---|---|
| production | If true then run in production mode where macros cannot be updated and templates, once cached in memory, are assumed not to change. If false then any change to a template will be detected and cause a change to the UI. |
| templates | Directory where the template files are located. |
The RDF data store for the registry.
| Key property | Meaning |
|---|---|
| location | The directory where the RDF database is held. |
| textIndex | Comma-separated list of predicates to include in the text index, can use prefixes from the default prefix set (rdf, rdfs, owl, xsd, dct, foaf, skos, ldp, void, prov, time, reg). |
| timeout | A global timeout for queries to the store. Set to "first,all" where first is the time until first result and all is the total query time allowed, both in ms. Should be set high enough to allow normal operation but can be useful to block roque ad hoc query requests. |
The (Lucene) free text index.
| Key property | Meaning |
|---|---|
| location | The directory where the text index is held. |
Component used to manage forwarding/proxy configurations. This will handle redirects itself and will implement any proxy forwarding by dynamically configuring a separate nginx instance.
| Key property | Meaning |
|---|---|
| proxyConfDir | Directory where nginx proxy scripts will be written. |
| proxyRestartScript | Shell script which will be used to cause nginx to reconsult the configuration scripts. |
This is a table containing abitrary key/value pairs, typically used to control external UI behaviour. It is possible to extend UI behaviour (though modification of the UI Velocity templates) and control those extensions by additions to this block. Additional configuration options can be added here without any change in the registry-core.war, they are simply passed through to the Velocity engine. The options supported by the current templates in registry-config-base are:
| Property | Meaning |
|---|---|
| suppressPasswordLogin | If set to true this removes the option to login or register through a password, leaving only the option to register via OAuth/OpenID. See the documentation for how to configure OAuth support. If set to true this also enables an admin menu item to generate temporary API passwords. |
| suppressLinkView | By default the item display shows two columns of information - item property/values and cross-item links. If this config property is set to true the links column is suppressed and the property/value table spans the whole display area. |
| showRegisterAsDatatable | If set to true then the display of register contents is rendered using a client-side datatable component that supports paging, search and reordering. Since this is purely client-side so to be effective the pageSize property of the registry should be set large enough that most registers will fit within one page |
| registryName | Provides a name for the registry that will be used in page titles. This avoids having to adapt the main.vm and header.vm templates when implementing a new registry instance. |
Logging of accesses to the registry is done by the nginx front end proxy. See the nginx configuration documentation.
In addition all requests that reach the registry itself will be logged. For requests from authenticated users the user name and their OpenID will be included, for other requests the originating IP address is (should be, when that's fixed!) recorded. This log will also include messages from internal software components. The registry logging is configured by log4j.properties within the registry application itself. Currently this will log to file (log/registry.log) and to standard output (which means it will be captured by the velocity logging).
The html styling for an registry pages requested in text/html format is handling by the velocity templating system. The builtin registry UI is implemented using these templates. The templates are all stored in the templates/ directory within the operating area. As usual, if this area does not exist at registry start up time then one will be created containing the default templates.
The key templates for customization are:
| Template | Use |
|---|---|
main.vm |
The entry point, the registry will invoke main.vm in response to any request to a registry resource in text/html format. |
header.vm |
Included in all top level templates to provide the HTML header information including page title and loading of style sheets. |
navbar.vm |
Included in all top level templates to render the top level navigation bar. |
footer.vm |
Included in all top level templates to provide the running foot, load all required javascript libraries and close the HTML page. |
root.vm |
Called by the main.vm to render the home page of the registry (root register). |
In addition to the calls of main.vm from the registry itself, requests to the /ui area of the registry are handled specially. They can either directly invoke a UI template or are served as files to provide the style sheets, images and javascript libraries needed for the UI.
| Request URI | Action |
|---|---|
/ui/template |
If a template called template.vm exists then it will be used to render the response, examples include about.vm and login.vm. |
/ui/* |
All other requests fall through to the underlying Tomcat engine, which in turn serves these files from the ui/ directory in the operating area. |
Configuring the document base from which Tomcat will serve the plain UI files is currently done by setting an alias in a META-INF/context.xml file in the web app itself. The configuration of aliases is Tomcat specific so that switching to a different application server would require some minor modifications to the web app.
When a velocity template is invoked the following environment variables will be bound
| Variable | Bound to | Instance of |
|---|---|---|
| $uri | The relative path of the resource being requested | String |
| $registry | The Registry object, access configuration info and invoke API operations from here | com.epimorphics.registry.core.Registry |
| $subject | The security subject, can test if the user is authenticated and check permissions | org.apache.shiro.subject.Subject |
| $requestor | Name of the person making the current request | String |
| $request | The incoming HTTP request | javax.servlet.http.HttpServletRequest |
| $language | The language code for the user's locale | String |