ansible-apim 3.1.0 release changes

CONTRIBUTING.md

@@ -17,19 +17,19 @@ We encourage you to report any problems in the WSO2 Ansible resources or their d
 ### Code Contributions
 If you like to contribute with a bug fix or a new feature, start by posting an issue and discussing the best way to implement it. 
 
-Unlike most projects, development for this repository is carried out on the **2.6.x branch**. This is because the master branch contains the latest stable release of the project. The code in 2.6.x is merged to the master branch after a final review and a round of testing.
+Unlike most projects, development for this repository is carried out on the **3.1.x branch**. This is because the master branch contains the latest stable release of the project. The code in 3.1.x is merged to the master branch after a final review and a round of testing.
 
 Please follow these guidelines when contributing to the code:
 1. Fork the current repository.
-2. Create a topic branch from the 2.6.x branch.
+2. Create a topic branch from the 3.1.x branch.
 3. Make commits in logical units.
 4. Before you send out the pull request, sync your forked repository with a remote repository. This makes your pull request simple and clear.
 
 ```bash
 git clone https://github.com/<user>/ansible-apim.git
 git remote add upstream https://github.com/wso2/ansible-apim.git
 git fetch upstream
-git checkout -b <topic-branch> upstream/2.6.x
+git checkout -b <topic-branch> upstream/3.1.x
 
 # add some work
 

README.md

@@ -9,7 +9,7 @@ This repository contains the Ansible scripts for installing and configuring WSO2
 
 ## Supported Ansible Versions
 
-- Ansible 2.8.0
+- Ansible 2.5 or higher
 
 ## Directory Structure
 ```
@@ -45,12 +45,11 @@ This repository contains the Ansible scripts for installing and configuring WSO2
 │   └── Pattern_5.md
 ├── files
 │   ├── lib
-│   │   ├── amazon-corretto-8.222.10.1-linux-x64.tar.gz
-│   │   └── mysql-connector-java-5.1.47-bin.jar
+│   │   ├── amazon-corretto-8.242.08.1-linux-x64.tar.gz
 │   └── packs
-│   │   ├── wso2am-3.0.0.zip
-│   │   ├── wso2am-analytics-3.0.0.zip
-│   │   └── wso2is-km-5.9.0.zip
+│   │   ├── wso2am-3.1.0.zip
+│   │   ├── wso2am-analytics-3.1.0.zip
+│   │   └── wso2is-km-5.10.0.zip
 │   ├── system
 │   │   └── etc
 │   │       ├── security
@@ -104,14 +103,13 @@ Packs could be either copied to a local directory, or downloaded from a remote l
 
 Copy the following files to `files/packs` directory.
 
-1. [WSO2 API Manager 3.0.0 package](https://wso2.com/api-management/install/)
-2. [WSO2 API Manager Analytics 3.0.0 package](https://wso2.com/api-management/install/analytics/)
-3. [WSO2 API Manager Identity Server as Key Manager 5.9.0 package](https://wso2.com/api-management/install/key-manager/)
+1. [WSO2 API Manager 3.1.0 package](https://wso2.com/api-management/install/)
+2. [WSO2 API Manager Analytics 3.1.0 package](https://wso2.com/api-management/install/analytics/)
+3. [WSO2 API Manager Identity Server as Key Manager 5.10.0 package](https://wso2.com/api-management/install/key-manager/)
 
 Copy the following files to `files/lib` directory.
 
-1. [MySQL Connector/J](https://dev.mysql.com/downloads/connector/j/5.1.html)
-2. [Amazon Corretto for Linux x64 JDK](https://docs.aws.amazon.com/corretto/latest/corretto-8-ug/downloads-list.html)
+1. [Amazon Corretto for Linux x64 JDK](https://docs.aws.amazon.com/corretto/latest/corretto-8-ug/downloads-list.html)
 
 Copy the miscellaneous files to `files/misc` directory. To enable file copying,  uncomment the `misc_file_list` in the yaml files under `group_vars` and add the miscellaneous files to the list.
 
@@ -128,9 +126,11 @@ In **group_vars**, change the values of the following variables in all groups:
 The existing Ansible scripts contain the configurations to set-up a single node WSO2 API Manager pattern. In order to deploy the pattern, you need to replace the `[ip_address]` and `[ssh_user]` given in the `inventory` file under `dev` folder by the IP of the location where you need to host the API Manager. An example is given below.
 ```
 [apim]
-wso2am ansible_host=172.28.128.4 ansible_user=vagrant
+apim_1 ansible_host=172.28.128.4 ansible_user=vagrant
 ```
 
+Update the hostname in dev/host_vars/apim_1.yml with ansible_host, 172.28.128.4
+
 Run the following command to run the scripts.
 
 `ansible-playbook -i dev site.yml`
@@ -172,6 +172,6 @@ System files that will be updated when performance tuning are enabled is availab
 
 The master branch of this repository contains the latest product version with the latest Ansible version. The Ansible resources for previous Ansible versions can be found in the branches. The following is an example.
 
-#### Ansible resources for API Manager 2.6.0 and Ansible 2.7
+#### Ansible resources for API Manager 3.0.0
 
-Branch name: 2.6.0_ansible2.7
+Branch name: 3.0.x

dev/group_vars/apim-analytics.yml

@@ -17,7 +17,7 @@
 wso2_group: wso2
 wso2_user: wso2carbon
 product_name: wso2am-analytics
-product_version: 3.0.0
+product_version: 3.1.0
 target: /mnt
 product_package_location: files
 misc_file_location: "{{ product_package_location }}/misc"
@@ -55,7 +55,7 @@ pack_location: local
 #remote_pack: "<URL_TO_APIM_ANALYTICS_PACK>"
 
 # JDK Distributions
-jdk_name: amazon-corretto-8.222.10.1-linux-x64
+jdk_name: amazon-corretto-8.242.08.1-linux-x64
 java_dir: /opt
 java_symlink: "{{ java_dir }}/java"
 java_home: "{{ java_dir }}/{{ jdk_name }}"

dev/group_vars/apim-is-as-km.yml

@@ -17,7 +17,7 @@
 wso2_group: wso2
 wso2_user: wso2carbon
 product_name: wso2is-km
-product_version: 5.9.0
+product_version: 5.10.0
 target: /mnt
 product_package_location: files
 misc_file_location: "{{ product_package_location }}/misc"
@@ -55,7 +55,7 @@ pack_location: local
 #remote_pack: "<URL_TO_IS_AS_KM_PACK>"
 
 # JDK Distributions
-jdk_name: amazon-corretto-8.222.10.1-linux-x64
+jdk_name: amazon-corretto-8.242.08.1-linux-x64
 java_dir: /opt
 java_symlink: "{{ java_dir }}/java"
 java_home: "{{ java_dir }}/{{ jdk_name }}"
@@ -65,56 +65,17 @@ wso2am_db_type: h2
 wso2am_db_url: jdbc:h2:./repository/database/WSO2AM_DB;DB_CLOSE_ON_EXIT=FALSE;MVCC=TRUE
 wso2am_db_username: wso2carbon
 wso2am_db_password: wso2carbon
-wso2am_db_validation_query: SELECT 1
 
 wso2shared_db_type: h2
 wso2shared_db_url: jdbc:h2:./repository/database/WSO2SHARED_DB;DB_CLOSE_ON_EXIT=FALSE
 wso2shared_db_username: wso2carbon
 wso2shared_db_password: wso2carbon
-wso2shared_db_validation_query: SELECT 1
 
-wso2config_db_type: h2
-wso2config_db_url: jdbc:h2:./repository/database/WSO2SHARED_DB;DB_CLOSE_ON_EXIT=FALSE;MVCC=TRUE
-wso2config_db_username: wso2carbon
-wso2config_db_password: wso2carbon
-wso2config_db_validation_query: SELECT 1
-
-wso2bps_db_type: h2
-wso2bps_db_url: jdbc:h2:file:./repository/database/jpadb;DB_CLOSE_ON_EXIT=FALSE;MVCC=TRUE
-wso2bps_db_username: wso2carbon
-wso2bps_db_password: wso2carbon
-wso2bps_db_driver: org.h2.Driver
-wso2bps_db_validation_query: SELECT 1
-
-wso2consent_db_id: WSO2CONSENT_DB
-wso2consent_db_type: h2
-wso2consent_db_url: jdbc:h2:./repository/database/WSO2AM_DB;DB_CLOSE_ON_EXIT=FALSE;MVCC=TRUE
-wso2consent_db_username: wso2carbon
-wso2consent_db_password: wso2carbon
-wso2consent_db_driver: org.h2.Driver
-wso2consent_db_validation_query: SELECT 1
-
-wso2identity_db_id: WSO2IDENTITY_DB
-wso2identity_db_type: h2
-wso2identity_db_url: jdbc:h2:./repository/database/WSO2AM_DB;DB_CLOSE_ON_EXIT=FALSE;MVCC=TRUE
-wso2identity_db_username: wso2carbon
-wso2identity_db_password: wso2carbon
-wso2identity_db_driver: org.h2.Driver
-wso2identity_db_validation_query: SELECT 1
-
-# Key Store and Trust Store configuration
+# Key Store configuration
 key_store_location: wso2carbon.jks
 key_store_password: wso2carbon
 key_store_key_password: wso2carbon
 key_store_key_alias: wso2carbon
-trust_store_location: client-truststore.jks
-trust_store_password: wso2carbon
-
-# The KeyStore which is used for encrypting/decrypting internal data. This block is read by Carbon Crypto Service
-internal_keystore_location: wso2carbon.jks
-internal_keystore_key_password: wso2carbon
-internal_keystore_password: wso2carbon
-internal_keystore_key_alias: wso2carbon
 
 # Traffic Manager configurations
 traffic_manager_url: 'tcp://localhost:9611'

dev/group_vars/apim.yml

@@ -17,7 +17,7 @@
 wso2_group: wso2
 wso2_user: wso2carbon
 product_name: wso2am
-product_version: 3.0.0
+product_version: 3.1.0
 target: /mnt
 product_package_location: files
 backup_dir: /tmp
@@ -59,7 +59,7 @@ pack_location: local
 #remote_pack: "<URL_TO_APIM_PACK>"
 
 # JDK Distributions
-jdk_name: amazon-corretto-8.222.10.1-linux-x64
+jdk_name: amazon-corretto-8.242.08.1-linux-x64
 java_dir: /opt
 java_symlink: "{{ java_dir }}/java"
 java_home: "{{ java_dir }}/{{ jdk_name }}"
@@ -130,13 +130,11 @@ wso2am_db_type: h2
 wso2am_db_url: jdbc:h2:./repository/database/WSO2AM_DB;DB_CLOSE_ON_EXIT=FALSE
 wso2am_db_username: wso2carbon
 wso2am_db_password: wso2carbon
-wso2am_db_validation_query: SELECT 1
 
 wso2shared_db_type: h2
 wso2shared_db_url: jdbc:h2:./repository/database/WSO2SHARED_DB;DB_CLOSE_ON_EXIT=FALSE
 wso2shared_db_username: wso2carbon
 wso2shared_db_password: wso2carbon
-wso2shared_db_validation_query: SELECT 1
 
 # Key Store and Trust Store configuration
 key_store_location: wso2carbon.jks

dev/host_vars/apim-analytics-dashboard_1.yml

@@ -25,7 +25,8 @@ thrift_data_receiver_ssl_port: 7711
 binary_data_receiver_tcp_port: 9611
 binary_data_receiver_ssl_port: 9711
 
-transport_listner_config_port: 9643
+transport_listner_config_port: 9089
+transport_listner_config_https_port: 9643
 
 # worker configurations
 # ip: ip of the worker node

dev/host_vars/apim-is-as-km_1.yml

@@ -18,7 +18,6 @@
 # requirements. If you need to add additional configurations, please use the custom.yml under each role.
 
 hostname: localhost
-ports_offset: 0
 
 # Auth Manager configurations
 admin_username: admin

dev/inventory

@@ -1,14 +1,14 @@
 [apim]
 apim_1 ansible_host=[ip_address] ansible_user=[ssh_user]
+
 # Example
+# [apim]
 # apim_1 ansible_host=172.28.128.4 ansible_user=vagrant
 
-[apim-analytics]
-apim-analytics-worker_1 ansible_host=[ip_address] ansible_user=[ssh_user]
 # Example
+# [apim-analytics]
 # apim-analytics-worker_1 ansible_host=172.28.128.4 ansible_user=vagrant
 
-[apim-is-as-km]
-apim-is-as-km_1 ansible_host=[ip_address] ansible_user=[ssh_user]
 # Example
+# [apim-is-as-km]
 # apim_is_as_km_1 ansible_host=172.28.128.4 ansible_user=vagrant

roles/apim-analytics-dashboard/templates/carbon-home/conf/dashboard/deployment.yaml.j2

@@ -22,7 +22,7 @@ wso2.carbon:
     # server name
   name: WSO2 API Manager Analytics Server
     # enable/disable hostname verifier
-  hostnameVerificationEnabled: true
+  hostnameVerificationEnabled: false
     # ports used by this server
   ports:
       # port offset
@@ -281,7 +281,7 @@ wso2.datasources:
         idleTimeout: 60000
         connectionTestQuery: {{ wso2_permissions_db_test_query }}
         validationTimeout: 30000
-        isAutoCommit: false
+        isAutoCommit: false 
 
   #Data source for APIM Analytics
   - name: APIM_ANALYTICS_DB
@@ -351,23 +351,31 @@ wso2.transport.http:
       value: true
 
   listenerConfigurations:
-    - id: "default-https"
+    - id: "default"
       host: "{{ hostname }}"
       port: {{ transport_listner_config_port }}
+
+    - id: "default-https"
+      host: "{{ hostname }}"
+      port: {{ transport_listner_config_https_port }}
       scheme: https
       keyStoreFile: "{{ key_store_location }}"
       keyStorePassword: {{ key_store_password }}
       certPass: {{ cert_password }}
 
 ## Dashboard data provider authorization
 data.provider.configs:
-  authorizingClass: org.wso2.carbon.dashboards.core.DashboardDataProviderAuthorizer
+  authorizingClass: org.wso2.analytics.apim.dashboards.core.data.provider.Authorizer
 
 ## Additional APIs that needs to be added to the server.
 ## Should be provided as a key value pairs { API context path: Microservice implementation class }
 ## The configured APIs will be available as https://{host}:{port}/analytics-dashboard/{API_context_path}
 additional.apis:
   /apis/analytics/v1.0/apim: org.wso2.analytics.apim.rest.api.proxy.ApimApi
+  /apis/v1.0/report: org.wso2.analytics.apim.rest.api.report.ReportApi
+
+report:
+  implClass: org.wso2.analytics.apim.rest.api.report.reportgen.DefaultReportGeneratorImpl
 
 ## Authentication configuration
 auth.configs:
@@ -376,23 +384,33 @@ auth.configs:
   properties:
     adminScope: apim_analytics:admin_carbon.super
     allScopes: apim_analytics:admin apim_analytics:product_manager apim_analytics:api_developer apim_analytics:app_developer apim_analytics:devops_engineer apim_analytics:analytics_viewer apim_analytics:everyone openid apim:api_view apim:subscribe
-    adminServiceBaseUrl: {{ admin_service_url }}
     adminUsername: {{ admin_service_username }}
     adminPassword: {{ admin_service_password }}
-    kmDcrUrl: {{ admin_service_url }}/client-registration/v0.15/register
-    kmTokenUrlForRedirection: {{ admin_service_url }}:9443/oauth2
+    kmDcrUrl: {{ admin_service_url }}/client-registration/v0.16/register
+    kmTokenUrlForRedirection: {{ admin_service_url }}/oauth2
     kmTokenUrl: {{ admin_service_url }}/oauth2
     kmUsername: {{ keymanager_username }}
     kmPassword: {{ keymanager_password }}
     portalAppContext: analytics-dashboard
     businessRulesAppContext : business-rules
-    cacheTimeout: 900
+    cacheTimeout: 30
     baseUrl: https://localhost:9643
     grantType: authorization_code
     publisherUrl: {{ admin_service_url }}
-    #storeUrl: {{ admin_service_url }}
+    devPortalUrl: {{ admin_service_url }}
+    externalLogoutUrl: {{ admin_service_url }}/oidc/logout
 
 wso2.dashboard:
   roles:
     creators:
       - apim_analytics:admin_carbon.super
+  themeConfigProviderClass: org.wso2.carbon.dashboards.core.DefaultDashboardThemeConfigProvider
+
+## RDBMS Data Provider configuration
+wso2.rdbms.data.provider:
+  timeTypes:
+    - DATE
+    - TIME
+    - DATETIME
+    - TIMESTAMP
+    - TIMESTAMP WITHOUT TIME ZONE

roles/apim-analytics-dashboard/templates/carbon-home/wso2/dashboard/bin/carbon.sh.j2

@@ -304,6 +304,7 @@ do
     -Djavax.net.ssl.keyStorePassword="wso2carbon" \
     -Djavax.net.ssl.trustStore="$CARBON_HOME/resources/security/client-truststore.jks" \
     -Djavax.net.ssl.trustStorePassword="wso2carbon" \
+    -Dio.netty.tryReflectionSetAccessible=false \
     org.wso2.carbon.launcher.Main $*
     status=$?
 done

roles/apim-analytics-dashboard/templates/wso2am-analytics-dashboard.service.j2

@@ -5,7 +5,7 @@ After=network.target
 [Service]
 ExecStart={{ carbon_home }}/wso2/dashboard/bin/carbon.sh start
 ExecStop={{ carbon_home }}/wso2/dashboard/bin/carbon.sh stop
-ExecRestart={{ carbon_home }}/wso2/dashboard/bin/carbon.sh restart
+ExecReload={{ carbon_home }}/wso2/dashboard/bin/carbon.sh restart
 PIDFile={{ carbon_home }}/wso2/dashboard/runtime.pid
 User={{ wso2_user }}
 Group={{ wso2_group }}