Change Carbon DB into Registry DB and UM DB

pattern-1/README.md

@@ -0,0 +1,132 @@
+# Pivotal Cloud Foundry Resources for WSO2 API Manager
+
+This repository contains resources required to build and install WSO2 API Manager in a Pivotal Cloud Foundry (PCF) environment. This document provides instructions you need to follow in order to deploy a WSO2 API Manager setup on PCF.
+
+## Prerequisites
+Before starting the installation process, ensure that the following prerequisites are completed:
+- A pre-configured Pivotal environment: For more information on the process for setting up the Pivotal environment on AWS, Azure, and GCP, see the documentation provided by Pivotal in [Architecture and Installation Overview](https://docs.pivotal.io/pivotalcf/2-4/installing/index.html).
+- An SQL Database (MySQL or MS SQL): This database should contain the tables required to run API Manager. The database schema required to populate the tables can be found within the <APIM_HOME>/dbscripts/ directory, where <APIM_HOME> refers to the [API Manager Binary](https://wso2.com/api-management/).
+- BOSH [CLI](https://bosh.io/docs/cli-v2/).
+- PCF Tile Generator [CLI](https://docs.pivotal.io/tiledev/2-3/tile-generator.html).
+
+## Setting up for BOSH Release
+The first step in running API Manager on PCF is creating a BOSH release. The following set of instructions should be followed in this process:
+1. Clone the PCF API Manager repository on GitHub by issuing the following command.
+    ```bash
+    git clone https://github.com/wso2/pivotal-cf-apim.git
+    ```
+    API Manager contains five main components named Publisher, Store, Gateway, Traffic Manager, and Key Manager. In a stand-alone APIM setup, these components are deployed in a single server. However, in a typical production setup, they need to be deployed in separate servers for better performance. Installing and configuring each or selected component/s in different servers is known as a distributed setup.
+
+    There are multiple ways in which these distributed setups can be arranged. These are known as [Deployment Patterns](https://docs.wso2.com/display/AM260/Deployment+Patterns#DeploymentPatterns-WSO2APIManagerdeploymentpatterns). The PCF resources for API Manager currently support Pattern 1 and Pattern 1.
+
+    The Deployment Architecture for Pattern 1 is as follows:
+
+    ![pattern-1](images/pattern-1.png "Pattern 1")
+
+2. Add distributions required to run Pattern 1 as follows:
+    1. Navigate to the `pattern-1` directory by issuing the following command.
+        ```bash
+        cd pivotal-cf-apim/pattern-1
+        ls 
+        ```
+        Observe that there are two subdirectories named `bosh-release` and `tile` within this directory.
+    2. Navigate into the `bosh-release` directory to view all of the resources required to deploy API Manager Pattern 1 on PCF.
+    3. To deploy Pattern 1, add the following files to the `dist` subdirectory inside the `bosh-deployment` directory.
+
+        * [mssql-jdbc-7.0.0.jre8.jar](https://www.microsoft.com/en-us/download/details.aspx?id=57175)
+        * [mysql-connector-java-5.1.45-bin.jar](https://dev.mysql.com/downloads/connector/j/)
+        * [OpenJDK8U-jdk_x64_linux_hotspot_8u192b12.tar.gz](https://adoptopenjdk.net/archive.html)
+        * [wso2am-2.6.0.zip](https://wso2.com/api-management/install/)
+        * [wso2is-km-5.7.0.zip](https://wso2.com/api-management/install/key-manager/)
+
+        JDBC Drivers for MySQL and MS SQL are added into the `dist` directory to add flexibility to the deployment. The tile cannot be changed once created, as the tile is immutable. However, options to switch between different databases can be provided.
+
+3. Create the BOSH release
+
+    In order to create the BOSH release, the resource provides two scripts with the deployment. The scripts must be run in the following order:
+    ```bash
+    ./create.sh
+    ```
+
+    This may take up to 20 minutes. After the build is complete, a bosh-release is completed in the root of the `bosh-deployment` directory.  This release pack is named `wso2am-2.6.0-bosh-release.tgz`.
+
+## Building the Pivotal Tile for API Manager
+1. Add the [routing-release](https://github.com/cloudfoundry/routing-release/releases/tag/0.178.0) provided by PCF to the root of the `tile` directory. This delivers HTTP and TCP routing for Cloud Foundry.
+2. Copy the bosh release from the `bosh-release` directory into the `tile` directory by issuing the following command.
+    ```bash
+    cd bosh-release
+    mv wso2am-2.6.0-bosh-release.tgz ../tile
+    cd ../tile
+    ```
+3. Run the build script to build the tile by issuing the following command.
+    ```bash
+    ./build.sh
+    ```
+    The build process takes up to 5 minutes. After the build is complete, a tile named `wso2apim-tile-0.0.1.pivotal` is created in the root of the `/tile/product` directory.
+
+## Install API Manager in PCF
+1. Log in to PCF Ops Manager and upload the tile built by clicking **Import a Product**.
+2. After the tile is uploaded, add the tile to the PCF environment by clicking the + icon next to it.
+3. After the tile is added to the environment, click on the **API Manager** tile in the PCF environment to add configurations to the setup.
+    ![API-Manager](images/new-tile.png "API Manager Tile")
+4. Set up the API Manager tile.
+    1. AZ and Network Assignments Page:
+        ![az-network-assignments](images/az-assignment.png "AZs and Network Assignments")
+        * Place singleton jobs in: Select the AZ in which the API Manager VM needs to run. The broker runs as a singleton job
+        * Balance other jobs in: Select any combination of AZs.
+        * Network: Select pcf-pas-network
+
+        Click Save.
+    2. Registry Database Connection Information:
+        * **JDBC URL**:
+            * **MySQL**: `jdbc:mysql://<hostname>:<port>/<db_name>?autoReconnect=true&amp;useSSL=false`
+            * **MS SQL**: `jdbc:sqlserver://<hostname>:<port>;databaseName=<db_name>;`
+        * **Driver Class Name**: Select the class name of the JDBC driver relevant to the database being used.
+        * **Validation Query**: `SELECT 1`
+        * **Username**: Username for database
+        * **Password**: Password for database
+
+        Click Save.
+    3. User Management Database Connection Information:
+        * **JDBC URL**:
+            * **MySQL**: `jdbc:mysql://<hostname>:<port>/<db_name>?autoReconnect=true&amp;useSSL=false`
+            * **MS SQL**: `jdbc:sqlserver://<hostname>:<port>;databaseName=<db_name>;`
+        * **Driver Class Name**: Select the class name of the JDBC driver relevant to the database being used.
+        * **Validation Query**: `SELECT 1`
+        * **Username**: Username for database
+        * **Password**: Password for database
+
+            Click Save.
+    4. API Manager Database Connection Information:
+        * **JDBC URL**:
+            * **MySQL**: `jdbc:mysql://<hostname>:<port>/<db_name>?autoReconnect=true&amp;useSSL=false`
+            * **MS SQL**: `jdbc:sqlserver://<hostname>:<port>;databaseName=<db_name>;`
+        * **Driver Class Name**: Select the class name of the JDBC driver relevant to the database being used.
+        * **Validation Query**: `SELECT 1`
+        * **Username**: Username for database
+        * **Password**: Password for database
+
+        Click Save.
+    5. API Manager - Analytics Clustering Database connection information
+        * **JDBC URL**:
+            * **MySQL**: `jdbc:mysql://<hostname>:<port>/<db_name>?autoReconnect=true&useSSL=false`
+            * **MS SQL**: `jdbc:sqlserver://<hostname>:<port>;databaseName=<db_name>;`
+        * **Driver Class Name**: Select the class name of the JDBC driver relevant to the database being used.
+        * **Validation Query**: `SELECT 1`
+        * **Username**: Username for database
+        * **Password**: Password for database
+
+        Note that the JDBC URL for MySQL does not contain `&amp;`. Instead, it indicates the `&` symbol. This is due to the fact that the first two configurations save the configuration data in XML format, and `&amp;` is used as an escape character. However, this configuration stores its data in YAML and therefore, an escape character is not required.
+
+        Click Save.
+
+    6. Return to the **Installation Dashboard** in Ops Manager and click **Review Pending Changes**.
+        ![review-changes](images/review-changes.png "Review Pending Changes")
+    7. Select the checkbox for API Manager and click **Apply Changes**.
+        ![apply-changes](images/apply-changes.png "Apply Changes")
+
+        The installation process may take around 25 minutes. After the installation is complete, the management console, publisher, and store can be accessed via the following URLs where domain_name refers to the **domain name** of the PCF environment.
+
+        * ``https://wso2apim.sys.<domain_name>/carbon/``
+        * ``https://wso2apim.sys.<domain_name>/publisher/``
+        * ``https://wso2apim.sys.<domain_name>/store/``

pattern-1/bosh-release/create.sh

@@ -105,70 +105,22 @@ if [ ! -d bosh-deployment ]; then
     git clone https://github.com/cloudfoundry/bosh-deployment bosh-deployment
 fi
 
-# create a directory to hold the configuration files for VirtualBox specific BOSH environment
-if [ ! -d vbox ]; then
-    echo "---> Creating environment directory..."
-    mkdir vbox
-fi
-
-# if forced, delete any existing BOSH environment
-if [ "$1" == "--force" ]; then
-    echo "---> Deleting existing BOSH environment..."
-    bosh delete-env bosh-deployment/bosh.yml \
-        --state vbox/state.json \
-        -o bosh-deployment/virtualbox/cpi.yml \
-        -o bosh-deployment/virtualbox/outbound-network.yml \
-        -o bosh-deployment/bosh-lite.yml \
-        -o bosh-deployment/bosh-lite-runc.yml \
-        -o bosh-deployment/jumpbox-user.yml \
-        --vars-store vbox/creds.yml \
-        -v director_name="Bosh Lite Director" \
-        -v internal_ip=192.168.50.6 \
-        -v internal_gw=192.168.50.1 \
-        -v internal_cidr=192.168.50.0/24 \
-        -v outbound_network_name=NatNetwork
-fi
-
-# create a new BOSH environment with BOSH Lite as the BOSH Director and VirtualBox as the IaaS
-echo "---> Creating the BOSH environment..."
-bosh create-env bosh-deployment/bosh.yml \
-    --state vbox/state.json \
-    -o bosh-deployment/virtualbox/cpi.yml \
-    -o bosh-deployment/virtualbox/outbound-network.yml \
-    -o bosh-deployment/bosh-lite.yml \
-    -o bosh-deployment/bosh-lite-runc.yml \
-    -o bosh-deployment/jumpbox-user.yml \
-    --vars-store vbox/creds.yml \
-    -v director_name="Bosh Lite Director" \
-    -v internal_ip=192.168.50.6 \
-    -v internal_gw=192.168.50.1 \
-    -v internal_cidr=192.168.50.0/24 \
-    -v outbound_network_name=NatNetwork
-
-# set an alias for the created BOSH environment
-echo "---> Setting alias for the environment..."
-bosh -e 192.168.50.6 alias-env vbox --ca-cert <(bosh int vbox/creds.yml --path /director_ssl/ca)
-
-# log into the created BOSH environment
-echo "---> Logging in..."
-bosh -e vbox login --client=admin --client-secret=$(bosh int vbox/creds.yml --path /admin_password)
-
 cd ..
 # add the locally available WSO2 product distribution(s) and dependencies as blobs to the BOSH Director
 echo "---> Adding blobs..."
 
 # add openjdk
-bosh -e vbox add-blob ${distributions}/${jdk_distribution} openjdk/${jdk_distribution}
+bosh add-blob ${distributions}/${jdk_distribution} openjdk/${jdk_distribution}
 # add wso2 product packs
-bosh -e vbox add-blob ${distributions}/${wso2_product_pack_identifier}.zip ${wso2_product}/${wso2_product_pack_identifier}.zip
-bosh -e vbox add-blob ${distributions}/${wso2_product_analytics_pack_identifier}.zip ${wso2_product}_analytics/${wso2_product_analytics_pack_identifier}.zip
+bosh add-blob ${distributions}/${wso2_product_pack_identifier}.zip ${wso2_product}/${wso2_product_pack_identifier}.zip
+bosh add-blob ${distributions}/${wso2_product_analytics_pack_identifier}.zip ${wso2_product}_analytics/${wso2_product_analytics_pack_identifier}.zip
 # add JDBC Drivers
-bosh -e vbox add-blob ${distributions}/${mysql_driver} jdbcdrivers/${mysql_driver}
-bosh -e vbox add-blob ${distributions}/${mssql_driver} jdbcdrivers/${mssql_driver}
+bosh add-blob ${distributions}/${mysql_driver} jdbcdrivers/${mysql_driver}
+bosh add-blob ${distributions}/${mssql_driver} jdbcdrivers/${mssql_driver}
 
 echo "---> Uploading blobs..."
-bosh -e vbox -n upload-blobs
+bosh -n upload-blobs
 
 # create the BOSH release
 echo "---> Creating bosh release..."
-bosh -e vbox create-release --force
+bosh create-release --tarball wso2am-2.6.0-bosh-release.tgz --force

pattern-1/bosh-release/jobs/nfs_server/templates/ctl.erb

@@ -52,7 +52,7 @@ log_debug "Kicking off ctl script as `whoami` with $1"
 dpkg -s nfs-kernel-server >/dev/null 2>&1
 if [ $? -ne 0 ] ; then
   apt-get update
-  apt-get install nfs-kernel-server
+  apt-get install nfs-kernel-server -y
 fi
 
 # create the share directory and set permission

pattern-1/bosh-release/jobs/wso2apim/spec

@@ -46,16 +46,27 @@ properties:
   wso2apim.address:
     description: wso2apim address
 
-  wso2apim.carbon_db.jdbc_url:
-    description: WSO2 Carbon database JDBC URL
-  wso2apim.carbon_db.driver:
-    description: WSO2 Carbon database database driver name
-  wso2apim.carbon_db.query:
-    description: WSO2 Carbon database database validation query
-  wso2apim.carbon_db.username:
-    description: WSO2 Carbon database username
-  wso2apim.carbon_db.password:
-    description: WSO2 Carbon database password
+  wso2apim.reg_db.jdbc_url:
+    description: WSO2 Registry database JDBC URL
+  wso2apim.reg_db.driver:
+    description: WSO2 Registry database database driver name
+  wso2apim.reg_db.query:
+    description: WSO2 Registry database database validation query
+  wso2apim.reg_db.username:
+    description: WSO2 Registry database username
+  wso2apim.reg_db.password:
+    description: WSO2 Registry database password
+
+  wso2apim.um_db.jdbc_url:
+    description: WSO2 User Management database JDBC URL
+  wso2apim.um_db.driver:
+    description: WSO2 User Management database database driver name
+  wso2apim.um_db.query:
+    description: WSO2 User Management database database validation query
+  wso2apim.um_db.username:
+    description: WSO2 User Management database username
+  wso2apim.um_db.password:
+    description: WSO2 User Management database password
 
   wso2apim.am_db.jdbc_url:
     description: WSO2 APIM database JDBC URL

pattern-1/bosh-release/jobs/wso2apim/templates/ctl.erb

@@ -68,7 +68,7 @@ dpkg -s nfs-common >/dev/null 2>&1
 if [ $? -ne 0 ] ; then
   log_debug "Installing NFS Client"
   sudo apt-get update
-  sudo apt-get install nfs-common
+  sudo apt-get install nfs-common -y
 fi
 
 mkdir -p ${NFS_SHARE_SERVER_DIR}

pattern-1/bosh-release/jobs/wso2apim/templates/repository/conf/datasources/master-datasources.xml

@@ -36,14 +36,14 @@
             </jndiConfig>
             <definition type="RDBMS">
                 <configuration>
-                    <url><%= p("wso2apim.carbon_db.jdbc_url") %></url>
-                    <username><%= p("wso2apim.carbon_db.username") %></username>
-                    <password><%= p("wso2apim.carbon_db.password") %></password>
-                    <driverClassName><%= p("wso2apim.carbon_db.driver") %></driverClassName>
+                    <url><%= p("wso2apim.um_db.jdbc_url") %></url>
+                    <username><%= p("wso2apim.um_db.username") %></username>
+                    <password><%= p("wso2apim.um_db.password") %></password>
+                    <driverClassName><%= p("wso2apim.um_db.driver") %></driverClassName>
                     <maxActive>50</maxActive>
                     <maxWait>60000</maxWait>
                     <testOnBorrow>true</testOnBorrow>
-                    <validationQuery><%= p("wso2apim.carbon_db.query") %></validationQuery>
+                    <validationQuery><%= p("wso2apim.um_db.query") %></validationQuery>
                     <validationInterval>30000</validationInterval>
                 </configuration>
             </definition>
@@ -57,14 +57,14 @@
             </jndiConfig>
             <definition type="RDBMS">
                 <configuration>
-                    <url><%= p("wso2apim.carbon_db.jdbc_url") %></url>
-                    <username><%= p("wso2apim.carbon_db.username") %></username>
-                    <password><%= p("wso2apim.carbon_db.password") %></password>
-                    <driverClassName><%= p("wso2apim.carbon_db.driver") %></driverClassName>
+                    <url><%= p("wso2apim.reg_db.jdbc_url") %></url>
+                    <username><%= p("wso2apim.reg_db.username") %></username>
+                    <password><%= p("wso2apim.reg_db.password") %></password>
+                    <driverClassName><%= p("wso2apim.reg_db.driver") %></driverClassName>
                     <maxActive>50</maxActive>
                     <maxWait>60000</maxWait>
                     <testOnBorrow>true</testOnBorrow>
-                    <validationQuery><%= p("wso2apim.carbon_db.query") %></validationQuery>
+                    <validationQuery><%= p("wso2apim.reg_db.query") %></validationQuery>
                     <validationInterval>30000</validationInterval>
                 </configuration>
             </definition>

pattern-1/bosh-release/jobs/wso2apim/templates/repository/conf/registry.xml

@@ -35,7 +35,7 @@
     </dbConfig>
     <remoteInstance url="https://publisher.apim-wso2.com">
         <id>gov</id>
-        <cacheId><%=properties.wso2apim.mysql.username %>@<%= p("wso2apim.carbon_db.jdbc_url") %></cacheId>
+        <cacheId><%= p("wso2apim.reg_db.username") %>@<%= p("wso2apim.reg_db.jdbc_url") %></cacheId>
         <dbConfig>govregistry</dbConfig>
         <readOnly>false</readOnly>
         <enableCache>true</enableCache>

pattern-1/bosh-release/jobs/wso2apim_analytics/spec

@@ -32,28 +32,6 @@ properties:
   wso2apim_analytics.address:
     description: wso2apim address
 
-  wso2apim.carbon_db.jdbc_url:
-    description: WSO2 Carbon JDBC database URL
-  wso2apim.carbon_db.driver:
-    description: WSO2 Carbon database database driver name
-  wso2apim.carbon_db.query:
-    description: WSO2 Carbon database database validation query
-  wso2apim.carbon_db.username:
-    description: WSO2 Carbon database username
-  wso2apim.carbon_db.password:
-    description: WSO2 Carbon database password
-
-  wso2apim.am_db.jdbc_url:
-    description: WSO2 APIM JDBC database URL
-  wso2apim.am_db.driver:
-    description: WSO2 APIM database database driver name
-  wso2apim.am_db.query:
-    description: WSO2 APIM database database validation query
-  wso2apim.am_db.username:
-    description: WSO2 APIM database username
-  wso2apim.am_db.password:
-    description: WSO2 APIM database password
-
   wso2apim.sp_cluster_db.jdbc_url:
     description: WSO2 APIM Analytics database JDBC URL
   wso2apim.sp_cluster_db.driver: