Part 7: Finishing Tasks

Finishing tasks consists of applying any customizations that may be needed in your environment.

In our case, we do the following “Branding” change for the <TARGET_DB_SID> environment:

Default Branding Change

• Login page Message (on the page)

cd $OA_MEDIA
mv oracle_white_logo_CUSTOM.png oracle_white_logo.png

Start the application tier processes.

• Site Name Profile Change

Login > System Administrator > Profile System

Search for the Site Name profile.

Update the site level profile option to: <TARGET_DB_SID> – Clone of <SOURCE DB SID> as of XX/XX/XX

Close the forms session, then connect again to validate the changes took effect.

Log File Locations

Different logs are created for Run Edition and Patch Edition Cloning. The log files are created in the following directories in the Run Edition File System:

<INST_TOP>/admin/log/clone/run

<INST_TOP>/admin/log/clone/patch

Potential Issues

• Per the instructions in MOS Doc. ID: 1383621.1, Cloning Oracle E-Business Suite Release 12.2 with Rapid Clone:

If the database version is 12c Release 1, you must add the following line in your sqlnet_ifile.ora after adcfgclone.pl execution completes:

SQLNET.ALLOWED_LOGON_VERSION_SERVER = 8 (if the initialization parameter SEC_CASE_SENSITIVE_LOGON is set to FALSE)

SQLNET.ALLOWED_LOGON_VERSION_SERVER = 10 (if SEC_CASE_SENSITIVE_LOGON is set to TRUE)

• Don’t use a HOSTNAME in UPPER CASE.

Bug 23343062 – Incorrect Patch File System context file with dualfs option when hostname is in uppercase

On a host with uppercase name, the dualfs option of Cloning may result in the context name and context file name being out of sync between the Run and Patch File System. This can happen if the context name or other context variables containing hostname are specified in the pairsfile with the host value in upper case. For example, on host TEST_HOST with database SID prod, if the pairsfile contains the entry s_contextname=prod_TEST_HOST, the Run File System context name and context file name will be set to prod_TEST_HOST and prod_TEST_HOST.xml respectively while the Patch File System context name and context file name will be set to prod_test_host and prod_test_host.xml respectively. This will lead to validation errors during adop execution. To avoid this issue, in case the s_contextname or other context variables containing the hostname are specified in the pairsfile, the specified value should always have the host in lower case.

• One recent change that came from either a CPU patch or AD-TXK patch affects cloning. You now need to create an etcc directory under the database Oracle Home, $ORACLE_HOME/appsutil/etcc.

From MOS Doc. ID: 1383621.1, Cloning Oracle E-Business Suite Release 12.2 with Rapid Clone:

“Run EBS Technology Codelevel Checker (ETCC) on the database tier.

On the database tier, run the EBS Technology Codelevel Checker (ETCC) described in MOS Doc. ID: 1594274.1, Oracle E-Business Suite Release 12.2: Consolidated List of Patches and Technology Bug Fixes to confirm that all required database patches have been applied. EBS Technology Codelevel Checker (ETCC) is available via Patch 17537119 and analyzes an Oracle Database Oracle Home, and warns of any missing database bug fixes required for Oracle E-Business Suite Release 12.2. It is run with the command checkDBpatch.sh (on UNIX) or checkDBpatch.cmd (on Windows). Further instructions are available in the patch readme. Ensure to install the latest version of ETCC into the directory:

 <RDBMS_ORACLE_HOME>/appsutil/etcc

Cloning/Patching Nuances

1. Before cloning a system with Rapid Clone, be sure to allow any active online patching cycles to run all the way through the final (cleanup) phase. In case patches are applied in Hotpatch or Downtime Mode, then you must run the cleanup phase of adop. Review Oracle E-Business Suite Maintenance Guide.
2. Then run fs_clone to synchronize with the other file system, to avoid the need for synchronization to be performed in the next patching cycle. For more information, review Oracle E-Business Suite Maintenance Guide.
3. A global (central) inventory is generally recommended for all Oracle E-Business Suite Release 12.2 application tier nodes and database tier nodes.

6.2 RUN ADPRECLONE ON RUN EDITION 

Run adpreclone.pl on the run edition file system in the Target System.

$ cd $ADMIN_SCRIPTS_HOME

$ adpreclone.pl appsTier

**require Apps User password and Weblogic AdminServer password

6.3 SHUTDOWN THE APPLICATION TIER PROCESSES

cd $ADMIN_SCRIPTS_HOME

./adstpall.sh apps/<apps_password>
**require Apps User password and Weblogic AdminServer password

6.4 Copy Target Run Edition Over Target Patch Edition

Now we need to copy the EBSapps directory from the run edition file system to the patch edition file system on the target application server.

Create the TARGET PATCH EDITION file system (in our case $ORACLE_HOME/<TARGET_DB_SID>/fs2), if it does not already exist:

cd $ORACLE_BASE
mkdir fs2
cd ./fs1
tar –cvf EBSapps.tar EBSapps
mv EBSapps.tar ../fs2
cd ../fs2
tar –xvf EBSapps.tar

Once this is complete, remove the TAR file:
rm –fr EBSapps.tar

IMPORTANT! – Unset the application environment if it set. This is done by logging out of the OS and logging back in making sure not to set the EBS environment.

$ cd $ORACLE_HOME/<TARGET_DB_SID>/fs2

6.5 Run ADCFGCLONE on Target Application Node on Patch Edition

The patch edition file system uses the RUN edition context XML file. At the prompt “Location of Run System Context File”, enter the absolute path to the context file for the run edition file system that was created in the previous step:

In our environment, the path is:
$ORACLE_HOME/<TARGET_DB_SID>/fs1/inst/apps/<TARGET_DB_SID>_apps1/appl/admin/<TARGET_DB_SID>_apps1.xml

Also the PATCH file system port pool MUST be different than RUN file system port pool (0).

Log on to the patch edition file system in the Target System as the applmgr user and enter the following commands:

$ cd $ORACLE_HOME/<TARGET_DB_SID>/fs2/EBSapps/

comn/clone/bin

$ perl adcfgclone.pl appsTier

                   Copyright (c) 2002, 2015 Oracle Corporation

             Redwood Shores, California, USA

          Oracle E-Business Suite Rapid Clone

                   Version 12.2

          adcfgclone Version 120.63.12020000.56

                ***********************************************************

In AD-TXK Delta 7, we recommend you clone the run and patch file systems in a single operation using the ‘dualfs’ option.

Separate cloning of the run and patch file systems will be deprecated

                ************************************************************

Enter the APPS password :

Enter the Weblogic AdminServer password :

Do you want to add a node (yes/no) [no] :

Running: Context clone…

Log file located at $ORACLE_HOME/<TARGET_DB_SID>/fs2/EBSapps/comn/clone/bin/CloneContext_1230145305.log

Target System File Edition type [run] : patch

Enter the full path of Run File System Context file : $ORACLE_HOME/<TARGET_DB_SID>/fs1/inst/apps/<TARGET_DB_SID>_apps1/appl/admin/<TARGET_DB_SID>_apps1.xml

Provide the values required for creation of the new APPL_TOP Context file.

Target System Fusion Middleware Home set to $ORACLE_HOME/<TARGET_DB_SID>/fs2/FMW_Home

Target System Web Oracle Home set to $ORACLE_HOME/<TARGET_DB_SID>/fs2/FMW_Home/webtier

Target System Appl TOP set to $ORACLE_HOME/<TARGET_DB_SID>/fs2/EBSapps/appl

Target System COMMON TOP set to $ORACLE_HOME/<TARGET_DB_SID>/fs2/EBSapps/comn

Target System Instance Top set to $ORACLE_HOME/<TARGET_DB_SID>/fs2/inst/apps/<TARGET_DB_SID>_apps1

Target System Port Pool [0-99] : 1

Checking the port pool 1

done: Port Pool 1 is free

UTL_FILE_DIR on database tier consists of the following directories.

1. /usr/tmp

2. /usr/tmp

3. $ORACLE_HOME/<TARGET_DB_SID>/12.1.0/appsutil/

outbound/<TARGET_DB_SID>_data1

4. /usr/tmp

Choose a value which will be set as APPLPTMP value on the target node [1] : 1

6.6 Start Application from Run File System and Review

Set RUN file system environment:

. ~/EBSapps.env

Make sure to select the RUN file system when prompted.

Start Application:

$ cd $ADMIN_SCRIPTS_HOME

$ ./ adstrtal.sh

**require Apps User password and Weblogic AdminServer password

From EBS side you can do these checks:

1. Login to the application. Check that Apache, forms sessions are coming up.
2. Run Synchronize views for Oracle Workflow

Request Name = Workflow Directory Services User/Role Validation

p_BatchSize = 10000

p_Check_Dangling = Yes

Add missing user/role assignments = Yes

Update WHO columns in WF tables = No

• Check Concurrent Manager processes.
• Run a simple Concurrent Program (like the Active Users report). Check if output and log files can be opened.
• If required, purge old log/out files from $APPLCSF/$APPLLOG and $APPLCSF/$APPLOUT directories as you may find that the output/log files have old dates.

Part 4: Copy Database Tier Node

For copying the database tier node from source to target database you can always follow the easiest approach of copying the database files from source to target database by first shutting down the source database system. But not all DBAs have the luxury of taking down their Production database at will to refresh a non-production database.

4.1 Stop Target Database – If Required

Login to the target Database node as the oracle software user.

$ sqlplus / as sysdba

SQL> shutdown immediate

4.2 Cleanup Target Database File

Delete all target data files.

Use a simple rm –rf command to delete these database files

4.3 Restore Your Production Database to the Target Database Server Using the Method of Your Choice

Part 5: Configure Target Database System

Log on to the Target System as the oracle user and enter the following commands to configure the adcfgclone dbTechStack

$ cd [RDBMS ORACLE_HOME]/appsutil/clone/bin

$ perl adcfgclone.pl dbTechStack

The log file is created in the <RDBMS_ORACLE_HOME>/appsutil/log/<CONTEXT_NAME> directory.

Answer the adcfgclone script questions and allow the script to run to completion.

5.1 De-Register the Production Nodes from the Configuration

As the APPS user, run the following command to de-register the current configuration:

Login from the target database node as the APPS user:

$ sqlplus apps/<PASSWORD>

SQL> exec fnd_conc_clone.setup_clean;

PL/SQL procedure successfully completed.

SQL> commit;

Commit complete.

5.2 Execute Autoconfig on Target Database Node

On the target database node from the $ORACLE_HOME/appsutil/bin directory, execute AutoConfig on the database tier by running the adconfig.pl script.

$ cd $ORACLE_HOME/appsutil/bin
$ perl adconfig.pl dbTier

Check the AutoConfig log for any errors.

Also check the CONTEXT_FILE that was generated under $ORACLE_HOME/appsutil

Part 6: Configure Target Application System

Configuring the target system changes instance settings from the source settings to the target settings that match the target hostname, target instance name, SID, paths and ports.

NOTES:

• At the prompt Do you want to add a node (yes/no), enter the value no.
• At the prompt: Target System Base Directory, enter the location of the base directory. For example: /u01/oracle/VIS.
• Provide the new port pools for the Run Edition File System and the Patch Edition File System.
• When asked the question Do you want to startup the Application Services for <TWO_TASK>? (y/n) you should answer y if you do not need to perform any further actions and n if there are other pending actions that need the Application services to be down.

6.1 Run ADCFGCLONE on Target Application Node Run Edition

Login to the Application node as the OS Apps user.

$ cd $COMMON_TOP/clone/bin

Example :

$ cd $ORACLE_HOME/<TARGET_DB_SID>/fs1/EBSapps/comn/

clone/bin

$ perl adcfgclone.pl appsTier

**This step requires the Apps User password AND the Weblogic Server Admin password

perl adcfgclone.pl appsTier

      Copyright (c) 2002, 2015 Oracle Corporation

             Redwood Shores, California, USA

            Oracle E-Business Suite Rapid Clone

                      Version 12.2

         adcfgclone Version 120.63.12020000.56

               ****************************************************

In AD-TXK Delta 7, we recommend you clone the run and patch file systems in a single operation using the ‘dualfs’ option.

Separate cloning of the run and patch file systems will be deprecated

                ***************************************************

Enter the APPS password :

Enter the Weblogic AdminServer password :

Do you want to add a node (yes/no) [no] :

Running: Context clone…

Log file located at $ORACLE_HOME/<TARGET_DB_SID>/fs1/EBSapps/comn/clone/bin/CloneContext_1230123711.log

Target System File Edition type [run] :

Provide the values required for creation of the new APPL_TOP Context file.

Target System Hostname (virtual or normal) [apps1] :

Target System Database SID : <TARGET_DB_SID>

Target System Database Server Node [apps1] : data1

Target System Database Domain Name [dg.local] :

Target System Base Directory : $ORACLE_HOME/<TARGET_DB_SID>

Target System Base Directory set to $ORACLE_HOME/<TARGET_DB_SID>

Target System Current File System Base set to $ORACLE_HOME/<TARGET_DB_SID>/fs1

Target System Other File System Base set to $ORACLE_HOME/<TARGET_DB_SID>/fs2

Target System Fusion Middleware Home set to $ORACLE_HOME/<TARGET_DB_SID>/fs1/FMW_Home

Target System Web Oracle Home set to $ORACLE_HOME/<TARGET_DB_SID>/fs1/FMW_Home/webtier

Target System Appl TOP set to $ORACLE_HOME/<TARGET_DB_SID>/fs1/EBSapps/appl

Target System COMMON TOP set to $ORACLE_HOME/<TARGET_DB_SID>/fs1/EBSapps/comn

Target System Instance Home Directory [$ORACLE_HOME/<TARGET_DB_SID>] :

Target System Instance Top set to $ORACLE_HOME/<TARGET_DB_SID>/fs1/inst/apps/<TARGET_DB_SID>_apps1

Do you want to preserve the Display [<Target_App_Server>:0.0] (y/n)  : n

Target System Display [apps1:0.0] : :1

Target System Root Service [enabled] :

Target System Web Entry Point Services [enabled] :

Target System Web Application Services [enabled] :

Target System Batch Processing Services [enabled] :

Target System Other Services [disabled] :

Do you want the target system to have the same port values as the source system (y/n) [y] ? : n

Target System Port Pool [0-99] : 0

Checking the port pool 0

done: Port Pool 0 is free

Report file located at $ORACLE_HOME/<TARGET_DB_SID>/fs1/inst/apps/<TARGET_DB_SID>_apps1/admin/out/portpool.lst

UTL_FILE_DIR on database tier consists of the following directories.

1. /usr/tmp

2. /usr/tmp

3. $ORACLE_HOME/<TARGET_DB_SID>/12.1.0/appsutil/outbound/<TARGET_DB_SID>_data1

4. /usr/tmp

Choose a value which will be set as APPLPTMP value on the target node [1] : 1

Part 3: Copy Application Node Data

The Application tier copy process is different in R12.2 compared to a R12.1.3 environment. In R12.2 we have two file systems, RUN and PATCH. During the copy process, we only need to copy the RUN file system.

3.1 Clone and Configure the Application Tier

Use the following instructions from MOS Doc. ID: 2552208.1, Cloning Oracle E-Business Suite Release 12.2 with Multitenant Database using Rapid Clone to clone and configure the application tier.

3.1.1 Copy the Application Tier File System from the Source Node to the Target Node:

Copy the application tier file system from the source node to the target node by running the following steps in the order listed. Ensure the application tier files copied to the target system are owned by the target applmgr user.

Before you begin, verify the permissions of the executables under ORACLE_HOME/bin that can potentially be owned by root, such as nmo, nmhs, and nmb and change the owner to the target applmgr user.

Note: In the copying tasks below, UNIX users should ensure that the symbolic links (soft links) are preserved when copying. On most UNIX platforms, you can accomplish this with the cp -prfH command. Consult the UNIX man page for the cp command to check the parameters available on your platform.

For example:
$ cd /Target_dest_dir/apps
$ cp -prfH /Source_dir/apps/.

Alternatively, you can use the tar command to compress the directories into a temporary staging area. UNIX users should ensure that the symbolic links (soft links) are preserved when compressing. On most UNIX platforms, this is the default behavior of the tar command. Consult the UNIX main page for the tar command to review the parameters available for your platform.

1. Log on to run edition file system in the source system application tier nodes as the applmgr user.
2. Copy the following application tier directories from the source node to the target run edition file system application tier node:

• <APPL_TOP>
• <COMMON_TOP>
• <OracleAS Tools 10.1.2 ORACLE_HOME>

The same operating system user must own both the run edition and patch edition file systems.
 

Note: In Release 12.2, you can set the base directory to any desired location. However, the subdirectory structure cannot be changed because of dependencies on both the WLS domain and the dual file system required for online patching. Also, the base directory must be the same across all nodes in multi-node configurations.

3.1.2 Dual File System Directory Structure

In Oracle E-Business Suite Release 12.2, the following directory structure exists to support the run edition and patch edition file systems:

Note: <s_base> and <sid> are user-defined values.

<s_base>/<sid>/fs1 (for example, /u01/122/prod/fs1)

<s_base>/<sid>/fs2 (for example, /u01/122/prod/fs2)

Two environment variables, $RUN_BASE and $PATCH_BASE, store these locations. The role (run or patch) of these two file systems switches every time a cutover phase completes.

Note: As Rapid Clone will create a replica of the source node, if the source run edition file system is the first file system (fs1), the target run edition file system will also be the first file system (fs1). Similarly, if the source run edition file system is the second file system (fs2), then the target run edition file system will also be the second file system (fs2). Therefore, when you perform a cloning task, you always clone and copy the source run edition file system to create the target run edition file system, but the directory location of the run edition file system can be pointing either to <s_base>/<sid>/fs1 or <s_base>/<sid>/fs2 based on the source run edition file system base directory.

When copying the files, use the values of $RUN_BASE and $PATCH_BASE variables to determine if the run edition file system should be copied to fs1 or fs2.

For example, the source run edition file system has the following values:

$RUN_BASE=/u01/122/prod/fs2
$PATCH_BASE=/u01/122/prod/fs1

The target <s_base> directory will be /d05/test. Copy the source run edition file system into the target /d05/test/fs2 directory to initially act as run edition file system.

3.2 Configure the Target System Application Tier Nodes

Note: If adcfgclone.pl is being run again after a failure, before a new cloning attempt you must run the steps in Appendix A of MOS Doc. ID:1383621.1, Cloning Oracle E-Business Suite Release 12.2 with Rapid Clone.

1. Log on to the run edition file system in the target system as the applmgr user and enter the following commands:

$ cd <COMMON_TOP>/clone/bin
$ perl adcfgclone.pl appsTier dualfs

• When prompted for the following, answer as shown:

• “Do you want to add a node (yes/no)”, enter a value of ‘no’.
• Target System Database SID, enter the name of the EBS pluggable database.
• “Target System Base Directory”, enter the location of the base directory. For example: /u02/r122.
• Provide the new port pools for the run edition file system and the patch edition file system.
• “Do you want to startup the Application Services for <TWO_TASK>? (y/n)” you should answer ‘y‘ if you do not need to perform any further actions and ‘n‘ if there are other pending actions which need the application services to be down.

Different logs are created for run edition and patch edition cloning. Log files are created in the following directories in the run edition file system:

• <INST_TOP>/admin/log/clone/run
• <INST_TOP>/admin/log/clone/patch
• If the April 2019 CPU patch (or later) is applied to the Oracle E-Business Suite instance, refer to Appendix B of MOS Doc. ID: 1383621.1, Cloning Oracle E-Business Suite Release 12.2 with Rapid Clone, to enable access to the WebLogic Admin console from hosts other than the application tier nodes.
• To complete the configuration on the primary application tier node, run the finishing tasks as listed in Section 4 of MOS Doc. ID: 1383621.1, Cloning Oracle E-Business Suite Release 12.2 with Rapid Clone.
• If you want to add secondary application tier nodes, run the steps in Section 5.3 of MOS Doc. ID: 1383621.1, Cloning Oracle E-Business Suite Release 12.2 with Rapid Clone.

3.3 Source Application Tier Data Copy Process

• Copy the Application Tier File System from the Source Run Edition File System to the Target Run Edition File System.
• Log on to the Run Edition File System on the Source System application tier nodes as the applmgr user.
• The next step is important: only copy the directories listed. Copy the following application tier directories from the Source Node to the Target Run Edition File System application tier node:

The EBSapps directory contains:

<APPL_TOP>
<COMMON_TOP>
<OracleAS Tools 10.1.2 ORACLE_HOME>

The EBSapps directory needs to be copied to the target file system ($RUN_BASE).

[<OS Apps User>@<Target_App_Server> fs1]$ pwd

$ORACLE_HOME/<SOURCE DB SID>/fs1

[<OS Apps User>@<Target_App_Server> fs1]$ ll

total 12

drwxrwxr-x.  5 <OS Apps User> dba 4096 Oct 19  2016 EBSapps      ßCOPY THIS FILE SYSTEM

drwxrwxr-x. 11 <OS Apps User> dba 4096 Dec 25  2016 FMW_Home           ß DO NOT COPY

drwxrwxr-x.  3 <OS Apps User> dba 4096 Oct 19  2016 inst               ß DO NOT COPY

[<OS Apps User>@<Target_App_Server> fs1]$

3.4 Cleanup Target Application File System

• Remove all data under the application directory structure on the target if required.
• COPY APPS FILESYSTEM BACKUP FROM SOURCE TO TARGET
• RESTORE THE APPS FILESYSTEM ON TARGET NODE
• Extract the backup to the target destination

Example: (If using the COLD BACK-UP method (Linux tar command)

$ nohup tar xvfz /tmp/dba_apps_22apr13.tar.gz &