Wrong ELF class: ELFCLASS64 after installing R1211 with rapidwiz

After installing R1211 with rapidwiz, I tried to run admrgpch to merge the R1211 preinstall=y patches listed in an earlier post. I received an error: admrgpch: fatal: libclntsh.so.10.1: wrong  ELF class: ELFCLASS64.  I found a related note on MOS, R12 – adcfgclone appsTier fails with error for ld.so.1 ‘wrong ELF class: ELFCLASS64’ [ID 1078123.1]. However, this did not fix the problem, but gave me a clue to what was wrong. It turns out the issue was the oraInventory for the Apps tier was not write-able by the applmgr user. I copied the old central oraInventory to oraInventoryback and then I made sure the oraInst.loc had the inst_group=dba. I changed the location of the oraInventory in all the Apps tier oraInst.loc locations to a write-able directory and reinstalled the Apps tier with rapidwiz. This solved the problem.

R12.1.1 preinstall=y Patches to Update the 1211 filesystem before the Upgrade

Apply only the patches for modules that are installed and in use. You may not need some of these patches.

7303029 Oracle E-Business Suite Consolidated Upgrade Patch 1 (CUP1) for R12.1.1  
7648869 RCA: SUPPLIER CONTACT DETAILS MISSING SOME TITLE IN R12 FORM AFTER UPGRADE. Payables
8230656 RCA BUG FOR BUG 7705629 Payments
8495719 EVENT CLASS OPTIONS ARE NOT MIGRATED FOR PAYABLES E-Business Tax
8764069 POST USERMIGRATE TO HASH PASSWORDS, AFTER 12.1 UPG, FNDCPASS FAILS DECRYPT Application Object Library
8781600 CFIX : CHANGE TRANSACTION_DATE IN XLA_EVENTS DURING AX UPGRADE Subledger Accounting
8781600 CFIX : CHANGE TRANSACTION_DATE IN XLA_EVENTS DURING AX UPGRADE Subledger Accounting
8850587 R12 UPGRAGE DEFAULT_PAYMENT_METHOD_CODE SETS TO WRONG VALUE Payments
8850587 R12 UPGRAGE DEFAULT_PAYMENT_METHOD_CODE SETS TO WRONG VALUE Payments
9082021 WRONG TAX CALCULATION FOR MIGRATED COMPOUNDED TAXES-CANADA E-Business Tax
9202645 UPGRADE SCRIPT ZXEXEMPTMIG1.SQL LONG RUNNING E-Business Tax
9290222 Some rules created from tax group migration are incorrectly enabled E-Business Tax
9560244 DURING UPGRADE TO R12, SCRIPT 5408422.SQL IS TAKING 1,5 HOURS Item Master
9726737 RCA: REVERSED TIPV AND NREC AMOUNTS ARE GOING WRONG WHILE UPGRADING Payables
9903933 ZXGEOTORATESMIG8.SQL FAILS IN 12.1.1 UPGRADE E-Business Tax
9918101 RCA:RECURRING INV TEMPLATE UPGRADE FROM 11I DOESN’T UPDATE EXT_BANK_ACCOUNT Payables
10011700 RCA:NEGATIVE CCIDS FOR INTRACOMPANY RULES UPGRADE FROM 11I TO R12 Financials Common Modules
10029457 TAX CODES WHICH ARE USED IN TAX GROUPS UPGRADED FROM 11I TO R12 UNDER DIFFERENT REGIME E-Business Tax
10041471 SKIP R12 UPGRADE CASH ACCOUNTING METHOD MIGRATION SCRIPTS ON ACCRUAL Receivables
10041471 SKIP R12 UPGRADE CASH ACCOUNTING METHOD MIGRATION SCRIPTS ON ACCRUAL Receivables
10163624 EMEA VAT UPG: JGZZVATUPGTRXDET.SQL TOOK 8+ HRS (PRE INSTALL) Regional Localizations
10163753 APPSST11202: BIV_B_AGE_H_SUM_MV FAILS DURING INDEX CREATION Service Intelligence
10163753 APPSST11202: BIV_B_AGE_H_SUM_MV FAILS DURING INDEX CREATION Service Intelligence
10198811 R12 UPGRADE DRIVER FAILES WITH ORA-01400: CANNOT INSERT NULL INTO( Subledger Accounting
10275581 QRE :R12.1.1 UPGRADE DRIVER (U6678700.DRV) FAILING FOR CSLKU.ODF Service
10420245 Pre-Install: Upgrade script jaircvrtvdff.sql hangs during 12.1.1 upgrade Financials For India
11653739 RCA : RCA FOR DUPLICATE PARTY SITES AFTER UPGRADE Payables
11653739 RCA : RCA FOR DUPLICATE PARTY SITES AFTER UPGRADE Payables
11767692 BUG TO STUB OUT GMDQCMI2.SQL AND GMDQCMIJ.sql OPM Product Development
11847569 PRE INSTALL PATCH FOR 10077152, INDEXES ON AP_LIABILITY_BALANCE Payables
12332819 In Oracle Purchasing, User was not able to approve a Complex PO with Advance Shipment. Purchasing
12344218 Invoice lines upgrade (APLNUPGB.PLS) taking longer time to complete Payables
12360278 In Oracle Purchasing, After patch 9593873, Note entered in Response section of notification was not getting updated in action history when document was rejected. Purchasing
12372035 AP > ZXPOTRXMIGUPD.SQL FAILS WITH ERROR ORA-00001: UNIQUE CONSTRAINT VIOLATED E-Business Tax
12382774 TAX GROUPS NOT CORRECTLY MIGRATED FROM R11I TO R12 E-Business Tax
12387835 MASTER ITEM MIGRATION FROM R11.5.7 TO R12 – SHELF LIFE NOTE MIGRATED TO ALL ORGS Inventory Management
12422840 GMDMIG.SQL SHOULD NOT BE A PART OF 12.1 OPM Product Development
12553436 SLA HOT PATCH FAILS WITH ORA-00001 UNIQUE CONSTRAINT (PA.PA_PRIM_REP_LEGER_TMP_U Project Costing
12578648 UPGRADE:APXLAPAY.SQL:PERFORMANCE PROBLEM WAS NOT FIXED Payables
12648752 RCA : POPULATE CORRECT VALUE OF FRZ_FLG AND ASSOC_CHILD_FRZN_FLG DURING UPGRADE E-Business Tax
12682665 Migrated tax groups were not working properly incase the tax group length is more than 40 E-Business Tax
12837833 RECOVERY RATE RULES ARE NOT GETTING HONORED E-Business Tax
12837833 RECOVERY RATE RULES ARE NOT GETTING HONORED E-Business Tax
12877002 INCORRECT TAXABLE BASE AMOUNT ON TAX DISTRIBUTIONS Payables
12877002 INCORRECT TAXABLE BASE AMOUNT ON TAX DISTRIBUTIONS Payables
12877611 RCA – CONTACT POINTS INACTIVATED WHEN CONTACTS ARE ENDDATED DURING UPGRADE Payables
12912302 ECE PAYABLES VAT REGISTER DISPLAY NO DATA Regional Localizations
12930626 1OFF:7216659:12.1.3:PA/GMS: R12 SLA UPGRADE ISSUE IF ENHANCED PERIOD PROC Project Costing
12944782 COMPOUNDED TAX RATE IS NOT CALCULATED CORRECTLY IN R12 E-Business Tax
12944782 COMPOUNDED TAX RATE IS NOT CALCULATED CORRECTLY IN R12 E-Business Tax
12990345 GROUP BY ERROR HAPPENING IN APSTCA01.SQL Payables
12990345 GROUP BY ERROR HAPPENING IN APSTCA01.SQL Payables
12992125 RCA BUG: BANK ACCOUNT NAME IS APPENDED WITH OU NAME DURING UPGRADE. Payments
13016846 RCA Upgraded untranferred accounting headers have incorrect GL transfer flag Payables
13016846 RCA Upgraded untranferred accounting headers have incorrect GL transfer flag Payables
13019385 AFTER R12 UPGRADE USER CANNOT OVERRIDE THE CALCULATED TAX LINES E-Business Tax
13040251 Performance issue in upgrade accounting Payables
13040251 Performance issue in upgrade accounting Payables
13243594 AFTER MIGRATION ORDERS WITH STATUS OF RTW CANNOT BE TRANSACTED OPM Logistics
13248921 EMEA VAT UPG: JGZZVATUPGTRXDET.SQL FAILS DURING UPGRADE TO R12.1.1 (PRE INSTALL) Regional Localizations
13335019 UPGRADE: GMIG3MIB.SQL TAKING 8 HRS OPM Inventory
13356426 MODIFY AX UPGRADE SCRIPT XLAAXUAR.SQL TO MAP INTRA ACCOUNTING_CLASS_CODE Subledger Accounting
13356426 MODIFY AX UPGRADE SCRIPT XLAAXUAR.SQL TO MAP INTRA ACCOUNTING_CLASS_CODE Subledger Accounting
13390676 APINTBAL.SQL SCRIPT RUNS FOR MORE THAN 48 HOURS DURING TEST UPGRADE Subledger Accounting
13397299 UPGRADE: GMIG3MIB.SQL TAKING 8 HRS Inventory Management
13403786 R12.1.1 UPGRADE DRIVER U6678700.DRV HANGS AT OZFUPFUT.SQL SINCE MORE THAN 24 HRS Trade Management
13451293 RECOVERY RATE RULES ARE NOT GETTING HONORED E-Business Tax
13451293 RECOVERY RATE RULES ARE NOT GETTING HONORED E-Business Tax
13509912 CONSOLIDATED ROOT CAUSE CODEFIX: ACCOUNTING & UPGRADE RELATED ISSUES Project Costing
13509912 CONSOLIDATED ROOT CAUSE CODEFIX: ACCOUNTING & UPGRADE RELATED ISSUES Project Costing
13544907 RCA BUG:PAYEE UPG SCRIPT CREATES INACTIVE PAYEES FOR INACTIVE SUPPLIERS/SITES Payments
13545598 In Oracle Purchasing,approval screen shows up only with buttons with no text on them Purchasing
13557244 NEED TO UPGRADE ICMS,IPI DISTRIBUTIONS CREATED BY JL LOCALIZATION AS ITEM IN R12 Payables
13557244 NEED TO UPGRADE ICMS,IPI DISTRIBUTIONS CREATED BY JL LOCALIZATION AS ITEM IN R12 Payables
13577549 ATT MOBILITY UPGRADE: SCRIPT APSTCA01.SQL Payables
13577549 ATT MOBILITY UPGRADE: SCRIPT APSTCA01.SQL Payables
13598940 UPG:AR120CASHLNPU.SQL FAILED DURING ADPATCH ON XLA5584908.DRV Receivables
13598940 UPG:AR120CASHLNPU.SQL FAILED DURING ADPATCH ON XLA5584908.DRV Receivables
13684949 UNABLE TO SEE VOLUME TIERS AFTER 12.1 UPGRADE Trade Management
13695167 Consolidated pre-upgrade fixes Cash Management
13774403 RCA:INVOICE INCLUDES PREPAY FLAG IS NOT POPULATING FOR INV DISTS WHILE UPGRADE Payables
13774403 RCA:INVOICE INCLUDES PREPAY FLAG IS NOT POPULATING FOR INV DISTS WHILE UPGRADE Payables
13793279 Automatic Receipts gives ‘Instrument Assignment not Valid Payer’ Receivables
13793279 Automatic Receipts gives ‘Instrument Assignment not Valid Payer’ Receivables
13799611 In Oracle Purchasing, release form closed abruptly when distribution window was closed Purchasing
13820304 PERFORMANCE HIT DURING UPGRADE TO 12.1.1 IN ZXTAXDEFMIGB.PLS E-Business Tax
13833174 In Oracle Purchasing, BWC, Submission check fails for Complex Work Order with Fixed Price Services due to in consistant amount calculation at Line level and Distributions Purchasing
13837821 PERFORMANCE ISSUE WITH APILNUPG.SQL SCRIPT DURING R12.1.1 UPGRADE-PART-2 Payables
13837821 PERFORMANCE ISSUE WITH APILNUPG.SQL SCRIPT DURING R12.1.1 UPGRADE-PART-2 Payables
13856969 This is the pre-req patch (to be applied Before Inline Migration) for GMF customers upgrading to R12.1 OPM Financials
13864126 APINTBAL.SQL NOT RE-RUNNABLE AND FAILS WITH U1 ERROR Subledger Accounting
13917170 LOCATION Tax not migrated to R12 setup because of multiple Location Tax-Type Tax Codes in 11i setup E-Business Tax
13917170 LOCATION Tax not migrated to R12 setup because of multiple Location Tax-Type Tax Codes in 11i setup E-Business Tax
13931180 In Oracle Purchasing, get FRM-41056 error when enter item information on Quotations form Purchasing
13978746 ACCRUAL LINES MISSING IN XDL AFTER PATCH 13444893:R12.AP.B Payables
13978746 ACCRUAL LINES MISSING IN XDL AFTER PATCH 13444893:R12.AP.B Payables
14026963 COUNTERS MIGRATION FROM 1159 TO R12 – “CSI_MIGRATE_CS_COUNTERS”(CSIMCTRB) ERRORS Install Base

iProcurement Data Missing after Upgrade to Release 12.1.3

You either have not run the extractor or have lots of data exceptions that need to be fixed. This process should help.

1. Run the extractor to ensure that the iProcurement extracted catalog data is updated. See the MOS note below for instructions to run the extractor.

 2. Apply the pre-upgrade patch (4914492). It inserts a new entry in the eContent Manager menu called Release 12 Data Migration, which you can use to run the data exceptions report and/or the pre-upgrade.

 3. Run the exceptions report prior to the pre-upgrade. The report lists data that cannot be automatically upgraded and must be fixed before the upgrade. The pre-upgrade processes the catalog data to the new data model to reduce upgrade downtime. If there are still exceptions, it also updates the exceptions report. The exceptions report divides exceptions into two categories: those to be fixed using an XML file and reloaded into the catalog, and those to be fixed by correcting system default values.

iProcurement Setup & Usage: Internal Catalog Extract Process [ID 215530.1]

Install the R12.1.1 Upgrade File System on Solaris with rapidwiz

This install is to prepare for the R12.1.1 upgrade. As mentioned previously, you are required to install the R12.1.1 Upgrade Filesystem as part of the upgrade to R12.1.1.

You must use root, or sudo to run rapidwiz, if you have applmgr and oracle users for the apps stack and the database. If there is only one user, you don’t need root to run the install, but you may need someone with root access to run root.sh after the install.

For Solaris you’ll need the following from edelivery.oracle.com:

 

 

 

 

 

 

These downloaded packs are different for Linux, so be careful to download the right files from edelivery.oracle.com.

When you unzip these, you’ll end up with four directories:

                oraApps

                oraAS

                oraDB

                startCD

There is no way to avoid installing the 11106 database home. You can throw it away when you’re done, as in our case, we already have the database upgraded to 11203. Oracle really should add another install option along with the rapidwiz -techstack option that would allow us to be able to install the Upgrade Filesystem without the database tech stack, something like, rapidwiz -appstack, hint, hint Max and Elke.

First, start Exceed.

To get the tool bar, start exceed and right mouse click the exceed icon on the status bar and go to tools and select show tool bar.

 To use Putty, you load or create a session and on the right hand side expand SSH and there is either X11 or Tunnels depending on your version.

Select X11 forwarding and then go back to session and save it. 

 Then, set the DISPLAY variable to point back to my PC.

DISPLAY=100.100.5.77:0.0

There is no option to just install the apps stack, so rapidwiz will install both the database home, for 12.1.1, the version of the db is 11.1.0.6, and the apps stack, APPL_TOP,  OC4J, Forms homes and Developer 10g homes.

I think it’s funny that the RDBMS version is listed as 9.2.0.5 in the first rapidwiz screen, even though it’s really 11.2.0.6.

The bigger issue when installing the Upgrade File System is usually we already have an upgraded RDBMS stack, in our case it’s 11.2.0.3, but we have no other choice but to install 11.1.0.6 and throw it away. It would be great in the future if rapidwiz would give us the option to not install the database stack. Not only does it take up space and we have to remove it when the install is done, but we have to point the apps to the correct RDBMS home by editing the context file and running AutoConfig.

To start rapidwiz, since I don’t have the root login, I can use sudo, as in the following screenshot:

 sudo  has to be setup by the Unix sysadmin and you have to use the full path name to the rapidwiz executable, as shown above. The examples are from a real client install, so I blanked out the client specific info. In this example, the instance name is VALID and the two application servers are named testdb4, this also is the database server, and bppwebt1 is the second application server.

The next figure shows the technology stack that will be installed. The installed database version is not 9205, but really 11.1.0.6. As mentioned previously, I have to install this, but I’ll throw it away when I’m done, because I already have the database upgraded to 11.2.0.3. In a future post, I’ll talk about how to upgrade the EBS database to 11203.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Use the same port pool as the current database port pool, in our case port pool 0, in order to match the existing port pool.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Name the SID the same and specify the oracle user and group the same as before, so we can hook into the existing 11203 database.

 

Make sure to match the existing APPS password and GUEST password.

 

 

 

 

 

 

 

 

Define the Primary Applications Node

 

 

 

 

 

 

 

 

And “Edit Services” so only the services intended to run are specified.

 

 

 

 

 

 

 

 

In the above case, we only specified the Concurrent Processing to run on the Primary Apps node. Notice that when you disable the Root Service, all the other services except Batch Processing services become disabled.

Then, add another node, in our case we have two apps nodes. This step has to be performed when running rapidwiz on both nodes, so that each node is aware of the other node.

On the second Apps node we want all the services except Concurrent Processing. If we wanted to setup Parallel Concurrent Processing, we would specify Batch Processing services as well as the other services on the second node.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Now, I have to repeat these steps on the other node.

 

Second node configuration

 

 

 

 

 

 

 

 

We had to create the oravalid user on bppwebt1, even though there is no database home on this node, because rapidwiz needed to write the XML file to the /tmp directory as the oravalid user.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

All services except concurrent processing are enabled on bppwebt1.

 

 

 

 

 

 

 

 

Click “Add Server” to add the testdb4 server, this screen is after we added the server with the following screenshot:

 

 

 

 

 

 

 

 

Disable all but Concurrent Processing on testdb4, by clicking “Edit Services” in the next screen:

 

 

 

 

 

 

 

 

and, enable or disable the necessary services. In this case, all services are disabled except “Batch Processing Services”.

 

 

 

 

 

 

 

After the install finishes, the “Validate System Configuration” window appears, showing the system is installed correctly.

 

 

 

 

 

 

 

 

This nest figure illustrates the installed configuration:

The Big Picture of the R12.1.3 Upgrade – Part I

It’s coming! Don’t close your eyes. Face it head on. Ease your worries by reading this, Part I of a three part overview of the upgrade from Release 11.5.10.2 to Release 12.1.3. I’ll describe the tried and true tips and techniques  – everything from “oops – did you know about these extra patches?” to ways to decrease your down time.

The Big Picture shows at a high level the tasks and challenges of the R12.1.3 upgrade. The Big Picture describes how to get started, how to motivate business users to upgrade, and whether to re-implement or upgrade. It covers the technical upgrade steps with a few functional steps; understanding these steps helps all team members work toward a better upgrade.

Oracle Support Compliance – Release 11.5.10.2 Support Milestones

Premium Support for E-Business Suite 11.5.10.2 ended November 30, 2010.

• First year of 11.5.10.2 EBS Extended Support fees are waived.

• Mandatory Patches – Minimum Baseline Patches for Extended Support  11.5.10.2 – After December 1, 2010 (Note 883202.1)

Release 12.0 Support Milestones

• February 1, 2012 – Premium Support Ends (R12.0.6) (845809.1)

 Release 12.1 Support Milestones

• July 1, 2011 – You must apply R12.ATG_PF.B.delta.2 (R12.1.2 ATG RUP) (845809.1)

• February 1, 2012 – You must apply R12.ATG_PF.B.delta3 (R12.1.3 ATG RUP) (1066312.1)

• February 1, 2013 – You must upgrade to EBS R12.1.3

 

 The Upgrade is Iterative

The Upgrade is an Iterative Process, until all the issues are resolved and new functionality is working.

evolve

 

 

 

 

 

 

 

The Technical Upgrade is like Evolution – It’s an Iterative Approach

The faster you can iterate, the faster you can resolve problems and proceed to the production upgrade. For these reasons, the upgrade machine should be the fastest, baddest machine available. Remember users will want to perform testing, gap analysis and training, and developers will want to recode their customizations. For this reason, you’ll need to have other machines available to clone the latest upgrade pass. The less time you spend cloning, use snapshots and very fast disks, the more time you’ll have to work on the upgrade.

In general, the iterative cycle is as follows:

  1. Follow the steps in the upgrade guide
  2. Run patchsets.sh
  3. Run Patch Wizard – • Patch Wizard Update for 11i (Patch 9803629)
  4. Apply any new patches
  5. Iterate back to Step 3 and run Patch Wizard, until there are no new patches

 

How We Learned to do the R12 Upgrade

We started with the upgrade to 12.0.4 from 11.5.10.2 and learned with each book we wrote. Our initial upgrades were based on VISION upgrades with base 11.5.10.2 patches. The same basic way Oracle initially tests the upgrade. We’ve done many technical upgrade assessments, functional assessments and customization assessments.  The technical assessment is an important step to establish your patch levels for each module. If you want to continue receiving support from Oracle, then you’ll need to be patched with the Mandatory patches.

First Pass Upgrade to 12.1.1

Real customer data introduces new combinations of patch levels and possible data issues. This gave us greater insight into the upgrade issues. We then introduced the “First Pass” upgrade. We come on-site and upgrade your instance in 2 weeks. The whole assessment -1st pass upgrade usually takes 4-6 weeks, nt including customizations. This gives us many data points in our analysis of what patches are required. The 1st pass upgrade gives your functional analysts an instance to perform gap analysis and the developers an instance to see what customizations are broken.

The combination of assessments and 1st Pass upgrades help define the P3 Upgrade Methodology.

From the assessments and 1st pass upgrades we gathered new issues /solutions and developed an upgrade methodology that we use at every customer. The following picture shows the book covers for the project plan, the overview for managers and team members and the detailed, step-by-step instructions to complete the upgrade to R12.1.3.

With another release and more clients patch levels/issues, we’ve identified even more issues. The following are the pictures for the covers for the project plan, the overview and detailed upgrade guide.

tailored books

 

Plan

Decision: re-implement or upgrade

Understand the hardware requirements and the upgrade path

Procure upgrade hardware

Purge unnecessary data

Train the functional super users in the new features of Release 12.1

Create an upgraded instance for gap analysis

Start with an assessment and a 1st pass upgrade.

Technical Assessment

Determine future capacity requirements, tech stack version compatibility and patch levels, including CPUs and PSUs. We review current issues from log files, unresolved service requests, and identify potential issues with the R12.1 Upgrade.

Architecture – Hardware Assessment

Review hardware configurations including options like: RAC vs SMP, Shared Application Tier with Distributed Processing, Parallel Concurrent Processing, SAN vs JBOD, RMAN vs Snapshots.

If the plan includes buying new hardware, consider migrating from the current 32-bit platform to a 64-bit platform.

Functional Assessment

The R12 Upgrade is not just a technical upgrade

The functional upgrade consists of mapping new business requirements with new functional features in R12.

Identify AS-IS Processes

Determine TO-BE Processes

Evaluate Potential Data Issues

Which New Features may replace customizations?

Estimate R12 New Features Training Needs

Recommend “Best Practices”

 

Example of Functional Issues

SQL> select count(*), country from ap_bank_branches group by country;

COUNT(*) COUNTRY

1081   (null)

112 US

3 NZ

226 CA

1 BB

2 GB

1 DE

 

The best solution is to have the AP functional superusers update the banks to their proper value. However, the following SQL statement can be used temporarily:

update ap_bank_branches set country=’US’ where country is null;

There are many other functional checks that are provided as a part of the functional assessment.  We also provide a summary of functional new features for your installed modules.

Customization Assessment

CEMLI = Configurations, Extensions, Modifications, Localizations, and Integrations

The CEMLI Upgrade Assessment includes determining technical impact of Oracle E-Business Suite Release 12.1 on CEMLIs, upgrading CEMLIs to the new technology stack, retrofit of CEMLIs for compatibility and usability on Oracle E-Business Suite Release 12.1.

Identify all customizations

Check-in all customizations into configuration management

Determine customizations that are replaced by new R12.1 functionality

Re-Code customizations

Prepare Customization Upgrade Implementation Plan

Customization Configuration Management

 

Start by identifying all customizations. In some environments the customizations aren’t well documented and some customizations may have been lost due to previous patching. Check-in all customizations into configuration management. Customizations are easier to customize if you can find them and have some version control.

Determine the customizations that are replaced by new R12.1 functionality. This requires an analyst that knows the new functionality of R12.1 and understands the customizations.

Lastly, determine the customizations that need to be fixed or added to the upgrade to preserve or extend the process alignment.

Prepare

Train the Super Users and Technical Staff

Buy Hardware

Create Upgraded Instance for detailed gap analysis

Practice Testing

Practice Upgrades

Use the Maintenance Wizard (215527.1)

Step-by-step, graphical user interface for performing upgrade tasks

Consolidates instructions from multiple sources to present a comprehensive upgrade picture

Reduces upgrade tasks by filtering out those that do not apply to you (using TUMS)

Indicates critical patches that your system requires

Can automatically execute upgrade tasks for you

Run patchsets.sh

Use Patch Wizard from Oracle Application Manager (OAM)

We recommend that when you finish your upgrade to what you believe is the latest version of 12.1.3, with all the patches and Family Packs identified from patchsets.sh, this book, and your own research, you should run Patch Wizard again to see if additional patches are found.

Note that Patch Wizard may require patches for both Release 11i and Release 12 (9643141, 10629956).

If possible complete the following prior to the R12.1 upgrade weekend:

Upgrade the Database to 11.2.0.2

Migrate to OATM

Install the R12.1.1 software

Run Downtime Reducing steps

Run pre-upgrade verification steps

Technical Upgrade – Details

R12.1 Upgrade Paths

Path A    DB 9iR2, 10gR2  Apps 11.5.7 or 11.5.8

DB Upgrade & Apps Upgrade need to be completed during the same downtime window.

Path B     If the DB already at 11gR1, Apps 11.5.9.2 or 11.5.10.2

Only upgrade the Apps Stack

Path C     Upgrade the DB & Apps in different phases

If upgrading from a release prior to 11.5.7, the upgrade path may require an interim upgrade to Release 11.5.10.2. Because of the significant downtime required to upgrade from Release 11.0 to Release 12, it may be more feasible to first upgrade to Release 11.5.10.2 and then some time later upgrade to Release 12.  This requires the functional users to learn Release 11.5.10.2, and perform all the testing for another upgrade.  The amount of work necessary to perform two rounds of system acceptance testing may justify another day or two of downtime, so that the upgrade from Release 11.0 to Release 12.1 can be completed in one longer period of downtime.

Upgrade Paths

The light green circles indicate the most documented upgrade path from 11.5.10.2 to 12.1.1 and 12.1.3.

These bubbles show the upgrade paths. If your initial release is 11.0.3, you will need to upgrade to an interim release, 11.5.10.2, before you can upgrade to 12.1.1. The following chart lists the initial release, interim release and final release, with the associated patch number,

Initial Release                Interim Release               Final Release           R12 Patch

11.0,  11.5.1 – 11.5.6            Release 11.5.10 CU2            Release 12.0.0              4440000

11.5.7. 11.5.8, 11.5.9* or 11.5.10*                                     Release 12.0.0              4440000

11.5.7, 11.5.8, 11.5.9.2, 11.5.10.2                                      Release 12.0.4              6394500

11.5.9*, 11.5.10*                                                                  Release 12.1                  6678700

12.1.1                                                                                     Release 12.1.2               7303033

12.1.1                                                                                     Release 12.1.3               9239090

* includes CU1 and CU2 (consolidated update)

Figure 4 indicates that a direct upgrade path exists from Release 11.5.7 to Release 12.0.0.

The Applications Upgrade path is constrained by the database release. The following chart shows the application release and the database versions that are certified on Solaris.  If the application release is 11.5.10.2, then you can upgrade the database to 11gR2 before the application upgrade, saving downtime during upgrade weekend, if you plan you use 11gR2, and you should always try to use the latest certified version of the database.

Release                  Certified Database Versions on Solaris

12.1                                        10gR2, 11gR2 and the 64 bit versions

12.0                                        10gR2, 11gR2 and the 64 bit versions

11.5.10.2                               10gR2 or 11gR2 and the 64 bit versions

11.5.9.2                                 10gR2 and the 64 bit versions

11.5.7                                     8.1.7, 9.0.1 9.2, and 9.2-64 bit

We can see that there is no certified database version that is certified with both Release 11.5.7 and Release 12.1. Therefore, we can’t do the database upgrade before the downtime window.

Overview of the R12.1 Technical Upgrade

The database upgrade is a bit more complicated if you’re running 9.2.0.6, because of Daylight Savings Time.

  • The database installed by the 11.5.10.2 RapidWiz is Version 9.2.0.6. This database version does not support Daylight Savings Time (DST). Therefore, we have two choices:
  • Upgrade the database to Version 9.2.0.8, which has support for DST, and then upgrade to Version 11.2.0.1, or

Perform the Upgrade

11i pre-reqs for Release 12.1.1

11870353, 5880762, 7477784, 7721754, 7828862, 8579398, 8757781, 8761881, 8798855, 8845395, 8908907, 8990356, 8991381, 9003549, 9053932, 9109247, 9128838, 9187813, 9288021, 9304675, 9442701, 9446543, 9476923, 9535311, 9685457, 9725579, 9747572, 9871422, 9889680, 3865683, 6408117, 8242248, 6024690, 4619025, 5368595, 5357791, 5970422, 59105548, 5194357, 5230979, 4396821, 5377946, 6741394, 6505416, 7418579, 4963569, 5259121, 4551977, 4607647, 6027561, 5760729, 5382135, 4699061, 6696828, 4582937, 4507073, 8340090, 4350832, 4563075, 4582937, 4607647, 4699061, 4939444, 4963569, 4969938, 5259121, 5382135, 6349338, 6351946, 6694260, 6696828, 8340090, 8487779, 10258309, 6264601, 3153717, 4252634

This list of patches continually changes. You should run Patch Wizard to determine any missing patches for your environment. When you’ve determined your patch list, cut and paste the patch list into the Patch Search form, on My Oracle Support.

Prepare for Fusion

Nadia Benedjedou lays out short and long term goals for migrating to fusion:

The steps Nadia lays out steps to prepare your organization to migrate to Fusion include:

  •  Implement OBIEE
  •  Implement Identity Management
  •  Start using BI Publisher
  •  Use Enterprise Manager, if you’re not already, to manage more of the enterprise.
  •  Move your customizations to a Fusion compatible framework, JDeveloper and ADF.
  •  Centralize and Consolidate your data using Oracle’s  Master Data Management.

In the coming months, I’ll discuss Upgrade by Request in detail, how to create 4 merged patches that contain all the patches for the 12.1.3 upgrade to help reduce downtime and how to use the Oracle supplied post upgrade verification tasks to validate your R12.1.3 upgrade. I’ll also drill down into major upgrade issues, such as HR_HOOKS, Analytic Workspaces, adgrants.sql.

TruTek Consulting is “Key” to Upgrade Success

115102 Upgrade Client in PA going Live on R1213 – Utility company

“I think all of TruTek’s assistance and especially the upgrade class were key to making this project successful.” Dan 2-15-12

115102 Upgrade Client in CA going live on R1213 – Research Organization

“This latest upgrade test on the new production hardware went very well.  I don’t see any problem getting the  upgrade done over a 3 day weekend.  I have you guys to thank for that.”  Terry 1-31-12

Slow TNS connections due to Network issue

Using OEL and 11g, an old / invalid DNS ip address in the first line of resolv.conf adds 60 seconds to each connection, causes very slow LOCAL=NO connections. The ip address in resolv.conf are checked from the top down. If the first IP Address is BAD, it waits 60 seconds to timeout. During the EBS 1211 MP it adds about 35 hours with 125K jobs, for a total of 53 hours. It runs in 18 hours using the correct DNS server in resolv.conf. It happens that the DB and AS are on the same server and should logically use a BEQUETH connection to eliminate the network performance issue.

The other option is to turn off DNS in the /etc/nsswitch.conf.

Connection Point – Atlanta

I’m speaking at the Atlanta Connection Point next week on Nov15th and 16th.

I have two presentations:

The R12.1.3 Technical Upgrade Overview

and,

 The Big Picture of the R12.1 Upgrade

It’s coming! Don’t close your eyes. Face it head on. Ease your worries by attending this walkthrough of an upgrade from Release 11.5.10.2 to Release 12.1.3. We’ll describe the tried and true tips and techniques when we upgrade – everything from “oops – did you know about these extra patches?” to ways to decrease your down time.

The Big Picture shows at a high level the tasks and challenges of the R12.1.3 upgrade. The Big Picture describes how to get started, how to motivate business users to upgrade, and whether to re-implement or upgrade. It covers the technical upgrade steps with a few functional steps; understanding these steps helps all team members work toward a better upgrade.

 

Recent email asks, “Why not wait for 12.2 to upgrade from 11.5.10.2”

One of the important things to consider with 11.5.10.2 is support. I think Extended Support ends Nov 2013 and without support I’m sure your business will want to upgrade.

You could wait for 12.2, but I wouldn’t want to implement a product with a brand new application server, until a few bugs settle out. 12.2 will use Weblogics.

The 12.1 upgrade is a functional upgrade and the 12.2 is a tech stack upgrade, probably with 11.2.0.3 for the DB. The big deal is WLS and the administration of possibly much more complicated configurations.

At OOW, a development manager at Oracle said that all the platforms, Peoplesoft, JDEdwards all have to run on WLS before they can attempt a “fusion” of the products. This seems a bit obvious, but illustrates the fact that Oracle will be really pushing us hard to 12.2.

If you really want to take on the complexity of educating your users in a completely new way of accounting and trying to resolve new undocumented AS issues at the same time by doing a 12.2 upgrade, good luck.

I think the more sensible approach would be to upgrade to 12.1.3 and then 2 years later upgrade to 12.2. Since, it’s primarily a tech stack upgrade with bug fixes, the users should not need to be retrained.

Meetings with Gustavo and Max at Oracle

The two Senior Development Managers I met this morning were Max and Gustavo.  We talked about 12.1.3, 12.2, Fusion Applications and Fusion Middleware. This is  my recollection of our discussion.

First, topics from 12.1.3:

OATM – TruTek suggests that OATM be run we before the upgrade to 12.1.3, but  running OATM after the upgrade will remove fragmentation by running “ALTER TABLESPACE MOVE TABLE” on all the tables in each tablespace .  However, in the interest of minimizing downtime during the upgrade, TruTek recommends running OATM before the upgrade and run OATM  after the upgrade, to remove any fragmentation, if present.

Parallel Concurrent Processing and Dead Connection Detection using 11g – with versions of the database before 11gR1, dead connection detection was dependent on TCP Keepalive settings. Now, with 11gR1 and above the dia0 database process eliminates dead connections within 10 seconds and sometimes seems almost immediate.  Special setup of TCP parameters is no longer necessary to perform PCP failover.

Edition Based Redifinition (EBR) was introduced in 11gR2.  Hot Patching uses this database feature in 12.2 with the introduction of “adop” AD Online Patching, the new admin tool for patching that keeps a copy of the Apps Tier for ADOP patching.

TruTek suggested an enhancement to adpatch: to allow a different number of workers for different phases. For example, normal patch jobs run better with fewer workers, in part because of less contention for deferred jobs, while the ad parallel compile runs fastest using all available CPUs.

Topics from 12.2:

In 12.0 and 12.1 the application server is 10gAS and the configuration parameters are incorporated into AutoConfig, leaving out the AS Control management console. In 12.2 the application server is WebLogic and while some parameters, such as DBC configuration parameters for AOL/J are still managed by AutoConfig, the WebLogic Server (WLS) is managed through the WLS Console. This allows admins to configure domains, managed servers,  server clusters, machines, node managers, virtual hosts, work managers and security realms.

All the AOL/J components are essentially the same, including calls to FND_GLOBAL.APPS_INITIALIZE. This sets the FND_GLOBAL_USER_ID, Function Security, NLS and Multi Org parameters. The call to the NLS setting is especially expensive, which is why the JDBC Connection pool will save this session info and try to reuse it if possible.

In 12.2 the OHS will connect to Weblogic with a reverse proxy using the Apache Plugin that supports the APG14 protocol.

12.2 supports EBR and needs the functionality of 11gR2 for EBR. Therefore, 10gR2 and 11gR1 will not support 12.2. Steven Chan’s blog mentions that 11.2.0.3 is on the horizon.

Fusion Applications:

Get ready for Fusion by upgrading to 12.1.3

Introduce SOA to begin to replace existing non-Fusion compatible customizations

Co-Exist with Fusion to take advantage of new functionality in Fusion and use SOA interface with EBS.

ADF does not support flexfields, but OAF still supports flexfields. These pictures are from Nadia’s presentation at Open World. You can launch ADF pages from OAF:

ADF objects can be embedded in in OAF pages.

ADF no longer supports flexfields, but OAF still supports flexfields.

Begin using ADF extensions for EBS to help protect your investment, better enable co-existence with Fusion Applications and ensure your upgrade path to Fusion in the future.

 

 Fusion Middleware:

Prepare for Fusion by implementing Identity Management and Access Manager:

 

 Prepare for Fusion by Implementing OBIEE:

 

 Prepare for Fusion by implementing BI Publisher: