java.lang.ExceptionInInitializerError while trying to Access login page

Today morning, we (the DBAs) were faced with a strange issue. On Saturday (the 14th of February - (St.) Valentine's day!!), the UNIX boxes hosting our critical test instance went down for scheduled maintenance without any prior information. So, we could not bring down our databases and applications.

As soon as the unix box came up, we encountered this issue. Let me briefly describe the environment, the issue, the analysis performed and of course, the solution to this most vexing problem.

Environment:

Apps Version: 11.5.10.2 with ATG RUP 5

DB Version: 10.2.0.3

Architecture: Two Node instance with database + admin on one tier and web server on another.

PS: Another instance (development) exists on the same server with the same configuration, which was working fine.

Issue:

When trying to access the login page of Oracle Apps, received the below error. Even OAM login page was throwing the same error.

Request URI: /OA_HTML/AppsLocalLogin.jsp
Exception:

java.lang.ExceptionInInitializerError

Request URI:/OA_HTML/AppsLocalLogin.jsp
Exception:

java.lang.NoClassDefFoundError

As soon as the services were bounced, we were receiving the first error. After some time, we received the second error (yes, truth is indeed stranger than fiction!!).

Analysis:

Since, we were receiving "NoClassDefFoundError", performed the following:

1. Checked the CLASSPATH and AF_CLASSPATH variable and ensured that every path is valid.

2. Checked all the class paths defined by wrapper.classpath in jserv.properties using the command below:

for i in `grep classpath $IAS_ORACLE_HOME/Apache/Jserv/etc/jserv.properties grep -v "#" cut -d '=' -f2`
do
ls -ld $i >> /tmp/check_class_path.txt
if [[ $? -eq 0 ]]; then
continue
else
printf "\n\n '${i}' is not proper. \n\n"
fi
done

I compared the output generated from the above script with that from the development instance that was working and found no discrepancies.

3. Ran aoljtest.jsp and found that it was running fine till the step "Verify Guest User", where it was throwing "Page cannot be found" error.

4. The above error set me thinking and I checked the GUEST username and password in the

• dbc file
• database (profile option)
• CONTEXT_FILE
• output of the below query "select fnd_web_sec.validate_login ('GUEST','<PASSWORD>') from dual;"

and found it to be consistent.

5. Checked a lot of Metalink Notes and documents and blogs and what not, but to no avail.

6. Enabled debug in :

• jserv.conf
• jserv.properties
• httpd.conf

but no log file was informative enough to point me in the right direction.

7. Meanwhile, the error log had been throwing the below error, which I analysed but could not make much headway and found it to be actually misleading.

[Sun Feb 15 23:13:20 2009] [error] [client <IP_ADDRESS>] (78)File name too long: access to /redirectURL$url=http_3A$$ <hostname.domain_name>_3A<port_number>$oa*_servlets$ oracle.apps.fnd.sso.AppsLogin_3FrequestUrl$=$oa*_servlets$weboam$ oam$oamLogin_26cancelUrl$=http_3A$$ <hostname.domain_name>_3A<port_number>$oa*_servlets$ oracle.apps.fnd.sso.AppsLogin$forward=oam$oamLogin failed

8. After exhausting all the options, I decided to take a break and promptly headed for lunch!

9. Post lunch, I tried to re-examine the issue from a different perspective. As point 3 above states, I was receiving an error while trying to check the Guest username using aoljtest.jsp. Again checked the guest username and password in the palces mentioned in point 4 above and found them to be consistent.

10. Finally (actually, I had found this note earlier, but ignored it because I found the guest username and password to be correct), I found Note: 458064.1, which I found relevant on three counts:

• It mentions about the "java.lang.ExceptionInInitializerError" error.
• we were also on ATG RUP 5 (5473858), as the note mentions.
• It also mentions that "Verify Guest user" ends with an error (though different from the one that I was receiving).

So, I decided to follow the solution mentioned in that note.

Solution:

1. a. Login directly to Forms via http://<hostname>:<port>/dev60cgi/f60cgi

b. Set the value of system profile "Password Case Option" to "Insensitive"

c. Save and close Oracle Applications

2. On the operating system as the apps user (e.g. applmgr) execute these commands:

a. cd $FND_TOP/secure/<SID>_<hostname>

b. java oracle.apps.fnd.security.AdminAppServer <apps username>/<apps password> UPDATE GUEST_USER_PWD=GUEST/<PASSWORD ALL UPPERCASE> DB_HOST=<host> DB_NAME=<SID> DB_PORT=<port>

The values for GUEST_USER_PWD, DB_HOST, DB_NAME, and DB_PORT can be found in the $FND_TOP/secure/<SID>_hostname/<SID>.dbc file if required.

3. Verify that the following things are true:

a. The GUEST password is updated and all upper-case in the $FND_TOP/secure/<SID>_<hostname>/<SID>.dbc file.

b. That the profile GUEST_USER_PWD is set correctly after the password change. The SQL to run is:

set serveroutput on
set echo on
set timing on
set feedback on
set long 10000
set linesize 120
set pagesize 132
column SHORT_NAME format A30
column NAME format A40
column LEVEL_SET format a15
column CONTEXT format a30
column VALUE format A60 wrap
select p.profile_option_name SHORT_NAME,
n.user_profile_option_name NAME,
decode(v.level_id,
10001, 'Site',
10002, 'Application',
10003, 'Responsibility',
10004, 'User',
10005, 'Server',
10007, 'SERVRESP',
'UnDef') LEVEL_SET,
decode(to_char(v.level_id),
'10001', '',
'10002', app.application_short_name,
'10003', rsp.responsibility_key,
'10005', svr.node_name,
'10006', org.name,
'10004', usr.user_name,
'10007', 'Serv/resp',
'UnDef') "CONTEXT",
v.profile_option_value VALUE
from fnd_profile_options p,
fnd_profile_option_values v,
fnd_profile_options_tl n,
fnd_user usr,
fnd_application app,
fnd_responsibility rsp,
fnd_nodes svr,
hr_operating_units org
where p.profile_option_id = v.profile_option_id (+)
and p.profile_option_name = n.profile_option_name
and upper(n.user_profile_option_name) = 'GUEST_USER_PWD'
and usr.user_id (+) = v.level_value
and rsp.application_id (+) = v.level_value_application_id
and rsp.responsibility_id (+) = v.level_value
and app.application_id (+) = v.level_value
and svr.node_id (+) = v.level_value
and org.organization_id (+) = v.level_value
order by short_name, level_set;

c. That the following SQL returns 'Y':

SELECT fnd_web_sec.validate_login('GUEST','') FROM dual;


4. Bounced the services and retried the login, which worked like a charm.