Question:
We upgraded our BW system kernel from 620 to 640. We brought the system back up and it was running fine - until last night. For some reason SAP 'disappeared' (for the lack of a better term) so we tried starting/stopping SAP. No luck. The logs say SAP is up but there are no work processes.
Server: HPUX 11i
Oracle:9.2.0.6
Latest starsap.log:
sapbde:bdeadm 42> more startsap_sapbde_22.log
Trace of system startup/check of R/3 System BDE on Wed Sep 14 10:09:56 CDT 2005
Called command: startsap_sapbde_22
Starting SAP-Collector Daemon
------------------------------
saposcol already running
Checking SAP R/3 BDE Database
------------------------------
Database is not available via /usr/sap/BDE/SYS/exe/run/R3trans -d -w
Starting SAP R/3 BDE Database
------------------------------
Startup-Log is written to /home/bdeadm/startdb.log
Database started
Checking SAP R/3 BDE Database
------------------------------
Database is running
Starting SAP R/3 Instance
------------------------------
SAP-R/3-Startup Program Rel 640 V1.8 (2003/04/24)
-------------------------------------------------
Starting at 2005/09/14 10:10:11
Startup Profile: "/usr/sap/BDE/SYS/profile/START_DVEBMGS22_sapbde"
Execute Pre-Startup Commands
----------------------------
(28241) Local: /usr/sap/BDE/SYS/exe/run/sapmscsa -n pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMG
S22_sapbde
/usr/sap/BDE/SYS/exe/run/sapmscsa: SCSA is attached and useable.
/usr/sap/BDE/SYS/exe/run/sapmscsa: finished.
(28241) Local: rm -f ms.sapBDE_DVEBMGS22
(28241) Local: ln -s -f /usr/sap/BDE/SYS/exe/run/msg_server ms.sapBDE_DVEBMGS22
(28241) Local: rm -f dw.sapBDE_DVEBMGS22
(28241) Local: ln -s -f /usr/sap/BDE/SYS/exe/run/disp+work dw.sapBDE_DVEBMGS22
(28241) Local: rm -f co.sapBDE_DVEBMGS22
(28241) Local: ln -s -f /usr/sap/BDE/SYS/exe/run/rslgcoll co.sapBDE_DVEBMGS22
(28241) Local: rm -f se.sapBDE_DVEBMGS22
(28241) Local: ln -s -f /usr/sap/BDE/SYS/exe/run/rslgsend se.sapBDE_DVEBMGS22
Starting Programs
-----------------
(28269) Starting: local ms.sapBDE_DVEBMGS22 pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(28270) Starting: local dw.sapBDE_DVEBMGS22 pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(28271) Starting: local co.sapBDE_DVEBMGS22 -F pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(28272) Starting: local se.sapBDE_DVEBMGS22 -F pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(28241) Waiting for Child Processes to terminate.
(28241) **** 2005/09/14 10:10:14 Child 28270 terminated with Status 1 . ****
Instance on host sapbde started
But it didn't start:
sapbde:bdeadm 41> ps -u bdeadm
PID TTY TIME COMMAND
28147 ? 0:00 oracle
28151 ? 0:00 oracle
24063 ? 0:00 saposcol
28271 ? 0:00 rslgcoll
28153 ? 0:00 oracle
28269 ? 0:00 msg_server
28159 ? 0:00 oracle
28161 ? 0:00 oracle
28155 ? 0:00 oracle
28157 ? 0:00 oracle
28241 ? 0:00 sapstart
28134 ? 0:00 oracle
28272 ? 0:00 rslgsend
26176 pts/tb 0:00 csh
28132 ? 0:00 oracle
28329 pts/tb 0:00 ps
28149 ? 0:00 oracle
Where can I look from here?
Thanks
-vince
Answer:
Is there anything that helps in
/usr/sap/BDE/DVEBMGS22/work/dev_w0
Answer:
cd /usr/sap/BDE/DVEBMGS22/work
ls -ltr
check last logs for more info.
_________________
SapFans Moderator
NetWeaver ‘04–SAP Web AS for ORACLE certified
Search: /forums/search.php
SAP Notes: http://service.sap.com/notes
SAP Help: http://help.sap.com
Basic Rules: /forums/viewtopic.php?t=222759
Answer:
ls -lrt
lrwxrwxrwx 1 bdeadm sapsys 35 Sep 14 10:41 ms.sapBDE_DVEBMGS22 -> /usr/sap/BDE/SYS/exe/run/msg_server
lrwxrwxrwx 1 bdeadm sapsys 33 Sep 14 10:41 se.sapBDE_DVEBMGS22 -> /usr/sap/BDE/SYS/exe/run/rslgsend
lrwxrwxrwx 1 bdeadm sapsys 33 Sep 14 10:41 co.sapBDE_DVEBMGS22 -> /usr/sap/BDE/SYS/exe/run/rslgcoll
lrwxrwxrwx 1 bdeadm sapsys 34 Sep 14 10:41 dw.sapBDE_DVEBMGS22 -> /usr/sap/BDE/SYS/exe/run/disp+work
-rwxr--r-- 1 bdeadm sapsys 14 Sep 14 10:41 kill.sap
-rw-rw-rw- 1 bdeadm sapsys 158 Sep 14 10:41 stderr1
-rw-rw-rw- 1 bdeadm sapsys 0 Sep 14 10:41 stdout1
-rw-rw-rw- 1 bdeadm sapsys 158 Sep 14 10:41 sapstart1.trc
-rw-rw-rw- 1 bdeadm sapsys 12 Sep 14 10:41 sapstart.sem
-rw-rw-rw- 1 bdeadm sapsys 193 Sep 14 10:41 INSTSTAT
-rw-r--r-- 1 bdeadm sapsys 757 Sep 14 10:41 dev_ms
-rwxr--r-- 1 bdeadm sapsys 56 Sep 14 10:41 shutdown.sap
-rw-rw-rw- 1 bdeadm sapsys 0 Sep 14 10:41 stdout2
-rw-rw-rw- 1 bdeadm sapsys 158 Sep 14 10:41 sapstart2.trc
-rw-rw-rw- 1 bdeadm sapsys 172 Sep 14 10:41 sapstart3.trc
-rw-rw-rw- 1 bdeadm sapsys 172 Sep 14 10:41 stderr3
-rw-rw-rw- 1 bdeadm sapsys 0 Sep 14 10:41 stdout3
-rw-rw-rw- 1 bdeadm sapsys 172 Sep 14 10:41 sapstart4.trc
-rw-rw-rw- 1 bdeadm sapsys 0 Sep 14 10:41 stdout4
-rw-rw-rw- 1 bdeadm sapsys 172 Sep 14 10:41 stderr4
-rw-rw-rw- 1 bdeadm sapsys 158 Sep 14 10:41 stderr2
-rw-rw-rw- 1 bdeadm sapsys 2042 Sep 14 10:41 sapstart.log
-rw-r--r-- 1 bdeadm sapsys 997 Sep 14 10:41 dev_disp
sapbde:bdeadm 65> more dev_disp
---------------------------------------------------
trc file: "dev_disp", trc level: 1, release: "640"
---------------------------------------------------
Wed Sep 14 10:41:59 2005
kernel runs with dp version 129(ext=102) (@(#) DPLIB-INT-VERSION-129)
length of sys_adm_ext is 312 bytes
sysno 22
sid BDE
systemid 273 (HP/9000 (PA-RISC) with HP-UX)
relno 6400
patchlevel 0
patchno 80
intno 20020600
make: single threaded, ASCII, 64 bit
pid 10717
***LOG Q00=> DpSapEnvInit, DPStart (22 10717) [dpxxdisp.c 1094]
shared lib "dw_xml.sl" version 80 successfully loaded
shared lib "dw_xtc.sl" version 80 successfully loaded
shared lib "dw_stl.sl" version 80 successfully loaded
shared lib "dw_gui.sl" version 80 successfully loaded
MtxInit: -2 0 0
*** ERROR => DpSysAdmExtCreate: ShmCreate (CREATE) [dpxxtool2.c 516]
*** ERROR => dispatcher already running ???? [dpxxtool2.c 517]
*** ERROR => I better EXIT before I do any (more) damage [dpxxtool2.c 518]
sapbde:bdeadm 66> more sapstart.log
SAP-R/3-Startup Program Rel 640 V1.8 (2003/04/24)
-------------------------------------------------
Starting at 2005/09/14 10:41:55
Startup Profile: "/usr/sap/BDE/SYS/profile/START_DVEBMGS22_sapbde"
Execute Pre-Startup Commands
----------------------------
(10647) Local: /usr/sap/BDE/SYS/exe/run/sapmscsa -n pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMG
S22_sapbde
(10647) Local: rm -f ms.sapBDE_DVEBMGS22
(10647) Local: ln -s -f /usr/sap/BDE/SYS/exe/run/msg_server ms.sapBDE_DVEBMGS22
(10647) Local: rm -f dw.sapBDE_DVEBMGS22
(10647) Local: ln -s -f /usr/sap/BDE/SYS/exe/run/disp+work dw.sapBDE_DVEBMGS22
(10647) Local: rm -f co.sapBDE_DVEBMGS22
(10647) Local: ln -s -f /usr/sap/BDE/SYS/exe/run/rslgcoll co.sapBDE_DVEBMGS22
(10647) Local: rm -f se.sapBDE_DVEBMGS22
(10647) Local: ln -s -f /usr/sap/BDE/SYS/exe/run/rslgsend se.sapBDE_DVEBMGS22
Starting Programs
-----------------
(10716) Starting: local ms.sapBDE_DVEBMGS22 pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(10716) New Child Process created.
(10716) Starting local Command:
Command: ms.sapBDE_DVEBMGS22
pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(10717) Starting: local dw.sapBDE_DVEBMGS22 pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(10717) New Child Process created.
(10717) Starting local Command:
Command: dw.sapBDE_DVEBMGS22
pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(10718) New Child Process created.
(10718) Starting local Command:
Command: co.sapBDE_DVEBMGS22
-F
pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(10718) Starting: local co.sapBDE_DVEBMGS22 -F pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(10721) New Child Process created.
(10721) Starting local Command:
Command: se.sapBDE_DVEBMGS22
-F
pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(10721) Starting: local se.sapBDE_DVEBMGS22 -F pf=/usr/sap/BDE/SYS/profile/BDE_DVEBMGS22_sapbde
(10647) Waiting for Child Processes to terminate.
(10647) **** 2005/09/14 10:41:59 Child 10717 terminated with Status 1 . ****
I knew the logs where there but there doesn't seem to be any information.
Answer:
We ended up doing the Microsoft solution - ie reboot (even though it is HP-UX). SAP came up fine. Now we are a bit concerned that our BQA and BPR systems will have this issue since they were given the newer kernels a week or so after BDE.
Thanks for everyones input.
Answer:
We ended up doing the Microsoft solution - ie reboot (even though it is HP-UX). SAP came up fine. Now we are a bit concerned that our BQA and BPR systems will have this issue since they were given the newer kernels a week or so after BDE.
Thanks for everyones input.
When SAP went down abruptly it did not release the shared memory and that is whay you could not restart SAP. By rebooting the system you released the sahred memory and wer able to start SAP.
If you issue startsap and it states that it is already running but at OS level you don't see any SAP processes active that is the sign that when SAP crashed, it did not release the shared memory. You could have cleaned / release the memory via cleanipc SAP command or ipcrm unix command.
Answer:
When SAP went down abruptly it did not release the shared memory and that is whay you could not restart SAP. By rebooting the system you released the sahred memory and wer able to start SAP.
If you issue startsap and it states that it is already running but at OS level you don't see any SAP processes active that is the sign that when SAP crashed, it did not release the shared memory. You could have cleaned / release the memory via cleanipc SAP command or ipcrm unix command.
Thanks for the information. When I showed it to the other BASIS guys, esp the unix one, he recalled the UNIX command. But it is one of those things you MIGHT run into once in a great while.
I'd forward you a drink if there was a way to do it.
Thanks again
-vince