— ## Applies to:
Oracle Server – Enterprise Edition – Version 11.2.0.1 and later Information in this document applies to any platform.
Symptoms
Clusterware fails to start up due to ASM instance fails to start with ORA-4031:
[root@racnode1 bin]# ./crsctl start cluster CRS-2672: Attempting to start 'ora.asm' on 'racnode1' CRS-5017: The resource action "ora.asm start" encountered the following error: ORA-04031: unable to allocate 16416 bytes of shared memory ("shared pool","unknown object","sga heap(1,0)","dbktb: trace buffer") . For details refer to "(:CLSN00107:)" in "/u01/app/grid/11.2.0/log/mcxoradb/agent/ohasd/oraagent_oracle/oraagent_oracle.log". CRS-2674: Start of 'ora.asm' on 'racnode1' failed CRS-2679: Attempting to clean 'ora.asm' on 'racnode1' CRS-2681: Clean of 'ora.asm' on 'racnode1' succeeded CRS-4000: Command Start failed, or completed with errors.
Symptoms
Cause
Solution
Cause
The problem is caused by db_cache_size is wrongly set to 1GB for ASM instance which in turn causing ASM fails to start with ORA-4031.
Per alert_+ASM1.log:
`Sun Sep 16 19:24:42 2012
MEMORY_TARGET defaulting to 411041792.
* instance_number obtained from CSS = 1, checking for the existence of node 0…
* node 0 does not exist. instance_number = 1
Starting ORACLE instance (normal)
…
Mon Sep 17 11:26:56 2012
ALTER SYSTEM SET db_cache_size='1G' SCOPE=SPFILE SID='*';
Mon Sep 17 11:27:33 2012
ALTER SYSTEM SET db_cache_size='1G' SCOPE=SPFILE;
Mon Sep 17 14:15:16 2012
ALTER SYSTEM SET db_cache_size='1G' SCOPE=SPFILE SID='+ASM1';
…
Mon Sep 17 14:18:39 2012
Shutting down instance (abort)
License high water mark = 11
Mon Sep 17 14:18:44 2012
MEMORY_TARGET defaulting to 1543503872.
* instance_number obtained from CSS = 1, checking for the existence of node 0…
* node 0 does not exist. instance_number = 1
Starting ORACLE instance (normal)
Errors in file /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_ora_3376.trc (incident=116161):
ORA-04031: unable to allocate 16416 bytes of shared memory ("shared pool","unknown object","sga heap(1,0)","dbktb: trace buffer")
Incident details in: /u01/app/oracle/diag/asm/+asm/+ASM1/incident/incdir_116161/+ASM1_ora_3376_i116161.trc
Use ADRCI or Support Workbench to package the incident.`
ASM instance is not a database instance, it does not require db_cache_size to be set. By default it is 0. By setting this parameter, it reduces the shared_pool_size which can be used by ASM instance, hence lead to ORA-4031.
Solution
1. Stop clusterware processes on all nodes as root user: # crsctl stop crs -f
2. Start clusterware on node 1 in exclusive mode as root user: # crsctl start crs -excl
3. Create a pfile (for example: /tmp/asmpfile.ora) with the content from alert_+ASM<x>.log under the section "System parameters with non-default values:", here is a simple example, please ensure to alter it per your ASM requirement:
`MEMORY_TARGET=1543503872
large_pool_size = 12M
instance_type = "asm"
remote_login_passwordfile= "EXCLUSIVE"
asm_power_limit = 1
diagnostic_dest = "/u01/app/oracle"`
Save the file, then start ASM instance manually as grid user, recreate spfile on its original diskgroup, eg:
$ sqlplus / as sysasm SQL> startup pfile='/tmp/asmpfile.ora'; SQL> create spfile='+DATADG' from pfile='/tmp/asmpfile.ora'; SQL> shutdown immediate
4. Stop clusterware on node 1 and restart clusterware on all nodes: as root user: # crsctl stop crs -f # crsctl start crs
5. Wait for a while, verify the clusterware status via: $ crsctl stat res -t