ORA-00494 enqueue [CF] held for too long故障分析处理(一)

2015-07-16 12:09:16 · 作者: · 浏览: 15

今天接应用人员电话反应,一套备份数据库在加完表空间后出现异常,现象是数据库能查询,但是update很慢。


?



?2、进入数据库中手动切换日志(alter system switch logfile)很慢几近于不动。



3、分析日志发现LGWR进程一直在等待CF enqueue,
Dump continued from file: /oracle/app/oracle/diag/rdbms/racdb/racdb/trace/racdb_lgwr_5854.trc
?ORA-00494: enqueue [CF] held for too long (more than 900 seconds) by 'inst 1, osid 7955'



?========= Dump for incident 380258 (ORA 494) ========
?----- Beginning of Customized Incident Dump(s) -----
?-------------------------------------------------------------------------------
?ENQUEUE [CF] HELD FOR TOO LONG



?enqueue holder: 'inst 1, osid 7955'
?
?4、进一步分析发现很多次control file sequential read等待,即处于IO等待状态,
?为什么出现control file sequential read,可能控制文件不在了,或者控制文件或其快照所在目录掉了。
Current Wait Stack:
? Not in wait; last wait ended 17 min 35 sec ago
?There are 40 sessions blocked by this session.
?Dumping one waiter:
? inst: 1, sid: 1653, ser: 1
? wait event: 'enq: CF - contention'
? ? p1: 'name|mode'=0x43460005
? ? p2: '0'=0x0
? ? p3: 'operation'=0x0
? row_wait_obj#: 4294967295, block#: 0, row#: 0, file# 0
? min_blocked_time: 898 secs, waiter_cache_ver: 407
?Wait State:
? fixed_waits=0 flags=0x21 boundary=0x0/-1
?Session Wait History:
? ? elapsed time of 17 min 35 sec since last wait
? 0: waited for 'control file sequential read'
? ? file#=0x0, block#=0x12, blocks=0x1
? ? wait_id=10168 seq_num=10169 snap_id=1
? ? wait times: snap=0.000352 sec, exc=0.000352 sec, total=0.000352 sec
? ? wait times: max=infinite
? ? wait counts: calls=0 os=0
? ? occurred after 0.000018 sec of elapsed time
? 1: waited for 'control file sequential read'
? ? file#=0x0, block#=0x10, blocks=0x1
? ? wait_id=10167 seq_num=10168 snap_id=1
? ? wait times: snap=0.000320 sec, exc=0.000320 sec, total=0.000320 sec
? ? wait times: max=infinite
? ? wait counts: calls=0 os=0
? ? occurred after 0.000018 sec of elapsed time


?



?5、进入RMAN中查看控制文件备份路径均为/oradata/racdbdb_rman_bak目录,详细如下:
?RMAN> show all;



使用目标数据库控制文件替代恢复目录
db_unique_name 为 racdb 的数据库的 RMAN 配置参数为:
CONFIGURE RETENTION POLICY TO REDUNDANCY 1;
?CONFIGURE BACKUP OPTIMIZATION OFF; # default
?CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default
?CONFIGURE CONTROLFILE AUTOBACKUP OFF;
?CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '/oradata/racdbdb_rman_bak/conf_%T_%F';
?CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default
?CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
?CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default
?CONFIGURE MAXSETSIZE TO UNLIMITED; # default
?CONFIGURE ENCRYPTION FOR DATABASE OFF; # default
?CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default
?CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default
?CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default
?CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/oradata/racdbdb_rman_bak/snap_racdb.ctl';



?6、查看目录发现/oradata目录掉了
?bash-3.00$ df -h
?Filesystem? ? ? ? ? ? size? used? avail capacity? Mounted on
?/dev/dsk/c0t0d0s0? ? ? 96G? ? 47G? ? 48G? ? 50%? ? /
?/devices? ? ? ? ? ? ? ? 0K? ? 0K? ? 0K? ? 0%? ? /devices
?ctfs? ? ? ? ? ? ? ? ? ? 0K? ? 0K? ? 0K? ? 0%? ? /system/contract
?proc? ? ? ? ? ? ? ? ? ? 0K? ? 0K? ? 0K? ? 0%? ? /proc
?mnttab? ? ? ? ? ? ? ? ? 0K? ? 0K? ? 0K? ? 0%? ? /etc/mnttab
?swap? ? ? ? ? ? ? ? ? ? 87G? 1.9M? ? 87G? ? 1%? ? /etc/svc/volatile
?objfs? ? ? ? ? ? ? ? ? ? 0K? ? 0K? ? 0K? ? 0%? ? /system/object
?sharefs? ? ? ? ? ? ? ? ? 0K? ? 0K? ? 0K? ? 0%? ? /etc/dfs/sharetab
?fd? ? ? ? ? ? ? ? ? ? ? 0K? ? 0K? ? 0K? ? 0%? ? /dev/fd
?swap? ? ? ? ? ? ? ? ? ? 87G? 120K? ? 87G? ? 1%? ? /tmp
?swap