Transaction recovery: lock conflict caught and ignored
April 30, 2013 Leave a comment
ALERT.LOG:
..... Transaction recovery: lock conflict caught and ignored
.....
And also some incident files are being created in $ORACLE_BASE/diag/rdbms/dbname/instancename/incident folder.
In my case the error started after SUPPLEMENTAL LOGGING enabled in a RAC environment. After disabling it the messages have not disappeared, but incident files are no longer being created.
1. Dead Trasaction
SQL> select b.name useg, b.inst# instid, b.status$ status, a.ktuxeusn
xid_usn, a.ktuxeslt xid_slot, a.ktuxesqn xid_seq, a.ktuxesiz undoblocks,a.ktuxesta txstatus
from x$ktuxe a, undo$ b
where a.ktuxecfl like ‘%DEAD%’
and a.ktuxeusn = b.us#;
USEG INSTID STATUS XID_USN XID_SLOT XID_SEQ UNDOBLOCKS TXSTATUS _SYSSMU7_881277423$ 1 3 7 13 1829999 1 ACTIVE _SYSSMU8_4204495590$ 1 3 8 32 3045564 1 ACTIVE _SYSSMU10_1314081219$ 1 3 10 3 11844457 1 ACTIVE
Transaction id is XID_USN.XID_SLOT.XID_SEQ
So in our case, for the first row it will be 7.13.1829999
2. Read transaction table from undo header.
ALTER SYSTEM DUMP UNDO HEADER ‘_SYSSMU7_881277423$’;
….
TRN TBL::
index state cflags wrap# uel scn dba parent-xid nub stmt_num cmt————————————————————————————————
0×00 9 0×03 0x1bf45c 0x000b 0×0000.789de808 0x00c242eb 0×0000.000.00000000 0×00000001 0x00c242eb 1367258143
0×01 9 0×00 0x1c031b 0×0014 0×0000.789e6018 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258225
0×02 9 0×00 0x1c147a 0x000e 0×0000.789e694b 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258230
0×03 9 0×00 0x1c06f9 0×0016 0×0000.789e601c 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258225
0×04 9 0×00 0x1c06c8 0×0009 0×0000.789e3566 0x00c242f9 0×0000.000.00000000 0×00000001 0×00000000 1367258192
0×05 9 0×00 0x1c1167 0×0015 0×0000.789e357f 0x00c242ec 0×0000.000.00000000 0×00000001 0×00000000 1367258192
0×06 9 0×00 0x1c2716 0×0017 0×0000.789e69e1 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258230
0×07 9 0×00 0x1c1045 0x000c 0×0000.789e1bdb 0x00c242eb 0×0000.000.00000000 0×00000001 0×00000000 1367258170
0×08 9 0×00 0x1c2614 0×0005 0×0000.789e357e 0x00c242ec 0×0000.000.00000000 0×00000001 0×00000000 1367258192
0×09 9 0×00 0x1bfa03 0×0021 0×0000.789e3574 0x00c242f9 0×0000.000.00000000 0×00000001 0×00000000 1367258192
0x0a 9 0×00 0x1bf712 0x001e 0×0000.789e3246 0x00c242f1 0×0000.000.00000000 0×00000001 0×00000000 1367258190
0x0b 9 0×00 0x1c1e01 0×0007 0×0000.789e1bd9 0x00c242eb 0×0000.000.00000000 0×00000001 0×00000000 1367258170
0x0c 9 0×00 0x1c08e0 0x000a 0×0000.789e3244 0x00c242f1 0×0000.000.00000000 0×00000006 0×00000000 1367258190
0x0d 10 0×90 0x1bec6f 0×0038 0×0000.789e783e 0x00c242fb 0×0000.000.00000000 0×00000001 0x00c242fb 0
0x0e 9 0×00 0x1c068e 0×0010 0×0000.789e694d 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258230
0x0f 9 0×00 0x1c151d 0×0012 0×0000.789e3578 0x00c242ec 0×0000.000.00000000 0×00000001 0×00000000 1367258192
0×10 9 0×00 0x1c26bc 0×0006 0×0000.789e69df 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258230
0×11 9 0×00 0x1c16eb 0×0000 0×0000.789cbd77 0x00c242eb 0×0000.000.00000000 0×00000001 0×00000000 1367257923
0×12 9 0×00 0x1c082a 0x001d 0×0000.789e357c 0x00c242ec 0×0000.000.00000000 0×00000001 0×00000000 1367258192
0×13 9 0×00 0x1c1459 0x001f 0×0000.789e7891 0x00c242fc 0×0000.000.00000000 0×00000001 0×00000000 1367258238
0×14 9 0×00 0x1c14b8 0×0003 0×0000.789e601a 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258225
0×15 9 0×00 0x1c0457 0×0020 0×0000.789e39d3 0x00c242ec 0×0000.000.00000000 0×00000001 0×00000000 1367258195
0×16 9 0×00 0x1c1326 0×0002 0×0000.789e601d 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258225
0×17 9 0×00 0x1c0db5 0x001c 0×0000.789e788a 0x00c242fc 0×0000.000.00000000 0×00000001 0×00000000 1367258238
0×18 9 0×00 0x1bffe4 0x001b 0×0000.789e400d 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258200
0×19 9 0×00 0x1c16e3 0×0001 0×0000.789e5fd2 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258225
0x1a 9 0×00 0x1bdbb2 0×0018 0×0000.789e400b 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258200
0x1b 9 0×00 0x1c1141 0×0019 0×0000.789e453a 0x00c242fa 0×0000.000.00000000 0×00000001 0×00000000 1367258204
0x1c 9 0×00 0x1bc9a0 0×0013 0×0000.789e788e 0x00c242fc 0×0000.000.00000000 0×00000001 0×00000000 1367258238
0x1d 9 0×00 0x1c02ef 0×0008 0×0000.789e357d 0x00c242ec 0×0000.000.00000000 0×00000001 0×00000000 1367258192
0x1e 9 0×00 0x1c0b6e 0×0004 0×0000.789e3250 0x00c242f9 0×0000.000.00000000 0×00000009 0×00000000 1367258190
0x1f 9 0×00 0x1c00ad 0xffff 0×0000.789e78a1 0x00c242fc 0×0000.000.00000000 0×00000001 0×00000000 1367258238
0×20 9 0×00 0x1c166c 0x001a 0×0000.789e39dd 0x00c242fa 0×0000.000.00000000 0×00000002 0×00000000 1367258195
0×21 9 0×00 0x1c160b 0x000f 0×0000.789e3576 0x00c242ec 0×0000.000.00000000 0×00000001 0×00000000 1367258192
EXT TRN CTL::
usn: 7
State# 10 means active transaction.
dba points to starting UNDO block address.
usn: Undo segment number
usn.index.wrap# gives transaction id.
An active transaction 0×0007.00d.001bec6f is available in slot 0x0d which has a dba of 0x00c242fb (12731131 in decimal)
3. Reading UNDO Block:
Identify fileID and blockID:
fileID:
select DBMS_UTILITY.DATA_BLOCK_ADDRESS_FILE(12731131) from x$dual;
3
blockID:
select DBMS_UTILITY.DATA_BLOCK_ADDRESS_BLOCK(12731131) from x$dual;
148219
Dumping block
alter system dump datafile 3 block 148219;
UNDO BLK:
xid: 0×0007.00d.001bec6f seq: 0x41f9 cnt: 0×6 irb: 0×5 icl: 0×0 flg: 0×0000
Rec Offset Rec Offset Rec Offset Rec Offset Rec Offset—————————————————————————
0×01 0x1f98 0×02 0x1f2c 0×03 0x1d7c 0×04 0x1d10 0×05 0x1ca0
0×06 0x1bfc
*—————————–* Rec #0×1 slt: 0x0d objn: 0(0×00000000) objd: 0 tblspc: 0(0×00000000)
* Layer: 5 (Transaction Undo) opc: 7 rci 0×00
Undo type: Regular undo Begin trans Last buffer split: NoTemp Object: No
Tablespace Undo: No
rdba: 0x00000000Ext idx: 0
flg2: 0
*—————————–
uba: 0x00c242fa.41f9.37 ctl max scn: 0×0000.789b7668 prv tx scn: 0×0000.789bb8d7
txn start scn: scn: 0×0000.789e783e logon user: 88
prev brb: 12731116 prev bcl: 0
*—————————–* Rec #0×2 slt: 0x0d objn: 110769(0x0001b0b1) objd: 110769 tblspc: 6(0×00000006)
* Layer: 11 (Row) opc: 1 rci 0×00
Undo type: Regular undo User Undo Applied Last buffer split: NoTemp Object: No
Tablespace Undo: No
rdba: 0×00000000
*—————————–
KDO undo record:
KTB Redo
op: 0×04 ver: 0×01
compat bit: 4 (post-11) padding: 1op: L itl: xid: 0×0012.01c.00322281 uba: 0x0102c5f0.3fa9.0a
flg: C— lkc: 0 scn: 0×0000.789ca3f4
KDO Op code: LKR row dependencies Disabled
xtype: XA flags: 0×00000000 bdba: 0x038180fc hdba: 0x018d64e2
itli: 1 ispac: 0 maxfr: 4858
tabn: 0 slot: 14 to: 0
*—————————–* Rec #0×3 slt: 0x0d objn: 110769(0x0001b0b1) objd: 110769 tblspc: 6(0×00000006)
* Layer: 11 (Row) opc: 1 rci 0×02
Undo type: Regular undo User Undo Applied Last buffer split: NoTemp Object: No
Tablespace Undo: No
rdba: 0×00000000
*—————————–
KDO undo record:
KTB Redo
op: 0×02 ver: 0×01
compat bit: 4 (post-11) padding: 1op: C uba: 0x00c242fb.41f9.02
KDO Op code: URP row dependencies Disabled
xtype: XA flags: 0×00000000 bdba: 0x038180fc hdba: 0x018d64e2
itli: 1 ispac: 0 maxfr: 4858
tabn: 0 slot: 14(0xe) flag: 0x2c lock: 1 ckix: 0
ncol: 9 nnew: 6 size: 0
col 1: [ 7] 78 71 04 1d 13 01 01
col 2: [ 2] c1 13
col 3: [ 1] 80
col 4: [16] 10 e5 00 2e 10 d1 10 d0 10 d7 10 e3 10 db 10 d8
col 5: [174]
10 d0 10 ed 10 d0 10 e0 10 d8 10 e1 00 20 10 d0 00 2e 10 e0 00 2e 00 20 10
de 10 e0 10 dd 10 d9 10 e3 10 e0 10 d0 10 e2 10 e3 10 e0 10 d8 10 e1 00 20
10 e1 10 d0 10 d2 10 d0 10 db 10 dd 10 eb 10 d8 10 d4 10 d1 10 dd 00 20 10
dc 10 d0 10 ec 10 d8 10 da 10 d8 10 e1 00 20 10 e3 10 e4 10 e0 10 dd 10 e1
00 20 10 d2 10 d0 10 db 10 dd 10 db 10 eb 10 d8 10 d4 10 d1 10 d4 10 da 10
e1 00 20 10 d1 10 d0 10 e2 10 dd 10 dc 00 20 10 d2 10 d8 10 dd 10 e0 10 d2
10 d8 00 20 10 de 10 d4 10 e0 10 d0 10 dc 10 d8 10 eb 10 d4 10 e1 00 2e
col 6: [36]
00 54 00 01 04 0c 00 00 00 02 00 00 00 01 00 00 09 07 b0 63 00 10 09 00 00
00 00 00 00 00 00 00 00 00 00 00
*—————————–* Rec #0×4 slt: 0x0d objn: 89834(0x00015eea) objd: 93214 tblspc: 6(0×00000006)
* Layer: 11 (Row) opc: 1 rci 0×03
Undo type: Regular undo User Undo Applied Last buffer split: NoTemp Object: No
Tablespace Undo: No
rdba: 0×00000000
*—————————–
KDO undo record:
KTB Redo
op: 0×04 ver: 0×01
compat bit: 4 (post-11) padding: 1op: L itl: xid: 0x000c.017.000d65d6 uba: 0x0103df2c.22a5.20
flg: C— lkc: 0 scn: 0×0000.789c4694
KDO Op code: LKR row dependencies Disabled
xtype: XA flags: 0×00000000 bdba: 0×03833994 hdba: 0x0181f832
itli: 1 ispac: 0 maxfr: 4858
tabn: 0 slot: 7 to: 0
*—————————–* Rec #0×5 slt: 0x0d objn: 89834(0x00015eea) objd: 93214 tblspc: 6(0×00000006)
* Layer: 11 (Row) opc: 1 rci 0×04
Undo type: Regular undo Last buffer split: NoTemp Object: No
Tablespace Undo: No
rdba: 0×00000000
*—————————–
KDO undo record:
KTB Redo
op: 0×02 ver: 0×01
compat bit: 4 (post-11) padding: 1op: C uba: 0x00c242fb.41f9.04
KDO Op code: LMN row dependencies Disabled
xtype: XA flags: 0×00000000 bdba: 0×03833994 hdba: 0x0181f832
itli: 1 ispac: 0 maxfr: 4858
*—————————–* Rec #0×6 slt: 0x0d objn: 89703(0x00015e67) objd: 92020 tblspc: 6(0×00000006)
* Layer: 11 (Row) opc: 1 rci 0×05
Undo type: Regular undo User Undo Applied Last buffer split: NoTemp Object: No
Tablespace Undo: No
rdba: 0×00000000
*—————————–
KDO undo record:
irb points to last UNDO RECORD in UNDO block.
rci points to previous UNDO RECORD. if rci=0, it’s the first UNDO RECORD.
Recovery operation starts from irb and chain is followed by rci until rci is zero.
The transaction starts recovery from UNDO RECORD of 0×5.
4. Reading UNDO Records:
* Rec #0×5 slt: 0x0d objn: 89834(0x00015eea) objd: 93214 tblspc: 6(0×00000006)* Layer: 11 (Row) opc: 1 rci 0×04
….* Rec #0×4 slt: 0x0d objn: 89834(0x00015eea) objd: 93214 tblspc: 6(0×00000006)
* Layer: 11 (Row) opc: 1 rci 0×03
….* Rec #0×3 slt: 0x0d objn: 110769(0x0001b0b1) objd: 110769 tblspc: 6(0×00000006)
* Layer: 11 (Row) opc: 1 rci 0×02
…* Rec #0×2 slt: 0x0d objn: 110769(0x0001b0b1) objd: 110769 tblspc: 6(0×00000006)
* Layer: 11 (Row) opc: 1 rci 0×00
…
objn means object id.
5. Find these objects
The following objects need recovery:
select * from dba_objects
where object_id in (89834,110769);
………………………………………………………..
This problem is Oracle Bug:9857702:
..... Affects: Product (Component) Oracle Server (Rdbms) Range of versions believed to be affected Versions >= 11.1 but BELOW 12.1 Versions confirmed as being affected •11.2.0.1 •11.1.0.7 Platforms affected Generic (all / most platforms affected) Fixed: This issue is fixed in •12.1 (Future Release) •11.2.0.2 (Server Patch Set) •11.1.0.7.8 Patch Set Update •11.1.0.7 Patch 40 on Windows Platforms .....
6. Workaround:
- Recreate objects that need recovery.
- Or drop them