目录
1、问题现象
2022.01.18日晚22点,青包前端无法刷新看板,时间节点是22点
2、分析现象
此时间节点正好是数据库系统默认的统计信息采集时间
(1)查看altert.log
发现有报错,这个时间点有锁导致了错误,继续跟踪j001_163530.trc文件
(2)查看j001_163530.trc文件
(3)再次查看其他时间点的ORA-04021报错:
2022.1.17 22点,2022.1.18 22点,以及2021.12.4,2021.11.29 2021.9.19 这几天发生过。
从上述看,大约和mes_dept等表有冲突。
3、分析awr日志
发现一个问题,有两条update sql语句在17点左右一直存在,并未释放,分析可能没有commit;
4、解决或优化思路:
(1)优化SQL
根据上述的描述,update这个语句有些问题,需要开发排查一下。
SELECT MESDEPT.ROW_ID , MESDEPT.DEPTID , MESDEPT.FAID , MESDEPT.DEPTCODE , MESDEPT.DEPTNAME, MESDEPT.DEPTNAMEJC , MESDEPT.STATE , MESDEPT.FACTORYID , MESDEPT.COID , MESDEPT.EMPLOYEE , MESDEPT.FSAP , MESDEPT.FGX , MESDEPT.FWX , MESDEPT.FWC , MESDEPT.FCJ , MESDEPT.FAALLCODE , MESDEPT.MTCODE , MESDEPT.UPDATE_DATE , MESDEPT.GXSX , MESDEPT.XHBS , MESDEPT.REMARK , MESDEPT.ENABLEFLAG , MESDEPT.CREATEBY , MESDEPT.LASTUPDATEBY , MESDEPT.BUFFER1 , MESDEPT.BUFFER2 , MESDEPT.BUFFER3 , MESDEPT.BUFFER4 , MESDEPT.BUFFER5 , MESDEPT.BUFFER6 , MESDEPT.BUFFER7 , MESDEPT.BUFFER8 , MESDEPT.BUFFER9 , MESDEPT.BUFFER10 , MESDEPT.BUFFER11 , MESDEPT.BUFFER12 , MESDEPT.BUFFER13 , MESDEPT.BUFFER14 , MESDEPT.BUFFER15 , MESDEPT.BUFFER16 , MESDEPT.BUFFER17 , MESDEPT.BUFFER18 , MESDEPT.BUFFER19 , MESDEPT.BUFFER20 , MESDEPT.BUFFER21 , MESDEPT.BUFFER22 , MESDEPT.BUFFER23 , MESDEPT.BUFFER24 , MESDEPT.BUFFER25 , MESDEPT.BUFFER26 , MESDEPT.BUFFER27 , MESDEPT.BUFFER28 , MESDEPT.BUFFER29 , MESDEPT.BUFFER30 , MESDEPT.FNET_W EIGHT, MESDEPT.FMATE_WEIGHT, MESDEPT.FGEPAN_WEIGHT, MESDEPT.FGEDU_WEIGHT, MESDEPT.FGROSS_WEIGHT FROM MES_DEPT MESDEPT WHERE MESDEPT.DEPTID IN ( SELECT DISTINCT cDeptCode as DEPTID from sb_Department START WITH sysDeptID in ( SELECT DISTINCT A.sysDeptID FROM sa_RoleDept A INNER JOIN sb_Department B ON A.sysDeptID=B.sysDeptID INNER JOIN sa_UserRole C ON A.sysRoleID=C.sysRoleID INNER JOIN sb_Role D ON C.sysRoleID=D.sysRoleID where A.cFuncCode='F2752' and B.bDel=0 and B.bUse=1 and D.bDel=0 and D.bUse=1 and C.sysUserID='1788' union select to_number(B.cFeature31) from sb_User A INNER JOIN sb_Person B ON A.sysPerID=B.cFeature00 where A.ID='1788' union select sysdeptid from sb_department where cdeptcode in ( select proid from mes_wc_proper where enableflag='Y' and personcode in (select cusername from sb_user where id='1788') ) and buse=1 and bdel=0 ) CONNECT BY sysFaID = PRIOR sysDeptID ) AND ENABLEFLAG='Y' AND NVL(MESDEPT.STATE, '0') = '1' AND FGX IS NOT NULL AND FCJ IS NOT NULL ORDER BY FAID , to_number(nvl(gxsx, 0))
update时间过长,可以排查一下:
UPDATE MES_SO_MAIN SET LASTUPDATEDATE = TO_DATE('2022-01-18 17:59:07', 'YYYY-MM-DD HH24:MI:SS'), LASTUPDATEBY = '16000112', DOCSTATE = '45' where 1 = 1 and DOCENTRY='1086202201180035' UPDATE MES_SO_MAIN SET LASTUPDATEDATE = TO_DATE('2022-01-18 17:58:32', 'YYYY-MM-DD HH24:MI:SS'), LASTUPDATEBY = '16000112', DOCSTATE = '45' where 1 = 1 and DOCENTRY='1086202201180037'
(2)调整信息统计采集时间
当前是周一到周五晚上10点-2点 (4个小时),可以将收集改到其他业务不繁忙的时间进行。
(3)应急处理(因为会占用CPU),如果要终止,执行如下操作
select * from v$session_longops where opname like '%Gather%'; select distinct sid, serial# from v$session_longops where opname like '%Gather%'; alter system kill session '142,569'; alter system kill session '19,729'; alter system kill session '144,659'; alter system kill session '17,265';