循序渐进调优union相关的sql(r2笔记23天)
时间:2022-05-04
本文章向大家介绍循序渐进调优union相关的sql(r2笔记23天)，主要内容包括其使用实例、应用技巧、基本知识点总结和需要注意事项，具有一定的参考价值，需要的朋友可以参考一下。
今天在生产中发现一条sql语句消耗了大量的cpu资源。使用top -c来查看。
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
17895 oratestdb 25 0 12.4g 217m 38m R 99.9 0.1 1232:43 oracleTESTDB (LOCAL=NO)
12318 oratestdb 18 0 12.2g 35m 28m R 54.0 0.0 10:47.89 oracleTESTDB (LOCAL=NO)
26316 oratestdb 18 0 12.3g 118m 28m D 21.2 0.1 0:40.33 oracleTESTDB (LOCAL=NO) 查看进程对应的session正在运行的sql语句。 看这条语句倒也不复杂，account_id对应的是主键，查询应该是毫秒级的，但是查看生产中执行的效率，平均在3-5秒左右。
SELECT TEST_TEST_SUBSCRIBER_FA_V.MOBILE_NO, TEST_ACCOUNT.ACCOUNT_ID FROM TEST_ACCOUNT ,TEST_TEST_SUBSCRIBER_FA_V
WHERE TEST_ACCOUNT.ACCOUNT_ID = TEST_TEST_SUBSCRIBER_FA_V.BAN AND TEST_ACCOUNT.ACCOUNT_ID = xxxxx查看这条语句对应的执行计划信息，如下：
PLAN_TABLE_OUTPUT
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID 6b1yv0ucyjsk3, child number 0
-------------------------------------
/* TEST_TESTGetSubscriberNumber_selectGetMobileNo_0 */ SELECT
TEST_TEST_SUBSCRIBER_FA_V.MOBILE_NO, TEST_ACCOUNT.ACCOUNT_ID FROM
TEST_ACCOUNT ,TEST_TEST_SUBSCRIBER_FA_V WHERE TEST_ACCOUNT.ACCOUNT_ID =
TEST_TEST_SUBSCRIBER_FA_V.BAN AND TEST_ACCOUNT.ACCOUNT_ID = :1
Plan hash value: 1685959897
---------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
---------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | | 56898 (100)| |
| 1 | NESTED LOOPS | | 536K| 61M| | 56898 (1)| 00:11:23 |
|* 2 | INDEX UNIQUE SCAN | TEST_ACCOUNT_PK | 1 | 6 | | 1 (0)| 00:00:01 |
| 3 | VIEW | TEST_TEST_SUBSCRIBER_FA_V |536K| 58M| | 56897 (1)| 00:11:23 |
| 4 | SORT UNIQUE | | 536K| 98M| 113M| 56897 (100)| 00:11:23 |
| 5 | UNION-ALL | | | | | | |
| 6 | NESTED LOOPS | | | | | | |
| 7 | NESTED LOOPS | | 1 | 89 | | 3 (0)| 00:00:01 |
| 8 | NESTED LOOPS | | 1 | 31 | | 2 (0)| 00:00:01 |
|* 9 | INDEX RANGE SCAN | TEST_PAY_CHANNEL_1IX | 1 | 12 | | 1 (0)| 00:00:01 |
|* 10 | TABLE ACCESS BY INDEX ROWID| TEST_DISTRIBUTE | 1 | 19 | | 1 (0)| 00:00:01 |
|* 11 | INDEX RANGE SCAN | TEST_DISTRIBUTE_3IX | 1 | | | 1 (0)| 00:00:01 |
|* 12 | INDEX UNIQUE SCAN | SUBSCRIBER_PK | 1 | | | 1 (0)| 00:00:01 |
|* 13 | TABLE ACCESS BY INDEX ROWID | SUBSCRIBER | 1 | 58 | | 1 (0)| 00:00:01 |
|* 14 | VIEW | | 536K| 98M| | 34296 (1)| 00:06:52 |
|* 15 | WINDOW SORT PUSHED RANK | | 536K| 45M| 51M| 34296 (1)| 00:06:52 |
|* 16 | HASH JOIN | | 536K| 45M| 28M| 23341 (1)| 00:04:41 |
| 17 | INDEX FULL SCAN | TEST_PAY_CHANNEL_1IX | 1259K| 14M| | 1254 (1)| 00:00:16 |
|* 18 | HASH JOIN | | 536K| 39M| 26M| 18391 (1)| 00:03:41 |
|* 19 | TABLE ACCESS FULL | SUBSCRIBER | 397K| 21M| | 8527 (1)| 00:01:43 |
|* 20 | TABLE ACCESS FULL | TEST_DISTRIBUTE | 1696K| 30M| | 6046 (1)| 00:01:13 |
---------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
2 - access("TEST_ACCOUNT"."ACCOUNT_ID"=TO_NUMBER(:1))
9 - access("CPC"."BAN"=TO_NUMBER(:1))
10 - filter("EG_DIST_TYPE"='D')
11 - access("CPC"."PYM_CHANNEL_NO"="ED"."TARGET_PCN" AND "ED"."EXPIRATION_DATE" IS NULL)
filter("ED"."EXPIRATION_DATE" IS NULL)
12 - access("ED"."AGREEMENT_NO"="SUBSCRIBER"."SUBSCRIBER_NO")
13 - filter(("SUBSCRIBER"."SUB_STATUS"<>'C' AND "SUBSCRIBER"."SUB_STATUS"<>'L' AND
"SUBSCRIBER"."SUB_STATUS"<>'T'))
14 - filter(("RANK"=1 AND "BAN"=TO_NUMBER(:1)))
15 - filter(ROW_NUMBER() OVER ( PARTITION BY "SUBSCRIBER"."SUBSCRIBER_NO" ORDER BY
"ED"."EXPIRATION_DATE")<=1)
16 - access("CPC"."PYM_CHANNEL_NO"="ED"."TARGET_PCN")
18 - access("ED"."AGREEMENT_NO"="SUBSCRIBER"."SUBSCRIBER_NO")
19 - filter(("SUBSCRIBER"."SUB_STATUS"='C' OR "SUBSCRIBER"."SUB_STATUS"='L' OR
"SUBSCRIBER"."SUB_STATUS"='T'))
20 - filter("EG_DIST_TYPE"='D')
---从执行计划可以看出，瓶颈就在于有两个表走了全表扫描。那两个表数据也不少。
---统计信息如下：
Elapsed: 00:00:03.53
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
96346 consistent gets
0 physical reads
0 redo size
613 bytes sent via SQL*Net to client
520 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
1 rows processed
其中TEST_TEST_SUBSCRIBER_FA_V 是一个视图，里面使用了Union,（另外关于这个union的地方和开发确认过，暂时还不能改为union all)。
把视图的内容填进去，sql语句就成了如下的样子，在数据库里执行的时候也基本是这个样子的。对于在查询中没有用到的字段都给注释掉了。标注为灰色。
SELECT MOBILE_NO, TEST_ACCOUNT.ACCOUNT_ID
FROM
(
SELECT cpc.ban,
subscriber.prim_resource_val MOBILE_NO,
-- subscriber.init_act_date,
-- SUBSCRIBER.CUSTOMER_ID,
-- subscriber.subscriber_no,
-- SUBSCRIBER.SUBSCRIBER_TYPE,
-- SUBSCRIBER.SUB_STATUS,
-- SUBSCRIBER.SUB_STS_RSN_CD,
-- SUBSCRIBER.SUB_STATUS_DATE,
-- SUBSCRIBER.EFFECTIVE_DATE,
1 RANK
FROM subscriber, TEST_distribute ed, TEST_pay_channel cpc
WHERE cpc.pym_channel_no = ed.target_pcn
AND ed.agreement_no = subscriber.subscriber_no
AND eg_dist_type = 'D'
AND ed.expiration_date IS NULL
AND SUBSCRIBER.SUB_STATUS NOT IN ('C', 'L', 'T')
)temp, TEST_ACCOUNT
where TEST_ACCOUNT.account_id=temp.ban and TEST_ACCOUNT.account_id=10001245
UNION
SELECT MOBILE_NO, TEST_ACCOUNT.ACCOUNT_ID
FROM
(
SELECT "BAN","PRIM_RESOURCE_VAL" MOBILE_NO--,"INIT_ACT_DATE","CUSTOMER_ID","SUBSCRIBER_NO","SUBSCRIBER_TYPE","SUB_STATUS","SUB_STS_RSN_CD","SUB_STATUS_DATE","EFFECTIVE_DATE","RANK"
FROM (SELECT cpc.ban,
subscriber.prim_resource_val ,
-- subscriber.init_act_date,
-- SUBSCRIBER.CUSTOMER_ID,
-- subscriber.subscriber_no,
-- SUBSCRIBER.SUBSCRIBER_TYPE,
-- SUBSCRIBER.SUB_STATUS,
-- SUBSCRIBER.SUB_STS_RSN_CD,
-- SUBSCRIBER.SUB_STATUS_DATE,
-- SUBSCRIBER.EFFECTIVE_DATE,
ROW_NUMBER ()
OVER (
PARTITION BY subscriber.subscriber_no
ORDER BY
ed.expiration_date, subscriber.subscriber_no DESC)
AS RANK
FROM subscriber, TEST_distribute ed, TEST_pay_channel cpc
WHERE cpc.pym_channel_no = ed.target_pcn
AND ed.agreement_no = subscriber.subscriber_no
AND eg_dist_type = 'D'
AND SUBSCRIBER.SUB_STATUS IN ('C', 'L', 'T'))
WHERE RANK = 1
)temp, TEST_ACCOUNT
where TEST_ACCOUNT.account_id=temp.ban and TEST_ACCOUNT.account_id=10001245
查看走全表扫描的两个表，TEST_pay_channel上有一个索引列中含有ban,从如上的执行计划中看出，没有尝试走索引，而是走了全表扫描，受此影响，TEST_distribute 也做了全表扫描。
根据查询条件TEST_ACCOUNT.account_id和TEST_pay_channel的ban是关联的，如果可以走索引的话，效率会大大提高。
TEST_PAY_CHANNEL和 TEST_DISTRIBUTE 中索引的信息如下：
INDEX_NAME TABLESPACE INDEX_TYPE UNIQUENES PAR COLUMN_LIST TABLE_TYPE STATUS NUM_ROWS LAST_ANAL G
------------------------------ ---------- ---------- --------- --- ------------------------------ ---------- ------ ---------- --------- -
TEST_PAY_CHANNEL_1IX INDXS01 NORMAL NONUNIQUE NO BAN,BEN, CHANNEL_NO TABLE VALID 1229972 01-JUL-14 N
TEST_DISTRIBUTE_3IX INDXS01 NORMAL NONUNIQUE NO TARGET_PCN,AGR_NO,EFFECT_DATE TABLE VALID 1597330 01-JUL-14 N
如果根据ban来关联的话，至少可以走一个range index scan或者skip/scan了.
如果可以走索引的话。条件 “cpc.pym_channel_no = ed.target_pcn”对应的索引就可以激活了，test_distribute表也就不走全表扫描了,可以走range scan/或者skip scan了。
现在的工作就是来看看能不能不改动逻辑来看看能做些什么。
改动后的sql语句如下。首先是弃用了原有的视图。然后在union的两个子查询中使用TEST_ACCOUNT来和TEST_PAY_CHANNEL来做关联。虽然关联的表多了一个，但是因为都可以走索引，比全表扫描要效率高很多了。
SELECT MOBILE_NO, ACCOUNT_ID
FROM
(
select prim_resource_val MOBILE_NO,ACCOUNT_ID,rank
from (
SELECT cpc.ban,TEST_ACCOUNT.ACCOUNT_ID,
subscriber.prim_resource_val,
1 RANK
FROM subscriber, TEST_distribute ed, TEST_pay_channel cpc,TEST_ACCOUNT
WHERE cpc.pym_channel_no = ed.target_pcn
AND ed.agreement_no = subscriber.subscriber_no
AND eg_dist_type = 'D'
AND ed.expiration_date IS NULL
AND SUBSCRIBER.SUB_STATUS NOT IN ('C', 'L', 'T')
AND TEST_ACCOUNT.ACCOUNT_ID = cpc.BAN AND TEST_ACCOUNT.ACCOUNT_ID = 10001245
)TEST_TEST_SUBSCRIBER
UNION
select TEST_TEST_SUBSCRIBER.prim_resource_val MOBILE_NO,TEST_TEST_SUBSCRIBER.ACCOUNT_ID,rank
from
(SELECT cpc.ban, subscriber.prim_resource_val, TEST_ACCOUNT.ACCOUNT_ID ,
ROW_NUMBER ()
OVER (
PARTITION BY subscriber.subscriber_no
ORDER BY
ed.expiration_date, subscriber.subscriber_no DESC)
AS RANK
FROM subscriber, TEST_distribute ed, TEST_pay_channel cpc,TEST_ACCOUNT
WHERE cpc.pym_channel_no = ed.target_pcn
AND ed.agreement_no = subscriber.subscriber_no
AND eg_dist_type = 'D'
AND SUBSCRIBER.SUB_STATUS IN ('C', 'L', 'T')
AND TEST_ACCOUNT.ACCOUNT_ID = cpc.BAN AND TEST_ACCOUNT.ACCOUNT_ID =10001245
) TEST_TEST_SUBSCRIBER
) where rank=1
来看看执行计划和统计信息和执行速度。
首先是执行速度，用了0.01秒，相比原来的3秒左右提高了不少，毫秒级的速度。
cpu的消耗一下子到了11，提高了很多倍
两个全本走全表扫描的表，现在都走了index range scan.
统计信息的提升，也提高了很多。从原来的96346降低到了目前的23
Elapsed: 00:00:00.01
Execution Plan
----------------------------------------------------------
Plan hash value: 1502756759
------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 256 | 11 (28)| 00:00:01 |
| 1 | VIEW | | 2 | 256 | 11 (28)| 00:00:01 |
| 2 | SORT UNIQUE | | 2 | 184 | 11 (64)| 00:00:01 |
| 3 | UNION-ALL | | | | | |
| 4 | NESTED LOOPS | | | | | |
| 5 | NESTED LOOPS | | 1 | 56 | 4 (0)| 00:00:01 |
| 6 | NESTED LOOPS | | 1 | 37 | 3 (0)| 00:00:01 |
| 7 | NESTED LOOPS | | 1 | 18 | 2 (0)| 00:00:01 |
|* 8 | INDEX UNIQUE SCAN | TEST_ACCOUNT_PK | 1 | 6 | 1 (0)| 00:00:01 |
|* 9 | INDEX RANGE SCAN | TEST_PAY_CHANNEL_1IX | 1 | 12 | 1 (0)| 00:00:01 |
|* 10 | TABLE ACCESS BY INDEX ROWID | TEST_DISTRIBUTE | 1 | 19 | 1 (0)| 00:00:01 |
|* 11 | INDEX RANGE SCAN | TEST_DISTRIBUTE_3IX | 1 | | 1 (0)| 00:00:01 |
|* 12 | INDEX UNIQUE SCAN | SUBSCRIBER_PK | 1 | | 1 (0)| 00:00:01 |
|* 13 | TABLE ACCESS BY INDEX ROWID | SUBSCRIBER | 1 | 19 | 1 (0)| 00:00:01 |
|* 14 | VIEW | | 1 | 128 | 5 (20)| 00:00:01 |
|* 15 | WINDOW SORT PUSHED RANK | | 1 | 56 | 5 (20)| 00:00:01 |
| 16 | NESTED LOOPS | | | | | |
| 17 | NESTED LOOPS | | 1 | 56 | 4 (0)| 00:00:01 |
| 18 | NESTED LOOPS | | 1 | 37 | 3 (0)| 00:00:01 |
| 19 | NESTED LOOPS | | 1 | 18 | 2 (0)| 00:00:01 |
|* 20 | INDEX UNIQUE SCAN | TEST_ACCOUNT_PK | 1 | 6 | 1 (0)| 00:00:01 |
|* 21 | INDEX RANGE SCAN | TEST_PAY_CHANNEL_1IX | 1 | 12 | 1 (0)| 00:00:01 |
|* 22 | TABLE ACCESS BY INDEX ROWID| TEST_DISTRIBUTE | 1 | 19 | 1 (0)| 00:00:01 |
|* 23 | INDEX RANGE SCAN | TEST_DISTRIBUTE_3IX | 1 | | 1 (0)| 00:00:01 |
|* 24 | INDEX UNIQUE SCAN | SUBSCRIBER_PK | 1 | | 1 (0)| 00:00:01 |
|* 25 | TABLE ACCESS BY INDEX ROWID | SUBSCRIBER | 1 | 19 | 1 (0)| 00:00:01 |
------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
8 - access("TEST_ACCOUNT"."ACCOUNT_ID"=10001245)
9 - access("CPC"."BAN"=10001245)
10 - filter("EG_DIST_TYPE"='D')
11 - access("CPC"."PYM_CHANNEL_NO"="ED"."TARGET_PCN" AND "ED"."EXPIRATION_DATE" IS NULL)
filter("ED"."EXPIRATION_DATE" IS NULL)
12 - access("ED"."AGREEMENT_NO"="SUBSCRIBER"."SUBSCRIBER_NO")
13 - filter("SUBSCRIBER"."SUB_STATUS"<>'C' AND "SUBSCRIBER"."SUB_STATUS"<>'L' AND
"SUBSCRIBER"."SUB_STATUS"<>'T')
14 - filter("RANK"=1)
15 - filter(ROW_NUMBER() OVER ( PARTITION BY "SUBSCRIBER"."SUBSCRIBER_NO" ORDER BY
"ED"."EXPIRATION_DATE")<=1)
20 - access("TEST_ACCOUNT"."ACCOUNT_ID"=10001245)
21 - access("CPC"."BAN"=10001245)
22 - filter("EG_DIST_TYPE"='D')
23 - access("CPC"."PYM_CHANNEL_NO"="ED"."TARGET_PCN")
24 - access("ED"."AGREEMENT_NO"="SUBSCRIBER"."SUBSCRIBER_NO")
25 - filter("SUBSCRIBER"."SUB_STATUS"='C' OR "SUBSCRIBER"."SUB_STATUS"='L' OR
"SUBSCRIBER"."SUB_STATUS"='T')
Statistics
----------------------------------------------------------
1 recursive calls
0 db block gets
23 consistent gets
3 physical reads
0 redo size
613 bytes sent via SQL*Net to client
520 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
1 rows processed
有了一定的数据保证，就需要在测试环境中进行了模拟测试，可以比对是否和预期的一样有极大的提升。
- HDU----(2157)How many ways??(快速矩阵幂)
- 试试Linux下的ip命令
- hdu---(2604)Queuing(矩阵快速幂)
- centos7下卸载python后yum不能使用的恢复方法
- hdu---（5038）Grade（胡搞）
- shell生成随机字符的几种方法
- Python时间获取及转换
- spark streaming知识总结[优化]
- 让你真正明白spark streaming
- Centos7 firewalld防火墙基本操作
- Spark Sql系统入门4：spark应用程序中使用spark sql
- Flume+Kafka收集Docker容器内分布式日志应用实践
- CentOS7搭建ELK日志集中分析平台
- Centos安装sshfs实现挂载目录
- JavaScript 教程
- JavaScript 编辑工具
- JavaScript 与HTML
- JavaScript 与Java
- JavaScript 数据结构
- JavaScript 基本数据类型
- JavaScript 特殊数据类型
- JavaScript 运算符
- JavaScript typeof 运算符
- JavaScript 表达式
- JavaScript 类型转换
- JavaScript 基本语法
- JavaScript 注释
- Javascript 基本处理流程
- Javascript 选择结构
- Javascript if 语句
- Javascript if 语句的嵌套
- Javascript switch 语句
- Javascript 循环结构
- Javascript 循环结构实例
- Javascript 跳转语句
- Javascript 控制语句总结
- Javascript 函数介绍
- Javascript 函数的定义
- Javascript 函数调用
- Javascript 几种特殊的函数
- JavaScript 内置函数简介
- Javascript eval() 函数
- Javascript isFinite() 函数
- Javascript isNaN() 函数
- parseInt() 与 parseFloat()
- escape() 与 unescape()
- Javascript 字符串介绍
- Javascript length属性
- javascript 字符串函数
- Javascript 日期对象简介
- Javascript 日期对象用途
- Date 对象属性和方法
- Javascript 数组是什么
- Javascript 创建数组
- Javascript 数组赋值与取值
- Javascript 数组属性和方法
- XSS Cheat Sheet
- 免费代理池的实现与优化
- 基于Django和clean-blog前端框架的博客系统
- 应急响应笔记之Linux篇
- Access数据库相关知识
- 11个技巧让你编写出更好的Python代码
- 芯片探针到基因组区段坐标的映射
- CDH5升级到CDP7.1
- 底层架构真的折磨死个人（急，在线等）
- Python基础语法(五)—常用模块和模块的安装和导入
- Python计算文件或字符串的MD5/SHA
- .NETCore中实现ObjectId反解
- GSE16561数据集的文章图表复现，小众的illumina表达量芯片
- 数据挖掘课程能带给你什么收获
- AkShare-股票数据-龙虎榜-机构席位成交明细