XLOG段文件跳号现象分析

时间:2022-07-28
本文章向大家介绍XLOG段文件跳号现象分析，主要内容包括其使用实例、应用技巧、基本知识点总结和需要注意事项，具有一定的参考价值，需要的朋友可以参考一下。

一、原理

当执行promote时，我们经常看到的结果是：生成一个新XLOG文件，名称为：时间线加1，段文件名为之前的段文件号。那么做这个动作的时机是什么时候呢？是否只有这一种现象，会不会有其他现象？先透露下，当执行promote动作前，最后一个XLOG日志是SWITCH时，段文件号会加1。下面我们对其流程做详细分析，并通过gdb理解其原理。

做这个动作的函数是exitArchiveRecovery，调用时机为startup进程退出的时刻，见堆栈：

Breakpoint 1, exitArchiveRecovery (endTLI=1, endOfLog=23053968) at xlog.c:5475
5475    InArchiveRecovery = false;
(gdb) bt
#0  exitArchiveRecovery (endTLI=1, endOfLog=23053968) at xlog.c:5475
#1  0x0815fc4b in StartupXLOG () at xlog.c:7460
#2  0x083dbc56 in StartupProcessMain () at startup.c:207
#3  0x08173cbf in AuxiliaryProcessMain (argc=2, argv=0xbfa8a2f4) at bootstrap.c:451
#4  0x083dab93 in StartChildProcess (type=StartupProcess) at postmaster.c:5386
#5  0x083d5d86 in PostmasterMain (argc=1, argv=0xa22e7e8) at postmaster.c:1369
#6  0x0831c76c in main (argc=1, argv=0xa22e7e8) at main.c:228

具体代码行为恢复完成之后：

StartupXLOG->读取checkpoint->恢复->exitArchiveRecovery：EndOfLog为当前回放日志最后的位置，EndOfLogTLI为当前退出时回放日志的时间线。当data目录下有standby.signal文件即该机器是备时ArchiveRecoveryRequested为TRUE。

exitArchiveRecovery函数调用流程如下：首先通过endOfLog即回放最后的位置计算出段文件日志号：endLogSegNo= (endOfLog - 1) / (wal_segment_size)；startLogSegNo= endOfLog / (wal_segment_size)。如果endLogSegNo等于startLogSegNo，表示回放位置为文件中间位置，在调用XLogFileCopy生成一个新文件，并将上个XLOG文件内容拷贝到新文件中；段文件号相同，时间线加1。如果endLogSegNo不等于startLogSegNo，即回放位置正好是文件大小的末尾处，或者正好是SWITCH这个日志，那么调用XLogFileInit函数进行初始化文件：

XlogFileCopy函数调用：调用XLogFilePath函数获取源XLOG文件名，OpenTransientFile打开该文件，创建并打开一个临时XLOG文件pg_wal/xlogtemp.pid，pid为startup进程的ID号。sizeof(buffer)为一页大小8192字节，从源文件每次read一页数据并将之write到xlogtemp文件，最后一页数据如果不够8192字节，则有多少读取多少并写入文件。当文件拷贝完成后，执行一次sync。最后调用InstallXLogFileSegment将文件重命名。

InstallXLogFileSegment函数：XlogFileCopy调用时，find_free为false，直接将文件重命名为时间线加1的文件名；XLogFileInit调用时为TRUE，将段文件号加1后（注意这里不是加1，是因为正好是文件末尾，求得的是下一个段文件号，只是现象上看是加1），重命名为时间线加1的文件，会先stat下这个文件，该流程返回值是2即该文件不存在，所以不会再将segno加1，直接跳过虚框内的步骤，进入重命名流程durable_like_or_rename。

XLogFileInit函数的调用：首先获取新文件的文件名，即时间线加1，段文件名为原文件名，本次exitArchiveRecovery函数的调用流程中，use_existent为TRUE所以会视图打开该文件。当然因为该文件不存在所以打开失败。然后创建并打开一个临时文件xlogtemp.pid，将该文件全部清0，最后sync。之后调用InstallXLogFileSegment函数重命名。最后打开新文件以供之后使用。

二、GDB跟踪-lsn位置在xlog文件中间

1、进入第一个断点，即函数入口

Breakpoint 1, exitArchiveRecovery (endTLI=1, endOfLog=23053968) at xlog.c:5475
5475    InArchiveRecovery = false;
(gdb) bt
#0  exitArchiveRecovery (endTLI=1, endOfLog=23053968) at xlog.c:5475
#1  0x0815fc4b in StartupXLOG () at xlog.c:7460
#2  0x083dbc56 in StartupProcessMain () at startup.c:207
#3  0x08173cbf in AuxiliaryProcessMain (argc=2, argv=0xbfa8a2f4) at bootstrap.c:451
#4  0x083dab93 in StartChildProcess (type=StartupProcess) at postmaster.c:5386
#5  0x083d5d86 in PostmasterMain (argc=1, argv=0xa22e7e8) at postmaster.c:1369
#6  0x0831c76c in main (argc=1, argv=0xa22e7e8) at main.c:228

2、接着计算出段文件号，这两个值相等，即执行promote时，lsn最后位置在文件中间。

5507    if (endLogSegNo == startLogSegNo)
(gdb) 
5517             XLogSegmentOffset(endOfLog, wal_segment_size));
(gdb) p endLogSegNo
$1 = 1
(gdb) p startLogSegNo
$2 = 1

3、lsn在文件中间，调用XlogFileCopy，upto为lsn在最后文件的偏移

XLogFileCopy (destsegno=1, srcTLI=1, srcsegno=1, upto=6276752) at xlog.c:3405
3405    XLogFilePath(path, srcTLI, srcsegno, wal_segment_size);
(gdb) p 23053968%(16*1024*1024)
$3 = 6276752

4、打开原文件000000010000000000000001，以及临时文件xlogtemp.29683

3406    srcfd = OpenTransientFile(path, O_RDONLY | PG_BINARY);
(gdb) p path
$4 = "pg_wal/00000001", '0' <repeats 15 times>, "1", '00' <repeats 869 times>, "b221250277gmEb0405ĥ0000000130221250277bnEb0405ĥ00000001X221250277$207Eb0405ĥ00000001", '00' <repeats 13 times>"270, 04246X22125027700000000000000 0027004000000000000000000n0000000027004246270221250277dd25b0005ĥ352320ab341n0000~16bb000000"
(gdb) n
(gdb) p tmppath
$5 = "pg_wal/xlogtemp.29683", '00' <repeats 1002 times>
(gdb) n
3420  fd = OpenTransientFile(tmppath, O_RDWR | O_CREAT | O_EXCL | PG_BINARY);

5、循环进行拷贝，一次拷贝一页8192字节

3429    for (nbytes = 0; nbytes < wal_segment_size; nbytes += sizeof(buffer))
(gdb) p sizeof(buffer)
$6 = 8192
(gdb) n
3433      nread = upto - nbytes;
(gdb) 
3439      if (nread < sizeof(buffer))
(gdb) p nread
$7 = 6276752
(gdb) p 6276752/8192
$8 = 766

6、InstallXLogFileSegment函数重命名，path为000000020000000000000001

Breakpoint 2, InstallXLogFileSegment (segno=0xbfa86968, tmppath=0xbfa88974 "pg_wal/xlogtemp.29683", find_free=false, max_segno=0, use_lock=false) at xlog.c:3545
3545  XLogFilePath(path, ThisTimeLineID, *segno, wal_segment_size);

(gdb) p path
$10 = "pg_wal/00000002", '0' <repeats 15 times>, "100 21222251322"[0000000000277 `b204211250277347e2502770500000003000000 NҬ030000000200000005000000;0000000000000030f25027720632`b00000000350h250277350h250277350h250277240.{262d000000363s00000000000024020[264030000000000000003000000240362:26603000000220251t267030000002403243227003000000220213T27103000000 361032720300000020250=27303000000 323343273322"[2968300000000n", '00' <repeats 15 times>, "01000000373333yb050000000000000001000000270240250277310h25027722517`b363s000000000000d", '00' <repeats 27 times>"350"...
(gdb) n

7、将临时文件重命名为000000020000000000000001

3579    if (durable_link_or_rename(tmppath, path, LOG) != 0)
(gdb) p path
$11 = "pg_wal/00000002", '0' <repeats 15 times>, "100 21222251322"[0000000000277 `b204211250277347e2502770500000003000000 NҬ030000000200000005000000;0000000000000030f25027720632`b00000000350h250277350h250277350h250277240.{262d000000363s00000000000024020[264030000000000000003000000240362:26603000000220251t267030000002403243227003000000220213T27103000000 361032720300000020250=27303000000 323343273322"[2968300000000n", '00' <repeats 15 times>, "01000000373333yb050000000000000001000000270240250277310h25027722517`b363s000000000000d", '00' <repeats 27 times>"350"...
(gdb) n

三、GDB跟踪-lsn位置在xlog文件尾或最后一个为SWITCH

1、lsn位于文件尾，调用函数XLogFileInit

Breakpoint 3, exitArchiveRecovery (endTLI=3, endOfLog=50331648) at xlog.c:5475
5475    InArchiveRecovery = false;
(gdb) n
5480    UpdateMinRecoveryPoint(InvalidXLogRecPtr, true);
(gdb) 
5486    if (readFile >= 0)
(gdb) 
5488      close(readFile);
(gdb) 
5489      readFile = -1;
(gdb) 
5498    XLByteToPrevSeg(endOfLog, endLogSegNo, wal_segment_size);
(gdb) 
5499    XLByteToSeg(endOfLog, startLogSegNo, wal_segment_size);
(gdb) 
5507    if (endLogSegNo == startLogSegNo)
(gdb) 
5525      bool    use_existent = true;
(gdb) 
5528      fd = XLogFileInit(startLogSegNo, &use_existent, true);
(gdb)

2、path为000000040000000000000003,该文件不存在

Breakpoint 2, XLogFileInit (logsegno=3, use_existent=0xbfbd366f, use_lock=true) at xlog.c:3216
3216    XLogFilePath(path, ThisTimeLineID, logsegno, wal_segment_size);
(gdb) n
3221    if (*use_existent)
(gdb) p path
$1 = "pg_wal/00000004", '0' <repeats 15 times>, "30000000000346!`bX627527760000000a00000030333177t364257`00270331177t00000000a00000000224M0000000000h2275277: `b4000000330652752773306527527700000000h2275277L274M000400000000000000?000000a000000b327527720632`b00000000330652752773306527527733065275277373210}tX25wb0400000000000000177323abX00000000000000b", '00' <repeats 19 times>, "b0000000000000060", '00' <repeats 11 times>, "recovering 00000322"[000027004246277 `b`6275277360253abb", '00' <repeats 11 times>...
(gdb) n
3223      fd = BasicOpenFile(path, O_RDWR | PG_BINARY | get_sync_bit(sync_method));
(gdb) 
3224      if (fd < 0)
(gdb) 
3226        if (errno != ENOENT)
(gdb) p fd
$2 = -1
(gdb) p errno
$3 = 2

3、创建并打开临时文件，将zbuffer.data清0，然后一页一页的将文件清0

3243    snprintf(tmppath, MAXPGPATH, XLOGDIR "/xlogtemp.%d", (int) getpid());
(gdb) 
3245    unlink(tmppath);
(gdb) 
3248    fd = BasicOpenFile(tmppath, O_RDWR | O_CREAT | O_EXCL | PG_BINARY);
(gdb) 
3249    if (fd < 0)
(gdb) 
3254    memset(zbuffer.data, 0, XLOG_BLCKSZ);
(gdb) p fd
$4 = 3
3269      for (nbytes = 0; nbytes < wal_segment_size; nbytes += XLOG_BLCKSZ)
(gdb) 
3271        errno = 0;
(gdb) 
3272      if (write(fd, zbuffer.data, XLOG_BLCKSZ) != XLOG_BLCKSZ)

4、进入InstallXLogFileSegment 函数，000000040000000000000003文件stat失败，调用durable_link_or_rename重命名。

InstallXLogFileSegment (segno=0xbfbd0de0, tmppath=0xbfbd2de8 "pg_wal/xlogtemp.31765", find_free=true, max_segno=45, use_lock=true) at xlog.c:3550
3550    if (use_lock)
(gdb) n
3551      LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
(gdb) 
3553    if (!find_free)
(gdb) 
3561      while (stat(path, &stat_buf) == 0)
(gdb) p path
$5 = "pg_wal/00000004", '0' <repeats 15 times>, "300 21222251322"[0000000000277 `b370-275277Wn2752770500000003000000 NҬ030000000300000005000000;00000000000000210n27527720632`b00000000Xr275277Xr275277Xr275277240.{262d00000025|00000000000024020[264030000000000000003000000240362:26603000000220251t267030000002403243227003000000220213T27103000000 361032720300000020250=27303000000 323343273322"[3176500000000n", '00' <repeats 15 times>, "01000000373333yb050000000000000003000000(E27527770r27527722517`b25|000000000000"...
(gdb) n
3579    if (durable_link_or_rename(tmppath, path, LOG) != 0)
(gdb) p errno
$6 = 2

随机文章
本站知识点必读
最近更新