You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Current »

This article describes only MVP/prototype of WAL reader, usage details will be changed

General description

Ignite WAL reader is console utility prototype for Ignite developers. This utility will allow developers to check WAL content and perform WAL analysis.

Utility converts WAL binary files into human readable records and dumps it to StdOut.

The IDEA of this utility is very simple - get the possibility to look into  your WAL. 
 

But it also requires understanding what's going on in WAL mechanism (core WAL record description can be found in Ignite Persistent Store - under the hood in section WAL structure )

Usage

 

Some tips how to use converter:
Example of command line:

 

java -cp "lib/ignite-dev-utils-2.3.0-SNAPSHOT.jar:lib/ignite-indexing-2.3.0-SNAPSHOT.jar:lib/ignite-core-2.3.0-SNAPSHOT.jar:~/.m2/repository/com/h2database/h2/1.4.195/h2-1.4.195.jar" org.apache.ignite.development.utils.IgniteWalConverter 4096 ~/IdeaProjects/incubator-ignite/work/db/wal/127_0_0_1_47503 ~/IdeaProjects/incubator-ignite/work/db/wal/archive/127_0_0_1_47503

 

You need to pass as argument:
1. Page size.
2. Path to WAL dir.
3. (Optional) Path to Archive WAL dir.

 

Format of output

Output consist from lines, every new record discovered causes new line

(Record source) (record string representation)

Record source - W - Work directory, A - Archive directory

Record string representation starts from record type name. Then record parameters are dumped. Record dump is determined by record class itself.

Note record may provide not all, but only printable human readable details. Record may hide long binary objects which non appropriate for human readable form.

Example of output

To view example of output click below.

 

 

[W] InitNewPageRecord [newPageId=0001001c00000067, super=PageDeltaRecord
[grpId=2141373875, pageId=0001001c00000067, super=WALRecord [size=41,
chainSize=0, pos=FileWALPointer [idx=1, fileOffset=25406, len=41,
forceFlush=false], type=INIT_NEW_PAGE_RECORD]]]
[W] InitNewPageRecord [newPageId=0001001c00000068, super=PageDeltaRecord
[grpId=2141373875, pageId=0001001c00000068, super=WALRecord [size=41,
chainSize=0, pos=FileWALPointer [idx=1, fileOffset=25447, len=41,
forceFlush=false], type=INIT_NEW_PAGE_RECORD]]]
[W] PagesListAddPageRecord [dataPageId=0001001c0000002a,
super=PageDeltaRecord [grpId=2141373875, pageId=0001001c00000068,
super=WALRecord [size=37, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=25488, len=37, forceFlush=false], type=PAGES_LIST_ADD_PAGE]]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=0001001c0000002a,
effectivePageId=0000001c0000002a, grpId=2141373875], page = [
Header [
type=1 (DataPageIO),
ver=1,
crc=0,
pageId=281595235794986(offset=0, flags=1, partId=28, index=42)
],
DataPageIO [
0001001c0000002a [4049, 4002, 3955, 3908, 3861, 3814, 3767, 3720, 3673,
3626][][free=3552, actualFree=-544
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=25525, len=4125, forceFlush=false], type=PAGE_RECORD]]]
[W] InsertRecord [idx=12, io=CacheIdAwareDataLeafIO[ver=1],
rightId=0000000000000000, super=PageDeltaRecord [grpId=2141373875,
pageId=0001001c00000004, super=WALRecord [size=59, chainSize=0,
pos=FileWALPointer [idx=1, fileOffset=29650, len=59, forceFlush=false],
type=BTREE_PAGE_INSERT]]]
[W] DataRecord [writeEntries=[DataEntry [cacheId=-1368047377, op=CREATE,
writeVer=GridCacheVersion [topVer=116892426, order=1505412445450,
nodeOrder=2], partId=28, partCnt=69]], super=WALRecord [size=100,
chainSize=0, pos=FileWALPointer [idx=1, fileOffset=29709, len=100,
forceFlush=false], type=DATA_RECORD]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=000100000000000a,
effectivePageId=000000000000000a, grpId=2141373875], page = [
Header [
type=20 (PagePartitionCountersIO),
ver=1,
crc=0,
pageId=281474976710666(offset=0, flags=1, partId=0, index=10)
],
PagePartitionCounters [
count=1,
lastFlag=true,
nextCountersPageId=0000000000000000,
size={
-1368047377=313
}
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=29809, len=4125, forceFlush=false], type=PAGE_RECORD]]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=0001000200000010,
effectivePageId=0000000200000010, grpId=2141373875], page = [
Header [
type=20 (PagePartitionCountersIO),
ver=1,
crc=0,
pageId=281483566645264(offset=0, flags=1, partId=2, index=16)
],
PagePartitionCounters [
count=1,
lastFlag=true,
nextCountersPageId=0000000000000000,
size={
-1368047377=313
}
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=33934, len=4125, forceFlush=false], type=PAGE_RECORD]]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=000100050000000d,
effectivePageId=000000050000000d, grpId=2141373875], page = [
Header [
type=20 (PagePartitionCountersIO),
ver=1,
crc=0,
pageId=281496451547149(offset=0, flags=1, partId=5, index=13)
],
PagePartitionCounters [
count=1,
lastFlag=true,
nextCountersPageId=0000000000000000,
size={
-1368047377=313
}
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=38059, len=4125, forceFlush=false], type=PAGE_RECORD]]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=0001000900000003,
effectivePageId=0000000900000003, grpId=2141373875], page = [
Header [
type=12 (PagesListMetaIO),
ver=1,
crc=0,
pageId=281513631416323(offset=0, flags=1, partId=9, index=3)
],
PagesListMeta [
nextMetaPageId=0000000000000000,
count=230,
bucketData={
bucket=163, list=GridLongList [idx=6,
arr=[281513631416558,281513631416559,281513631416560,281513631416561,281513631416562,281513631416563]]
bucket=166, list=GridLongList [idx=8,
arr=[281513631416550,281513631416551,281513631416552,281513631416553,281513631416554,281513631416555,281513631416556,281513631416557]]
bucket=169, list=GridLongList [idx=8,
arr=[281513631416542,281513631416543,281513631416544,281513631416545,281513631416546,281513631416547,281513631416548,281513631416549]]
bucket=172, list=GridLongList [idx=8,
arr=[281513631416534,281513631416535,281513631416536,281513631416537,281513631416538,281513631416539,281513631416540,281513631416541]]
bucket=175, list=GridLongList [idx=8,
arr=[281513631416526,281513631416527,281513631416528,281513631416529,281513631416530,281513631416531,281513631416532,281513631416533]]
bucket=178, list=GridLongList [idx=8,
arr=[281513631416512,281513631416513,281513631416520,281513631416521,281513631416522,281513631416523,281513631416524,281513631416525]]
bucket=181, list=GridLongList [idx=8,
arr=[281513631416510,281513631416511,281513631416514,281513631416515,281513631416516,281513631416517,281513631416518,281513631416519]]
bucket=184, list=GridLongList [idx=8,
arr=[281513631416500,281513631416503,281513631416504,281513631416505,281513631416506,281513631416507,281513631416508,281513631416509]]
bucket=187, list=GridLongList [idx=8,
arr=[281513631416494,281513631416495,281513631416496,281513631416497,281513631416498,281513631416499,281513631416501,281513631416502]]
bucket=190, list=GridLongList [idx=8,
arr=[281513631416484,281513631416485,281513631416486,281513631416487,281513631416488,281513631416489,281513631416492,281513631416493]]
bucket=193, list=GridLongList [idx=8,
arr=[281513631416448,281513631416449,281513631416480,281513631416481,281513631416482,281513631416483,281513631416490,281513631416491]]
bucket=196, list=GridLongList [idx=8,
arr=[281513631416446,281513631416447,281513631416474,281513631416475,281513631416476,281513631416477,281513631416478,281513631416479]]
bucket=199, list=GridLongList [idx=8,
arr=[281513631416444,281513631416445,281513631416468,281513631416469,281513631416470,281513631416471,281513631416472,281513631416473]]
bucket=202, list=GridLongList [idx=8,
arr=[281513631416436,281513631416437,281513631416462,281513631416463,281513631416464,281513631416465,281513631416466,281513631416467]]
bucket=205, list=GridLongList [idx=8,
arr=[281513631416434,281513631416435,281513631416456,281513631416457,281513631416458,281513631416459,281513631416460,281513631416461]]
bucket=209, list=GridLongList [idx=8,
arr=[281513631416426,281513631416427,281513631416450,281513631416451,281513631416452,281513631416453,281513631416454,281513631416455]]
bucket=212, list=GridLongList [idx=8,
arr=[281513631416418,281513631416419,281513631416438,281513631416439,281513631416440,281513631416441,281513631416442,281513631416443]]
bucket=215, list=GridLongList [idx=8,
arr=[281513631416416,281513631416417,281513631416428,281513631416429,281513631416430,281513631416431,281513631416432,281513631416433]]
bucket=218, list=GridLongList [idx=8,
arr=[281513631416414,281513631416415,281513631416420,281513631416421,281513631416422,281513631416423,281513631416424,281513631416425]]
bucket=221, list=GridLongList [idx=8,
arr=[281513631416406,281513631416407,281513631416408,281513631416409,281513631416410,281513631416411,281513631416412,281513631416413]]
bucket=224, list=GridLongList [idx=8,
arr=[281513631416398,281513631416399,281513631416400,281513631416401,281513631416402,281513631416403,281513631416404,281513631416405]]
bucket=227, list=GridLongList [idx=8,
arr=[281513631416390,281513631416391,281513631416392,281513631416393,281513631416394,281513631416395,281513631416396,281513631416397]]
bucket=230, list=GridLongList [idx=8,
arr=[281513631416381,281513631416382,281513631416383,281513631416384,281513631416385,281513631416386,281513631416387,281513631416388]]
bucket=233, list=GridLongList [idx=8,
arr=[281513631416373,281513631416374,281513631416375,281513631416376,281513631416377,281513631416378,281513631416379,281513631416380]]
bucket=236, list=GridLongList [idx=8,
arr=[281513631416365,281513631416366,281513631416367,281513631416368,281513631416369,281513631416370,281513631416371,281513631416372]]
bucket=239, list=GridLongList [idx=8,
arr=[281513631416351,281513631416352,281513631416359,281513631416360,281513631416361,281513631416362,281513631416363,281513631416364]]
bucket=242, list=GridLongList [idx=8,
arr=[281513631416348,281513631416349,281513631416353,281513631416354,281513631416355,281513631416356,281513631416357,281513631416358]]
bucket=245, list=GridLongList [idx=8,
arr=[281513631416335,281513631416336,281513631416337,281513631416338,281513631416339,281513631416340,281513631416346,281513631416347]]
bucket=248, list=GridLongList [idx=8,
arr=[281513631416326,281513631416327,281513631416328,281513631416329,281513631416330,281513631416331,281513631416343,281513631416344]]
}
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=42184, len=4125, forceFlush=false], type=PAGE_RECORD]]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=000100090000000d,
effectivePageId=000000090000000d, grpId=2141373875], page = [
Header [
type=20 (PagePartitionCountersIO),
ver=1,
crc=0,
pageId=281513631416333(offset=0, flags=1, partId=9, index=13)
],
PagePartitionCounters [
count=1,
lastFlag=true,
nextCountersPageId=0000000000000000,
size={
-1368047377=198
}
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=46309, len=4125, forceFlush=false], type=PAGE_RECORD]]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=0001000900000000,
effectivePageId=0000000900000000, grpId=2141373875], page = [
Header [
type=14 (PagePartitionMetaIO),
ver=1,
crc=0,
pageId=281513631416320(offset=0, flags=1, partId=9, index=0)
],
PageMeta[
treeRoot=281513631416323,
lastSuccessfulFullSnapshotId=0,
lastSuccessfulSnapshotId=0,
nextSnapshotTag=1,
lastSuccessfulSnapshotTag=0,
lastAllocatedPageCount=0,
candidatePageCount=0
],
PagePartitionMeta[
size=198,
updateCounter=198,
globalRemoveId=1505412422861000000,
partitionState=1(OWNING),
countersPageId=281513631416333
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=50434, len=4125, forceFlush=false], type=PAGE_RECORD]]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=0001000b0000000c,
effectivePageId=0000000b0000000c, grpId=2141373875], page = [
Header [
type=20 (PagePartitionCountersIO),
ver=1,
crc=0,
pageId=281522221350924(offset=0, flags=1, partId=11, index=12)
],
PagePartitionCounters [
count=1,
lastFlag=true,
nextCountersPageId=0000000000000000,
size={
-1368047377=313
}
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=54559, len=4125, forceFlush=false], type=PAGE_RECORD]]]
[W] PageSnapshot [fullPageId = FullPageId [pageId=0001000c0000000f,
effectivePageId=0000000c0000000f, grpId=2141373875], page = [
Header [
type=20 (PagePartitionCountersIO),
ver=1,
crc=0,
pageId=281526516318223(offset=0, flags=1, partId=12, index=15)
],
PagePartitionCounters [
count=1,
lastFlag=true,
nextCountersPageId=0000000000000000,
size={
-1368047377=313
}
]],
super = [WALRecord [size=4125, chainSize=0, pos=FileWALPointer [idx=1,
fileOffset=58684, len=4125, forceFlush=false], type=PAGE_RECORD]]]

 

 

 

 

 

References

http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-PDS-WAL-analysis-with-human-readable-records-td22277.html

Unable to render Jira issues macro, execution error.

Unable to render Jira issues macro, execution error.


  • No labels

Apache Ignite