1. Master status for dsjhd113,60000,1402039442506 as of Fri Jun 06 18:20:08 CST 2014
  2. Version Info:
  3. ===========================================================
  4. HBase 0.96.0.2.0.6.0-76-hadoop2
  5. Subversion git://c64-s11/grid/0/workspace/BIGTOP-bigwheel-GA-HDP_RPM_REPO/label/c64-s11/bigtop-0.5/build/hbase/rpm/BUILD/hbase-0.96.0.2.0.6.0 -r e6d7a56f72914d01e55c0478d74e5cfd3778f231
  6. Compiled by jenkins on Thu Oct 17 18:15:20 PDT 2013
  7. Hadoop 2.2.0.2.0.6.0-76
  8. Subversion [email protected]:hortonworks/hadoop.git -r 8656b1cfad13b03b29e98cad042626205e7a1c86
  9. Compiled by jenkins on 2013-10-18T00:19Z
  10. Tasks:
  11. ===========================================================
  12. Task: RpcServer.handler=0,port=60000
  13. Status: WAITING:Waiting for a call
  14. Running for 10565s
  15. Task: RpcServer.handler=1,port=60000
  16. Status: WAITING:Waiting for a call
  17. Running for 10565s
  18. Task: RpcServer.handler=2,port=60000
  19. Status: WAITING:Waiting for a call
  20. Running for 10565s
  21. Task: RpcServer.handler=3,port=60000
  22. Status: WAITING:Waiting for a call
  23. Running for 10565s
  24. Task: RpcServer.handler=4,port=60000
  25. Status: WAITING:Waiting for a call
  26. Running for 10565s
  27. Task: RpcServer.handler=5,port=60000
  28. Status: WAITING:Waiting for a call
  29. Running for 10565s
  30. Task: RpcServer.handler=6,port=60000
  31. Status: WAITING:Waiting for a call
  32. Running for 10565s
  33. Task: RpcServer.handler=7,port=60000
  34. Status: WAITING:Waiting for a call
  35. Running for 10565s
  36. Task: RpcServer.handler=8,port=60000
  37. Status: WAITING:Waiting for a call
  38. Running for 10565s
  39. Task: RpcServer.handler=9,port=60000
  40. Status: WAITING:Waiting for a call
  41. Running for 10565s
  42. Task: RpcServer.handler=10,port=60000
  43. Status: WAITING:Waiting for a call
  44. Running for 10565s
  45. Task: RpcServer.handler=11,port=60000
  46. Status: WAITING:Waiting for a call
  47. Running for 10565s
  48. Task: RpcServer.handler=12,port=60000
  49. Status: WAITING:Waiting for a call
  50. Running for 10565s
  51. Task: RpcServer.handler=13,port=60000
  52. Status: RUNNING:Servicing call from 132.228.248.123:52589: ListTableDescriptorsByNamespace
  53. Running for 10565s
  54. Task: RpcServer.handler=14,port=60000
  55. Status: WAITING:Waiting for a call
  56. Running for 10565s
  57. Task: RpcServer.handler=15,port=60000
  58. Status: WAITING:Waiting for a call
  59. Running for 10565s
  60. Task: RpcServer.handler=16,port=60000
  61. Status: WAITING:Waiting for a call
  62. Running for 10565s
  63. Task: RpcServer.handler=17,port=60000
  64. Status: WAITING:Waiting for a call
  65. Running for 10565s
  66. Task: RpcServer.handler=18,port=60000
  67. Status: RUNNING:Servicing call from 132.228.248.123:52604: ListTableDescriptorsByNamespace
  68. Running for 10565s
  69. Task: RpcServer.handler=19,port=60000
  70. Status: RUNNING:Servicing call from 132.228.248.123:52580: ListTableDescriptorsByNamespace
  71. Running for 10565s
  72. Task: RpcServer.handler=20,port=60000
  73. Status: WAITING:Waiting for a call
  74. Running for 10565s
  75. Task: RpcServer.handler=21,port=60000
  76. Status: WAITING:Waiting for a call
  77. Running for 10565s
  78. Task: RpcServer.handler=22,port=60000
  79. Status: WAITING:Waiting for a call
  80. Running for 10565s
  81. Task: RpcServer.handler=23,port=60000
  82. Status: WAITING:Waiting for a call
  83. Running for 10565s
  84. Task: RpcServer.handler=24,port=60000
  85. Status: WAITING:Waiting for a call
  86. Running for 10565s
  87. Task: RpcServer.handler=25,port=60000
  88. Status: RUNNING:Servicing call from 132.228.248.123:52596: ListTableDescriptorsByNamespace
  89. Running for 10565s
  90. Task: RpcServer.handler=26,port=60000
  91. Status: WAITING:Waiting for a call
  92. Running for 10565s
  93. Task: RpcServer.handler=27,port=60000
  94. Status: RUNNING:Servicing call from 132.228.248.123:52589: ListTableDescriptorsByNamespace
  95. Running for 10565s
  96. Task: RpcServer.handler=28,port=60000
  97. Status: WAITING:Waiting for a call
  98. Running for 10565s
  99. Task: RpcServer.handler=29,port=60000
  100. Status: WAITING:Waiting for a call
  101. Running for 10565s
  102. Task: RpcServer.handler=30,port=60000
  103. Status: WAITING:Waiting for a call
  104. Running for 10565s
  105. Task: RpcServer.handler=31,port=60000
  106. Status: WAITING:Waiting for a call
  107. Running for 10565s
  108. Task: RpcServer.handler=32,port=60000
  109. Status: WAITING:Waiting for a call
  110. Running for 10565s
  111. Task: RpcServer.handler=33,port=60000
  112. Status: WAITING:Waiting for a call
  113. Running for 10565s
  114. Task: RpcServer.handler=34,port=60000
  115. Status: WAITING:Waiting for a call
  116. Running for 10565s
  117. Task: RpcServer.handler=35,port=60000
  118. Status: WAITING:Waiting for a call
  119. Running for 10565s
  120. Task: RpcServer.handler=36,port=60000
  121. Status: WAITING:Waiting for a call
  122. Running for 10565s
  123. Task: RpcServer.handler=37,port=60000
  124. Status: WAITING:Waiting for a call
  125. Running for 10565s
  126. Task: RpcServer.handler=38,port=60000
  127. Status: RUNNING:Servicing call from 132.228.248.123:52600: ListTableDescriptorsByNamespace
  128. Running for 10565s
  129. Task: RpcServer.handler=39,port=60000
  130. Status: WAITING:Waiting for a call
  131. Running for 10565s
  132. Task: RpcServer.handler=40,port=60000
  133. Status: WAITING:Waiting for a call
  134. Running for 10565s
  135. Task: RpcServer.handler=41,port=60000
  136. Status: WAITING:Waiting for a call
  137. Running for 10565s
  138. Task: RpcServer.handler=42,port=60000
  139. Status: WAITING:Waiting for a call
  140. Running for 10565s
  141. Task: RpcServer.handler=43,port=60000
  142. Status: WAITING:Waiting for a call
  143. Running for 10565s
  144. Task: RpcServer.handler=44,port=60000
  145. Status: RUNNING:Servicing call from 132.228.248.123:52600: ListTableDescriptorsByNamespace
  146. Running for 10565s
  147. Task: RpcServer.handler=45,port=60000
  148. Status: WAITING:Waiting for a call
  149. Running for 10565s
  150. Task: RpcServer.handler=46,port=60000
  151. Status: WAITING:Waiting for a call
  152. Running for 10565s
  153. Task: RpcServer.handler=47,port=60000
  154. Status: WAITING:Waiting for a call
  155. Running for 10565s
  156. Task: RpcServer.handler=48,port=60000
  157. Status: RUNNING:Servicing call from 132.228.248.123:52584: ListTableDescriptorsByNamespace
  158. Running for 10565s
  159. Task: RpcServer.handler=49,port=60000
  160. Status: WAITING:Waiting for a call
  161. Running for 10565s
  162. Task: RpcServer.handler=50,port=60000
  163. Status: WAITING:Waiting for a call
  164. Running for 10565s
  165. Task: RpcServer.handler=51,port=60000
  166. Status: WAITING:Waiting for a call
  167. Running for 10565s
  168. Task: RpcServer.handler=52,port=60000
  169. Status: WAITING:Waiting for a call
  170. Running for 10565s
  171. Task: RpcServer.handler=53,port=60000
  172. Status: WAITING:Waiting for a call
  173. Running for 10565s
  174. Task: RpcServer.handler=54,port=60000
  175. Status: RUNNING:Servicing call from 132.228.248.123:52596: ListTableDescriptorsByNamespace
  176. Running for 10565s
  177. Task: RpcServer.handler=55,port=60000
  178. Status: WAITING:Waiting for a call
  179. Running for 10565s
  180. Task: RpcServer.handler=56,port=60000
  181. Status: WAITING:Waiting for a call
  182. Running for 10565s
  183. Task: RpcServer.handler=57,port=60000
  184. Status: RUNNING:Servicing call from 132.228.248.123:52577: ListTableDescriptorsByNamespace
  185. Running for 10565s
  186. Task: RpcServer.handler=58,port=60000
  187. Status: RUNNING:Servicing call from 132.228.248.123:52604: ListTableDescriptorsByNamespace
  188. Running for 10565s
  189. Task: RpcServer.handler=59,port=60000
  190. Status: WAITING:Waiting for a call
  191. Running for 10565s
  192. Task: Replication.RpcServer.handler=0,port=60000
  193. Status: WAITING:Waiting for a call
  194. Running for 10565s
  195. Task: Replication.RpcServer.handler=1,port=60000
  196. Status: WAITING:Waiting for a call
  197. Running for 10565s
  198. Task: Replication.RpcServer.handler=2,port=60000
  199. Status: WAITING:Waiting for a call
  200. Running for 10565s
  201. Servers:
  202. ===========================================================
  203. dsjhd076,60020,1402027922130: requestsPerSecond=285198.0, numberOfOnlineRegions=391, usedHeapMB=9940, maxHeapMB=16332, numberOfStores=535, numberOfStorefiles=1413, storefileUncompressedSizeMB=13194842, storefileSizeMB=2510416, compressionRatio=0.1903, memstoreSizeMB=877, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6201918, rootIndexSizeKB=33277, totalStaticIndexSizeKB=20600067, totalStaticBloomSizeKB=36594476, totalCompactingKVs=393668602, currentCompactedKVs=298781948, compactionProgressPct=0.7589682, coprocessors=[]
  204. dsjhd055,60020,1402027951094: requestsPerSecond=328.0, numberOfOnlineRegions=389, usedHeapMB=8289, maxHeapMB=16332, numberOfStores=535, numberOfStorefiles=851, storefileUncompressedSizeMB=13355512, storefileSizeMB=2554818, compressionRatio=0.1913, memstoreSizeMB=621, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=4394552, rootIndexSizeKB=38034, totalStaticIndexSizeKB=20866881, totalStaticBloomSizeKB=36815194, totalCompactingKVs=479738188, currentCompactedKVs=479738188, compactionProgressPct=1.0, coprocessors=[]
  205. dsjhd074,60020,1402039899044: requestsPerSecond=653.0, numberOfOnlineRegions=351, usedHeapMB=7170, maxHeapMB=16332, numberOfStores=508, numberOfStorefiles=788, storefileUncompressedSizeMB=12716455, storefileSizeMB=2356905, compressionRatio=0.1853, memstoreSizeMB=399, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=5037169, rootIndexSizeKB=34313, totalStaticIndexSizeKB=20463008, totalStaticBloomSizeKB=32405318, totalCompactingKVs=505141420, currentCompactedKVs=505141420, compactionProgressPct=1.0, coprocessors=[]
  206. dsjhd075,60020,1402027953331: requestsPerSecond=269.0, numberOfOnlineRegions=389, usedHeapMB=7767, maxHeapMB=16332, numberOfStores=557, numberOfStorefiles=883, storefileUncompressedSizeMB=13914907, storefileSizeMB=2629550, compressionRatio=0.1890, memstoreSizeMB=413, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6307135, rootIndexSizeKB=37493, totalStaticIndexSizeKB=22046104, totalStaticBloomSizeKB=37025777, totalCompactingKVs=659623182, currentCompactedKVs=659623182, compactionProgressPct=1.0, coprocessors=[]
  207. dsjhd078,60020,1402027921335: requestsPerSecond=666.0, numberOfOnlineRegions=388, usedHeapMB=6692, maxHeapMB=16332, numberOfStores=540, numberOfStorefiles=887, storefileUncompressedSizeMB=13351402, storefileSizeMB=2555262, compressionRatio=0.1914, memstoreSizeMB=627, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=7102881, rootIndexSizeKB=38573, totalStaticIndexSizeKB=20689894, totalStaticBloomSizeKB=37597883, totalCompactingKVs=158498513, currentCompactedKVs=158498513, compactionProgressPct=1.0, coprocessors=[]
  208. dsjhd057,60020,1402039842287: requestsPerSecond=431.0, numberOfOnlineRegions=351, usedHeapMB=2127, maxHeapMB=16332, numberOfStores=487, numberOfStorefiles=789, storefileUncompressedSizeMB=12426629, storefileSizeMB=2335082, compressionRatio=0.1879, memstoreSizeMB=656, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=5288972, rootIndexSizeKB=33268, totalStaticIndexSizeKB=19578441, totalStaticBloomSizeKB=33462794, totalCompactingKVs=711157130, currentCompactedKVs=640427134, compactionProgressPct=0.9005424, coprocessors=[]
  209. dsjhd068,60020,1402038513322: requestsPerSecond=474.0, numberOfOnlineRegions=351, usedHeapMB=3689, maxHeapMB=16332, numberOfStores=471, numberOfStorefiles=792, storefileUncompressedSizeMB=11800330, storefileSizeMB=2254462, compressionRatio=0.1911, memstoreSizeMB=362, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6098047, rootIndexSizeKB=32137, totalStaticIndexSizeKB=18195643, totalStaticBloomSizeKB=33492854, totalCompactingKVs=592761684, currentCompactedKVs=592761684, compactionProgressPct=1.0, coprocessors=[]
  210. dsjhd062,60020,1402027917800: requestsPerSecond=592.0, numberOfOnlineRegions=389, usedHeapMB=8051, maxHeapMB=16332, numberOfStores=543, numberOfStorefiles=851, storefileUncompressedSizeMB=13974248, storefileSizeMB=2621347, compressionRatio=0.1876, memstoreSizeMB=869, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6727636, rootIndexSizeKB=36288, totalStaticIndexSizeKB=22118194, totalStaticBloomSizeKB=37155772, totalCompactingKVs=804952991, currentCompactedKVs=804952991, compactionProgressPct=1.0, coprocessors=[]
  211. dsjhd059,60020,1402027941533: requestsPerSecond=308.0, numberOfOnlineRegions=389, usedHeapMB=8334, maxHeapMB=16332, numberOfStores=538, numberOfStorefiles=861, storefileUncompressedSizeMB=13199517, storefileSizeMB=2500137, compressionRatio=0.1894, memstoreSizeMB=426, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6914927, rootIndexSizeKB=36146, totalStaticIndexSizeKB=20710708, totalStaticBloomSizeKB=36002786, totalCompactingKVs=435381501, currentCompactedKVs=435381501, compactionProgressPct=1.0, coprocessors=[]
  212. dsjhd066,60020,1402027920604: requestsPerSecond=484.0, numberOfOnlineRegions=390, usedHeapMB=6343, maxHeapMB=16332, numberOfStores=541, numberOfStorefiles=869, storefileUncompressedSizeMB=13245107, storefileSizeMB=2513471, compressionRatio=0.1898, memstoreSizeMB=521, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6413922, rootIndexSizeKB=36406, totalStaticIndexSizeKB=20851595, totalStaticBloomSizeKB=35851661, totalCompactingKVs=323319647, currentCompactedKVs=247970609, compactionProgressPct=0.76695186, coprocessors=[]
  213. dsjhd052,60020,1402039731411: requestsPerSecond=594.0, numberOfOnlineRegions=351, usedHeapMB=6907, maxHeapMB=16332, numberOfStores=497, numberOfStorefiles=832, storefileUncompressedSizeMB=11519771, storefileSizeMB=2227394, compressionRatio=0.1934, memstoreSizeMB=1311, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6349749, rootIndexSizeKB=31350, totalStaticIndexSizeKB=17600323, totalStaticBloomSizeKB=33354160, totalCompactingKVs=1271691778, currentCompactedKVs=1271691778, compactionProgressPct=1.0, coprocessors=[]
  214. dsjhd070,60020,1402027928334: requestsPerSecond=339.0, numberOfOnlineRegions=388, usedHeapMB=8344, maxHeapMB=16332, numberOfStores=527, numberOfStorefiles=865, storefileUncompressedSizeMB=13001803, storefileSizeMB=2506474, compressionRatio=0.1928, memstoreSizeMB=691, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6522753, rootIndexSizeKB=35687, totalStaticIndexSizeKB=19934802, totalStaticBloomSizeKB=37072136, totalCompactingKVs=591807087, currentCompactedKVs=591807087, compactionProgressPct=1.0, coprocessors=[]
  215. dsjhd069,60020,1402032388543: requestsPerSecond=214.0, numberOfOnlineRegions=351, usedHeapMB=3036, maxHeapMB=16332, numberOfStores=501, numberOfStorefiles=783, storefileUncompressedSizeMB=12556044, storefileSizeMB=2344028, compressionRatio=0.1867, memstoreSizeMB=858, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6823330, rootIndexSizeKB=33864, totalStaticIndexSizeKB=20052582, totalStaticBloomSizeKB=32489808, totalCompactingKVs=274994501, currentCompactedKVs=274994501, compactionProgressPct=1.0, coprocessors=[]
  216. dsjhd063,60020,1402039188138: requestsPerSecond=299.0, numberOfOnlineRegions=351, usedHeapMB=5791, maxHeapMB=16332, numberOfStores=505, numberOfStorefiles=795, storefileUncompressedSizeMB=12624689, storefileSizeMB=2378884, compressionRatio=0.1884, memstoreSizeMB=706, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=5367856, rootIndexSizeKB=31968, totalStaticIndexSizeKB=20066251, totalStaticBloomSizeKB=33340458, totalCompactingKVs=492221600, currentCompactedKVs=492221600, compactionProgressPct=1.0, coprocessors=[]
  217. dsjhd054,60020,1402027922467: requestsPerSecond=1076.0, numberOfOnlineRegions=388, usedHeapMB=11263, maxHeapMB=16332, numberOfStores=546, numberOfStorefiles=882, storefileUncompressedSizeMB=13467633, storefileSizeMB=2550587, compressionRatio=0.1894, memstoreSizeMB=1274, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6953014, rootIndexSizeKB=36103, totalStaticIndexSizeKB=21074984, totalStaticBloomSizeKB=36714006, totalCompactingKVs=275450057, currentCompactedKVs=275450057, compactionProgressPct=1.0, coprocessors=[]
  218. dsjhd067,60020,1402027944119: requestsPerSecond=869.0, numberOfOnlineRegions=390, usedHeapMB=2848, maxHeapMB=16332, numberOfStores=552, numberOfStorefiles=878, storefileUncompressedSizeMB=13256892, storefileSizeMB=2514078, compressionRatio=0.1896, memstoreSizeMB=1216, storefileIndexSizeMB=0, readRequestsCount=54, writeRequestsCount=6255995, rootIndexSizeKB=36050, totalStaticIndexSizeKB=20885937, totalStaticBloomSizeKB=35550005, totalCompactingKVs=391266872, currentCompactedKVs=391266872, compactionProgressPct=1.0, coprocessors=[]
  219. dsjhd060,60020,1402027925238: requestsPerSecond=618.0, numberOfOnlineRegions=389, usedHeapMB=10196, maxHeapMB=16332, numberOfStores=540, numberOfStorefiles=856, storefileUncompressedSizeMB=13242092, storefileSizeMB=2529191, compressionRatio=0.1910, memstoreSizeMB=591, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6639840, rootIndexSizeKB=35666, totalStaticIndexSizeKB=20587296, totalStaticBloomSizeKB=37031791, totalCompactingKVs=375974916, currentCompactedKVs=375974916, compactionProgressPct=1.0, coprocessors=[]
  220. dsjhd056,60020,1402027921412: requestsPerSecond=570.0, numberOfOnlineRegions=389, usedHeapMB=9162, maxHeapMB=16332, numberOfStores=532, numberOfStorefiles=860, storefileUncompressedSizeMB=13242766, storefileSizeMB=2508637, compressionRatio=0.1894, memstoreSizeMB=977, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=5874910, rootIndexSizeKB=35815, totalStaticIndexSizeKB=20704668, totalStaticBloomSizeKB=36409248, totalCompactingKVs=208286299, currentCompactedKVs=208286299, compactionProgressPct=1.0, coprocessors=[]
  221. dsjhd064,60020,1402027918813: requestsPerSecond=153863.0, numberOfOnlineRegions=392, usedHeapMB=10299, maxHeapMB=16332, numberOfStores=555, numberOfStorefiles=1018, storefileUncompressedSizeMB=13796015, storefileSizeMB=2608225, compressionRatio=0.1891, memstoreSizeMB=297, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=7124943, rootIndexSizeKB=35382, totalStaticIndexSizeKB=21830978, totalStaticBloomSizeKB=36957582, totalCompactingKVs=135472294, currentCompactedKVs=130549503, compactionProgressPct=0.9636621, coprocessors=[]
  222. dsjhd073,60020,1402039874233: requestsPerSecond=762.0, numberOfOnlineRegions=351, usedHeapMB=4429, maxHeapMB=16332, numberOfStores=498, numberOfStorefiles=798, storefileUncompressedSizeMB=12122837, storefileSizeMB=2321883, compressionRatio=0.1915, memstoreSizeMB=383, storefileIndexSizeMB=0, readRequestsCount=100, writeRequestsCount=5114202, rootIndexSizeKB=32562, totalStaticIndexSizeKB=18921431, totalStaticBloomSizeKB=33381664, totalCompactingKVs=471874244, currentCompactedKVs=471874244, compactionProgressPct=1.0, coprocessors=[]
  223. dsjhd051,60020,1402036579439: requestsPerSecond=589.0, numberOfOnlineRegions=352, usedHeapMB=5045, maxHeapMB=16332, numberOfStores=492, numberOfStorefiles=796, storefileUncompressedSizeMB=12215192, storefileSizeMB=2322454, compressionRatio=0.1901, memstoreSizeMB=465, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=6889980, rootIndexSizeKB=32192, totalStaticIndexSizeKB=18926837, totalStaticBloomSizeKB=34115119, totalCompactingKVs=489759604, currentCompactedKVs=489759604, compactionProgressPct=1.0, coprocessors=[]
  224. dsjhd061,60020,1402027921226: requestsPerSecond=0.0, numberOfOnlineRegions=389, usedHeapMB=4931, maxHeapMB=16332, numberOfStores=540, numberOfStorefiles=860, storefileUncompressedSizeMB=13503175, storefileSizeMB=2565486, compressionRatio=0.1900, memstoreSizeMB=227, storefileIndexSizeMB=0, readRequestsCount=218, writeRequestsCount=5671262, rootIndexSizeKB=37518, totalStaticIndexSizeKB=20970351, totalStaticBloomSizeKB=37459647, totalCompactingKVs=518045130, currentCompactedKVs=518045130, compactionProgressPct=1.0, coprocessors=[]
  225. dsjhd058,60020,1402047076406: requestsPerSecond=1218.0, numberOfOnlineRegions=351, usedHeapMB=4283, maxHeapMB=16332, numberOfStores=480, numberOfStorefiles=811, storefileUncompressedSizeMB=11731762, storefileSizeMB=2265024, compressionRatio=0.1931, memstoreSizeMB=798, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=508542, rootIndexSizeKB=32525, totalStaticIndexSizeKB=17787321, totalStaticBloomSizeKB=34423162, totalCompactingKVs=353247546, currentCompactedKVs=353247546, compactionProgressPct=1.0, coprocessors=[]
  226. dsjhd065,60020,1402027967439: requestsPerSecond=717.0, numberOfOnlineRegions=390, usedHeapMB=9718, maxHeapMB=16332, numberOfStores=544, numberOfStorefiles=872, storefileUncompressedSizeMB=13938670, storefileSizeMB=2626918, compressionRatio=0.1885, memstoreSizeMB=1120, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=5130145, rootIndexSizeKB=36165, totalStaticIndexSizeKB=21887923, totalStaticBloomSizeKB=37974426, totalCompactingKVs=808162374, currentCompactedKVs=808162374, compactionProgressPct=1.0, coprocessors=[]
  227. dsjhd077,60020,1402027915865: requestsPerSecond=580.0, numberOfOnlineRegions=388, usedHeapMB=6226, maxHeapMB=16332, numberOfStores=546, numberOfStorefiles=857, storefileUncompressedSizeMB=13911984, storefileSizeMB=2610002, compressionRatio=0.1876, memstoreSizeMB=887, storefileIndexSizeMB=0, readRequestsCount=38315885, writeRequestsCount=7521917, rootIndexSizeKB=35143, totalStaticIndexSizeKB=22130809, totalStaticBloomSizeKB=36778986, totalCompactingKVs=197310894, currentCompactedKVs=197310894, compactionProgressPct=1.0, coprocessors=[]
  228. dsjhd053,60020,1402027976229: requestsPerSecond=389.0, numberOfOnlineRegions=389, usedHeapMB=8669, maxHeapMB=16332, numberOfStores=543, numberOfStorefiles=870, storefileUncompressedSizeMB=13402607, storefileSizeMB=2533951, compressionRatio=0.1891, memstoreSizeMB=372, storefileIndexSizeMB=0, readRequestsCount=0, writeRequestsCount=5185420, rootIndexSizeKB=37047, totalStaticIndexSizeKB=21053319, totalStaticBloomSizeKB=36264735, totalCompactingKVs=431095832, currentCompactedKVs=431095832, compactionProgressPct=1.0, coprocessors=[]
  229. Regions-in-transition:
  230. ===========================================================
  231. Executors:
  232. ===========================================================
  233. Status for executor: Executor-6-MASTER_TABLE_OPERATIONS-dsjhd113:60000
  234. =======================================
  235. 0 events queued, 0 running
  236. Status for executor: Executor-5-M_LOG_REPLAY_OPS-dsjhd113:60000
  237. =======================================
  238. 0 events queued, 0 running
  239. Status for executor: Executor-3-MASTER_SERVER_OPERATIONS-dsjhd113:60000
  240. =======================================
  241. 0 events queued, 0 running
  242. Status for executor: Executor-1-MASTER_OPEN_REGION-dsjhd113:60000
  243. =======================================
  244. 0 events queued, 0 running
  245. Status for executor: Executor-2-MASTER_CLOSE_REGION-dsjhd113:60000
  246. =======================================
  247. 0 events queued, 0 running
  248. Status for executor: Executor-4-MASTER_META_SERVER_OPERATIONS-dsjhd113:60000
  249. =======================================
  250. 0 events queued, 0 running
  251. Stacks:
  252. ===========================================================
  253. Process Thread Dump:
  254. 112 active threads
  255. Thread 1419 (IPC Client (71257789) connection to dsjhd113/132.228.248.123:60000 from hbase):
  256. State: RUNNABLE
  257. Blocked count: 1
  258. Waited count: 1
  259. Stack:
  260. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  261. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  262. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  263. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  264. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  265. org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
  266. org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
  267. org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
  268. org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
  269. java.io.FilterInputStream.read(FilterInputStream.java:116)
  270. java.io.FilterInputStream.read(FilterInputStream.java:116)
  271. org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:553)
  272. java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  273. java.io.BufferedInputStream.read(BufferedInputStream.java:237)
  274. java.io.DataInputStream.readInt(DataInputStream.java:370)
  275. org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1057)
  276. org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:719)
  277. Thread 1418 (IPC Client (71257789) connection to dsjhd061/132.228.248.71:60020 from hbase):
  278. State: RUNNABLE
  279. Blocked count: 0
  280. Waited count: 0
  281. Stack:
  282. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  283. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  284. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  285. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  286. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  287. org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)
  288. org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
  289. org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
  290. org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
  291. java.io.FilterInputStream.read(FilterInputStream.java:116)
  292. java.io.FilterInputStream.read(FilterInputStream.java:116)
  293. org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:553)
  294. java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  295. java.io.BufferedInputStream.read(BufferedInputStream.java:237)
  296. java.io.DataInputStream.readInt(DataInputStream.java:370)
  297. org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1057)
  298. org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:719)
  299. Thread 1405 (843405603@qtp-1312568829-42):
  300. State: RUNNABLE
  301. Blocked count: 26
  302. Waited count: 29
  303. Stack:
  304. sun.management.ThreadImpl.getThreadInfo1(Native Method)
  305. sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:156)
  306. sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:121)
  307. org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:165)
  308. org.apache.hadoop.hbase.master.MasterDumpServlet.doGet(MasterDumpServlet.java:80)
  309. javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
  310. javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
  311. org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  312. org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
  313. org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
  314. org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  315. org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1081)
  316. org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  317. org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
  318. org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
  319. org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
  320. org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
  321. org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
  322. org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
  323. org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
  324. Thread 1382 (431225297@qtp-1312568829-37):
  325. State: TIMED_WAITING
  326. Blocked count: 15
  327. Waited count: 248
  328. Stack:
  329. java.lang.Object.wait(Native Method)
  330. org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1432)
  331. org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1650)
  332. org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1708)
  333. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.listTableDescriptorsByNamespace(MasterProtos.java:40612)
  334. org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.listTableDescriptorsByNamespace(HConnectionManager.java:2054)
  335. org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2268)
  336. org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2265)
  337. org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:116)
  338. org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:94)
  339. org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3124)
  340. org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2264)
  341. org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:461)
  342. org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:270)
  343. org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
  344. org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372)
  345. org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
  346. javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
  347. javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
  348. org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  349. Thread 1198 (600134808@qtp-1312568829-34):
  350. State: TIMED_WAITING
  351. Blocked count: 25
  352. Waited count: 490
  353. Stack:
  354. java.lang.Object.wait(Native Method)
  355. org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1432)
  356. org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1650)
  357. org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1708)
  358. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.listTableDescriptorsByNamespace(MasterProtos.java:40612)
  359. org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$5.listTableDescriptorsByNamespace(HConnectionManager.java:2054)
  360. org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2268)
  361. org.apache.hadoop.hbase.client.HBaseAdmin$22.call(HBaseAdmin.java:2265)
  362. org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:116)
  363. org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:94)
  364. org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3124)
  365. org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2264)
  366. org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:461)
  367. org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:270)
  368. org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
  369. org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372)
  370. org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
  371. javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
  372. javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
  373. org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
  374. Thread 942 (MASTER_TABLE_OPERATIONS-dsjhd113:60000-0):
  375. State: WAITING
  376. Blocked count: 37
  377. Waited count: 52
  378. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@6c13cbd0
  379. Stack:
  380. sun.misc.Unsafe.park(Native Method)
  381. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  382. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  383. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  384. java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
  385. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  386. java.lang.Thread.run(Thread.java:662)
  387. Thread 409 (IPC Parameter Sending Thread #1):
  388. State: TIMED_WAITING
  389. Blocked count: 1514
  390. Waited count: 669156
  391. Stack:
  392. sun.misc.Unsafe.park(Native Method)
  393. java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196)
  394. java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424)
  395. java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323)
  396. java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874)
  397. java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945)
  398. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  399. java.lang.Thread.run(Thread.java:662)
  400. Thread 156 (CatalogJanitor-dsjhd113:60000):
  401. State: TIMED_WAITING
  402. Blocked count: 6643
  403. Waited count: 9574
  404. Stack:
  405. java.lang.Object.wait(Native Method)
  406. org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
  407. org.apache.hadoop.hbase.Chore.run(Chore.java:88)
  408. java.lang.Thread.run(Thread.java:662)
  409. Thread 155 (dsjhd113,60000,1402039442506-BalancerChore):
  410. State: TIMED_WAITING
  411. Blocked count: 102117
  412. Waited count: 183590
  413. Stack:
  414. java.lang.Object.wait(Native Method)
  415. org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
  416. org.apache.hadoop.hbase.Chore.run(Chore.java:88)
  417. java.lang.Thread.run(Thread.java:662)
  418. Thread 154 (dsjhd113,60000,1402039442506-ClusterStatusChore):
  419. State: TIMED_WAITING
  420. Blocked count: 353
  421. Waited count: 503
  422. Stack:
  423. java.lang.Object.wait(Native Method)
  424. org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
  425. org.apache.hadoop.hbase.Chore.run(Chore.java:88)
  426. java.lang.Thread.run(Thread.java:662)
  427. Thread 123 (MASTER_SERVER_OPERATIONS-dsjhd113:60000-4):
  428. State: WAITING
  429. Blocked count: 248
  430. Waited count: 246
  431. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@b240236
  432. Stack:
  433. sun.misc.Unsafe.park(Native Method)
  434. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  435. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  436. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  437. java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
  438. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  439. java.lang.Thread.run(Thread.java:662)
  440. Thread 122 (MASTER_SERVER_OPERATIONS-dsjhd113:60000-3):
  441. State: WAITING
  442. Blocked count: 380
  443. Waited count: 417
  444. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@b240236
  445. Stack:
  446. sun.misc.Unsafe.park(Native Method)
  447. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  448. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  449. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  450. java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
  451. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  452. java.lang.Thread.run(Thread.java:662)
  453. Thread 121 (MASTER_SERVER_OPERATIONS-dsjhd113:60000-2):
  454. State: WAITING
  455. Blocked count: 279
  456. Waited count: 254
  457. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@b240236
  458. Stack:
  459. sun.misc.Unsafe.park(Native Method)
  460. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  461. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  462. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  463. java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
  464. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  465. java.lang.Thread.run(Thread.java:662)
  466. Thread 120 (MASTER_SERVER_OPERATIONS-dsjhd113:60000-1):
  467. State: WAITING
  468. Blocked count: 546
  469. Waited count: 748
  470. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@b240236
  471. Stack:
  472. sun.misc.Unsafe.park(Native Method)
  473. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  474. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  475. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  476. java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
  477. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  478. java.lang.Thread.run(Thread.java:662)
  479. Thread 119 (MASTER_SERVER_OPERATIONS-dsjhd113:60000-0):
  480. State: WAITING
  481. Blocked count: 249
  482. Waited count: 236
  483. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@b240236
  484. Stack:
  485. sun.misc.Unsafe.park(Native Method)
  486. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  487. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  488. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  489. java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
  490. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
  491. java.lang.Thread.run(Thread.java:662)
  492. Thread 115 (master:dsjhd113:60000.archivedHFileCleaner):
  493. State: TIMED_WAITING
  494. Blocked count: 631998
  495. Waited count: 1243640
  496. Stack:
  497. java.lang.Object.wait(Native Method)
  498. org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
  499. org.apache.hadoop.hbase.Chore.run(Chore.java:88)
  500. java.lang.Thread.run(Thread.java:662)
  501. Thread 116 (snapshot-hfile-cleaner-cache-refresher):
  502. State: TIMED_WAITING
  503. Blocked count: 64
  504. Waited count: 103
  505. Stack:
  506. java.lang.Object.wait(Native Method)
  507. java.util.TimerThread.mainLoop(Timer.java:509)
  508. java.util.TimerThread.run(Timer.java:462)
  509. Thread 111 (master:dsjhd113:60000.oldLogCleaner):
  510. State: TIMED_WAITING
  511. Blocked count: 10113
  512. Waited count: 15584
  513. Stack:
  514. java.lang.Object.wait(Native Method)
  515. org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
  516. org.apache.hadoop.hbase.Chore.run(Chore.java:88)
  517. java.lang.Thread.run(Thread.java:662)
  518. Thread 114 (snapshot-log-cleaner-cache-refresher):
  519. State: TIMED_WAITING
  520. Blocked count: 36
  521. Waited count: 103
  522. Stack:
  523. java.lang.Object.wait(Native Method)
  524. java.util.TimerThread.mainLoop(Timer.java:509)
  525. java.util.TimerThread.run(Timer.java:462)
  526. Thread 113 (master:dsjhd113:60000-EventThread):
  527. State: WAITING
  528. Blocked count: 0
  529. Waited count: 3
  530. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@69e1e999
  531. Stack:
  532. sun.misc.Unsafe.park(Native Method)
  533. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  534. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  535. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  536. org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
  537. Thread 112 (master:dsjhd113:60000-SendThread(dsjhd113:2181)):
  538. State: RUNNABLE
  539. Blocked count: 7
  540. Waited count: 0
  541. Stack:
  542. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  543. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  544. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  545. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  546. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  547. org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  548. org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1072)
  549. Thread 110 (dsjhd113,60000,1402039442506.splitLogManagerTimeoutMonitor):
  550. State: TIMED_WAITING
  551. Blocked count: 0
  552. Waited count: 10494
  553. Stack:
  554. java.lang.Object.wait(Native Method)
  555. org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
  556. org.apache.hadoop.hbase.Chore.run(Chore.java:88)
  557. java.lang.Thread.run(Thread.java:662)
  558. Thread 109 (org.apache.hadoop.hdfs.PeerCache@2b1bed2):
  559. State: TIMED_WAITING
  560. Blocked count: 0
  561. Waited count: 88
  562. Stack:
  563. java.lang.Thread.sleep(Native Method)
  564. org.apache.hadoop.hdfs.PeerCache.run(PeerCache.java:252)
  565. org.apache.hadoop.hdfs.PeerCache.access$000(PeerCache.java:39)
  566. org.apache.hadoop.hdfs.PeerCache$1.run(PeerCache.java:135)
  567. java.lang.Thread.run(Thread.java:662)
  568. Thread 102 (1294712427@qtp-1312568829-3-EventThread):
  569. State: WAITING
  570. Blocked count: 0
  571. Waited count: 3
  572. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@4c5b55a9
  573. Stack:
  574. sun.misc.Unsafe.park(Native Method)
  575. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  576. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  577. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  578. org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
  579. Thread 101 (1294712427@qtp-1312568829-3-SendThread(dsjhd113:2181)):
  580. State: RUNNABLE
  581. Blocked count: 2
  582. Waited count: 0
  583. Stack:
  584. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  585. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  586. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  587. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  588. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  589. org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  590. org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1072)
  591. Thread 97 (Timer-0):
  592. State: TIMED_WAITING
  593. Blocked count: 1
  594. Waited count: 354
  595. Stack:
  596. java.lang.Object.wait(Native Method)
  597. java.util.TimerThread.mainLoop(Timer.java:509)
  598. java.util.TimerThread.run(Timer.java:462)
  599. Thread 96 (234794758@qtp-1312568829-1 - Acceptor0 [email protected]:60010):
  600. State: RUNNABLE
  601. Blocked count: 3
  602. Waited count: 1
  603. Stack:
  604. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  605. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  606. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  607. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  608. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  609. org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
  610. org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
  611. org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
  612. org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
  613. org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
  614. Thread 12 (master:dsjhd113:60000):
  615. State: TIMED_WAITING
  616. Blocked count: 897
  617. Waited count: 105599
  618. Stack:
  619. java.lang.Object.wait(Native Method)
  620. org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:92)
  621. org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:56)
  622. org.apache.hadoop.hbase.master.HMaster.loop(HMaster.java:734)
  623. org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:604)
  624. java.lang.Thread.run(Thread.java:662)
  625. Thread 93 (JvmPauseMonitor):
  626. State: TIMED_WAITING
  627. Blocked count: 3
  628. Waited count: 21118
  629. Stack:
  630. java.lang.Thread.sleep(Native Method)
  631. org.apache.hadoop.hbase.util.JvmPauseMonitor$Monitor.run(JvmPauseMonitor.java:159)
  632. java.lang.Thread.run(Thread.java:662)
  633. Thread 92 (Replication.RpcServer.handler=2,port=60000):
  634. State: WAITING
  635. Blocked count: 0
  636. Waited count: 1
  637. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@43277a30
  638. Stack:
  639. sun.misc.Unsafe.park(Native Method)
  640. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  641. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  642. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  643. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  644. Thread 91 (Replication.RpcServer.handler=1,port=60000):
  645. State: WAITING
  646. Blocked count: 0
  647. Waited count: 1
  648. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@43277a30
  649. Stack:
  650. sun.misc.Unsafe.park(Native Method)
  651. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  652. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  653. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  654. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  655. Thread 90 (Replication.RpcServer.handler=0,port=60000):
  656. State: WAITING
  657. Blocked count: 0
  658. Waited count: 1
  659. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@43277a30
  660. Stack:
  661. sun.misc.Unsafe.park(Native Method)
  662. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  663. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  664. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  665. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  666. Thread 89 (RpcServer.handler=59,port=60000):
  667. State: WAITING
  668. Blocked count: 8
  669. Waited count: 1634
  670. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  671. Stack:
  672. sun.misc.Unsafe.park(Native Method)
  673. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  674. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  675. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  676. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  677. Thread 88 (RpcServer.handler=58,port=60000):
  678. State: BLOCKED
  679. Blocked count: 128
  680. Waited count: 1866
  681. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  682. Blocked by 87 (RpcServer.handler=57,port=60000)
  683. Stack:
  684. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  685. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  686. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  687. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  688. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  689. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  690. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  691. Thread 87 (RpcServer.handler=57,port=60000):
  692. State: TIMED_WAITING
  693. Blocked count: 38
  694. Waited count: 2006
  695. Stack:
  696. java.lang.Object.wait(Native Method)
  697. org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1432)
  698. org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1650)
  699. org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1708)
  700. org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:27308)
  701. org.apache.hadoop.hbase.protobuf.ProtobufUtil.get(ProtobufUtil.java:1281)
  702. org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:752)
  703. org.apache.hadoop.hbase.client.HTable$3.call(HTable.java:750)
  704. org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:116)
  705. org.apache.hadoop.hbase.client.HTable.get(HTable.java:755)
  706. org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:134)
  707. org.apache.hadoop.hbase.master.TableNamespaceManager.get(TableNamespaceManager.java:118)
  708. org.apache.hadoop.hbase.master.HMaster.getNamespaceDescriptor(HMaster.java:3103)
  709. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3123)
  710. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  711. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  712. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  713. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  714. Thread 86 (RpcServer.handler=56,port=60000):
  715. State: WAITING
  716. Blocked count: 200
  717. Waited count: 2013
  718. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  719. Stack:
  720. sun.misc.Unsafe.park(Native Method)
  721. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  722. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  723. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  724. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  725. Thread 85 (RpcServer.handler=55,port=60000):
  726. State: WAITING
  727. Blocked count: 14
  728. Waited count: 1647
  729. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  730. Stack:
  731. sun.misc.Unsafe.park(Native Method)
  732. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  733. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  734. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  735. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  736. Thread 84 (RpcServer.handler=54,port=60000):
  737. State: BLOCKED
  738. Blocked count: 131
  739. Waited count: 1846
  740. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  741. Blocked by 87 (RpcServer.handler=57,port=60000)
  742. Stack:
  743. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  744. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  745. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  746. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  747. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  748. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  749. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  750. Thread 83 (RpcServer.handler=53,port=60000):
  751. State: WAITING
  752. Blocked count: 134
  753. Waited count: 1870
  754. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  755. Stack:
  756. sun.misc.Unsafe.park(Native Method)
  757. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  758. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  759. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  760. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  761. Thread 82 (RpcServer.handler=52,port=60000):
  762. State: WAITING
  763. Blocked count: 14
  764. Waited count: 1642
  765. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  766. Stack:
  767. sun.misc.Unsafe.park(Native Method)
  768. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  769. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  770. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  771. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  772. Thread 81 (RpcServer.handler=51,port=60000):
  773. State: WAITING
  774. Blocked count: 141
  775. Waited count: 1882
  776. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  777. Stack:
  778. sun.misc.Unsafe.park(Native Method)
  779. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  780. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  781. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  782. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  783. Thread 80 (RpcServer.handler=50,port=60000):
  784. State: WAITING
  785. Blocked count: 247
  786. Waited count: 2111
  787. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  788. Stack:
  789. sun.misc.Unsafe.park(Native Method)
  790. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  791. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  792. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  793. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  794. Thread 79 (RpcServer.handler=49,port=60000):
  795. State: WAITING
  796. Blocked count: 158
  797. Waited count: 1912
  798. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  799. Stack:
  800. sun.misc.Unsafe.park(Native Method)
  801. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  802. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  803. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  804. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  805. Thread 78 (RpcServer.handler=48,port=60000):
  806. State: BLOCKED
  807. Blocked count: 9
  808. Waited count: 1582
  809. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  810. Blocked by 87 (RpcServer.handler=57,port=60000)
  811. Stack:
  812. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  813. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  814. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  815. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  816. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  817. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  818. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  819. Thread 77 (RpcServer.handler=47,port=60000):
  820. State: WAITING
  821. Blocked count: 2
  822. Waited count: 1624
  823. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  824. Stack:
  825. sun.misc.Unsafe.park(Native Method)
  826. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  827. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  828. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  829. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  830. Thread 76 (RpcServer.handler=46,port=60000):
  831. State: WAITING
  832. Blocked count: 383
  833. Waited count: 2337
  834. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  835. Stack:
  836. sun.misc.Unsafe.park(Native Method)
  837. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  838. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  839. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  840. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  841. Thread 75 (RpcServer.handler=45,port=60000):
  842. State: WAITING
  843. Blocked count: 8
  844. Waited count: 1635
  845. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  846. Stack:
  847. sun.misc.Unsafe.park(Native Method)
  848. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  849. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  850. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  851. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  852. Thread 74 (RpcServer.handler=44,port=60000):
  853. State: BLOCKED
  854. Blocked count: 139
  855. Waited count: 1874
  856. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  857. Blocked by 87 (RpcServer.handler=57,port=60000)
  858. Stack:
  859. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  860. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  861. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  862. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  863. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  864. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  865. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  866. Thread 73 (RpcServer.handler=43,port=60000):
  867. State: WAITING
  868. Blocked count: 509
  869. Waited count: 2583
  870. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  871. Stack:
  872. sun.misc.Unsafe.park(Native Method)
  873. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  874. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  875. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  876. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  877. Thread 72 (RpcServer.handler=42,port=60000):
  878. State: WAITING
  879. Blocked count: 7
  880. Waited count: 1634
  881. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  882. Stack:
  883. sun.misc.Unsafe.park(Native Method)
  884. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  885. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  886. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  887. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  888. Thread 71 (RpcServer.handler=41,port=60000):
  889. State: WAITING
  890. Blocked count: 129
  891. Waited count: 1876
  892. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  893. Stack:
  894. sun.misc.Unsafe.park(Native Method)
  895. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  896. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  897. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  898. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  899. Thread 70 (RpcServer.handler=40,port=60000):
  900. State: WAITING
  901. Blocked count: 244
  902. Waited count: 2103
  903. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  904. Stack:
  905. sun.misc.Unsafe.park(Native Method)
  906. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  907. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  908. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  909. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  910. Thread 69 (RpcServer.handler=39,port=60000):
  911. State: WAITING
  912. Blocked count: 126
  913. Waited count: 1869
  914. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  915. Stack:
  916. sun.misc.Unsafe.park(Native Method)
  917. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  918. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  919. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  920. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  921. Thread 68 (RpcServer.handler=38,port=60000):
  922. State: BLOCKED
  923. Blocked count: 129
  924. Waited count: 1857
  925. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  926. Blocked by 87 (RpcServer.handler=57,port=60000)
  927. Stack:
  928. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  929. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  930. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  931. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  932. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  933. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  934. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  935. Thread 67 (RpcServer.handler=37,port=60000):
  936. State: WAITING
  937. Blocked count: 148
  938. Waited count: 1888
  939. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  940. Stack:
  941. sun.misc.Unsafe.park(Native Method)
  942. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  943. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  944. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  945. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  946. Thread 66 (RpcServer.handler=36,port=60000):
  947. State: WAITING
  948. Blocked count: 145
  949. Waited count: 1871
  950. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  951. Stack:
  952. sun.misc.Unsafe.park(Native Method)
  953. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  954. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  955. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  956. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  957. Thread 65 (RpcServer.handler=35,port=60000):
  958. State: WAITING
  959. Blocked count: 380
  960. Waited count: 2375
  961. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  962. Stack:
  963. sun.misc.Unsafe.park(Native Method)
  964. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  965. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  966. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  967. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  968. Thread 64 (RpcServer.handler=34,port=60000):
  969. State: WAITING
  970. Blocked count: 7
  971. Waited count: 1631
  972. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  973. Stack:
  974. sun.misc.Unsafe.park(Native Method)
  975. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  976. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  977. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  978. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  979. Thread 63 (RpcServer.handler=33,port=60000):
  980. State: WAITING
  981. Blocked count: 130
  982. Waited count: 1878
  983. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  984. Stack:
  985. sun.misc.Unsafe.park(Native Method)
  986. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  987. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  988. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  989. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  990. Thread 62 (RpcServer.handler=32,port=60000):
  991. State: WAITING
  992. Blocked count: 121
  993. Waited count: 1858
  994. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  995. Stack:
  996. sun.misc.Unsafe.park(Native Method)
  997. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  998. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  999. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1000. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1001. Thread 61 (RpcServer.handler=31,port=60000):
  1002. State: WAITING
  1003. Blocked count: 119
  1004. Waited count: 1856
  1005. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1006. Stack:
  1007. sun.misc.Unsafe.park(Native Method)
  1008. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1009. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1010. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1011. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1012. Thread 60 (RpcServer.handler=30,port=60000):
  1013. State: WAITING
  1014. Blocked count: 119
  1015. Waited count: 1855
  1016. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1017. Stack:
  1018. sun.misc.Unsafe.park(Native Method)
  1019. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1020. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1021. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1022. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1023. Thread 59 (RpcServer.handler=29,port=60000):
  1024. State: WAITING
  1025. Blocked count: 146
  1026. Waited count: 1910
  1027. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1028. Stack:
  1029. sun.misc.Unsafe.park(Native Method)
  1030. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1031. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1032. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1033. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1034. Thread 58 (RpcServer.handler=28,port=60000):
  1035. State: WAITING
  1036. Blocked count: 128
  1037. Waited count: 1870
  1038. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1039. Stack:
  1040. sun.misc.Unsafe.park(Native Method)
  1041. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1042. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1043. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1044. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1045. Thread 57 (RpcServer.handler=27,port=60000):
  1046. State: BLOCKED
  1047. Blocked count: 135
  1048. Waited count: 1838
  1049. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  1050. Blocked by 87 (RpcServer.handler=57,port=60000)
  1051. Stack:
  1052. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  1053. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  1054. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  1055. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  1056. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  1057. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  1058. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  1059. Thread 56 (RpcServer.handler=26,port=60000):
  1060. State: WAITING
  1061. Blocked count: 247
  1062. Waited count: 2113
  1063. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1064. Stack:
  1065. sun.misc.Unsafe.park(Native Method)
  1066. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1067. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1068. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1069. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1070. Thread 55 (RpcServer.handler=25,port=60000):
  1071. State: BLOCKED
  1072. Blocked count: 135
  1073. Waited count: 1853
  1074. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  1075. Blocked by 87 (RpcServer.handler=57,port=60000)
  1076. Stack:
  1077. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  1078. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  1079. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  1080. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  1081. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  1082. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  1083. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  1084. Thread 54 (RpcServer.handler=24,port=60000):
  1085. State: WAITING
  1086. Blocked count: 132
  1087. Waited count: 1874
  1088. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1089. Stack:
  1090. sun.misc.Unsafe.park(Native Method)
  1091. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1092. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1093. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1094. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1095. Thread 53 (RpcServer.handler=23,port=60000):
  1096. State: WAITING
  1097. Blocked count: 127
  1098. Waited count: 1872
  1099. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1100. Stack:
  1101. sun.misc.Unsafe.park(Native Method)
  1102. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1103. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1104. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1105. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1106. Thread 52 (RpcServer.handler=22,port=60000):
  1107. State: WAITING
  1108. Blocked count: 373
  1109. Waited count: 2364
  1110. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1111. Stack:
  1112. sun.misc.Unsafe.park(Native Method)
  1113. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1114. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1115. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1116. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1117. Thread 51 (RpcServer.handler=21,port=60000):
  1118. State: WAITING
  1119. Blocked count: 2
  1120. Waited count: 1621
  1121. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1122. Stack:
  1123. sun.misc.Unsafe.park(Native Method)
  1124. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1125. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1126. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1127. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1128. Thread 50 (RpcServer.handler=20,port=60000):
  1129. State: WAITING
  1130. Blocked count: 252
  1131. Waited count: 2110
  1132. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1133. Stack:
  1134. sun.misc.Unsafe.park(Native Method)
  1135. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1136. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1137. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1138. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1139. Thread 49 (RpcServer.handler=19,port=60000):
  1140. State: BLOCKED
  1141. Blocked count: 239
  1142. Waited count: 2034
  1143. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  1144. Blocked by 87 (RpcServer.handler=57,port=60000)
  1145. Stack:
  1146. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  1147. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  1148. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  1149. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  1150. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  1151. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  1152. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  1153. Thread 48 (RpcServer.handler=18,port=60000):
  1154. State: BLOCKED
  1155. Blocked count: 388
  1156. Waited count: 2358
  1157. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  1158. Blocked by 87 (RpcServer.handler=57,port=60000)
  1159. Stack:
  1160. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  1161. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  1162. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  1163. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  1164. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  1165. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  1166. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  1167. Thread 47 (RpcServer.handler=17,port=60000):
  1168. State: WAITING
  1169. Blocked count: 23
  1170. Waited count: 1658
  1171. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1172. Stack:
  1173. sun.misc.Unsafe.park(Native Method)
  1174. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1175. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1176. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1177. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1178. Thread 46 (RpcServer.handler=16,port=60000):
  1179. State: WAITING
  1180. Blocked count: 133
  1181. Waited count: 1884
  1182. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1183. Stack:
  1184. sun.misc.Unsafe.park(Native Method)
  1185. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1186. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1187. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1188. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1189. Thread 45 (RpcServer.handler=15,port=60000):
  1190. State: WAITING
  1191. Blocked count: 367
  1192. Waited count: 2333
  1193. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1194. Stack:
  1195. sun.misc.Unsafe.park(Native Method)
  1196. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1197. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1198. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1199. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1200. Thread 44 (RpcServer.handler=14,port=60000):
  1201. State: WAITING
  1202. Blocked count: 127
  1203. Waited count: 1870
  1204. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1205. Stack:
  1206. sun.misc.Unsafe.park(Native Method)
  1207. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1208. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1209. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1210. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1211. Thread 43 (RpcServer.handler=13,port=60000):
  1212. State: BLOCKED
  1213. Blocked count: 143
  1214. Waited count: 1860
  1215. Blocked on org.apache.hadoop.hbase.master.TableNamespaceManager@67e24f6d
  1216. Blocked by 87 (RpcServer.handler=57,port=60000)
  1217. Stack:
  1218. org.apache.hadoop.hbase.master.TableNamespaceManager.isTableAvailableAndInitialized(TableNamespaceManager.java:250)
  1219. org.apache.hadoop.hbase.master.HMaster.isTableNamespaceManagerReady(HMaster.java:3141)
  1220. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3120)
  1221. org.apache.hadoop.hbase.master.HMaster.listTableDescriptorsByNamespace(HMaster.java:3024)
  1222. org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:38261)
  1223. org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  1224. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  1225. Thread 42 (RpcServer.handler=12,port=60000):
  1226. State: WAITING
  1227. Blocked count: 134
  1228. Waited count: 1887
  1229. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1230. Stack:
  1231. sun.misc.Unsafe.park(Native Method)
  1232. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1233. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1234. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1235. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1236. Thread 41 (RpcServer.handler=11,port=60000):
  1237. State: WAITING
  1238. Blocked count: 243
  1239. Waited count: 2096
  1240. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1241. Stack:
  1242. sun.misc.Unsafe.park(Native Method)
  1243. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1244. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1245. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1246. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1247. Thread 40 (RpcServer.handler=10,port=60000):
  1248. State: WAITING
  1249. Blocked count: 365
  1250. Waited count: 2349
  1251. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1252. Stack:
  1253. sun.misc.Unsafe.park(Native Method)
  1254. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1255. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1256. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1257. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1258. Thread 39 (RpcServer.handler=9,port=60000):
  1259. State: WAITING
  1260. Blocked count: 8
  1261. Waited count: 1630
  1262. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1263. Stack:
  1264. sun.misc.Unsafe.park(Native Method)
  1265. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1266. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1267. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1268. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1269. Thread 38 (RpcServer.handler=8,port=60000):
  1270. State: WAITING
  1271. Blocked count: 245
  1272. Waited count: 2110
  1273. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1274. Stack:
  1275. sun.misc.Unsafe.park(Native Method)
  1276. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1277. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1278. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1279. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1280. Thread 37 (RpcServer.handler=7,port=60000):
  1281. State: WAITING
  1282. Blocked count: 129
  1283. Waited count: 1876
  1284. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1285. Stack:
  1286. sun.misc.Unsafe.park(Native Method)
  1287. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1288. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1289. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1290. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1291. Thread 36 (RpcServer.handler=6,port=60000):
  1292. State: WAITING
  1293. Blocked count: 127
  1294. Waited count: 1871
  1295. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1296. Stack:
  1297. sun.misc.Unsafe.park(Native Method)
  1298. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1299. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1300. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1301. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1302. Thread 35 (RpcServer.handler=5,port=60000):
  1303. State: WAITING
  1304. Blocked count: 135
  1305. Waited count: 1888
  1306. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1307. Stack:
  1308. sun.misc.Unsafe.park(Native Method)
  1309. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1310. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1311. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1312. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1313. Thread 34 (RpcServer.handler=4,port=60000):
  1314. State: WAITING
  1315. Blocked count: 0
  1316. Waited count: 1618
  1317. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1318. Stack:
  1319. sun.misc.Unsafe.park(Native Method)
  1320. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1321. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1322. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1323. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1324. Thread 33 (RpcServer.handler=3,port=60000):
  1325. State: WAITING
  1326. Blocked count: 133
  1327. Waited count: 1882
  1328. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1329. Stack:
  1330. sun.misc.Unsafe.park(Native Method)
  1331. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1332. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1333. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1334. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1335. Thread 32 (RpcServer.handler=2,port=60000):
  1336. State: WAITING
  1337. Blocked count: 120
  1338. Waited count: 1865
  1339. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1340. Stack:
  1341. sun.misc.Unsafe.park(Native Method)
  1342. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1343. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1344. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1345. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1346. Thread 31 (RpcServer.handler=1,port=60000):
  1347. State: WAITING
  1348. Blocked count: 13
  1349. Waited count: 1638
  1350. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1351. Stack:
  1352. sun.misc.Unsafe.park(Native Method)
  1353. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1354. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1355. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1356. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1357. Thread 30 (RpcServer.handler=0,port=60000):
  1358. State: WAITING
  1359. Blocked count: 362
  1360. Waited count: 2340
  1361. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@54c707c1
  1362. Stack:
  1363. sun.misc.Unsafe.park(Native Method)
  1364. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1365. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1366. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1367. org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1826)
  1368. Thread 13 (RpcServer.listener,port=60000):
  1369. State: RUNNABLE
  1370. Blocked count: 420
  1371. Waited count: 0
  1372. Stack:
  1373. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1374. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1375. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1376. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1377. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1378. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1379. org.apache.hadoop.hbase.ipc.RpcServer$Listener.run(RpcServer.java:660)
  1380. Thread 26 (RpcServer.responder):
  1381. State: RUNNABLE
  1382. Blocked count: 0
  1383. Waited count: 0
  1384. Stack:
  1385. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1386. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1387. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1388. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1389. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1390. org.apache.hadoop.hbase.ipc.RpcServer$Responder.doRunLoop(RpcServer.java:847)
  1391. org.apache.hadoop.hbase.ipc.RpcServer$Responder.run(RpcServer.java:830)
  1392. Thread 29 (main-EventThread):
  1393. State: WAITING
  1394. Blocked count: 890
  1395. Waited count: 16872
  1396. Waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@56ce3b62
  1397. Stack:
  1398. sun.misc.Unsafe.park(Native Method)
  1399. java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
  1400. java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
  1401. java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
  1402. org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:491)
  1403. Thread 28 (main-SendThread(dsjhd112:2181)):
  1404. State: RUNNABLE
  1405. Blocked count: 5945
  1406. Waited count: 1
  1407. Stack:
  1408. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1409. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1410. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1411. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1412. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1413. org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
  1414. org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1072)
  1415. Thread 25 (Timer for 'HBase' metrics system):
  1416. State: TIMED_WAITING
  1417. Blocked count: 1
  1418. Waited count: 1058
  1419. Stack:
  1420. java.lang.Object.wait(Native Method)
  1421. java.util.TimerThread.mainLoop(Timer.java:509)
  1422. java.util.TimerThread.run(Timer.java:462)
  1423. Thread 24 (ganglia):
  1424. State: WAITING
  1425. Blocked count: 1056
  1426. Waited count: 1057
  1427. Waiting on org.apache.hadoop.metrics2.impl.SinkQueue@67071c84
  1428. Stack:
  1429. java.lang.Object.wait(Native Method)
  1430. java.lang.Object.wait(Object.java:485)
  1431. org.apache.hadoop.metrics2.impl.SinkQueue.waitForData(SinkQueue.java:114)
  1432. org.apache.hadoop.metrics2.impl.SinkQueue.consumeAll(SinkQueue.java:83)
  1433. org.apache.hadoop.metrics2.impl.MetricsSinkAdapter.publishMetricsFromQueue(MetricsSinkAdapter.java:127)
  1434. org.apache.hadoop.metrics2.impl.MetricsSinkAdapter$1.run(MetricsSinkAdapter.java:86)
  1435. Thread 23 (RpcServer.reader=9,port=60000):
  1436. State: RUNNABLE
  1437. Blocked count: 42
  1438. Waited count: 45
  1439. Stack:
  1440. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1441. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1442. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1443. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1444. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1445. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1446. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1447. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1448. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1449. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1450. java.lang.Thread.run(Thread.java:662)
  1451. Thread 22 (RpcServer.reader=8,port=60000):
  1452. State: RUNNABLE
  1453. Blocked count: 43
  1454. Waited count: 46
  1455. Stack:
  1456. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1457. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1458. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1459. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1460. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1461. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1462. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1463. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1464. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1465. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1466. java.lang.Thread.run(Thread.java:662)
  1467. Thread 21 (RpcServer.reader=7,port=60000):
  1468. State: RUNNABLE
  1469. Blocked count: 42
  1470. Waited count: 48
  1471. Stack:
  1472. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1473. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1474. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1475. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1476. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1477. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1478. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1479. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1480. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1481. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1482. java.lang.Thread.run(Thread.java:662)
  1483. Thread 20 (RpcServer.reader=6,port=60000):
  1484. State: RUNNABLE
  1485. Blocked count: 42
  1486. Waited count: 49
  1487. Stack:
  1488. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1489. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1490. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1491. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1492. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1493. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1494. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1495. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1496. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1497. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1498. java.lang.Thread.run(Thread.java:662)
  1499. Thread 19 (RpcServer.reader=5,port=60000):
  1500. State: RUNNABLE
  1501. Blocked count: 43
  1502. Waited count: 44
  1503. Stack:
  1504. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1505. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1506. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1507. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1508. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1509. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1510. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1511. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1512. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1513. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1514. java.lang.Thread.run(Thread.java:662)
  1515. Thread 18 (RpcServer.reader=4,port=60000):
  1516. State: RUNNABLE
  1517. Blocked count: 43
  1518. Waited count: 50
  1519. Stack:
  1520. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1521. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1522. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1523. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1524. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1525. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1526. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1527. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1528. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1529. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1530. java.lang.Thread.run(Thread.java:662)
  1531. Thread 17 (RpcServer.reader=3,port=60000):
  1532. State: RUNNABLE
  1533. Blocked count: 47
  1534. Waited count: 44
  1535. Stack:
  1536. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1537. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1538. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1539. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1540. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1541. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1542. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1543. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1544. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1545. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1546. java.lang.Thread.run(Thread.java:662)
  1547. Thread 16 (RpcServer.reader=2,port=60000):
  1548. State: RUNNABLE
  1549. Blocked count: 42
  1550. Waited count: 44
  1551. Stack:
  1552. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1553. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1554. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1555. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1556. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1557. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1558. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1559. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1560. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1561. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1562. java.lang.Thread.run(Thread.java:662)
  1563. Thread 15 (RpcServer.reader=1,port=60000):
  1564. State: RUNNABLE
  1565. Blocked count: 42
  1566. Waited count: 47
  1567. Stack:
  1568. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1569. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1570. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1571. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1572. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1573. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1574. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1575. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1576. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1577. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1578. java.lang.Thread.run(Thread.java:662)
  1579. Thread 14 (RpcServer.reader=0,port=60000):
  1580. State: RUNNABLE
  1581. Blocked count: 42
  1582. Waited count: 43
  1583. Stack:
  1584. sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
  1585. sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
  1586. sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
  1587. sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
  1588. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
  1589. sun.nio.ch.SelectorImpl.select(SelectorImpl.java:84)
  1590. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.doRunLoop(RpcServer.java:552)
  1591. org.apache.hadoop.hbase.ipc.RpcServer$Listener$Reader.run(RpcServer.java:538)
  1592. java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
  1593. java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
  1594. java.lang.Thread.run(Thread.java:662)
  1595. Thread 5 (Signal Dispatcher):
  1596. State: RUNNABLE
  1597. Blocked count: 0
  1598. Waited count: 0
  1599. Stack:
  1600. Thread 3 (Finalizer):
  1601. State: WAITING
  1602. Blocked count: 580
  1603. Waited count: 579
  1604. Waiting on java.lang.ref.ReferenceQueue$Lock@3e152f4
  1605. Stack:
  1606. java.lang.Object.wait(Native Method)
  1607. java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
  1608. java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
  1609. java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
  1610. Thread 2 (Reference Handler):
  1611. State: WAITING
  1612. Blocked count: 888
  1613. Waited count: 878
  1614. Waiting on java.lang.ref.Reference$Lock@3e0d1329
  1615. Stack:
  1616. java.lang.Object.wait(Native Method)
  1617. java.lang.Object.wait(Object.java:485)
  1618. java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
  1619. Thread 1 (main):
  1620. State: WAITING
  1621. Blocked count: 19
  1622. Waited count: 12
  1623. Waiting on java.lang.Thread@7d9ccb73
  1624. Stack:
  1625. java.lang.Object.wait(Native Method)
  1626. java.lang.Thread.join(Thread.java:1186)
  1627. java.lang.Thread.join(Thread.java:1239)
  1628. org.apache.hadoop.hbase.util.HasThread.join(HasThread.java:89)
  1629. org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:190)
  1630. org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:134)
  1631. org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
  1632. org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
  1633. org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2787)
  1634. Master configuration:
  1635. ===========================================================
  1636. <?xml version="1.0" encoding="UTF-8" standalone="no"?><configuration>
  1637. <property><name>hbase.regionserver.catalog.timeout</name><value>600000</value><source>hbase-default.xml</source></property>
  1638. <property><name>hbase.status.publisher.class</name><value>org.apache.hadoop.hbase.master.ClusterStatusPublisher$MulticastPublisher</value><source>hbase-default.xml</source></property>
  1639. <property><name>hbase.regions.slop</name><value>0.2</value><source>hbase-default.xml</source></property>
  1640. <property><name>dfs.image.transfer.bandwidthPerSec</name><value>0</value><source>hdfs-default.xml</source></property>
  1641. <property><name>hbase.zookeeper.leaderport</name><value>3888</value><source>hbase-default.xml</source></property>
  1642. <property><name>hbase.regionserver.info.port</name><value>60030</value><source>hbase-default.xml</source></property>
  1643. <property><name>dfs.block.access.token.lifetime</name><value>600</value><source>hdfs-default.xml</source></property>
  1644. <property><name>hadoop.proxyuser.hive.hosts</name><value>dsjhd112</value><source>core-site.xml</source></property>
  1645. <property><name>fs.AbstractFileSystem.file.impl</name><value>org.apache.hadoop.fs.local.LocalFs</value><source>core-default.xml</source></property>
  1646. <property><name>hbase.rs.cacheblocksonwrite</name><value>false</value><source>hbase-default.xml</source></property>
  1647. <property><name>dfs.domain.socket.path</name><value>/var/lib/hadoop-hdfs/dn_socket</value><source>hdfs-site.xml</source></property>
  1648. <property><name>hadoop.ssl.keystores.factory.class</name><value>org.apache.hadoop.security.ssl.FileBasedKeyStoresFactory</value><source>core-default.xml</source></property>
  1649. <property><name>hbase.rpc.server.engine</name><value>org.apache.hadoop.hbase.ipc.ProtobufRpcServerEngine</value><source>hbase-default.xml</source></property>
  1650. <property><name>hadoop.http.authentication.kerberos.keytab</name><value>${user.home}/hadoop.keytab</value><source>core-default.xml</source></property>
  1651. <property><name>io.seqfile.sorter.recordlimit</name><value>1000000</value><source>core-default.xml</source></property>
  1652. <property><name>hbase.data.umask.enable</name><value>false</value><source>hbase-default.xml</source></property>
  1653. <property><name>s3.blocksize</name><value>67108864</value><source>core-default.xml</source></property>
  1654. <property><name>hbase.master.dns.nameserver</name><value>default</value><source>hbase-default.xml</source></property>
  1655. <property><name>dfs.namenode.num.checkpoints.retained</name><value>2</value><source>hdfs-default.xml</source></property>
  1656. <property><name>hbase.lease.recovery.dfs.timeout</name><value>64000</value><source>hbase-default.xml</source></property>
  1657. <property><name>dfs.namenode.delegation.token.renew-interval</name><value>86400000</value><source>hdfs-default.xml</source></property>
  1658. <property><name>hbase.status.listener.class</name><value>org.apache.hadoop.hbase.client.ClusterStatusListener$MulticastListener</value><source>hbase-default.xml</source></property>
  1659. <property><name>io.map.index.interval</name><value>128</value><source>core-default.xml</source></property>
  1660. <property><name>hfile.format.version</name><value>2</value><source>hbase-default.xml</source></property>
  1661. <property><name>nfs3.mountd.port</name><value>4242</value><source>core-default.xml</source></property>
  1662. <property><name>s3.client-write-packet-size</name><value>65536</value><source>core-default.xml</source></property>
  1663. <property><name>hbase.master.catalog.timeout</name><value>600000</value><source>hbase-default.xml</source></property>
  1664. <property><name>hbase.auth.token.max.lifetime</name><value>604800000</value><source>hbase-default.xml</source></property>
  1665. <property><name>hbase.hregion.majorcompaction.jitter</name><value>0.50</value><source>hbase-default.xml</source></property>
  1666. <property><name>dfs.namenode.http-address</name><value>dsjhd112,dsjhd113:50070</value><source>hdfs-site.xml</source></property>
  1667. <property><name>ha.zookeeper.session-timeout.ms</name><value>5000</value><source>core-default.xml</source></property>
  1668. <property><name>hbase.master.loadbalancer.class</name><value>org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer</value><source>hbase-default.xml</source></property>
  1669. <property><name>hbase.hstore.flush.retries.number</name><value>120</value><source>hbase-site.xml</source></property>
  1670. <property><name>hadoop.hdfs.configuration.version</name><value>1</value><source>hdfs-default.xml</source></property>
  1671. <property><name>s3.replication</name><value>3</value><source>core-default.xml</source></property>
  1672. <property><name>hfile.block.bloom.cacheonwrite</name><value>false</value><source>hbase-default.xml</source></property>
  1673. <property><name>dfs.datanode.balance.bandwidthPerSec</name><value>6250000</value><source>hdfs-site.xml</source></property>
  1674. <property><name>hadoop.ssl.enabled</name><value>false</value><source>core-default.xml</source></property>
  1675. <property><name>dfs.journalnode.rpc-address</name><value>0.0.0.0:8485</value><source>hdfs-default.xml</source></property>
  1676. <property><name>hbase.lease.recovery.timeout</name><value>900000</value><source>hbase-default.xml</source></property>
  1677. <property><name>dfs.datanode.readahead.bytes</name><value>4193404</value><source>hdfs-default.xml</source></property>
  1678. <property><name>ipc.client.connect.max.retries.on.timeouts</name><value>45</value><source>core-default.xml</source></property>
  1679. <property><name>hbase.hregion.memstore.flush.size</name><value>134217728</value><source>hbase-site.xml</source></property>
  1680. <property><name>dfs.client.failover.max.attempts</name><value>15</value><source>hdfs-default.xml</source></property>
  1681. <property><name>dfs.namenode.checkpoint.dir</name><value>/data/data1/namesecondary</value><source>hdfs-site.xml</source></property>
  1682. <property><name>hadoop.proxyuser.hcat.groups</name><value>users</value><source>core-site.xml</source></property>
  1683. <property><name>hbase.regionserver.logroll.period</name><value>3600000</value><source>hbase-default.xml</source></property>
  1684. <property><name>dfs.namenode.replication.work.multiplier.per.iteration</name><value>2</value><source>hdfs-default.xml</source></property>
  1685. <property><name>fs.trash.interval</name><value>360</value><source>core-site.xml</source></property>
  1686. <property><name>ha.health-monitor.check-interval.ms</name><value>1000</value><source>core-default.xml</source></property>
  1687. <property><name>hadoop.jetty.logs.serve.aliases</name><value>true</value><source>core-default.xml</source></property>
  1688. <property><name>hadoop.http.authentication.kerberos.principal</name><value>HTTP/_HOST@LOCALHOST</value><source>core-default.xml</source></property>
  1689. <property><name>hbase.dynamic.jars.dir</name><value>${hbase.rootdir}/lib</value><source>hbase-default.xml</source></property>
  1690. <property><name>s3native.blocksize</name><value>67108864</value><source>core-default.xml</source></property>
  1691. <property><name>hbase.regionserver.dns.interface</name><value>default</value><source>hbase-default.xml</source></property>
  1692. <property><name>dfs.namenode.edits.dir</name><value>${dfs.namenode.name.dir}</value><source>hdfs-default.xml</source></property>
  1693. <property><name>ha.health-monitor.sleep-after-disconnect.ms</name><value>1000</value><source>core-default.xml</source></property>
  1694. <property><name>dfs.encrypt.data.transfer</name><value>false</value><source>hdfs-default.xml</source></property>
  1695. <property><name>dfs.datanode.http.address</name><value>0.0.0.0:50075</value><source>hdfs-site.xml</source></property>
  1696. <property><name>hbase.hstore.blockingStoreFiles</name><value>20</value><source>hbase-site.xml</source></property>
  1697. <property><name>dfs.namenode.write.stale.datanode.ratio</name><value>1.0f</value><source>hdfs-site.xml</source></property>
  1698. <property><name>dfs.client.use.datanode.hostname</name><value>false</value><source>hdfs-default.xml</source></property>
  1699. <property><name>hbase.hstore.bytes.per.checksum</name><value>16384</value><source>hbase-default.xml</source></property>
  1700. <property><name>hbase.client.write.buffer</name><value>2097152</value><source>hbase-default.xml</source></property>
  1701. <property><name>hadoop.security.instrumentation.requires.admin</name><value>false</value><source>core-default.xml</source></property>
  1702. <property><name>hadoop.security.authorization</name><value>false</value><source>core-site.xml</source></property>
  1703. <property><name>dfs.namenode.fs-limits.min-block-size</name><value>1048576</value><source>hdfs-default.xml</source></property>
  1704. <property><name>dfs.client.failover.connection.retries.on.timeouts</name><value>0</value><source>hdfs-default.xml</source></property>
  1705. <property><name>hadoop.security.group.mapping.ldap.search.filter.group</name><value>(objectClass=group)</value><source>core-default.xml</source></property>
  1706. <property><name>dfs.namenode.safemode.extension</name><value>30000</value><source>hdfs-default.xml</source></property>
  1707. <property><name>hbase.zookeeper.property.clientPort</name><value>2181</value><source>hbase-site.xml</source></property>
  1708. <property><name>hbase.regionserver.info.port.auto</name><value>false</value><source>hbase-default.xml</source></property>
  1709. <property><name>hbase.coprocessor.region.classes</name><value>org.apache.hadoop.hbase.security.access.AccessController</value><source>hbase-site.xml</source></property>
  1710. <property><name>dfs.datanode.sync.behind.writes</name><value>false</value><source>hdfs-default.xml</source></property>
  1711. <property><name>dfs.https.server.keystore.resource</name><value>ssl-server.xml</value><source>hdfs-default.xml</source></property>
  1712. <property><name>hadoop.security.group.mapping.ldap.search.attr.group.name</name><value>cn</value><source>core-default.xml</source></property>
  1713. <property><name>dfs.namenode.replication.min</name><value>1</value><source>hdfs-default.xml</source></property>
  1714. <property><name>hbase.master.logcleaner.ttl</name><value>600000</value><source>hbase-default.xml</source></property>
  1715. <property><name>fs.client.resolve.remote.symlinks</name><value>true</value><source>core-default.xml</source></property>
  1716. <property><name>s3native.bytes-per-checksum</name><value>512</value><source>core-default.xml</source></property>
  1717. <property><name>hbase.regionserver.optionallogflushinterval</name><value>1000</value><source>hbase-default.xml</source></property>
  1718. <property><name>tfile.fs.output.buffer.size</name><value>262144</value><source>core-default.xml</source></property>
  1719. <property><name>fs.AbstractFileSystem.hdfs.impl</name><value>org.apache.hadoop.fs.Hdfs</value><source>core-default.xml</source></property>
  1720. <property><name>hbase.regionserver.codecs</name><value>snappy</value><source>hbase-site.xml</source></property>
  1721. <property><name>dfs.namenode.safemode.min.datanodes</name><value>0</value><source>hdfs-default.xml</source></property>
  1722. <property><name>hadoop.security.uid.cache.secs</name><value>14400</value><source>core-default.xml</source></property>
  1723. <property><name>hbase.rootdir</name><value>hdfs://rtpha/apps/hbase/data</value><source>hbase-site.xml</source></property>
  1724. <property><name>dfs.client.https.need-auth</name><value>false</value><source>hdfs-default.xml</source></property>
  1725. <property><name>dfs.client.write.exclude.nodes.cache.expiry.interval.millis</name><value>600000</value><source>hdfs-default.xml</source></property>
  1726. <property><name>dfs.client.https.keystore.resource</name><value>ssl-client.xml</value><source>hdfs-default.xml</source></property>
  1727. <property><name>dfs.namenode.max.objects</name><value>0</value><source>hdfs-default.xml</source></property>
  1728. <property><name>hbase.zookeeper.dns.interface</name><value>default</value><source>hbase-default.xml</source></property>
  1729. <property><name>hadoop.ssl.client.conf</name><value>ssl-client.xml</value><source>core-default.xml</source></property>
  1730. <property><name>zookeeper.session.timeout</name><value>30000</value><source>hbase-site.xml</source></property>
  1731. <property><name>dfs.namenode.safemode.threshold-pct</name><value>1.0f</value><source>hdfs-site.xml</source></property>
  1732. <property><name>hbase.hstore.compaction.max</name><value>10</value><source>hbase-default.xml</source></property>
  1733. <property><name>mapred.task.id</name><value>hb_m_dsjhd113,60000,1402039442506</value><source>programatically</source></property>
  1734. <property><name>dfs.blocksize</name><value>134217728</value><source>hdfs-site.xml</source></property>
  1735. <property><name>hbase.security.authorization</name><value>true</value><source>hbase-site.xml</source></property>
  1736. <property><name>hbase.master.info.bindAddress</name><value>0.0.0.0</value><source>hbase-default.xml</source></property>
  1737. <property><name>io.native.lib.available</name><value>true</value><source>core-default.xml</source></property>
  1738. <property><name>dfs.client-write-packet-size</name><value>65536</value><source>hdfs-default.xml</source></property>
  1739. <property><name>hbase.hstore.compactionThreshold</name><value>3</value><source>hbase-site.xml</source></property>
  1740. <property><name>dfs.namenode.name.dir</name><value>/data/data1/namenode,/data/data2/namenode</value><source>hdfs-site.xml</source></property>
  1741. <property><name>dfs.ha.log-roll.period</name><value>120</value><source>hdfs-default.xml</source></property>
  1742. <property><name>io.storefile.bloom.block.size</name><value>131072</value><source>hbase-default.xml</source></property>
  1743. <property><name>dfs.client.failover.sleep.base.millis</name><value>500</value><source>hdfs-default.xml</source></property>
  1744. <property><name>dfs.datanode.directoryscan.threads</name><value>1</value><source>hdfs-default.xml</source></property>
  1745. <property><name>hbase.offheapcache.percentage</name><value>0</value><source>hbase-default.xml</source></property>
  1746. <property><name>dfs.permissions.enabled</name><value>true</value><source>hdfs-site.xml</source></property>
  1747. <property><name>hbase.thrift.maxQueuedRequests</name><value>1000</value><source>hbase-default.xml</source></property>
  1748. <property><name>dfs.support.append</name><value>true</value><source>hdfs-site.xml</source></property>
  1749. <property><name>hbase.thrift.minWorkerThreads</name><value>16</value><source>hbase-default.xml</source></property>
  1750. <property><name>ipc.client.connection.maxidletime</name><value>30000</value><source>core-site.xml</source></property>
  1751. <property><name>fs.checkpoint.size</name><value>0.5</value><source>core-site.xml</source></property>
  1752. <property><name>hbase.data.umask</name><value>000</value><source>hbase-default.xml</source></property>
  1753. <property><name>dfs.namenode.invalidate.work.pct.per.iteration</name><value>0.32f</value><source>hdfs-default.xml</source></property>
  1754. <property><name>dfs.blockreport.intervalMsec</name><value>21600000</value><source>hdfs-default.xml</source></property>
  1755. <property><name>fs.s3.sleepTimeSeconds</name><value>10</value><source>core-default.xml</source></property>
  1756. <property><name>dfs.namenode.replication.considerLoad</name><value>true</value><source>hdfs-default.xml</source></property>
  1757. <property><name>dfs.client.block.write.retries</name><value>3</value><source>hdfs-default.xml</source></property>
  1758. <property><name>hadoop.ssl.server.conf</name><value>ssl-server.xml</value><source>core-default.xml</source></property>
  1759. <property><name>hbase.balancer.period</name><value>300000</value><source>hbase-default.xml</source></property>
  1760. <property><name>dfs.namenode.name.dir.restore</name><value>false</value><source>hdfs-default.xml</source></property>
  1761. <property><name>dfs.datanode.hdfs-blocks-metadata.enabled</name><value>false</value><source>hdfs-default.xml</source></property>
  1762. <property><name>ha.zookeeper.parent-znode</name><value>/hadoop-ha</value><source>core-default.xml</source></property>
  1763. <property><name>io.seqfile.lazydecompress</name><value>true</value><source>core-default.xml</source></property>
  1764. <property><name>hbase.zookeeper.property.maxClientCnxns</name><value>300</value><source>hbase-default.xml</source></property>
  1765. <property><name>dfs.https.enable</name><value>false</value><source>hdfs-default.xml</source></property>
  1766. <property><name>dfs.namenode.shared.edits.dir</name><value>qjournal://dsjhd112:8485;dsjhd113:8485;dsjhd051:8485/rtpha</value><source>hdfs-site.xml</source></property>
  1767. <property><name>fail.fast.expired.active.master</name><value>false</value><source>hbase-default.xml</source></property>
  1768. <property><name>dfs.replication</name><value>3</value><source>hdfs-site.xml</source></property>
  1769. <property><name>hbase.client.scanner.caching</name><value>100</value><source>hbase-site.xml</source></property>
  1770. <property><name>ipc.client.tcpnodelay</name><value>false</value><source>core-default.xml</source></property>
  1771. <property><name>hbase.ipc.client.tcpnodelay</name><value>true</value><source>hbase-site.xml</source></property>
  1772. <property><name>dfs.namenode.accesstime.precision</name><value>0</value><source>hdfs-site.xml</source></property>
  1773. <property><name>s3.stream-buffer-size</name><value>4096</value><source>core-default.xml</source></property>
  1774. <property><name>hbase.client.retries.number</name><value>350</value><source>programatically</source></property>
  1775. <property><name>nfs3.server.port</name><value>2049</value><source>core-default.xml</source></property>
  1776. <property><name>hbase.client.scanner.timeout.period</name><value>60000</value><source>hbase-default.xml</source></property>
  1777. <property><name>io.compression.codecs</name><value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.SnappyCodec</value><source>core-site.xml</source></property>
  1778. <property><name>hbase.zookeeper.useMulti</name><value>true</value><source>hbase-site.xml</source></property>
  1779. <property><name>hbase.zookeeper.dns.nameserver</name><value>default</value><source>hbase-default.xml</source></property>
  1780. <property><name>io.file.buffer.size</name><value>131072</value><source>core-site.xml</source></property>
  1781. <property><name>hfile.index.block.max.size</name><value>131072</value><source>hbase-default.xml</source></property>
  1782. <property><name>hbase.regionserver.hlog.reader.impl</name><value>org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader</value><source>hbase-default.xml</source></property>
  1783. <property><name>dfs.namenode.audit.loggers</name><value>default</value><source>hdfs-default.xml</source></property>
  1784. <property><name>dfs.namenode.checkpoint.txns</name><value>1000000</value><source>hdfs-default.xml</source></property>
  1785. <property><name>hbase.zookeeper.property.dataDir</name><value>${hbase.tmp.dir}/zookeeper</value><source>hbase-default.xml</source></property>
  1786. <property><name>hbase.hregion.memstore.mslab.enabled</name><value>true</value><source>hbase-site.xml</source></property>
  1787. <property><name>hbase.regionserver.dns.nameserver</name><value>default</value><source>hbase-default.xml</source></property>
  1788. <property><name>hfile.block.index.cacheonwrite</name><value>false</value><source>hbase-default.xml</source></property>
  1789. <property><name>hfile.block.cache.size</name><value>0.0</value><source>programatically</source></property>
  1790. <property><name>dfs.ha.tail-edits.period</name><value>60</value><source>hdfs-default.xml</source></property>
  1791. <property><name>hadoop.security.authentication</name><value>simple</value><source>core-site.xml</source></property>
  1792. <property><name>fs.s3.buffer.dir</name><value>${hadoop.tmp.dir}/s3</value><source>core-default.xml</source></property>
  1793. <property><name>hbase.hstore.checksum.algorithm</name><value>CRC32</value><source>hbase-default.xml</source></property>
  1794. <property><name>hbase.master.info.port</name><value>60010</value><source>hbase-default.xml</source></property>
  1795. <property><name>dfs.namenode.avoid.read.stale.datanode</name><value>true</value><source>hdfs-site.xml</source></property>
  1796. <property><name>dfs.client.file-block-storage-locations.timeout</name><value>60</value><source>hdfs-default.xml</source></property>
  1797. <property><name>hbase.rest.threads.min</name><value>2</value><source>hbase-default.xml</source></property>
  1798. <property><name>dfs.datanode.drop.cache.behind.writes</name><value>false</value><source>hdfs-default.xml</source></property>
  1799. <property><name>tfile.fs.input.buffer.size</name><value>262144</value><source>core-default.xml</source></property>
  1800. <property><name>dfs.block.access.token.enable</name><value>true</value><source>hdfs-site.xml</source></property>
  1801. <property><name>dfs.journalnode.http-address</name><value>0.0.0.0:8480</value><source>hdfs-site.xml</source></property>
  1802. <property><name>hadoop.proxyuser.hive.groups</name><value>users</value><source>core-site.xml</source></property>
  1803. <property><name>ftp.blocksize</name><value>67108864</value><source>core-default.xml</source></property>
  1804. <property><name>hbase.local.dir</name><value>${hbase.tmp.dir}/local/</value><source>hbase-default.xml</source></property>
  1805. <property><name>dfs.datanode.data.dir</name><value>/data/data1,/data/data2,/data/data3,/data/data4,/data/data5,/data/data6</value><source>hdfs-site.xml</source></property>
  1806. <property><name>hadoop.policy.file</name><value>hbase-policy.xml</value><source>hbase-default.xml</source></property>
  1807. <property><name>ha.failover-controller.cli-check.rpc-timeout.ms</name><value>20000</value><source>core-default.xml</source></property>
  1808. <property><name>dfs.namenode.rpc-address.rtpha.nn2</name><value>dsjhd113:8020</value><source>hdfs-site.xml</source></property>
  1809. <property><name>dfs.namenode.rpc-address.rtpha.nn1</name><value>dsjhd112:8020</value><source>hdfs-site.xml</source></property>
  1810. <property><name>dfs.namenode.max.extra.edits.segments.retained</name><value>10000</value><source>hdfs-default.xml</source></property>
  1811. <property><name>dfs.https.port</name><value>50470</value><source>hdfs-site.xml</source></property>
  1812. <property><name>dfs.namenode.replication.interval</name><value>3</value><source>hdfs-default.xml</source></property>
  1813. <property><name>dfs.namenode.https-address</name><value>dsjhd112,dsjhd113:50470</value><source>hdfs-site.xml</source></property>
  1814. <property><name>mapreduce.jobtracker.webinterface.trusted</name><value>false</value><source>core-site.xml</source></property>
  1815. <property><name>ipc.client.kill.max</name><value>10</value><source>core-default.xml</source></property>
  1816. <property><name>dfs.ha.automatic-failover.enabled</name><value>true</value><source>hdfs-site.xml</source></property>
  1817. <property><name>dfs.image.transfer.timeout</name><value>600000</value><source>hdfs-default.xml</source></property>
  1818. <property><name>zookeeper.znode.acl.parent</name><value>acl</value><source>hbase-default.xml</source></property>
  1819. <property><name>dfs.client.failover.sleep.max.millis</name><value>15000</value><source>hdfs-default.xml</source></property>
  1820. <property><name>hbase.superuser</name><value>hbase</value><source>hbase-site.xml</source></property>
  1821. <property><name>dfs.default.chunk.view.size</name><value>32768</value><source>hdfs-default.xml</source></property>
  1822. <property><name>hbase.auth.key.update.interval</name><value>86400000</value><source>hbase-default.xml</source></property>
  1823. <property><name>hbase.server.compactchecker.interval.multiplier</name><value>1000</value><source>hbase-default.xml</source></property>
  1824. <property><name>hadoop.http.filter.initializers</name><value>org.apache.hadoop.http.lib.StaticUserWebFilter</value><source>core-default.xml</source></property>
  1825. <property><name>dfs.datanode.failed.volumes.tolerated</name><value>0</value><source>hdfs-site.xml</source></property>
  1826. <property><name>hadoop.http.authentication.type</name><value>simple</value><source>core-default.xml</source></property>
  1827. <property><name>hbase.regionserver.regionSplitLimit</name><value>2147483647</value><source>hbase-default.xml</source></property>
  1828. <property><name>hbase.rest.port</name><value>8080</value><source>hbase-default.xml</source></property>
  1829. <property><name>dfs.datanode.data.dir.perm</name><value>750</value><source>hdfs-site.xml</source></property>
  1830. <property><name>ipc.server.listen.queue.size</name><value>128</value><source>core-default.xml</source></property>
  1831. <property><name>file.stream-buffer-size</name><value>4096</value><source>core-default.xml</source></property>
  1832. <property><name>hbase.storescanner.parallel.seek.threads</name><value>10</value><source>hbase-default.xml</source></property>
  1833. <property><name>dfs.namenode.fs-limits.max-directory-items</name><value>0</value><source>hdfs-default.xml</source></property>
  1834. <property><name>io.mapfile.bloom.size</name><value>1048576</value><source>core-default.xml</source></property>
  1835. <property><name>ftp.replication</name><value>3</value><source>core-default.xml</source></property>
  1836. <property><name>dfs.datanode.dns.nameserver</name><value>default</value><source>hdfs-default.xml</source></property>
  1837. <property><name>hbase.client.pause</name><value>100</value><source>hbase-default.xml</source></property>
  1838. <property><name>hadoop.proxyuser.hcat.hosts</name><value>dsjhd112</value><source>core-site.xml</source></property>
  1839. <property><name>dfs.replication.max</name><value>50</value><source>hdfs-site.xml</source></property>
  1840. <property><name>hbase.metrics.exposeOperationTimes</name><value>true</value><source>hbase-default.xml</source></property>
  1841. <property><name>dfs.stream-buffer-size</name><value>4096</value><source>hdfs-default.xml</source></property>
  1842. <property><name>dfs.namenode.backup.address</name><value>0.0.0.0:50100</value><source>hdfs-default.xml</source></property>
  1843. <property><name>dfs.hosts.exclude</name><value>/etc/hadoop/conf/dfs.exclude</value><source>hdfs-site.xml</source></property>
  1844. <property><name>hadoop.util.hash.type</name><value>murmur</value><source>core-default.xml</source></property>
  1845. <property><name>dfs.block.access.key.update.interval</name><value>600</value><source>hdfs-default.xml</source></property>
  1846. <property><name>dfs.datanode.dns.interface</name><value>default</value><source>hdfs-default.xml</source></property>
  1847. <property><name>dfs.datanode.use.datanode.hostname</name><value>false</value><source>hdfs-default.xml</source></property>
  1848. <property><name>dfs.client.read.shortcircuit</name><value>true</value><source>hdfs-site.xml</source></property>
  1849. <property><name>dfs.namenode.backup.http-address</name><value>0.0.0.0:50105</value><source>hdfs-default.xml</source></property>
  1850. <property><name>hbase.snapshot.enabled</name><value>true</value><source>hbase-default.xml</source></property>
  1851. <property><name>hbase.rpc.timeout</name><value>60000</value><source>hbase-default.xml</source></property>
  1852. <property><name>ha.zookeeper.acl</name><value>world:anyone:rwcda</value><source>core-default.xml</source></property>
  1853. <property><name>dfs.namenode.enable.retrycache</name><value>true</value><source>hdfs-default.xml</source></property>
  1854. <property><name>io.map.index.skip</name><value>0</value><source>core-default.xml</source></property>
  1855. <property><name>net.topology.node.switch.mapping.impl</name><value>org.apache.hadoop.net.ScriptBasedMapping</value><source>core-default.xml</source></property>
  1856. <property><name>fs.s3.maxRetries</name><value>4</value><source>core-default.xml</source></property>
  1857. <property><name>dfs.namenode.logging.level</name><value>info</value><source>hdfs-default.xml</source></property>
  1858. <property><name>ha.failover-controller.new-active.rpc-timeout.ms</name><value>60000</value><source>core-default.xml</source></property>
  1859. <property><name>s3native.client-write-packet-size</name><value>65536</value><source>core-default.xml</source></property>
  1860. <property><name>hadoop.http.staticuser.user</name><value>dr.who</value><source>core-default.xml</source></property>
  1861. <property><name>ha.zookeeper.quorum</name><value>dsjhd051:2181,dsjhd112:2181,dsjhd113:2181</value><source>core-site.xml</source></property>
  1862. <property><name>hbase.ipc.client.fallback-to-simple-auth-allowed</name><value>false</value><source>hbase-default.xml</source></property>
  1863. <property><name>hadoop.http.authentication.simple.anonymous.allowed</name><value>true</value><source>core-default.xml</source></property>
  1864. <property><name>hadoop.fuse.timer.period</name><value>5</value><source>hdfs-default.xml</source></property>
  1865. <property><name>dfs.namenode.num.extra.edits.retained</name><value>1000000</value><source>hdfs-default.xml</source></property>
  1866. <property><name>hadoop.rpc.socket.factory.class.default</name><value>org.apache.hadoop.net.StandardSocketFactory</value><source>core-default.xml</source></property>
  1867. <property><name>hbase.hstore.blockingWaitTime</name><value>90000</value><source>hbase-default.xml</source></property>
  1868. <property><name>dfs.namenode.handler.count</name><value>100</value><source>hdfs-site.xml</source></property>
  1869. <property><name>fs.automatic.close</name><value>true</value><source>core-default.xml</source></property>
  1870. <property><name>dfs.datanode.directoryscan.interval</name><value>21600</value><source>hdfs-default.xml</source></property>
  1871. <property><name>hbase.coprocessor.abortonerror</name><value>false</value><source>hbase-default.xml</source></property>
  1872. <property><name>hbase.zookeeper.quorum</name><value>dsjhd051,dsjhd112,dsjhd113</value><source>hbase-site.xml</source></property>
  1873. <property><name>dfs.client.file-block-storage-locations.num-threads</name><value>10</value><source>hdfs-default.xml</source></property>
  1874. <property><name>hbase.defaults.for.version</name><value>0.96.0.2.0.6.0-76-hadoop2</value><source>hbase-default.xml</source></property>
  1875. <property><name>hbase.hregion.max.filesize</name><value>10737418240</value><source>hbase-site.xml</source></property>
  1876. <property><name>hadoop.security.group.mapping.ldap.directory.search.timeout</name><value>10000</value><source>core-default.xml</source></property>
  1877. <property><name>hbase.regionserver.logroll.errors.tolerated</name><value>2</value><source>hbase-default.xml</source></property>
  1878. <property><name>dfs.bytes-per-checksum</name><value>512</value><source>hdfs-default.xml</source></property>
  1879. <property><name>hbase.status.published</name><value>false</value><source>hbase-default.xml</source></property>
  1880. <property><name>dfs.namenode.checkpoint.max-retries</name><value>3</value><source>hdfs-default.xml</source></property>
  1881. <property><name>zookeeper.znode.parent</name><value>/hbase-unsecure</value><source>hbase-site.xml</source></property>
  1882. <property><name>ftp.stream-buffer-size</name><value>4096</value><source>core-default.xml</source></property>
  1883. <property><name>ha.health-monitor.rpc-timeout.ms</name><value>45000</value><source>core-default.xml</source></property>
  1884. <property><name>dfs.namenode.avoid.write.stale.datanode</name><value>true</value><source>hdfs-site.xml</source></property>
  1885. <property><name>dfs.namenode.edits.noeditlogchannelflush</name><value>false</value><source>hdfs-default.xml</source></property>
  1886. <property><name>hadoop.security.group.mapping.ldap.search.attr.member</name><value>member</value><source>core-default.xml</source></property>
  1887. <property><name>dfs.blockreport.initialDelay</name><value>120</value><source>hdfs-site.xml</source></property>
  1888. <property><name>io.compression.codec.bzip2.library</name><value>system-native</value><source>core-default.xml</source></property>
  1889. <property><name>hadoop.http.authentication.token.validity</name><value>36000</value><source>core-default.xml</source></property>
  1890. <property><name>dfs.namenode.delegation.token.max-lifetime</name><value>604800000</value><source>hdfs-default.xml</source></property>
  1891. <property><name>hbase.metrics.showTableName</name><value>true</value><source>hbase-default.xml</source></property>
  1892. <property><name>s3native.replication</name><value>3</value><source>core-default.xml</source></property>
  1893. <property><name>hbase.regionserver.hlog.writer.impl</name><value>org.apache.hadoop.hbase.regionserver.wal.ProtobufLogWriter</value><source>hbase-default.xml</source></property>
  1894. <property><name>hbase.zookeeper.property.syncLimit</name><value>5</value><source>hbase-default.xml</source></property>
  1895. <property><name>dfs.heartbeat.interval</name><value>3</value><source>hdfs-site.xml</source></property>
  1896. <property><name>dfs.ha.fencing.ssh.connect-timeout</name><value>30000</value><source>core-default.xml</source></property>
  1897. <property><name>hbase.thrift.htablepool.size.max</name><value>1000</value><source>hbase-default.xml</source></property>
  1898. <property><name>net.topology.impl</name><value>org.apache.hadoop.net.NetworkTopology</value><source>core-default.xml</source></property>
  1899. <property><name>dfs.nameservices</name><value>rtpha</value><source>hdfs-site.xml</source></property>
  1900. <property><name>zookeeper.znode.rootserver</name><value>root-region-server</value><source>hbase-default.xml</source></property>
  1901. <property><name>hbase.regionserver.msginterval</name><value>3000</value><source>hbase-default.xml</source></property>
  1902. <property><name>ha.failover-controller.graceful-fence.rpc-timeout.ms</name><value>5000</value><source>core-default.xml</source></property>
  1903. <property><name>hbase.table.lock.enable</name><value>true</value><source>hbase-default.xml</source></property>
  1904. <property><name>hbase.rest.threads.max</name><value>100</value><source>hbase-default.xml</source></property>
  1905. <property><name>dfs.client.read.shortcircuit.streams.cache.size</name><value>4096</value><source>hdfs-site.xml</source></property>
  1906. <property><name>hbase.master.logcleaner.plugins</name><value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner,org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner,org.apache.hadoop.hbase.master.snapshot.SnapshotLogCleaner</value><source>programatically</source></property>
  1907. <property><name>dfs.namenode.secondary.http-address</name><value>null</value><source>hdfs-site.xml</source></property>
  1908. <property><name>hbase.coprocessor.master.classes</name><value>org.apache.hadoop.hbase.security.access.AccessController</value><source>hbase-site.xml</source></property>
  1909. <property><name>ipc.client.idlethreshold</name><value>8000</value><source>core-site.xml</source></property>
  1910. <property><name>ipc.server.tcpnodelay</name><value>false</value><source>core-default.xml</source></property>
  1911. <property><name>ftp.bytes-per-checksum</name><value>512</value><source>core-default.xml</source></property>
  1912. <property><name>dfs.namenode.stale.datanode.interval</name><value>30000</value><source>hdfs-site.xml</source></property>
  1913. <property><name>s3.bytes-per-checksum</name><value>512</value><source>core-default.xml</source></property>
  1914. <property><name>hbase.rpc.shortoperation.timeout</name><value>10000</value><source>hbase-default.xml</source></property>
  1915. <property><name>hbase.storescanner.parallel.seek.enable</name><value>false</value><source>hbase-default.xml</source></property>
  1916. <property><name>dfs.ha.fencing.methods</name><value>shell(/bin/true)</value><source>hdfs-site.xml</source></property>
  1917. <property><name>dfs.namenode.http-address.rtpha.nn2</name><value>dsjhd113:50070</value><source>hdfs-site.xml</source></property>
  1918. <property><name>hbase.online.schema.update.enable</name><value>false</value><source>hbase-default.xml</source></property>
  1919. <property><name>dfs.namenode.http-address.rtpha.nn1</name><value>dsjhd112:50070</value><source>hdfs-site.xml</source></property>
  1920. <property><name>fs.s3.block.size</name><value>67108864</value><source>core-default.xml</source></property>
  1921. <property><name>dfs.client.failover.connection.retries</name><value>0</value><source>hdfs-default.xml</source></property>
  1922. <property><name>hbase.status.multicast.address.ip</name><value>226.1.1.3</value><source>hbase-default.xml</source></property>
  1923. <property><name>hadoop.rpc.protection</name><value>authentication</value><source>core-default.xml</source></property>
  1924. <property><name>dfs.namenode.retrycache.expirytime.millis</name><value>600000</value><source>hdfs-default.xml</source></property>
  1925. <property><name>hadoop.security.auth_to_local</name><value>
  1926. RULE:[2:$1@$0]([rn]m@.*)s/.*/yarn/
  1927. RULE:[2:$1@$0](jhs@.*)s/.*/mapred/
  1928. RULE:[2:$1@$0]([nd]n@.*)s/.*/hdfs/
  1929. RULE:[2:$1@$0](hm@.*)s/.*/hbase/
  1930. RULE:[2:$1@$0](rs@.*)s/.*/hbase/
  1931. DEFAULT</value><source>core-site.xml</source></property>
  1932. <property><name>dfs.secondary.namenode.kerberos.internal.spnego.principal</name><value>${dfs.web.authentication.kerberos.principal}</value><source>hdfs-default.xml</source></property>
  1933. <property><name>dfs.journalnode.edits.dir</name><value>/grid/0/hdfs/journal</value><source>hdfs-site.xml</source></property>
  1934. <property><name>ftp.client-write-packet-size</name><value>65536</value><source>core-default.xml</source></property>
  1935. <property><name>hbase.regionserver.checksum.verify</name><value>true</value><source>hbase-default.xml</source></property>
  1936. <property><name>fs.defaultFS</name><value>hdfs://rtpha</value><source>because fs.default.name is deprecated</source></property>
  1937. <property><name>file.client-write-packet-size</name><value>65536</value><source>core-default.xml</source></property>
  1938. <property><name>hbase.tmp.dir</name><value>/opt/hadoop/hbase</value><source>hbase-site.xml</source></property>
  1939. <property><name>hbase.zookeeper.property.initLimit</name><value>10</value><source>hbase-default.xml</source></property>
  1940. <property><name>fs.trash.checkpoint.interval</name><value>0</value><source>core-default.xml</source></property>
  1941. <property><name>hbase.cluster.distributed</name><value>true</value><source>hbase-site.xml</source></property>
  1942. <property><name>hadoop.http.authentication.signature.secret.file</name><value>${user.home}/hadoop-http-auth-signature-secret</value><source>core-default.xml</source></property>
  1943. <property><name>s3native.stream-buffer-size</name><value>4096</value><source>core-default.xml</source></property>
  1944. <property><name>hbase.thrift.maxWorkerThreads</name><value>1000</value><source>hbase-default.xml</source></property>
  1945. <property><name>dfs.namenode.checkpoint.edits.dir</name><value>${dfs.namenode.checkpoint.dir}</value><source>hdfs-site.xml</source></property>
  1946. <property><name>fs.permissions.umask-mode</name><value>022</value><source>hdfs-site.xml</source></property>
  1947. <property><name>hbase.hstore.compaction.kv.max</name><value>10</value><source>hbase-default.xml</source></property>
  1948. <property><name>hadoop.common.configuration.version</name><value>0.23.0</value><source>core-default.xml</source></property>
  1949. <property><name>hbase.bulkload.retries.number</name><value>0</value><source>hbase-default.xml</source></property>
  1950. <property><name>hadoop.security.group.mapping.ldap.ssl</name><value>false</value><source>core-default.xml</source></property>
  1951. <property><name>dfs.client.failover.proxy.provider.rtpha</name><value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value><source>hdfs-site.xml</source></property>
  1952. <property><name>io.serializations</name><value>org.apache.hadoop.io.serializer.WritableSerialization</value><source>core-site.xml</source></property>
  1953. <property><name>fs.df.interval</name><value>60000</value><source>core-default.xml</source></property>
  1954. <property><name>hbase.rest.readonly</name><value>false</value><source>hbase-default.xml</source></property>
  1955. <property><name>io.seqfile.compress.blocksize</name><value>1000000</value><source>core-default.xml</source></property>
  1956. <property><name>hbase.client.localityCheck.threadPoolSize</name><value>2</value><source>hbase-default.xml</source></property>
  1957. <property><name>ipc.client.connect.max.retries</name><value>50</value><source>core-site.xml</source></property>
  1958. <property><name>hadoop.security.groups.cache.secs</name><value>300</value><source>core-default.xml</source></property>
  1959. <property><name>dfs.namenode.delegation.key.update-interval</name><value>86400000</value><source>hdfs-default.xml</source></property>
  1960. <property><name>hbase.regionserver.port</name><value>60020</value><source>hbase-default.xml</source></property>
  1961. <property><name>dfs.datanode.available-space-volume-choosing-policy.balanced-space-preference-fraction</name><value>0.75f</value><source>hdfs-default.xml</source></property>
  1962. <property><name>hbase.regionserver.handler.count</name><value>60</value><source>hbase-site.xml</source></property>
  1963. <property><name>hadoop.security.group.mapping.ldap.search.filter.user</name><value>(&amp;(objectClass=user)(sAMAccountName={0}))</value><source>core-default.xml</source></property>
  1964. <property><name>dfs.image.compress</name><value>false</value><source>hdfs-default.xml</source></property>
  1965. <property><name>hbase.hregion.majorcompaction</name><value>604800000</value><source>hbase-site.xml</source></property>
  1966. <property><name>dfs.namenode.kerberos.internal.spnego.principal</name><value>${dfs.web.authentication.kerberos.principal}</value><source>hdfs-default.xml</source></property>
  1967. <property><name>fs.s3n.block.size</name><value>67108864</value><source>core-default.xml</source></property>
  1968. <property><name>hbase.regionserver.optionalcacheflushinterval</name><value>3600000</value><source>hbase-default.xml</source></property>
  1969. <property><name>fs.ftp.host</name><value>0.0.0.0</value><source>core-default.xml</source></property>
  1970. <property><name>hadoop.security.group.mapping</name><value>org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback</value><source>core-default.xml</source></property>
  1971. <property><name>dfs.datanode.address</name><value>0.0.0.0:50010</value><source>hdfs-site.xml</source></property>
  1972. <property><name>dfs.datanode.https.address</name><value>0.0.0.0:50475</value><source>hdfs-default.xml</source></property>
  1973. <property><name>file.replication</name><value>1</value><source>core-default.xml</source></property>
  1974. <property><name>hbase.rest.filter.classes</name><value>org.apache.hadoop.hbase.rest.filter.GzipFilter</value><source>hbase-default.xml</source></property>
  1975. <property><name>hbase.config.read.zookeeper.config</name><value>false</value><source>hbase-default.xml</source></property>
  1976. <property><name>dfs.datanode.drop.cache.behind.reads</name><value>false</value><source>hdfs-default.xml</source></property>
  1977. <property><name>hbase.regionserver.global.memstore.upperLimit</name><value>0.4</value><source>hbase-site.xml</source></property>
  1978. <property><name>hadoop.fuse.connection.timeout</name><value>300</value><source>hdfs-default.xml</source></property>
  1979. <property><name>hadoop.work.around.non.threadsafe.getpwuid</name><value>false</value><source>core-default.xml</source></property>
  1980. <property><name>mapreduce.client.genericoptionsparser.used</name><value>true</value><source>programatically</source></property>
  1981. <property><name>hadoop.tmp.dir</name><value>/tmp/hadoop-${user.name}</value><source>core-default.xml</source></property>
  1982. <property><name>dfs.client.block.write.replace-datanode-on-failure.policy</name><value>DEFAULT</value><source>hdfs-default.xml</source></property>
  1983. <property><name>hbase.hregion.preclose.flush.size</name><value>5242880</value><source>hbase-default.xml</source></property>
  1984. <property><name>hbase.regionserver.global.memstore.lowerLimit</name><value>0.38</value><source>hbase-site.xml</source></property>
  1985. <property><name>dfs.cluster.administrators</name><value> hdfs</value><source>hdfs-site.xml</source></property>
  1986. <property><name>hadoop.kerberos.kinit.command</name><value>kinit</value><source>core-default.xml</source></property>
  1987. <property><name>dfs.namenode.fs-limits.max-blocks-per-file</name><value>1048576</value><source>hdfs-default.xml</source></property>
  1988. <property><name>dfs.webhdfs.enabled</name><value>true</value><source>hdfs-site.xml</source></property>
  1989. <property><name>dfs.datanode.du.reserved</name><value>1073741824</value><source>hdfs-site.xml</source></property>
  1990. <property><name>fs.default.name</name><value>hdfs://rtpha</value><source>programatically</source></property>
  1991. <property><name>file.bytes-per-checksum</name><value>512</value><source>core-default.xml</source></property>
  1992. <property><name>hbase.master.dns.interface</name><value>default</value><source>hbase-default.xml</source></property>
  1993. <property><name>dfs.client.block.write.replace-datanode-on-failure.enable</name><value>true</value><source>hdfs-default.xml</source></property>
  1994. <property><name>net.topology.script.number.args</name><value>100</value><source>core-default.xml</source></property>
  1995. <property><name>dfs.namenode.decommission.interval</name><value>30</value><source>hdfs-default.xml</source></property>
  1996. <property><name>hbase.defaults.for.version.skip</name><value>true</value><source>hbase-site.xml</source></property>
  1997. <property><name>dfs.image.compression.codec</name><value>org.apache.hadoop.io.compress.DefaultCodec</value><source>hdfs-default.xml</source></property>
  1998. <property><name>hadoop.ssl.hostname.verifier</name><value>DEFAULT</value><source>core-default.xml</source></property>
  1999. <property><name>dfs.namenode.support.allow.format</name><value>true</value><source>hdfs-default.xml</source></property>
  2000. <property><name>ipc.client.connect.timeout</name><value>20000</value><source>core-default.xml</source></property>
  2001. <property><name>hbase.server.versionfile.writeattempts</name><value>3</value><source>hbase-default.xml</source></property>
  2002. <property><name>io.mapfile.bloom.error.rate</name><value>0.005</value><source>core-default.xml</source></property>
  2003. <property><name>dfs.permissions.superusergroup</name><value>hdfs</value><source>hdfs-site.xml</source></property>
  2004. <property><name>dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold</name><value>10737418240</value><source>hdfs-default.xml</source></property>
  2005. <property><name>hbase.regionserver.info.bindAddress</name><value>0.0.0.0</value><source>hbase-default.xml</source></property>
  2006. <property><name>hbase.master.hfilecleaner.plugins</name><value>org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner,org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner,org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner</value><source>programatically</source></property>
  2007. <property><name>ha.failover-controller.graceful-fence.connection.retries</name><value>1</value><source>core-default.xml</source></property>
  2008. <property><name>ha.health-monitor.connect-retry-interval.ms</name><value>1000</value><source>core-default.xml</source></property>
  2009. <property><name>hbase.security.authentication</name><value>simple</value><source>hbase-site.xml</source></property>
  2010. <property><name>io.seqfile.local.dir</name><value>${hadoop.tmp.dir}/io/local</value><source>core-default.xml</source></property>
  2011. <property><name>dfs.namenode.checkpoint.check.period</name><value>60</value><source>hdfs-default.xml</source></property>
  2012. <property><name>tfile.io.chunk.size</name><value>1048576</value><source>core-default.xml</source></property>
  2013. <property><name>file.blocksize</name><value>67108864</value><source>core-default.xml</source></property>
  2014. <property><name>io.skip.checksum.errors</name><value>false</value><source>core-default.xml</source></property>
  2015. <property><name>dfs.namenode.edits.journal-plugin.qjournal</name><value>org.apache.hadoop.hdfs.qjournal.client.QuorumJournalManager</value><source>hdfs-default.xml</source></property>
  2016. <property><name>hbase.zookeeper.peerport</name><value>2888</value><source>hbase-default.xml</source></property>
  2017. <property><name>dfs.datanode.handler.count</name><value>10</value><source>hdfs-default.xml</source></property>
  2018. <property><name>hbase.master.port</name><value>60000</value><source>hbase-default.xml</source></property>
  2019. <property><name>fs.ftp.host.port</name><value>21</value><source>core-default.xml</source></property>
  2020. <property><name>dfs.namenode.decommission.nodes.per.interval</name><value>5</value><source>hdfs-default.xml</source></property>
  2021. <property><name>dfs.namenode.fs-limits.max-component-length</name><value>0</value><source>hdfs-default.xml</source></property>
  2022. <property><name>dfs.namenode.checkpoint.period</name><value>21600</value><source>hdfs-site.xml</source></property>
  2023. <property><name>fs.AbstractFileSystem.viewfs.impl</name><value>org.apache.hadoop.fs.viewfs.ViewFs</value><source>core-default.xml</source></property>
  2024. <property><name>dfs.namenode.retrycache.heap.percent</name><value>0.03f</value><source>hdfs-default.xml</source></property>
  2025. <property><name>ipc.client.fallback-to-simple-auth-allowed</name><value>false</value><source>core-default.xml</source></property>
  2026. <property><name>dfs.ha.namenodes.rtpha</name><value>nn1,nn2</value><source>hdfs-site.xml</source></property>
  2027. <property><name>hbase.status.multicast.address.port</name><value>6100</value><source>hbase-default.xml</source></property>
  2028. <property><name>hbase.client.keyvalue.maxsize</name><value>10485760</value><source>hbase-site.xml</source></property>
  2029. <property><name>dfs.datanode.ipc.address</name><value>0.0.0.0:8010</value><source>hdfs-site.xml</source></property>
  2030. <property><name>hadoop.ssl.require.client.cert</name><value>false</value><source>core-default.xml</source></property>
  2031. <property><name>hbase.hregion.memstore.block.multiplier</name><value>8</value><source>hbase-site.xml</source></property>
  2032. <property><name>dfs.datanode.max.transfer.threads</name><value>1024</value><source>hdfs-site.xml</source></property>
  2033. <property><name>hbase.server.thread.wakefrequency</name><value>10000</value><source>hbase-default.xml</source></property>
  2034. </configuration>
  2035. Recent regionserver aborts:
  2036. ===========================================================
  2037. 2014-06-06T17:10:24 Region server dsjhd058,60020,1402038676433 reported a fatal error:
  2038. ABORTING region server dsjhd058,60020,1402038676433: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing dsjhd058,60020,1402038676433 as dead server
  2039. at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:339)
  2040. at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:254)
  2041. at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:1351)
  2042. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5087)
  2043. at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  2044. at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  2045. Cause:
  2046. org.apache.hadoop.hbase.YouAreDeadException: org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing dsjhd058,60020,1402038676433 as dead server
  2047. at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:339)
  2048. at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:254)
  2049. at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:1351)
  2050. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5087)
  2051. at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  2052. at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  2053. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  2054. at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
  2055. at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
  2056. at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
  2057. at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
  2058. at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
  2059. at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:235)
  2060. at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:991)
  2061. at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:838)
  2062. at java.lang.Thread.run(Thread.java:662)
  2063. Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.YouAreDeadException): org.apache.hadoop.hbase.YouAreDeadException: Server REPORT rejected; currently processing dsjhd058,60020,1402038676433 as dead server
  2064. at org.apache.hadoop.hbase.master.ServerManager.checkIsDead(ServerManager.java:339)
  2065. at org.apache.hadoop.hbase.master.ServerManager.regionServerReport(ServerManager.java:254)
  2066. at org.apache.hadoop.hbase.master.HMaster.regionServerReport(HMaster.java:1351)
  2067. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:5087)
  2068. at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
  2069. at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
  2070. at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1446)
  2071. at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1650)
  2072. at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1708)
  2073. at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerReport(RegionServerStatusProtos.java:5414)
  2074. at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:989)
  2075. ... 2 more
  2076. 2014-06-06T17:10:24 Region server dsjhd058,60020,1402038676433 reported a fatal error:
  2077. ABORTING region server dsjhd058,60020,1402038676433: IOE in log roller
  2078. Cause:
  2079. java.io.FileNotFoundException: File does not exist: hdfs://rtpha/apps/hbase/data/WALs/dsjhd058,60020,1402038676433/dsjhd058%2C60020%2C1402038676433.1402045785595
  2080. at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1110)
  2081. at org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102)
  2082. at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
  2083. at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102)
  2084. at org.apache.hadoop.hbase.regionserver.wal.FSHLog.rollWriter(FSHLog.java:527)
  2085. at org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:96)
  2086. at java.lang.Thread.run(Thread.java:662)
  2087. 2014-06-06T17:10:25 Region server dsjhd058,60020,1402038676433 reported a fatal error:
  2088. ABORTING region server dsjhd058,60020,1402038676433: regionserver:60020-0x1466fded954006b regionserver:60020-0x1466fded954006b received expired from ZooKeeper, aborting
  2089. Cause:
  2090. org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
  2091. at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:401)
  2092. at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:319)
  2093. at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
  2094. at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
  2095. Logs
  2096. ===========================================================
  2097. +++++++++++++++++++++++++++++++
  2098. /var/log/hbase/hbase-hbase-master-dsjhd113.log
  2099. +++++++++++++++++++++++++++++++
  2100. 2014-06-06 18:06:13,309 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd074%2C60020%2C1402039899044.1402044753437
  2101. 2014-06-06 18:06:13,312 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048318944
  2102. 2014-06-06 18:06:13,315 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048321383
  2103. 2014-06-06 18:06:13,318 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048323708
  2104. 2014-06-06 18:06:13,321 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048326167
  2105. 2014-06-06 18:06:13,323 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048328583
  2106. 2014-06-06 18:06:13,326 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048331641
  2107. 2014-06-06 18:06:13,329 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048333978
  2108. 2014-06-06 18:06:13,332 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048337079
  2109. 2014-06-06 18:06:13,334 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd077%2C60020%2C1402027915865.1402045440987
  2110. 2014-06-06 18:06:13,337 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd077%2C60020%2C1402027915865.1402045531965
  2111. 2014-06-06 18:06:13,340 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd077%2C60020%2C1402027915865.1402045576448
  2112. 2014-06-06 18:06:13,343 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd077%2C60020%2C1402027915865.1402045628744
  2113. 2014-06-06 18:06:13,346 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd077%2C60020%2C1402027915865.1402045714430
  2114. 2014-06-06 18:07:13,293 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd051%2C60020%2C1402036579439.1402045196462
  2115. 2014-06-06 18:07:13,297 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd052%2C60020%2C1402039731411.1402046845630
  2116. 2014-06-06 18:07:13,300 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd052%2C60020%2C1402039731411.1402046865873
  2117. 2014-06-06 18:07:13,303 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd052%2C60020%2C1402039731411.1402046873000
  2118. 2014-06-06 18:07:13,306 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd052%2C60020%2C1402039731411.1402046884848
  2119. 2014-06-06 18:07:13,309 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd052%2C60020%2C1402039731411.1402046969319
  2120. 2014-06-06 18:07:13,313 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd052%2C60020%2C1402039731411.1402047028326
  2121. 2014-06-06 18:07:13,315 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd053%2C60020%2C1402027976229.1402044988600
  2122. 2014-06-06 18:07:13,318 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd053%2C60020%2C1402027976229.1402045038538
  2123. 2014-06-06 18:07:13,321 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd053%2C60020%2C1402027976229.1402045219706
  2124. 2014-06-06 18:07:13,324 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402044762886
  2125. 2014-06-06 18:07:13,327 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd063%2C60020%2C1402039188138.1402045242796
  2126. 2014-06-06 18:07:13,330 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd063%2C60020%2C1402039188138.1402045388005
  2127. 2014-06-06 18:07:13,333 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd063%2C60020%2C1402039188138.1402045433680
  2128. 2014-06-06 18:07:13,336 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd066%2C60020%2C1402027920604.1402045883546
  2129. 2014-06-06 18:07:13,340 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd069%2C60020%2C1402032388543.1402045757267
  2130. 2014-06-06 18:07:13,343 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd069%2C60020%2C1402032388543.1402046061894
  2131. 2014-06-06 18:07:13,346 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd069%2C60020%2C1402032388543.1402046197639
  2132. 2014-06-06 18:07:13,349 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd069%2C60020%2C1402032388543.1402046215876
  2133. 2014-06-06 18:07:13,352 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd069%2C60020%2C1402032388543.1402046240616
  2134. 2014-06-06 18:07:13,355 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd069%2C60020%2C1402032388543.1402046294360
  2135. 2014-06-06 18:07:13,358 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd069%2C60020%2C1402032388543.1402046449669
  2136. 2014-06-06 18:07:13,361 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd069%2C60020%2C1402032388543.1402046475135
  2137. 2014-06-06 18:07:13,366 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048340021
  2138. 2014-06-06 18:07:13,370 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048342408
  2139. 2014-06-06 18:07:13,373 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048345085
  2140. 2014-06-06 18:07:13,377 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048347837
  2141. 2014-06-06 18:08:13,288 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048349938
  2142. 2014-06-06 18:08:13,291 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048351695
  2143. 2014-06-06 18:08:13,294 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048354186
  2144. 2014-06-06 18:10:05,365 DEBUG [AM.ZK.Worker-pool2-t394] master.AssignmentManager: Handling RS_ZK_REQUEST_REGION_SPLIT, server=dsjhd064,60020,1402027918813, region=5544bd37b36ea28e890ba839f4506af6, current_state={5544bd37b36ea28e890ba839f4506af6 state=OPEN, ts=1402048435327, server=dsjhd064,60020,1402027918813}
  2145. 2014-06-06 18:10:05,366 DEBUG [AM.ZK.Worker-pool2-t394] zookeeper.ZKAssign: master:60000-0x2466f52fa5c01c6 Transitioning 5544bd37b36ea28e890ba839f4506af6 from RS_ZK_REQUEST_REGION_SPLIT to RS_ZK_REGION_SPLITTING
  2146. 2014-06-06 18:10:05,369 DEBUG [AM.ZK.Worker-pool2-t394] zookeeper.ZKAssign: master:60000-0x2466f52fa5c01c6 Transitioned node 5544bd37b36ea28e890ba839f4506af6 from RS_ZK_REQUEST_REGION_SPLIT to RS_ZK_REGION_SPLITTING
  2147. 2014-06-06 18:10:05,369 INFO [AM.ZK.Worker-pool2-t394] master.RegionStates: Transitioned {69f69a03e5ac38c7c12003a6aee3df39 state=OFFLINE, ts=1402049405369, server=null} to {69f69a03e5ac38c7c12003a6aee3df39 state=SPLITTING_NEW, ts=1402049405369, server=dsjhd064,60020,1402027918813}
  2148. 2014-06-06 18:10:05,369 INFO [AM.ZK.Worker-pool2-t394] master.RegionStates: Transitioned {d1095a2fffc1a67c1fac9e700ef54679 state=OFFLINE, ts=1402049405369, server=null} to {d1095a2fffc1a67c1fac9e700ef54679 state=SPLITTING_NEW, ts=1402049405369, server=dsjhd064,60020,1402027918813}
  2149. 2014-06-06 18:10:05,369 INFO [AM.ZK.Worker-pool2-t394] master.RegionStates: Transitioned {5544bd37b36ea28e890ba839f4506af6 state=OPEN, ts=1402048435327, server=dsjhd064,60020,1402027918813} to {5544bd37b36ea28e890ba839f4506af6 state=SPLITTING, ts=1402049405369, server=dsjhd064,60020,1402027918813}
  2150. 2014-06-06 18:10:05,371 DEBUG [AM.ZK.Worker-pool2-t395] master.AssignmentManager: Handling RS_ZK_REGION_SPLITTING, server=dsjhd064,60020,1402027918813, region=5544bd37b36ea28e890ba839f4506af6, current_state={5544bd37b36ea28e890ba839f4506af6 state=SPLITTING, ts=1402049405369, server=dsjhd064,60020,1402027918813}
  2151. 2014-06-06 18:10:07,335 DEBUG [AM.ZK.Worker-pool2-t396] master.AssignmentManager: Handling RS_ZK_REGION_SPLIT, server=dsjhd064,60020,1402027918813, region=5544bd37b36ea28e890ba839f4506af6, current_state={5544bd37b36ea28e890ba839f4506af6 state=SPLITTING, ts=1402049405371, server=dsjhd064,60020,1402027918813}
  2152. 2014-06-06 18:10:07,335 INFO [AM.ZK.Worker-pool2-t396] master.RegionStates: Transitioned {5544bd37b36ea28e890ba839f4506af6 state=SPLITTING, ts=1402049407335, server=dsjhd064,60020,1402027918813} to {5544bd37b36ea28e890ba839f4506af6 state=SPLIT, ts=1402049407335, server=dsjhd064,60020,1402027918813}
  2153. 2014-06-06 18:10:07,335 INFO [AM.ZK.Worker-pool2-t396] master.RegionStates: Offlined 5544bd37b36ea28e890ba839f4506af6 from dsjhd064,60020,1402027918813
  2154. 2014-06-06 18:10:07,335 INFO [AM.ZK.Worker-pool2-t396] master.RegionStates: Transitioned {69f69a03e5ac38c7c12003a6aee3df39 state=SPLITTING_NEW, ts=1402049407335, server=dsjhd064,60020,1402027918813} to {69f69a03e5ac38c7c12003a6aee3df39 state=OPEN, ts=1402049407335, server=dsjhd064,60020,1402027918813}
  2155. 2014-06-06 18:10:07,335 INFO [AM.ZK.Worker-pool2-t396] master.RegionStates: Onlined 69f69a03e5ac38c7c12003a6aee3df39 on dsjhd064,60020,1402027918813
  2156. 2014-06-06 18:10:07,335 INFO [AM.ZK.Worker-pool2-t396] master.RegionStates: Transitioned {d1095a2fffc1a67c1fac9e700ef54679 state=SPLITTING_NEW, ts=1402049407335, server=dsjhd064,60020,1402027918813} to {d1095a2fffc1a67c1fac9e700ef54679 state=OPEN, ts=1402049407335, server=dsjhd064,60020,1402027918813}
  2157. 2014-06-06 18:10:07,336 INFO [AM.ZK.Worker-pool2-t396] master.RegionStates: Onlined d1095a2fffc1a67c1fac9e700ef54679 on dsjhd064,60020,1402027918813
  2158. 2014-06-06 18:10:07,336 DEBUG [AM.ZK.Worker-pool2-t396] master.AssignmentManager: Handling SPLIT event for 5544bd37b36ea28e890ba839f4506af6; deleting node
  2159. 2014-06-06 18:10:07,338 DEBUG [AM.ZK.Worker-pool2-t396] zookeeper.ZKAssign: master:60000-0x2466f52fa5c01c6 Deleted unassigned node 5544bd37b36ea28e890ba839f4506af6 in expected state RS_ZK_REGION_SPLIT
  2160. 2014-06-06 18:10:07,338 INFO [AM.ZK.Worker-pool2-t396] master.AssignmentManager: Handled SPLIT event; parent=ERR_BT_D_BF001_201406,0000000000000002014060517561739900000000000000000800000000000000000004744400000005,1402048433136.5544bd37b36ea28e890ba839f4506af6., daughter a=ERR_BT_D_BF001_201406,0000000000000002014060517561739900000000000000000800000000000000000004744400000005,1402049404816.69f69a03e5ac38c7c12003a6aee3df39., daughter b=ERR_BT_D_BF001_201406,00000000000000020140605181642335,I074,I07000000008000000000000000000053,1402049404816.d1095a2fffc1a67c1fac9e700ef54679., on dsjhd064,60020,1402027918813
  2161. 2014-06-06 18:10:25,948 DEBUG [CatalogJanitor-dsjhd113:60000] master.CatalogJanitor: Deleting region BT_D_BF001_201406,46003636179312,1402045922425.c52d75455c6e36c8fb3243959f4cb1a0. because daughter splits no longer hold references
  2162. 2014-06-06 18:10:25,948 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: ARCHIVING hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0
  2163. 2014-06-06 18:10:25,952 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Archiving [class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf]
  2164. 2014-06-06 18:10:25,965 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/09188096e8e647a5990120b5f7583bfd, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/09188096e8e647a5990120b5f7583bfd
  2165. 2014-06-06 18:10:25,969 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/233b2b45bef84a578b7e09221e593250, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/233b2b45bef84a578b7e09221e593250
  2166. 2014-06-06 18:10:25,973 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/2782f43fd3794df5b3952069bb554df6, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/2782f43fd3794df5b3952069bb554df6
  2167. 2014-06-06 18:10:25,977 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/3b45c8be193845ab8d46e3af5709eca4, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/3b45c8be193845ab8d46e3af5709eca4
  2168. 2014-06-06 18:10:25,980 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/4472e97d5aa2403bbd5df3531ce15329, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/4472e97d5aa2403bbd5df3531ce15329
  2169. 2014-06-06 18:10:25,984 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/6549a9f976e4407f82476b8852c9759d, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/6549a9f976e4407f82476b8852c9759d
  2170. 2014-06-06 18:10:25,988 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/6a4814415fa84da1b2da373704834a41, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/6a4814415fa84da1b2da373704834a41
  2171. 2014-06-06 18:10:25,992 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/7718f6d4de0143f1b1425dc3363ed9df, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/7718f6d4de0143f1b1425dc3363ed9df
  2172. 2014-06-06 18:10:25,996 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/81c34c3f26784f5daa3c2ddbe683933e, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/81c34c3f26784f5daa3c2ddbe683933e
  2173. 2014-06-06 18:10:25,999 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/8ed842bbf31d4da7b7cf4ef3d1611e33, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/8ed842bbf31d4da7b7cf4ef3d1611e33
  2174. 2014-06-06 18:10:26,003 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/a0a99388df4e49a580aa5b2ff6e8b707, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/a0a99388df4e49a580aa5b2ff6e8b707
  2175. 2014-06-06 18:10:26,007 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/a4ef628cdb0f4161b840420871ab360d, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/a4ef628cdb0f4161b840420871ab360d
  2176. 2014-06-06 18:10:26,010 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/bf308a440f034cb78e5d3d8c94a3feb8, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/bf308a440f034cb78e5d3d8c94a3feb8
  2177. 2014-06-06 18:10:26,014 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/d035722daf224d5289b155b17b005f7d, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/d035722daf224d5289b155b17b005f7d
  2178. 2014-06-06 18:10:26,018 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/d0d38e8c15c74bca90607753edd1bf23, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/d0d38e8c15c74bca90607753edd1bf23
  2179. 2014-06-06 18:10:26,022 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/e11cbfc28a4743a6aabdafbb6c470303, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/e11cbfc28a4743a6aabdafbb6c470303
  2180. 2014-06-06 18:10:26,026 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/f4d31cab976b4504b50829ca009a68f2, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/f4d31cab976b4504b50829ca009a68f2
  2181. 2014-06-06 18:10:26,030 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/fa4c35d84bcf432a957587970e6d4867, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0/cf/fa4c35d84bcf432a957587970e6d4867
  2182. 2014-06-06 18:10:26,032 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Deleted all region files in: hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/c52d75455c6e36c8fb3243959f4cb1a0
  2183. 2014-06-06 18:10:26,039 INFO [CatalogJanitor-dsjhd113:60000] catalog.MetaEditor: Deleted BT_D_BF001_201406,46003636179312,1402045922425.c52d75455c6e36c8fb3243959f4cb1a0.
  2184. 2014-06-06 18:10:26,043 DEBUG [CatalogJanitor-dsjhd113:60000] master.CatalogJanitor: Deleting region DPI_ORIGINAL_DETAIL,0615144540420140530074247793;3,1401820613695.92e752bd4fc0afbe1dece267ffa8e21a. because daughter splits no longer hold references
  2185. 2014-06-06 18:10:26,043 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: ARCHIVING hdfs://rtpha/apps/hbase/data/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a
  2186. 2014-06-06 18:10:26,047 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Archiving [class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D]
  2187. 2014-06-06 18:10:26,059 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/0624d17a1dc64628adf703e0b78bf3c3, to hdfs://rtpha/apps/hbase/data/archive/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/0624d17a1dc64628adf703e0b78bf3c3
  2188. 2014-06-06 18:10:26,063 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/1fbcb12740a843caa34e7132e313f8b4, to hdfs://rtpha/apps/hbase/data/archive/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/1fbcb12740a843caa34e7132e313f8b4
  2189. 2014-06-06 18:10:26,067 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/70c5979cdc1b4eb2bac6045de144ea94, to hdfs://rtpha/apps/hbase/data/archive/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/70c5979cdc1b4eb2bac6045de144ea94
  2190. 2014-06-06 18:10:26,071 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/79fb17a402d54e1b981441c337e6db6d, to hdfs://rtpha/apps/hbase/data/archive/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/79fb17a402d54e1b981441c337e6db6d
  2191. 2014-06-06 18:10:26,075 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/ddfd947bad8247ef83365eb93455c318, to hdfs://rtpha/apps/hbase/data/archive/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a/D/ddfd947bad8247ef83365eb93455c318
  2192. 2014-06-06 18:10:26,077 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Deleted all region files in: hdfs://rtpha/apps/hbase/data/data/default/DPI_ORIGINAL_DETAIL/92e752bd4fc0afbe1dece267ffa8e21a
  2193. 2014-06-06 18:10:26,081 INFO [CatalogJanitor-dsjhd113:60000] catalog.MetaEditor: Deleted DPI_ORIGINAL_DETAIL,0615144540420140530074247793;3,1401820613695.92e752bd4fc0afbe1dece267ffa8e21a.
  2194. 2014-06-06 18:10:26,087 INFO [CatalogJanitor-dsjhd113:60000] master.CatalogJanitor: Scanned 9780 catalog row(s), gc'd 0 unreferenced merged region(s) and 2 unreferenced parent region(s)
  2195. 2014-06-06 18:11:13,335 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd054%2C60020%2C1402027922467.1402045732230
  2196. 2014-06-06 18:11:13,339 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd059%2C60020%2C1402027941533.1402045232669
  2197. 2014-06-06 18:11:13,342 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd068%2C60020%2C1402038513322.1402045236289
  2198. 2014-06-06 18:11:13,345 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd068%2C60020%2C1402038513322.1402045337919
  2199. 2014-06-06 18:11:13,348 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd068%2C60020%2C1402038513322.1402045399892
  2200. 2014-06-06 18:11:13,350 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd070%2C60020%2C1402027928334.1402045312779
  2201. 2014-06-06 18:11:13,353 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd073%2C60020%2C1402039874233.1402044797547
  2202. 2014-06-06 18:11:13,356 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd073%2C60020%2C1402039874233.1402045004966
  2203. 2014-06-06 18:11:13,359 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd073%2C60020%2C1402039874233.1402045058240
  2204. 2014-06-06 18:12:13,294 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd051%2C60020%2C1402036579439.1402045234168
  2205. 2014-06-06 18:12:13,298 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd051%2C60020%2C1402036579439.1402045295397
  2206. 2014-06-06 18:12:13,301 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd062%2C60020%2C1402027917800.1402045424873
  2207. 2014-06-06 18:12:13,305 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd062%2C60020%2C1402027917800.1402045450854
  2208. 2014-06-06 18:12:13,308 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd062%2C60020%2C1402027917800.1402045572219
  2209. 2014-06-06 18:12:13,311 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd062%2C60020%2C1402027917800.1402045631206
  2210. 2014-06-06 18:12:13,314 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd062%2C60020%2C1402027917800.1402045722152
  2211. 2014-06-06 18:12:13,317 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048356263
  2212. 2014-06-06 18:13:13,330 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048404306
  2213. 2014-06-06 18:13:13,333 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048409431
  2214. 2014-06-06 18:13:13,337 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048414324
  2215. 2014-06-06 18:13:13,340 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048418192
  2216. 2014-06-06 18:13:13,344 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048422871
  2217. 2014-06-06 18:13:13,347 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048431169
  2218. 2014-06-06 18:13:13,350 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048435798
  2219. 2014-06-06 18:13:13,353 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048439181
  2220. 2014-06-06 18:13:13,357 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048443573
  2221. 2014-06-06 18:13:13,360 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048448330
  2222. 2014-06-06 18:13:13,363 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048453232
  2223. 2014-06-06 18:13:13,365 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048457409
  2224. 2014-06-06 18:13:13,368 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048358607
  2225. 2014-06-06 18:13:13,371 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048361006
  2226. 2014-06-06 18:13:13,374 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048363409
  2227. 2014-06-06 18:13:13,377 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048366086
  2228. 2014-06-06 18:13:13,380 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048368136
  2229. 2014-06-06 18:13:13,383 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048370503
  2230. 2014-06-06 18:13:13,385 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048372858
  2231. 2014-06-06 18:13:13,388 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048374780
  2232. 2014-06-06 18:13:13,391 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048377961
  2233. 2014-06-06 18:13:13,393 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048380427
  2234. 2014-06-06 18:13:13,396 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048382715
  2235. 2014-06-06 18:13:13,399 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048385145
  2236. 2014-06-06 18:13:13,402 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048387269
  2237. 2014-06-06 18:13:13,404 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048389348
  2238. 2014-06-06 18:13:13,407 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048391668
  2239. 2014-06-06 18:13:13,411 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048393897
  2240. 2014-06-06 18:13:13,416 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048396363
  2241. 2014-06-06 18:13:13,420 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048398455
  2242. 2014-06-06 18:13:13,424 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048400758
  2243. 2014-06-06 18:13:13,426 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048403003
  2244. 2014-06-06 18:13:13,429 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048405264
  2245. 2014-06-06 18:13:13,431 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048407693
  2246. 2014-06-06 18:13:13,434 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048410246
  2247. 2014-06-06 18:13:13,436 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048412354
  2248. 2014-06-06 18:13:13,439 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048414252
  2249. 2014-06-06 18:13:13,443 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048416296
  2250. 2014-06-06 18:13:13,447 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048418081
  2251. 2014-06-06 18:13:13,449 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048420247
  2252. 2014-06-06 18:13:13,454 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048422449
  2253. 2014-06-06 18:13:13,458 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048428306
  2254. 2014-06-06 18:13:13,460 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048430604
  2255. 2014-06-06 18:13:13,462 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048432636
  2256. 2014-06-06 18:13:13,465 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048434667
  2257. 2014-06-06 18:13:13,467 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048436255
  2258. 2014-06-06 18:13:13,469 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048438110
  2259. 2014-06-06 18:13:13,472 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048440067
  2260. 2014-06-06 18:13:13,474 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048442099
  2261. 2014-06-06 18:13:13,476 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048444210
  2262. 2014-06-06 18:13:13,479 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048446472
  2263. 2014-06-06 18:13:13,481 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048448642
  2264. 2014-06-06 18:13:13,483 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048451055
  2265. 2014-06-06 18:13:13,485 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048453212
  2266. 2014-06-06 18:13:13,488 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048455437
  2267. 2014-06-06 18:13:13,490 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048457451
  2268. 2014-06-06 18:13:13,492 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048459399
  2269. 2014-06-06 18:13:13,495 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048461584
  2270. 2014-06-06 18:13:13,497 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048463747
  2271. 2014-06-06 18:13:13,500 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048466067
  2272. 2014-06-06 18:13:13,502 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048468292
  2273. 2014-06-06 18:13:13,505 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048470343
  2274. 2014-06-06 18:13:13,507 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048472592
  2275. 2014-06-06 18:13:13,510 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048474951
  2276. 2014-06-06 18:13:13,513 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048477238
  2277. 2014-06-06 18:13:13,515 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048479335
  2278. 2014-06-06 18:13:13,518 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048481455
  2279. 2014-06-06 18:13:13,521 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048483610
  2280. 2014-06-06 18:13:13,523 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048485697
  2281. 2014-06-06 18:13:13,526 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048487881
  2282. 2014-06-06 18:13:13,528 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048489715
  2283. 2014-06-06 18:13:13,531 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048491548
  2284. 2014-06-06 18:13:13,534 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048493310
  2285. 2014-06-06 18:13:13,536 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048495628
  2286. 2014-06-06 18:13:13,562 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048497786
  2287. 2014-06-06 18:13:13,566 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048499379
  2288. 2014-06-06 18:13:13,568 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048501067
  2289. 2014-06-06 18:13:13,571 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048502819
  2290. 2014-06-06 18:13:13,574 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048504555
  2291. 2014-06-06 18:13:13,577 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048506785
  2292. 2014-06-06 18:13:13,580 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048508694
  2293. 2014-06-06 18:13:13,585 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048510729
  2294. 2014-06-06 18:13:13,587 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048512935
  2295. 2014-06-06 18:13:13,589 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048515890
  2296. 2014-06-06 18:13:13,591 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048518710
  2297. 2014-06-06 18:13:13,593 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048523891
  2298. 2014-06-06 18:13:48,897 DEBUG [RpcServer.handler=57,port=60000] client.HConnectionManager$HConnectionImplementation: Removed all cached region locations that map to dsjhd061,60020,1402027921226
  2299. 2014-06-06 18:14:13,327 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048461989
  2300. 2014-06-06 18:14:13,330 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048466817
  2301. 2014-06-06 18:14:13,333 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048471631
  2302. 2014-06-06 18:14:13,336 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048476684
  2303. 2014-06-06 18:14:13,339 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048481330
  2304. 2014-06-06 18:14:13,342 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048485970
  2305. 2014-06-06 18:14:13,345 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048490385
  2306. 2014-06-06 18:14:13,348 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048494612
  2307. 2014-06-06 18:14:13,351 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048499111
  2308. 2014-06-06 18:14:13,353 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048503095
  2309. 2014-06-06 18:14:13,356 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048507639
  2310. 2014-06-06 18:14:13,359 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048512413
  2311. 2014-06-06 18:14:13,362 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048531567
  2312. 2014-06-06 18:14:13,365 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048551423
  2313. 2014-06-06 18:14:13,368 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048571173
  2314. 2014-06-06 18:14:49,265 DEBUG [RpcServer.handler=57,port=60000] client.HConnectionManager$HConnectionImplementation: Removed all cached region locations that map to dsjhd061,60020,1402027921226
  2315. 2014-06-06 18:15:13,330 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048519249
  2316. 2014-06-06 18:15:13,333 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048535326
  2317. 2014-06-06 18:15:13,337 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048560010
  2318. 2014-06-06 18:15:13,340 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048596472
  2319. 2014-06-06 18:15:13,344 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048626528
  2320. 2014-06-06 18:15:13,348 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048650702
  2321. 2014-06-06 18:15:13,351 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048931511
  2322. 2014-06-06 18:15:13,354 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048936590
  2323. 2014-06-06 18:15:13,357 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048941809
  2324. 2014-06-06 18:15:13,360 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048594603
  2325. 2014-06-06 18:15:13,363 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048611087
  2326. 2014-06-06 18:15:13,365 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048628148
  2327. 2014-06-06 18:15:13,368 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048635438
  2328. 2014-06-06 18:15:13,371 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048650860
  2329. 2014-06-06 18:15:13,373 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048929446
  2330. 2014-06-06 18:15:25,933 DEBUG [CatalogJanitor-dsjhd113:60000] master.CatalogJanitor: Deleting region BT_D_BF001_201406,4600304159432142014060517055720500000000000000000808000000000000000005668700000006410174664,1402046055308.bf89ff438e6961193b0bb7b772e2fbd2. because daughter splits no longer hold references
  2331. 2014-06-06 18:15:25,933 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: ARCHIVING hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2
  2332. 2014-06-06 18:15:25,938 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Archiving [class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf]
  2333. 2014-06-06 18:15:25,952 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/18986793f0ea43599bebe6ad4c4df099, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/18986793f0ea43599bebe6ad4c4df099
  2334. 2014-06-06 18:15:25,957 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/1c454591b5a2427fadabc5211d138c01, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/1c454591b5a2427fadabc5211d138c01
  2335. 2014-06-06 18:15:25,960 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/1ca167a494e14a9f88c68b53e4dcc8cf, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/1ca167a494e14a9f88c68b53e4dcc8cf
  2336. 2014-06-06 18:15:25,964 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/3af727f20bd1411aa45dd87736a2c3e9, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/3af727f20bd1411aa45dd87736a2c3e9
  2337. 2014-06-06 18:15:25,968 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/3eeb9fe2252749af989b3b53e7eb49c9, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/3eeb9fe2252749af989b3b53e7eb49c9
  2338. 2014-06-06 18:15:25,972 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/57701292ab1f41988ab3e32c7393871a, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/57701292ab1f41988ab3e32c7393871a
  2339. 2014-06-06 18:15:25,977 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/5de7a228c52646a39e7e0d81e201ff9d, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/5de7a228c52646a39e7e0d81e201ff9d
  2340. 2014-06-06 18:15:25,981 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/5e4b9612a2334cf99d336ca1d510b129, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/5e4b9612a2334cf99d336ca1d510b129
  2341. 2014-06-06 18:15:25,985 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/69301f0dd10245369ec7f6fc00280405, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/69301f0dd10245369ec7f6fc00280405
  2342. 2014-06-06 18:15:25,990 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/8fced1ba62d945679f1f927e6b0c7b2f, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/8fced1ba62d945679f1f927e6b0c7b2f
  2343. 2014-06-06 18:15:25,994 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/97bfe0707bb14738a59e8867f8a0d78d, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/97bfe0707bb14738a59e8867f8a0d78d
  2344. 2014-06-06 18:15:25,998 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/a72baaaec9f84302b96fe7fd92ce2077, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/a72baaaec9f84302b96fe7fd92ce2077
  2345. 2014-06-06 18:15:26,003 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/c7e43d1b578e4564a0b05068b9e91957, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/c7e43d1b578e4564a0b05068b9e91957
  2346. 2014-06-06 18:15:26,007 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/d11486209d214c5fb0e6b1d0f5d3578d, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/d11486209d214c5fb0e6b1d0f5d3578d
  2347. 2014-06-06 18:15:26,011 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/f009edcfec28494d906061c49278df80, to hdfs://rtpha/apps/hbase/data/archive/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2/cf/f009edcfec28494d906061c49278df80
  2348. 2014-06-06 18:15:26,013 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Deleted all region files in: hdfs://rtpha/apps/hbase/data/data/default/BT_D_BF001_201406/bf89ff438e6961193b0bb7b772e2fbd2
  2349. 2014-06-06 18:15:26,034 INFO [CatalogJanitor-dsjhd113:60000] catalog.MetaEditor: Deleted BT_D_BF001_201406,4600304159432142014060517055720500000000000000000808000000000000000005668700000006410174664,1402046055308.bf89ff438e6961193b0bb7b772e2fbd2.
  2350. 2014-06-06 18:15:26,039 DEBUG [CatalogJanitor-dsjhd113:60000] master.CatalogJanitor: Deleting region ERR_BT_D_BF001_201406,0000000000000002014060517561739900000000000000000800000000000000000004744400000005,1402048433136.5544bd37b36ea28e890ba839f4506af6. because daughter splits no longer hold references
  2351. 2014-06-06 18:15:26,040 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: ARCHIVING hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6
  2352. 2014-06-06 18:15:26,043 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Archiving [class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf]
  2353. 2014-06-06 18:15:26,056 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/0fdc490810b94a10ad117421d60cbf2d, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/0fdc490810b94a10ad117421d60cbf2d
  2354. 2014-06-06 18:15:26,060 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/6de1db61811b48809a733b21f032b6e4, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/6de1db61811b48809a733b21f032b6e4
  2355. 2014-06-06 18:15:26,064 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/7c1ec330954b49198304646fe07d41c7, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/7c1ec330954b49198304646fe07d41c7
  2356. 2014-06-06 18:15:26,068 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/96359a9a2d14442280a3365805a7ecee, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/96359a9a2d14442280a3365805a7ecee
  2357. 2014-06-06 18:15:26,073 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/9c63d6e7dae94effb97c30a77cfe7269, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/9c63d6e7dae94effb97c30a77cfe7269
  2358. 2014-06-06 18:15:26,077 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/a1fb57e26f9e47ee8f495a108ea5d75b, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/a1fb57e26f9e47ee8f495a108ea5d75b
  2359. 2014-06-06 18:15:26,081 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/a936551fe9264d10bedaa839a31be3de, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/a936551fe9264d10bedaa839a31be3de
  2360. 2014-06-06 18:15:26,085 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/ac4a1f7a101c4b389c4315f8765f53a9, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/ac4a1f7a101c4b389c4315f8765f53a9
  2361. 2014-06-06 18:15:26,089 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/af04730876cb4560ba214930431ff91f, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/af04730876cb4560ba214930431ff91f
  2362. 2014-06-06 18:15:26,093 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/dc65e7c7ef004bd980fa4d2f0da1b9e3, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/dc65e7c7ef004bd980fa4d2f0da1b9e3
  2363. 2014-06-06 18:15:26,097 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/ecda798867b24016807157f22732d59e, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/ecda798867b24016807157f22732d59e
  2364. 2014-06-06 18:15:26,102 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/eef82e61f779404eb5c42b0a18f8f54b, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/eef82e61f779404eb5c42b0a18f8f54b
  2365. 2014-06-06 18:15:26,106 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/fd44da5f4b3e400486481c8c85dffea9, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/fd44da5f4b3e400486481c8c85dffea9
  2366. 2014-06-06 18:15:26,110 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Finished archiving from class org.apache.hadoop.hbase.backup.HFileArchiver$FileablePath, file:hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/fe7d0d3a048848e684e4f09d605a939b, to hdfs://rtpha/apps/hbase/data/archive/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6/cf/fe7d0d3a048848e684e4f09d605a939b
  2367. 2014-06-06 18:15:26,112 DEBUG [CatalogJanitor-dsjhd113:60000] backup.HFileArchiver: Deleted all region files in: hdfs://rtpha/apps/hbase/data/data/default/ERR_BT_D_BF001_201406/5544bd37b36ea28e890ba839f4506af6
  2368. 2014-06-06 18:15:26,117 INFO [CatalogJanitor-dsjhd113:60000] catalog.MetaEditor: Deleted ERR_BT_D_BF001_201406,0000000000000002014060517561739900000000000000000800000000000000000004744400000005,1402048433136.5544bd37b36ea28e890ba839f4506af6.
  2369. 2014-06-06 18:15:26,117 INFO [CatalogJanitor-dsjhd113:60000] master.CatalogJanitor: Scanned 9778 catalog row(s), gc'd 0 unreferenced merged region(s) and 2 unreferenced parent region(s)
  2370. 2014-06-06 18:15:49,833 DEBUG [RpcServer.handler=57,port=60000] client.HConnectionManager$HConnectionImplementation: Removed all cached region locations that map to dsjhd061,60020,1402027921226
  2371. 2014-06-06 18:16:13,292 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048946361
  2372. 2014-06-06 18:16:13,296 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048950787
  2373. 2014-06-06 18:16:13,299 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048954665
  2374. 2014-06-06 18:16:13,302 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048959247
  2375. 2014-06-06 18:16:13,305 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048964174
  2376. 2014-06-06 18:16:13,308 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048968639
  2377. 2014-06-06 18:16:13,311 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048973028
  2378. 2014-06-06 18:16:13,314 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048977598
  2379. 2014-06-06 18:16:13,316 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048983229
  2380. 2014-06-06 18:16:13,319 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048987930
  2381. 2014-06-06 18:16:13,322 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402048997754
  2382. 2014-06-06 18:16:13,325 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049002092
  2383. 2014-06-06 18:16:13,327 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049007197
  2384. 2014-06-06 18:16:13,330 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049011687
  2385. 2014-06-06 18:16:13,332 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049016886
  2386. 2014-06-06 18:16:13,335 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049021827
  2387. 2014-06-06 18:16:13,338 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049026762
  2388. 2014-06-06 18:16:13,341 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049032132
  2389. 2014-06-06 18:16:13,344 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049037021
  2390. 2014-06-06 18:16:13,347 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048933338
  2391. 2014-06-06 18:16:13,350 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048935814
  2392. 2014-06-06 18:16:13,352 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048937911
  2393. 2014-06-06 18:16:13,355 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048940232
  2394. 2014-06-06 18:16:13,358 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048942766
  2395. 2014-06-06 18:16:13,360 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048944719
  2396. 2014-06-06 18:16:13,363 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048946386
  2397. 2014-06-06 18:16:13,367 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048948435
  2398. 2014-06-06 18:16:13,370 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048950236
  2399. 2014-06-06 18:16:13,373 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048952115
  2400. 2014-06-06 18:16:13,376 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048953784
  2401. 2014-06-06 18:16:13,378 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048955682
  2402. 2014-06-06 18:16:13,381 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048957714
  2403. 2014-06-06 18:16:13,384 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048959789
  2404. 2014-06-06 18:16:13,386 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048962263
  2405. 2014-06-06 18:16:13,389 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048964314
  2406. 2014-06-06 18:16:13,392 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048966244
  2407. 2014-06-06 18:16:13,394 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048968079
  2408. 2014-06-06 18:16:13,397 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048970142
  2409. 2014-06-06 18:16:13,400 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048972136
  2410. 2014-06-06 18:16:13,402 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048974098
  2411. 2014-06-06 18:16:13,404 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048976149
  2412. 2014-06-06 18:16:13,407 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048978357
  2413. 2014-06-06 18:16:13,410 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048980465
  2414. 2014-06-06 18:16:13,413 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048983378
  2415. 2014-06-06 18:16:13,415 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048985782
  2416. 2014-06-06 18:16:13,418 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048987914
  2417. 2014-06-06 18:16:13,420 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048992125
  2418. 2014-06-06 18:16:13,423 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048997124
  2419. 2014-06-06 18:16:50,911 DEBUG [RpcServer.handler=57,port=60000] client.HConnectionManager$HConnectionImplementation: Removed all cached region locations that map to dsjhd061,60020,1402027921226
  2420. 2014-06-06 18:17:13,287 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402044995434
  2421. 2014-06-06 18:17:13,290 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402045016308
  2422. 2014-06-06 18:17:13,293 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402045076854
  2423. 2014-06-06 18:17:13,296 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402045228388
  2424. 2014-06-06 18:17:13,299 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402045259832
  2425. 2014-06-06 18:17:13,302 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402045394327
  2426. 2014-06-06 18:17:13,305 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402045439203
  2427. 2014-06-06 18:17:13,308 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd057%2C60020%2C1402039842287.1402045556982
  2428. 2014-06-06 18:17:13,311 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd060%2C60020%2C1402027925238.1402045137219
  2429. 2014-06-06 18:17:13,313 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd060%2C60020%2C1402027925238.1402045230745
  2430. 2014-06-06 18:17:13,316 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd060%2C60020%2C1402027925238.1402045271441
  2431. 2014-06-06 18:17:13,343 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd060%2C60020%2C1402027925238.1402045395619
  2432. 2014-06-06 18:17:13,346 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd060%2C60020%2C1402027925238.1402045440327
  2433. 2014-06-06 18:17:13,349 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd060%2C60020%2C1402027925238.1402045549088
  2434. 2014-06-06 18:17:13,352 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd060%2C60020%2C1402027925238.1402045622002
  2435. 2014-06-06 18:17:13,355 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd060%2C60020%2C1402027925238.1402045718581
  2436. 2014-06-06 18:17:13,358 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd074%2C60020%2C1402039899044.1402044954679
  2437. 2014-06-06 18:17:13,361 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd074%2C60020%2C1402039899044.1402045036377
  2438. 2014-06-06 18:17:13,364 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd074%2C60020%2C1402039899044.1402045219697
  2439. 2014-06-06 18:17:13,367 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd074%2C60020%2C1402039899044.1402045245477
  2440. 2014-06-06 18:17:13,370 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd074%2C60020%2C1402039899044.1402045393308
  2441. 2014-06-06 18:17:13,373 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402048999129
  2442. 2014-06-06 18:17:13,376 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049001457
  2443. 2014-06-06 18:17:13,379 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049003628
  2444. 2014-06-06 18:17:13,381 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049005940
  2445. 2014-06-06 18:17:13,384 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049008308
  2446. 2014-06-06 18:17:13,387 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049010366
  2447. 2014-06-06 18:17:13,390 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049012487
  2448. 2014-06-06 18:17:13,393 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049014811
  2449. 2014-06-06 18:17:13,395 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049017100
  2450. 2014-06-06 18:17:13,398 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049019403
  2451. 2014-06-06 18:17:13,401 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049021840
  2452. 2014-06-06 18:17:13,404 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049024068
  2453. 2014-06-06 18:17:13,407 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049026394
  2454. 2014-06-06 18:17:13,410 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049028753
  2455. 2014-06-06 18:17:13,412 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049030958
  2456. 2014-06-06 18:17:13,415 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049033623
  2457. 2014-06-06 18:17:13,417 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049035965
  2458. 2014-06-06 18:17:13,420 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049038330
  2459. 2014-06-06 18:17:13,423 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049041236
  2460. 2014-06-06 18:17:13,426 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049043631
  2461. 2014-06-06 18:17:13,428 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049045793
  2462. 2014-06-06 18:17:13,431 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049047912
  2463. 2014-06-06 18:17:13,434 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049050293
  2464. 2014-06-06 18:17:13,436 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049052605
  2465. 2014-06-06 18:17:13,439 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049054781
  2466. 2014-06-06 18:17:13,442 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049057191
  2467. 2014-06-06 18:17:13,445 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049059395
  2468. 2014-06-06 18:17:13,447 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049061781
  2469. 2014-06-06 18:17:13,450 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049064109
  2470. 2014-06-06 18:17:13,453 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049066260
  2471. 2014-06-06 18:17:13,455 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049068577
  2472. 2014-06-06 18:17:13,459 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049070894
  2473. 2014-06-06 18:17:13,463 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049073286
  2474. 2014-06-06 18:17:13,466 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049075554
  2475. 2014-06-06 18:17:13,471 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049078156
  2476. 2014-06-06 18:17:13,474 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049080419
  2477. 2014-06-06 18:17:13,476 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049082884
  2478. 2014-06-06 18:17:13,479 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049085224
  2479. 2014-06-06 18:17:13,482 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049087507
  2480. 2014-06-06 18:17:13,484 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049089716
  2481. 2014-06-06 18:17:13,486 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049092246
  2482. 2014-06-06 18:17:13,488 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049094645
  2483. 2014-06-06 18:17:13,491 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049097072
  2484. 2014-06-06 18:17:13,494 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049099773
  2485. 2014-06-06 18:17:13,496 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049102351
  2486. 2014-06-06 18:17:13,499 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049104753
  2487. 2014-06-06 18:17:13,501 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049106711
  2488. 2014-06-06 18:17:13,504 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049108796
  2489. 2014-06-06 18:17:13,506 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049111658
  2490. 2014-06-06 18:17:13,509 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049113986
  2491. 2014-06-06 18:17:13,511 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049116420
  2492. 2014-06-06 18:17:13,514 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049118914
  2493. 2014-06-06 18:17:13,516 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049121116
  2494. 2014-06-06 18:17:13,519 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049123268
  2495. 2014-06-06 18:17:13,521 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049129255
  2496. 2014-06-06 18:17:13,524 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049131495
  2497. 2014-06-06 18:17:13,526 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049133745
  2498. 2014-06-06 18:17:13,529 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049135759
  2499. 2014-06-06 18:17:13,532 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049137916
  2500. 2014-06-06 18:17:13,535 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049140298
  2501. 2014-06-06 18:17:13,537 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049142508
  2502. 2014-06-06 18:17:13,540 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049144665
  2503. 2014-06-06 18:17:13,543 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd078%2C60020%2C1402027921335.1402045224394
  2504. 2014-06-06 18:17:13,545 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd078%2C60020%2C1402027921335.1402045247047
  2505. 2014-06-06 18:17:13,547 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd078%2C60020%2C1402027921335.1402045388045
  2506. 2014-06-06 18:17:13,550 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd078%2C60020%2C1402027921335.1402045420942
  2507. 2014-06-06 18:17:13,553 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd078%2C60020%2C1402027921335.1402045457402
  2508. 2014-06-06 18:18:01,014 DEBUG [RpcServer.handler=57,port=60000] client.HConnectionManager$HConnectionImplementation: Removed all cached region locations that map to dsjhd061,60020,1402027921226
  2509. 2014-06-06 18:18:13,286 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd055%2C60020%2C1402027951094.1402045177206
  2510. 2014-06-06 18:18:13,289 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd055%2C60020%2C1402027951094.1402045269050
  2511. 2014-06-06 18:18:13,292 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd055%2C60020%2C1402027951094.1402045421126
  2512. 2014-06-06 18:18:13,296 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd055%2C60020%2C1402027951094.1402045567664
  2513. 2014-06-06 18:18:13,299 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd055%2C60020%2C1402027951094.1402045678757
  2514. 2014-06-06 18:18:13,302 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049042715
  2515. 2014-06-06 18:18:13,305 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049047512
  2516. 2014-06-06 18:18:13,308 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049052445
  2517. 2014-06-06 18:18:13,311 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049057573
  2518. 2014-06-06 18:18:13,314 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049062520
  2519. 2014-06-06 18:18:13,316 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049067235
  2520. 2014-06-06 18:18:13,319 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049072382
  2521. 2014-06-06 18:18:13,322 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049077598
  2522. 2014-06-06 18:18:13,325 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049082878
  2523. 2014-06-06 18:18:13,328 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049087658
  2524. 2014-06-06 18:18:13,331 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049092838
  2525. 2014-06-06 18:18:13,334 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049098118
  2526. 2014-06-06 18:18:13,337 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049103936
  2527. 2014-06-06 18:18:13,339 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049108385
  2528. 2014-06-06 18:18:13,342 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049113782
  2529. 2014-06-06 18:18:13,345 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049119079
  2530. 2014-06-06 18:18:13,349 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049127594
  2531. 2014-06-06 18:18:13,352 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049132910
  2532. 2014-06-06 18:18:13,355 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049137472
  2533. 2014-06-06 18:18:13,358 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049142244
  2534. 2014-06-06 18:18:13,361 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049147238
  2535. 2014-06-06 18:18:13,364 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049147110
  2536. 2014-06-06 18:18:13,367 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049149138
  2537. 2014-06-06 18:18:13,371 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049151344
  2538. 2014-06-06 18:18:13,376 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049153622
  2539. 2014-06-06 18:18:13,379 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049155987
  2540. 2014-06-06 18:18:13,384 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049158145
  2541. 2014-06-06 18:18:13,389 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049160435
  2542. 2014-06-06 18:18:13,392 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049162560
  2543. 2014-06-06 18:18:13,395 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049164362
  2544. 2014-06-06 18:18:13,397 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049166423
  2545. 2014-06-06 18:18:13,400 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049168796
  2546. 2014-06-06 18:18:13,403 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049170833
  2547. 2014-06-06 18:19:11,170 DEBUG [RpcServer.handler=57,port=60000] client.HConnectionManager$HConnectionImplementation: Removed all cached region locations that map to dsjhd061,60020,1402027921226
  2548. 2014-06-06 18:19:13,291 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd054%2C60020%2C1402027922467.1402045803789
  2549. 2014-06-06 18:19:13,295 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049151909
  2550. 2014-06-06 18:19:13,298 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049156816
  2551. 2014-06-06 18:19:13,302 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd064%2C60020%2C1402027918813.1402049161745
  2552. 2014-06-06 18:19:13,305 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd066%2C60020%2C1402027920604.1402046117563
  2553. 2014-06-06 18:19:13,308 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd075%2C60020%2C1402027953331.1402045229582
  2554. 2014-06-06 18:19:13,311 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd075%2C60020%2C1402027953331.1402045268073
  2555. 2014-06-06 18:19:13,315 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd075%2C60020%2C1402027953331.1402045394363
  2556. 2014-06-06 18:19:13,317 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd075%2C60020%2C1402027953331.1402045440203
  2557. 2014-06-06 18:19:13,320 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd075%2C60020%2C1402027953331.1402045559507
  2558. 2014-06-06 18:19:13,323 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd075%2C60020%2C1402027953331.1402045624300
  2559. 2014-06-06 18:19:13,326 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049173142
  2560. 2014-06-06 18:19:13,329 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049175412
  2561. 2014-06-06 18:19:13,332 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049177798
  2562. 2014-06-06 18:19:13,335 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049180160
  2563. 2014-06-06 18:19:13,337 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049182815
  2564. 2014-06-06 18:19:13,340 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049185115
  2565. 2014-06-06 18:19:13,343 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049187804
  2566. 2014-06-06 18:19:13,346 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049190153
  2567. 2014-06-06 18:19:13,349 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049192222
  2568. 2014-06-06 18:19:13,352 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049194249
  2569. 2014-06-06 18:19:13,354 DEBUG [master:dsjhd113:60000.oldLogCleaner] master.ReplicationLogCleaner: Didn't find this log in ZK, deleting: dsjhd076%2C60020%2C1402027922130.1402049196275