2. Load Balancer Outline in
HBase
1. Bulk assignment
Load 2. Inline region placement
Balancer
3. Cluster-wide load balancing
3. Load Balancer Outline in
HBase
dedicated thread for the load balancing
› Step 1 : Thread sleep during period at which the
region balancer runs in the HMaster
› Step 2 : Generate a global load balancing plan
› Step 3 : Assignment Manager class executes the
plan
4. Step 1
package org.apache.hadoop.hbase.master.HMaster
private static Thread getAndStartBalancerChore(final HMaster master) {
String name = master.getServerName() + "-BalancerChore";
int period = master.getConfiguration().getInt("hbase.balancer.period", 300000);
// Start up the load balancer chore
Chore chore = new Chore(name, period, master) {
@Override
protected void chore() {
master.balance(); hbase-site.xml
} hbase.balancer.period
};
return Threads.setDaemonThreadRunning(chore);
} Period at which the region
balancer runs in the Master.
5. Load Balancer Outline in
HBase
dedicated thread for the load balancing
› Step 1 : Thread sleep during period at which the
region balancer runs in the HMaster
› Step 2 : Generate a global load balancing plan
› Step 3 : Assignment Manager class executes the
plan
6. Step 2
@Override
public boolean balance() {
if (!this.balanceSwitch) return false;
synchronized (this.balancer) {
List<RegionPlan> plans = this.balancer.balanceCluster(assignments);
if (plans != null && !plans.isEmpty()) {
for (RegionPlan plan: plans) {
LOG.info("balance " + plan);
this.assignmentManager.balance(plan);
}
return true;
}
8. this chart does not
Step 2 : Planning flow cover all steps.
needed to fill all
Region Server : 5 to min and none NO
Balancer Region Num : 16 left to drain all
Start MIN = 3 to MAX
MAX = 4
return
Get Server number YES Region
we need Plan
NO more to fill NO
numServers !
=0? MIN ?
YES YES
MIN = region / region server
grab one from each
MAX = region mod region server == 0 ? return null most loaded until
min : min + 1 enough
Walk down least
server NO loaded, filling each
outside to the MIN
range ?
YES we still have NO
Walk down most return null regions to dish
loaded, pruning out? return
each to the MAX Region
YES Plan
Walk down least
return
loaded, assigning to assign underloaded to
Region
each to fill up to MIN MAX
Plan
9. this chart does not
Step 2 : Planning flow cover all steps.
needed to fill all
Region Server : 5 to min and none NO
Balancer Region Num : 16 left to drain all
Start MIN = 3 to MAX
MAX = 4 1 2 3 4 5
return
Get Server number 5 5 5 1 0
YES Region
we need Plan
NO more to fill NO
numServers !
=0? MIN ?
YES YES
MIN = region / region server
grab one from each
MAX = region mod region server == 0 ? return null most loaded until
min : min + 1 enough
Walk down least
server NO loaded, filling each
outside to the MIN
range ?
YES we still have NO
Walk down most return null regions to dish
loaded, pruning out? return
each to the MAX Region
YES Plan
Walk down least
return
loaded, assigning to assign underloaded to
Region
each to fill up to MIN MAX
Plan
10. this chart does not
Step 2 : Planning flow cover all steps.
needed to fill all
Region Server : 5 to min and none NO
Balancer Region Num : 16 left to drain all
Start MIN = 3 to MAX
MAX = 4 1 2 3 4 5
return
Get Server number 4 4 4 1 0
YES Region
we need Plan
NO more to fill NO
numServers !
=0? MIN ?
YES YES
MIN = region / region server
grab one from each
MAX = region mod region server == 0 ? return null most loaded until
min : min + 1 enough
Walk down least
server NO loaded, filling each
outside to the MIN
range ?
YES we still have NO
Walk down most return null regions to dish
loaded, pruning out? return
each to the MAX Region
YES Plan
Walk down least
return
loaded, assigning to assign underloaded to
Region
each to fill up to MIN MAX
Plan
11. this chart does not
Step 2 : Planning flow cover all steps.
needed to fill all
Region Server : 5 to min and none NO
Balancer Region Num : 16 left to drain all
Start MIN = 3 to MAX
MAX = 4 1 2 3 4 5
return
Get Server number 4 4 4 3 1 YES Region
we need Plan
NO more to fill NO
numServers !
=0? MIN ?
YES YES
MIN = region / region server
grab one from each
MAX = region mod region server == 0 ? return null most loaded until
min : min + 1 enough
Walk down least
server NO loaded, filling each
outside to the MIN
range ?
YES we still have NO
Walk down most return null regions to dish
loaded, pruning out? return
each to the MAX Region
YES Plan
Walk down least
return
loaded, assigning to assign underloaded to
Region
each to fill up to MIN MAX
Plan
12. this chart does not
Step 2 : Planning flow cover all steps.
needed to fill all
Region Server : 5 to min and none NO
Balancer Region Num : 16 left to drain all
Start MIN = 3 to MAX
MAX = 4 1 2 3 4 5
return
Get Server number 3 3 3 3 1
YES Region
we need Plan
NO more to fill NO
numServers !
=0? MIN ?
YES YES
MIN = region / region server
grab one from each
MAX = region mod region server == 0 ? return null most loaded until
min : min + 1 enough
Walk down least
server NO loaded, filling each
outside to the MIN
range ?
YES we still have NO
Walk down most return null regions to dish
loaded, pruning out? return
each to the MAX Region
YES Plan
Walk down least
return
loaded, assigning to assign underloaded to
Region
each to fill up to MIN MAX
Plan
13. this chart does not
Step 2 : Planning flow cover all steps.
needed to fill all
Region Server : 5 to min and none NO
Balancer Region Num : 16 left to drain all
Start MIN = 3 to MAX
MAX = 4 1 2 3 4 5
return
Get Server number 3 3 3 3 3
YES Region
we need Plan
NO more to fill NO
numServers !
=0? MIN ?
YES YES
MIN = region / region server
grab one from each
MAX = region mod region server == 0 ? return null most loaded until
min : min + 1 enough
Walk down least
server NO loaded, filling each
outside to the MIN
range ?
YES we still have NO
Walk down most return null regions to dish
loaded, pruning out? return
each to the MAX Region
YES Plan
Walk down least
return
loaded, assigning to assign underloaded to
Region
each to fill up to MIN MAX
Plan
14. this chart does not
Step 2 : Planning flow cover all steps.
needed to fill all
Region Server : 5 to min and none NO
Balancer Region Num : 16 left to drain all
Start MIN = 3 to MAX
MAX = 4 1 2 3 4 5
return
Get Server number 4 3 3 3 3
YES Region
we need Plan
NO more to fill NO
numServers !
=0? MIN ?
YES YES
MIN = region / region server
grab one from each
MAX = region mod region server == 0 ? return null most loaded until
min : min + 1 enough
Walk down least
server NO loaded, filling each
outside to the MIN
range ?
YES we still have NO
Walk down most return null regions to dish
loaded, pruning out? return
each to the MAX Region
YES Plan
Walk down least
return
loaded, assigning to assign underloaded to
Region
each to fill up to MIN MAX
Plan
15. Load Balancer Outline in
HBase
dedicated thread for the load balancing
› Step 1 : Thread sleep during period at which the
region balancer runs in the HMaster
› Step 2 : Generate a global load balancing plan
› Step 3 : Assignment Manager class executes the
plan
