假如你要往swift集群里增加3T硬盘(标识d16),存储节点IP是192.168.12.104.为了让swift使用它,需要把它加到rings里,例如:
$ swift-ring-builder account.builder add z1-192.168.12.104:6002/d16 3000 $ swift-ring-builder container.builder add z1-192.168.12.104:6001/d16 3000 $ swift-ring-builder object.builder add z1-192.168.12.104:6000/d16 3000
我们使用3000作为权重,它表示3000GB(该磁盘的容量),这种权重设置方法值得借鉴。
然后,对rings进行rebalance,并且拷贝到所有存储节点:
$ swift-ring-builder account.builder rebalance $ swift-ring-builder container.builder rebalance $ swift-ring-builder object.builder rebalance $ scp account.ring.gz swift-node-1:/etc/swift/account.ring.gz $ scp container.ring.gz swift-node-1:/etc/swift/container.ring.gz $ scp object.ring.gz swift-node-1:/etc/swift/account.ring.gz $ scp account.ring.gz swift-node-2:/etc/swift/account.ring.gz $ scp container.ring.gz swift-node-2:/etc/swift/container.ring.gz $ scp object.ring.gz swift-node-2:/etc/swift/account.ring.gz ... $ scp account.ring.gz swift-node-10:/etc/swift/account.ring.gz $ scp container.ring.gz swift-node-10:/etc/swift/container.ring.gz $ scp object.ring.gz swift-node-10:/etc/swift/account.ring.gz
swift会适应rings改变的结果,每个人都很开心,是这样吗?
错了,此时swift集群性能会大大降低。一次性增加3T硬盘,会导致整个磁盘容量的数据在集群里转移,这个过程中性能降低。rebalancing过程如下:每个磁盘分配均衡的数据,假如整个集群的容量50%满了,这时有1.5TB的数据需要转移到新的磁盘上。在最理想的情况下,一个10GB的以太网也要花费20分钟时间去完成这个工作。在实际中,你往往发现数小时内性能都是降低的。
正确的处理方法是逐步转移数据,而不是一次性转移。例如,可以这样做:
$ swift-ring-builder account.builder add z1-192.168.12.104:6002/d16 25 $ swift-ring-builder container.builder add z1-192.168.12.104:6001/d16 25 $ swift-ring-builder object.builder add z1-192.168.12.104:6000/d16 25
现在使用25作为权重,所以只会有一小部分数据转移到新磁盘上。
跟往常一样,rebalance rings并且拷贝它们到存储节点上:
$ swift-ring-builder account.builder rebalance $ swift-ring-builder container.builder rebalance $ swift-ring-builder object.builder rebalance $ scp account.ring.gz swift-node-1:/etc/swift/account.ring.gz $ scp container.ring.gz swift-node-1:/etc/swift/container.ring.gz $ scp object.ring.gz swift-node-1:/etc/swift/account.ring.gz $ scp account.ring.gz swift-node-2:/etc/swift/account.ring.gz $ scp container.ring.gz swift-node-2:/etc/swift/container.ring.gz $ scp object.ring.gz swift-node-2:/etc/swift/account.ring.gz ...
过一会儿(例如一个小时)等事情平静下来,增加再多一点权重:
$ swift-ring-builder account.builder set_weight z1-192.168.12.104:6002/d16 50 $ swift-ring-builder container.builder set_weight z1-192.168.12.104:6001/d16 50 $ swift-ring-builder object.builder set_weight z1-192.168.12.104:6000/d16 50 $ swift-ring-builder account.builder rebalance $ swift-ring-builder container.builder rebalance $ swift-ring-builder object.builder rebalance $ scp account.ring.gz ...
然后等一个小时,再继续这个操作,如此循环往复。120小时过后,你的新磁盘就变成swift集群的正式成员了。
从集群里删除磁盘也是这个过程,避免一次性完全移除,而是通过逐步降低权重的方法,慢慢移除,避免数据震荡而影响整个集群的性能。
原文见:http://blog.swiftstack.com/blog/2012/04/09/swift-capacity-management/