Ceph性能测试二

在上一篇blog里，简单测试了下本地硬盘和分布式块存储的性能，从数据表现看分布式存储的IO吞吐能力大概是本地硬盘的1/7。正好邮件列表上有个同学也做了测试，内容如下：

I was doing some tests in my cluster with fio tool, one fio instance
with 70 jobs, each job writing 1GB random with 4K block size. I did this
test with 3 variations:

1- Creating 70 images, 60GB each, in the pool. Using rbd kernel module,
format and mount each image as ext4. Each fio job writing in a separate
image/directory. (ioengine=libaio, queue_depth=4, direct=1)

IOPS: 6542
AVG LAT: 41ms

2- Creating 1 large image 4,2TB in the pool. Using rbd kernel module,
format and mount the image as ext4. Each fio job writing in a separate
file in the same directory. (ioengine=libaio, queue_depth=4,direct=1)

IOPS: 5899
AVG LAT: 47ms

3- Creating 1 large image 4,2TB in the pool. Use ioengine rbd in fio to
access the image through librados. (ioengine=rbd, queue_depth=4,direct=1)

IOPS: 2638
AVG LAT: 96ms

且不管他的第3种情况，看前2种情况。第1种是fio对多个小镜像文件进行测试，IOPS是6542。第2种情况是fio对一个大镜像文件里的多个独立小文件进行测试，IOPS是5899。这个IOPS看起来还行，但是换算成吞吐量（throughput）也很一般。

他每次写入的block size是4K，乘以IOPS就是吞吐量：

6542*4/1024 = 25MB/s

5899*4/1024 = 23MB/s

从这个情况看也很一般。服务器的本地磁盘（SAS/SATA）可以轻易达到100MB+的吞吐量。若在生产环境使用分布式块存储，性能方面还是有压力。