文章目录
- 概念介绍
- layout
- file stripe
- 实例
- 参考链接
概念介绍
layout
cephfs file layout描述了一个file对象是怎么存储到rados中的,存储在文件的xattr中。文件xattr为ceph.file.layout,目录xattr为ceph.dir.layout
下面是一个layout示例:
ceph.file.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data"
可以看到主要有几个字段:
- stripe_unit:条带大小,单位bytes,条带划分的基本单位,也是一个文件对象逻辑的最细粒度切分
- stripe_count:连续几个stripe unit组成一个完整的stripe
- object_size:rados对象大小
- pool:顾名思义,表示文件存储在哪个存储池
file stripe
根据layout对一个文件进行逻辑切分
实例
下面通过官网上的一个例子来说明条带化是如何切分的:
- file size = 1 trillion = 1000000000000 bytes
- stripe_unit = 64KB = 65536 bytes
- stripe_count = 5 stripe units per stripe
- object_size = 64GB = 68719476736 bytes
- 每个object大小为64G,5个object为一组,则每个object set size大小为64*5=320G;所以1 trillion bytes文件会分为1000000000000/320G(343597383680 bytes)=2余312805232640,所以该文件被分割为2个完整的object set(320G),再加一个不完整的object set(312805232640 bytes)。
_________ _________ _________ _________ _________
/object 0\ /object 1\ /object 2\ /object 3\ /object 4\
+=========+ +=========+ +=========+ +=========+ +=========+
object set 0:complete object set
\=========/ \=========/ \=========/ \=========/ \=========/
_________ _________ _________ _________ _________
/object 5\ /object 6\ /object 7\ /object 8\ /object 9\
+=========+ +=========+ +=========+ +=========+ +=========+
object set 1:complete object set
\=========/ \=========/ \=========/ \=========/ \=========/
_________ _________ _________ _________ _________
/object 10\ /object 11\ /object 12\ /object 13\ /object 14\
+=========+ +=========+ +=========+ +=========+ +=========+
object set 2:incomplete object set
\=========/ \=========/ \=========/ \=========/ \=========/
- 一个stripe的大小是stripe unit*strip counts,本例中即64KB*5=320KB,那显然每个complete object set包含320G/320KB=1048576个stripe,而最后一个incomplete object set则包含312805232640 bytes/320KB=954605个stripe,且剩余312805232640 bytes%320KB=266240 bytes,所以三个object set的stripe范围分别是0~1048575,1048576~2097151,2097152~3051757
_________ _________ _________ _________ _________
/object 0\ /object 1\ /object 2\ /object 3\ /object 4\
+=========+ +=========+ +=========+ +=========+ +=========+
o stripe 0(complete)
b
j |---------| |---------| |---------| |---------| |---------|
e
c stripe 1(complete)
t
|---------| |---------| |---------| |---------| |---------|
s | . | | . | | . | | . | | . |
e . . . . .
t | . | | . | | . | | . | | . |
|---------| |---------| |---------| |---------| |---------|
0
stripe 1048575(complete)
\=========/ \=========/ \=========/ \=========/ \=========/
_________ _________ _________ _________ _________
/object 5\ /object 6\ /object 7\ /object 8\ /object 9\
+=========+ +=========+ +=========+ +=========+ +=========+
o stripe 1048576(complete)
b
j |---------| |---------| |---------| |---------| |---------|
e
c stripe 1048577(complete)
t
|---------| |---------| |---------| |---------| |---------|
s | . | | . | | . | | . | | . |
e . . . . .
t | . | | . | | . | | . | | . |
|---------| |---------| |---------| |---------| |---------|
1
stripe 2097151(complete)
\=========/ \=========/ \=========/ \=========/ \=========/
_________ _________ _________ _________ _________
/object 10\ /object 11\ /object 12\ /object 13\ /object 14\
+=========+ +=========+ +=========+ +=========+ +=========+
o stripe 2097152(complete)
b
j |---------| |---------| |---------| |---------| |---------|
e
c stripe 2097153(complete)
t
|---------| |---------| |---------| |---------| |---------|
s | . | | . | | . | | . | | . |
e . . . . .
t | . | | . | | . | | . | | . |
|---------| |---------| |---------| |---------| |---------|
2
stripe 3051756(complete)
|---------| |---------| |---------| |---------| |---------|
stripe 3051757(incomplete)
\=========/ \=========/ \=========/ \=========/ \=========/
- 最后是stripe unit填充,stripe_count=5,所有complete stripe都是有5个stripe unit,所以complete stripe共有3051757*5=15258785个stripe unit,每次按5个填充,编号为0~15258784;所以最后一个incomplete stripe的stripe unit则是从15258785开始,而其大小则为266240bytes,那么显然共有266240/64KB=4个stripe unit,并余有266240%64KB=4096bytes大小。至此全部划分已明了,最后一个incomplet stripe的大小是4096bytes。
所以这个1 trillion bytes大小的文件的完整stripe划分如下所示:
_________ _________ _________ _________ _________
/object 0\ /object 1\ /object 2\ /object 3\ /object 4\
+=========+ +=========+ +=========+ +=========+ +=========+
| stripe | | stripe | | stripe | | stripe | | stripe |
o | unit | | unit | | unit | | unit | | unit | stripe 0
b | 0 | | 1 | | 2 | | 3 | | 4 |
j |---------| |---------| |---------| |---------| |---------|
e | stripe | | stripe | | stripe | | stripe | | stripe |
c | unit | | unit | | unit | | unit | | unit | stripe 1
t | 5 | | 6 | | 7 | | 8 | | 9 |
|---------| |---------| |---------| |---------| |---------|
s | . | | . | | . | | . | | . |
e . . . . .
t | . | | . | | . | | . | | . |
|---------| |---------| |---------| |---------| |---------|
0 | stripe | | stripe | | stripe | | stripe | | stripe | stripe
| unit | | unit | | unit | | unit | | unit | 1048575
| 5242875 | | 5242876 | | 5242877 | | 5242878 | | 5242879 |
\=========/ \=========/ \=========/ \=========/ \=========/
_________ _________ _________ _________ _________
/object 5\ /object 6\ /object 7\ /object 8\ /object 9\
+=========+ +=========+ +=========+ +=========+ +=========+
| stripe | | stripe | | stripe | | stripe | | stripe | stripe
o | unit | | unit | | unit | | unit | | unit | 1048576
b | 5242880 | | 5242881 | | 5242882 | | 5242883 | | 5242884 |
j |---------| |---------| |---------| |---------| |---------|
e | stripe | | stripe | | stripe | | stripe | | stripe | stripe
c | unit | | unit | | unit | | unit | | unit | 1048577
t | 5242885 | | 5242886 | | 5242887 | | 5242888 | | 5242889 |
|---------| |---------| |---------| |---------| |---------|
s | . | | . | | . | | . | | . |
e . . . . .
t | . | | . | | . | | . | | . |
|---------| |---------| |---------| |---------| |---------|
1 | stripe | | stripe | | stripe | | stripe | | stripe | stripe
| unit | | unit | | unit | | unit | | unit | 2097151
| 10485755| | 10485756| | 10485757| | 10485758| | 10485759|
\=========/ \=========/ \=========/ \=========/ \=========/
_________ _________ _________ _________ _________
/object 10\ /object 11\ /object 12\ /object 13\ /object 14\
+=========+ +=========+ +=========+ +=========+ +=========+
| stripe | | stripe | | stripe | | stripe | | stripe | stripe
o | unit | | unit | | unit | | unit | | unit | 2097152
b | 10485760| | 10485761| | 10485762| | 10485763| | 10485764|
j |---------| |---------| |---------| |---------| |---------|
e | stripe | | stripe | | stripe | | stripe | | stripe | stripe
c | unit | | unit | | unit | | unit | | unit | 2097153
t | 10485765| | 10485766| | 10485767| | 10485768| | 10485769|
|---------| |---------| |---------| |---------| |---------|
s | . | | . | | . | | . | | . |
e . . . . .
t | . | | . | | . | | . | | . |
|---------| |---------| |---------| |---------| |---------|
2 | stripe | | stripe | | stripe | | stripe | | stripe | stripe
| unit | | unit | | unit | | unit | | unit | 3051756
| 15258780| | 15258781| | 15258782| | 15258783| | 15258784|
|---------| |---------| |---------| |---------| |---------|
| stripe | | stripe | | stripe | | stripe | | (partial| (partial
| unit | | unit | | unit | | unit | | stripe | stripe
| 15258785| | 15258786| | 15258787| | 15258788| | unit) | 3051757)
\=========/ \=========/ \=========/ \=========/ \=========/
参考链接
[1] cephfs file layout
[2] cephfs file stripe
更多推荐
cephfs file stripe简介
发布评论