问题: 使用“cinder show”查询一个不存在的volumeid,使用时间很长。
[root@node1 ~]# time cinder show aksjdfkajsdkfjsakdf
ERROR: No volume with a name or ID of 'aksjdfkajsdkfjsakdf' exists.
real 0m35.374s
user 0m0.855s
sys 0m0.107s
分析:
1. cinderclient.v2.shell.do_show说明
指令“cinder show”的入口函数是cinderclient.v2.shell.do_show
:
@utils.arg('volume',
metavar='<volume>',
help='Name or ID of volume.')
def do_show(cs, args):
"""Shows volume details."""
info = dict()
volume = utils.find_volume(cs, args.volume)
info.update(volume._info)
if 'readonly' in info['metadata']:
info['readonly'] = info['metadata']['readonly']
info.pop('links', None)
# 打印字典
utils.print_dict(info,
formatters=['metadata', 'volume_image_metadata',
'attachments'])
@utils.arg
是用于定义参数的标签。
指令参数根据必填、选填分为两种:
- 1)
@utils.arg('volume', metavar='<volume>', help='Name or ID of volume.')
必填参数,使用时不用带参数名。例如”cinder show 123123”,这里的123123就是<volume>。 - 2)
@utils.arg('--tenant', type=str, dest='tenant', nargs='?', metavar='<tenant>', help='Display information from single tenant (Admin only).')
选填参数,名字多了个破折号 --,指令使用时必须带参数名。例如”cinder list --tenant 123123”
。
根据参数值分类:
- 1)@utils.arg('volume', metavar='<volume>', nargs='+', help='Name or ID of volume or volumes to delete.')
如果设置nargs='+',代表值为数组型,以空格分隔,输入1个以上。如”cinder delete 111 2222 3333”,删除volume_id=111、volume_id=222、volume_id=333 的volume。
此外还有nargs='*' 表示参数可设置零个或多个;nargs=' '+' 表示参数可设置一个或多个;nargs='?' 表示参数可设置零个或一个。 - 2)@utils.arg('--metadata', type=str, nargs='*', metavar='<key=value>', default=None, help='Snapshot metadata key and value pairs. Default=None.')
设置metavar='<key=value>',值为键值对形式。比如”cinder storage-create --metadata storage_protocol='fc' storage_ip='172.24.1.21' ”
@utils.arg
定义的参数会传入def do_show(cs, args)
的args,通过args.xxxx
来调用。
2.utils.find_volume(cs, args.volume) 说明
cinderclient.utils.find_volume:
def find_volume(cs, volume):
"""Get a volume by name or ID."""
return find_resource(cs.volumes, volume)
def find_resource(manager, name_or_id):
"""Helper for the _find_* methods."""
# first try to get entity as integer id
try:
if isinstance(name_or_id, int) or name_or_id.isdigit():
return manager.get(int(name_or_id))
except exceptions.NotFound:
pass
else:
# now try to get entity as uuid
try:
uuid.UUID(name_or_id)
return manager.get(name_or_id)
except (ValueError, exceptions.NotFound):
pass
if sys.version_info <= (3, 0):
name_or_id = encodeutils.safe_decode(name_or_id)
try:
try:
resource = getattr(manager, 'resource_class', None)
name_attr = resource.NAME_ATTR if resource else 'name'
return manager.find(**{name_attr: name_or_id})
except exceptions.NotFound:
pass
# finally try to find entity by human_id
try:
return manager.find(human_id=name_or_id)
except exceptions.NotFound:
msg = "No %s with a name or ID of '%s' exists." % \
(manager.resource_class.__name__.lower(), name_or_id)
raise exceptions.CommandError(msg)
except exceptions.NoUniqueMatch:
msg = ("Multiple %s matches found for '%s', use an ID to be more"
" specific." % (manager.resource_class.__name__.lower(),
name_or_id))
raise exceptions.CommandError(msg)
调用了cinderclient.utils.find_resource(manager, name_or_id)
方法,find_resource
的逻辑是:
① 如果id是int型,直接调用VolumeManager.get。如果没查到,走下一步。
② id转换成uuid,直接调用VolumeManager.get。如果没查到,走下一步。
③ 如果volumes类定义了NAME_ATTR属性,用{name_attr: name_or_id}作为条件调用VolumeManager.find查询。如果没查到,走下一步。
④ 用{human_id:name_or_id}作为条件调用VolumeManager.find查询。
3.VolumeManager.get说明
cinderclient.v2.volumes.VolumeManager#get
def get(self, volume_id):
"""Get a volume.
:param volume_id: The ID of the volume to get.
:rtype: :class:`Volume`
"""
return self._get("/volumes/%s" % volume_id, "volume")
self._get调用cinderclient.base.Manager#_get
def _get(self, url, response_key=None):
resp, body = self.api.client.get(url)
if response_key:
return self.resource_class(self, body[response_key], loaded=True,
resp=resp)
else:
return self.resource_class(self, body, loaded=True, resp=resp)
self.resource_class在cinderclient.v2.volumes.VolumeManager
定义:resource_class = Volume
。所以返回值是body[response_key]转换出的cinderclient.v2.volumes.Volume(base.Resource)
对象。
4.VolumeManager.find说明
VolumeManager.find
调用cinderclient.base.ManagerWithFind#findall(self, **kwargs):
def find(self, **kwargs):
"""
Find a single item with attributes matching ``**kwargs``.
This isn't very efficient for search options which require the
Python side filtering(e.g. 'human_id')
"""
matches = self.findall(**kwargs)
num_matches = len(matches)
if num_matches == 0:
msg = "No %s matching %s." % (self.resource_class.__name__, kwargs)
raise exceptions.NotFound(404, msg)
elif num_matches > 1:
raise exceptions.NoUniqueMatch
else:
matches[0].append_request_ids(matches.request_ids)
return matches[0]
def findall(self, **kwargs):
"""
Find all items with attributes matching ``**kwargs``.
This isn't very efficient for search options which require the
Python side filtering(e.g. 'human_id')
"""
# Want to search for all tenants here so that when attempting to delete
# that a user like admin doesn't get a failure when trying to delete
# another tenant's volume by name.
search_opts = {'all_tenants': 1}
# Pass 'name' or 'display_name' search_opts to server filtering to
# increase search performance.
if 'name' in kwargs:
search_opts['name'] = kwargs['name']
elif 'display_name' in kwargs:
search_opts['display_name'] = kwargs['display_name']
found = common_base.ListWithMeta([], None)
searches = kwargs.items()
listing = self.list(search_opts=search_opts)
found.append_request_ids(listing.request_ids)
# Not all resources attributes support filters on server side
# (e.g. 'human_id' doesn't), so when doing findall some client
# side filtering is still needed.
for obj in listing:
try:
if all(getattr(obj, attr) == value
for (attr, value) in searches):
found.append(obj)
except AttributeError:
continue
return found
findall的逻辑是:
① 检查kwargs字典key有无'name',有则加入查询条件search_opts。
② 检查kwargs字典key有无'display_name',有则加入查询条件search_opts。
③ self.list(search_opts=search_opts) 用查询条件查询VolumeManager.list。如果前两步都为否,则查询条件为空,那么list方法将查询所有卷信息返回。cinder的db api有配置单次查询最大条数是1000(配置项osapi_max_limit,默认1000),而我们的卷目前有8k多条,其中2528条可用的,所以得查三次才能查出所有卷记录。这占用时间很长。
④ 对第三步查出的卷列表做遍历,用kwargs字典的key和value检查,符合getattr(obj, attr) == value的记录添加进found数组返回。
5.human_id说明
上文介绍cinderclient.utils.find_volume方法里用{human_id:name_or_id}作为条件调用VolumeManager.find查询。human_id是什么?我们可以看到cinderclient.v3.volumes.Volume继承自cinderclient.apiclient.base.Resource
,而Resource类里定义了HUMAN_ID = False
和NAME_ATTR = 'name'
,Resource类里还有一个human_id(self)
属性方法。
cinderclient.apiclient.base.Resource
:
class Resource(RequestIdMixin):
"""Base class for OpenStack resources (tenant, user, etc.).
This is pretty much just a bag for attributes.
"""
HUMAN_ID = False
NAME_ATTR = 'name'
@property
def human_id(self):
"""Human-readable ID which can be used for bash completion.
"""
if self.NAME_ATTR in self.__dict__ and self.HUMAN_ID:
return strutils.to_slug(getattr(self, self.NAME_ATTR))
return None
根据human_id(self)属性方法的逻辑,我们可以这么理解:如果在cinderclient.v3.volumes.Volume
类里有定义HUMAN_ID = True
,且定义了类的名字属性NAME_ATTR ,那么volume.human_id=strutils.to_slug(getattr(self, self.NAME_ATTR))
。strutils.to_slug
这个方法的作用是把参数转换成小写、移除non-word的字符(比如中文)、空格转连字符(-)、删除前后端空格。
那么现在我们基本明白human_id的作用了:
human_id是用于查询字符格式不标准的NAME_ATTR 的记录。db里可能存的NAME_ATTR包含了非不正常的字符,比如中文啊、多出的空格啊,cinderclient指令无法传递正确编码的参数到服务端查询。所以通过volume.human_id=strutils.to_slug(getattr(self, self.NAME_ATTR))方法设置human_id的值为NAME_ATTR(即name)转换格式后的值,这样对human_id来查询,就能查到了。比如下面的例子:
数据库里有个volume的display_name= miaomiao喵喵猫,
但是在linux显示乱码:
如果直接使用” cinder show miaomiao喵喵猫 ” 来查询,会提示错误 “ERROR: 'utf8' codec can't decode byte 0xdf in position 8: invalid continuation byte”,无法使用name来查询:
但只要我们在cinderclient.v3.volumes.Volume类里设置HUMAN_ID=True,strutils.to_slug(u'miaomiao喵喵猫') == 'miaomiao'
,即可使用`”cinder show miaomiao”查到结果。
但是我们看cinderclient.v3.volumes.Volume
没有复写HUMAN_ID,根据属性方法 human_id(self)
的逻辑,如果self.HUMAN_ID==False
,返回None作为volume的human_id属性值。所以,如果用{human_id:name_or_id}作为条件调用VolumeManager.find查询,getattr(obj, attr) == value是一定为False,find结果一定为空。
既然cinderclient.v3.volumes.Volume的human_id=None,为什么utils.find_resource(manager, name_or_id, **kwargs)
最后一步还要用human_id查询呢?用human_id查询是不是有什么特别的用途?
我用HUMAN_ID搜索cinderclient整个项目,发现只有cinderclient.apiclient.base.Resource里有用到,其它Resource的子类都没有复写。
用human_id搜索cinderclient整个项目,也没用对human_id特别赋值的地方。
这样看来,HUMAN_ID在cinderclient整个项目里的调用值都为False,human_id将始终未None,用{human_id:name_or_id}作为条件调用VolumeManager.find查询在当前的cinderclient里实在毫无意义。
解决方案:
可以考虑把{human_id:name_or_id}查询屏蔽。