OpenStack社区一直在尝试用Container技术来部署其各个组件,由此还发展出了kolla这个big tent项目,以及衍生项目kolla-ansible。
最近项目上有些调整,有了些自由时间,于是开始尝试使用container方式部署OpenStack的组件,自然是从最熟悉的Cinder开始。
惊喜的发现Cinder项目自身就有一个简单易上手的脚步,可以帮助理解以container部署Cinder的基本步骤和方式。
[请在ROOT用户下执行下面所有的命令,主要是docker和后面的local-attach都需要root或者sudo,直接使用root会少掉很多坑]
- git clone https://github.com/openstack/cinder
- cd contrib/block-box/
然后就是安装相关的依赖了,如make, docker,docker-compose
- sudo apt - get install build - essential
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04
https://www.digitalocean.com/community/tutorials/how-to-install-docker-compose-on-ubuntu-16-04
接下来就是build需要的docker image了
- make base
这个过程中,会clone loci-cinder项目,如果遇到不能clone这个项目,你可以把loci-cinder项目clone到本地,然后修改目录下的Makefile, 指向本地的loci-cinder就行。我就遇到这种问题,于是我做了如下修改
- # git diff Makefile
- diff --git a/contrib/block-box/Makefile b/contrib/block-box/Makefile
- index ca173cc..ff60a56 100644
- --- a/contrib/block-box/Makefile
- +++ b/contrib/block-box/Makefile
- @@ -1,12 +1,12 @@
- CINDER_BRANCH ?= master # master, stable/ocata, refs/changes/67/418167/1
- NAME_PREFIX ?= ""
- -PLATFORM ?= debian # ubuntu, centos
- +PLATFORM ?= ubuntu # debian, centos
- TAG ?= latest
- all: base lvm devbox
- base:
- - docker build https://git.openstack.org/openstack/loci-cinder.git\#:$(PLATFORM) --tag cinder:$(TAG) --build-arg PROJECT_REF=$(CINDER_BRANCH)
- + docker build /root/containerized/loci-cinder/$(PLATFORM) --tag cinder:$(TAG) --build-arg PROJECT_REF=$(CINDER_BRANCH)
- lvm:
- docker build -t cinder-lvm -f ./docker_files/Dockerfile.cinder-lvm .
Docker 的image build好过后,马上就可以使用docker-compose命令启动cinder的相关service了。
启动之前你要按照自己的需求,修改目录下的etc-cinder/cinder.conf 和 docker-compose.yml文件,你可以参考我的gist文件 cinder.conf 和 docker-compose.yml
docker-compose -f docker-compose.yml up -d
docker-compose up -d
Creating network "blockbox_cindernet" with driver "bridge"
Creating blockbox_mariadb_1 ...
Creating blockbox_rabbitmq_1 ...
Creating blockbox_mariadb_1
Creating blockbox_mariadb_1 ... done
Creating blockbox_rabbitmq_1 ... done
Creating blockbox_cinder-api_1 ... done
Creating blockbox_cinder-scheduler_1 ...
Creating blockbox_cinder-scheduler_1 ... done
Creating blockbox_cinder-volume_1 ...
Creating blockbox_cinder-volume_1 ... done
启动后,查看启动的container,发现cinder-api服务启动后又退出了。
- docker ps
最后发现,cinder api的相关code有变动,需要在 当前目录(contrib/block-box)的 etc-cinder/cinder.conf,加入下面绿色内容,cinder-api才能启动成功,不让会因为找不到key manager而退出:
- [database]
- connection = mysql+pymysql://cinder:password@mariadb/cinder?charset=utf8
- [key_manager]
- backend = cinder.keymgr.conf_key_mgr.ConfKeyManager
更改后,重新启动所有docker containers
- docker-compose -f docker-compose.yml down
- docker-compose -f docker-compose.yml up -d
等一段时间,重新检查所有的docker containers
- docker ps
- CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
- 67180a6e8056 cinder "bash -c 'pip inst..." 38 seconds ago Up 37 seconds blockbox_cinder-volume_1
- d963f200be2a cinder "cinder-scheduler" 39 seconds ago Up 38 seconds blockbox_cinder-scheduler_1
- 5166d688bbbd cinder "sh /init-scripts/..." 39 seconds ago Up 38 seconds blockbox_cinder-api_1
- a14757b40830 mariadb "docker-entrypoint..." 40 seconds ago Up 38 seconds 0.0.0.0:3306->3306/tcp blockbox_mariadb_1
- b23d77e10bad rabbitmq "docker-entrypoint..." 40 seconds ago Up 38 seconds 4369/tcp, 5671/tcp, 25672/tcp, 0.0.0.0:5672->5672/tcp blockbox_rabbitmq_1
所有服务都启动成功,后面使用cinder client请求cinder volume
TIPS: 可以使用下面命令查看docker里面的log。
- docker logs < container id > -f
接下来,在host上新建virtual environment并在venv中安装最新的cinder client(只有最新的cinder client才有对noauth这种认证方式的支持)
- # 创建venv
- virtualenv pyenv
- . pyenv/bin/activate
- git clone https://github.com/openstack/python-cinderclient
- pip install -e python-cinderclient
直接请求请求cinder api
- (pyenv) peter@ubuntu16:~/pyenv$ cinder --os-auth-type=noauth --os-endpoint="http://127.0.0.1:8776/v3" --os-user-id=admin --os-tenant-id=admin list
- +--------------------------------------+-----------+--------+------+-------------+----------+-------------+
- | ID | Status | Name | Size | Volume Type | Bootable | Attached to |
- +--------------------------------------+-----------+--------+------+-------------+----------+-------------+
- | 0fefa93d-475e-4cde-9246-cc5666fde991 | available | peter1 | 10 | - | false | |
- +--------------------------------------+-----------+--------+------+-------------+----------+-------------+
注意上面使用了--os-auth-type-noauth,这就是让它直接请求cinder api,而不是先到keystone去认证。使用--os-endpoint=127.0.0.1:8776/v3是因为cinder-api同时暴露在了host上的这个端口上。
这样,就可以使用很多cinder的命令,比如cinder create,list,snapshot-create等等,相关的storage就可以被这个standalone的cinder来管理了。
volume创建好后,这里不能给到VM直接使用,因为没有nova:)。尴尬,难道volume就不能给host用了吗?当然可以。opestack社区提供了一个cinder client的plugin,让volume可以直接attach给一个host
安装python-brick-cinderclient-ext
- git clone https: //github.com/openstack/python-brick-cinderclient-ext
- pip install - e python - brick - cinderclient - ext /
现在我把刚才创建的volume暴露给host使用(记得在host上安装open-iscsi包)
- cinder--os - auth - type = noauth--os - endpoint = "http://127.0.0.1:8776/v3"--os - user - id = admin--os - tenant - id = admin local - attach 0fefa93d - 475e-4cde - 9246 - cc5666fde991
注意,要保证--os-user-id和--os-tenant-id使用与create volume一样的参数,否则会出现NotVolumeFound类似的error
https://thenewstack.io/deploying-cinder-stand-alone-storage-service/
https://gorka.eguileor.com/standalone-cinder/
来源: http://www.cnblogs.com/sting2me/p/7625776.html