Cloud Foundry
OpenStack
最近在部署cloudfoundry测试环境，遇到的问题不少，不停地在google group、github以及cf官方之间穿梭，抽空总结出来，分享的同时让大家少走弯路。利用BOSH部署cf-release有几个步骤，BOSH-cli-->MIcroBosh-->Bosh-->cf-release，先安装 bosh-cli，通过bosh-cli部署Miro bosh，再通过Micro bosh部署 bosh集群，最后利用 bosh集群来部署cf-realse集群。这些相信大家都很了解了，一些背景知识本文不打算涉及，只专注于部署过程本身。
本文先详细讲解如何部署MicroBosh，迈出第一步。
注：基于openstack IAAS，版本Folsom 。
第一篇 ：部署MicroBOSH
一、BOSH 整体架构
官方的一张图
发现其实BOSH的架构与CloudFoundry几乎一样。
outer Shell部分
CLI=VMC ，指令入口
director=CloudController，控制器，接收CLI传成的指令，执行并返回
DB = CCDB ，存储着系统的一些数据
Blobstore = Blobstore or NFS，存储着package，jobs 以及uncompiled package，stemcell
MessageBus=NATS，消息总线，本身就是用NATS。与CF一样
HealtehMonitor = HealthManager ，健康检查，接收每个vm通过agent传回的数据。
Inner Shell = DEA ，vms就形如droplets
与CF相比，这里多了一个Worker，实际上可以理解为线程，诸如编译bosh-release，cf-release的工作，以及打包Jobs，都得另外启动虚拟机来完成的，这个虚拟机就是Worker，当然也是director通过 openstack CPI来创建的。在测试环境中，为了节省资源，将worker 数据设置为1就行。
CF架构中，droplets是由开发者开发上传的，而对应到bosh的vm，是由openstack创建的。BOSH与CF的架构几乎一致。因为都是VMware主导，这就不奇怪了。
二、环境要求
官方的推荐配置
| 组件 | 节点数目 | 操作系统 | 可否是物理机 |
|---|---|---|---|
| BOSH CLI | 1 | Ubuntu | 可以 |
| Micro BOSH | 1 | Ubuntu | 不可以 |
| BOSH | 6 | Ubuntu | 不可以 |
| Cloud Foundry | 20+ | Ubuntu | 不可以 |
| 合计： | 30- |
cloudfoundry 在测试环境，很多service是可选的。这个就可以少很多。具体的再定。还有隐藏的Worker，它完成任务就删掉，所有还得准备些冗余。
三、准备虚拟机镜像
CloudFoundry运行在ubuntu环境，虽然理论上其他Linux版本也可以。先准备一个ubuntu操作系统镜像，用于创建部署过程中需要的虚拟机，以及自定义stemcell模板的制作，推荐使用ubuntu 10.04 LTS版本，在openstack dashboard 上传一个镜像。
本文用的是openstack官方上的12.04版本。
四、创建BOSH-cli
有了系统镜像之后，就可以创建BOSH-cil虚拟机了。
4.1 用admin帐号创建一个flavor，测试环境，虚拟机要求不太高：1个vcpu，10G root disk, 10G ephemeral disk,1024M RAM就OK了。
4.2 回到租户帐号，创建密钥对test.pem，浏览器会自动下载到本地，请妥善保存。
4.3 创建虚拟机，在Access&Security 中选择密钥对 test，再创建。
4.4 登录虚拟机， 从本机登录，要安装ssh。
ssh -i test.pem ubuntu@ip
五、初始化 BOSH-cli环境
利用bosh-ci来部署micro bosh， 我们需要ruby环境，bosh-cli，bosh-agent ，bosh-deployer等。
5.1 更换软件源
这步可选，由于国内的网络因素，普遍认为163源比较靠谱。
cp /etc/apt/sources.list /etc/apt/sources.ilst.bak vim /etc/apt/sources.list
用以下源替代
deb http://mirrors.163.com/ubuntu/ precise main universe restricted multiverse deb-src http://mirrors.163.com/ubuntu/ precise main universe restricted multiverse deb http://mirrors.163.com/ubuntu/ precise-security universe main multiverse restricted deb-src http://mirrors.163.com/ubuntu/ precise-security universe main multiverse restricted deb http://mirrors.163.com/ubuntu/ precise-updates universe main multiverse restricted deb http://mirrors.163.com/ubuntu/ precise-proposed universe main multiverse restricted deb-src http://mirrors.163.com/ubuntu/ precise-proposed universe main multiverse restricted deb http://mirrors.163.com/ubuntu/ precise-backports universe main multiverse restricted deb-src http://mirrors.163.com/ubuntu/ precise-backports universe main multiverse restricted deb-src http://mirrors.163.com/ubuntu/ precise-updates universe main multiverse restricted更新软件源
apt-get update
5.2 安装ruby环境
方法一：
有一个现成的shell脚本，跑下就行了。脚本安装了ruby，以及一些必要的软件包，还有bosh源码，并编译。
sudo su - export ORIGUSER=ubuntu curl -s https://raw.github.com/KingJames3/bosh-getting-started/master/scripts/create-bosh-cli.sh | bash source /etc/profile
安装ruby时会比较慢，也是跟国内网络环境有关。如果觉得太慢，可以查看prepare_inception_openstack.sh脚本，一步步执行，眼见为实。安装到ruby时，如遇到
网络问题，建议改成淘宝的gem镜像，再执行一遍。最后一步的git clone bosh-release暂时不需要，过程也比较慢。这个shell也是为自定义stemcell准备的，
由于我们用现成的micro-bosh-stemcell，里面包含了bosh-release。所以建议把prepare_inception_openstack.sh下载下来，删除agent,openstack-
registry, bosh-release那几段，再执行。
gem source -r http://rubygems.org/
gem source -a http://ruby.taobao.org
环境安装完后，确认下。
ubuntu@bosh-cli:~/stemcell$ bosh -v
BOSH 1.0.3
ubuntu@bosh-cli:~/stemcell$ bosh help micro
micro
show micro bosh sub-commands
micro agent <args>
Send agent messages
micro apply <spec>
Apply spec
micro delete
Delete micro BOSH instance (including persistent disk)
micro deploy [<stemcell>] [--update]
Deploy a micro BOSH instance to the currently selected deployment
--update update existing instance
micro deployment [<name>]
Choose micro deployment to work with, or display current deployment
micro deployments
Show the list of deployments
micro status
Display micro BOSH deployment status
没问题后就可以开始干活了。
方法二：
bosh cli 就是需要有ruby环境，安装bosh_cli和bosh_deployer就行了。所以如果不嫌麻烦，可以自己一步步来。假设创建完ubuntu vm
1. 安装ruby环境
curl -L get.rvm.io | bash -s stable source /etc/profile.d/rvm.sh rvm install 1.9.3-p374 # 这里用p374，新版的p392在安装nokogiri时有问题，没有深入排查 rvm 1.9.3 rvm alias create default 1.9.3
2. 安装系统依赖包
openstack提供的ubuntu才200M+，是个很基本的系统，没有ruby环境，默认很多东西都不装的。所以要自己来。如果用的是ubuntu官方的镜像，估计就不会这么麻烦了。
系统默认安装的是ruby 1.8.7，可以直接跳至第3步。
rvm requirements
安装这些依赖包。
3. 安装bosh cli
gem install fog --no-ri --no-rdoc gem install bosh_cli gem install bosh_deployer
这样bosh cli就OK了，bosh help micro 下。
六、部署MicroBOSH
6.1 下载micro bosh stemcell
Mico bosh有单独的操作系统模板，跟ubuntu 10.04 LTS有什么不同呢。因为 Micro bosh其实是个功能完备的bosh，包括所有组件。理论上可以用来部署cf-release，
但是micro bosh是个单台虚拟机，性能很差，扛不动。但是可以用来自部署，就是部署自己，即分布式bosh，这叫做inception，盗梦空间。在官方文档以及各大cf会议上
都能听过这个词。
ubuntu@bosh-cli:~$ mkdir -p ~/stemcells ubuntu@bosh-cli:~$ cd stemcells ubuntu@bosh-cli:~$ bosh public stemcells --all +---------------------------------------------+-----------------------------+ | Name | Tags | +---------------------------------------------+-----------------------------+ | bosh-stemcell-0.3.0.tgz | vsphere | | bosh-stemcell-0.4.4.tgz | vsphere | | bosh-stemcell-0.4.7.tgz | vsphere | | bosh-stemcell-0.5.2.tgz | vsphere | | bosh-stemcell-aws-0.5.1.tgz | aws | | bosh-stemcell-aws-0.6.4.tgz | aws, stable | | bosh-stemcell-aws-0.6.7.tgz | aws | | bosh-stemcell-aws-0.7.0.tgz | aws, test | | bosh-stemcell-openstack-0.6.7.tgz | openstack | | bosh-stemcell-openstack-kvm-0.7.0.tgz | openstack, kvm, test | | bosh-stemcell-vsphere-0.6.1.tgz | vsphere | | bosh-stemcell-vsphere-0.6.2.tgz | vsphere | | bosh-stemcell-vsphere-0.6.3.tgz | vsphere | | bosh-stemcell-vsphere-0.6.4.tgz | vsphere, stable | | bosh-stemcell-vsphere-0.6.7.tgz | vsphere, stable | | bosh-stemcell-vsphere-0.7.0.tgz | vsphere, test | | micro-bosh-stemcell-aws-0.6.4.tgz | aws, micro, stable | | micro-bosh-stemcell-aws-0.7.0.tgz | aws, micro, test | | micro-bosh-stemcell-aws-0.8.1.tgz | aws, micro, test | | micro-bosh-stemcell-openstack-0.7.0.tgz | openstack, micro, test | | micro-bosh-stemcell-openstack-kvm-0.8.1.tgz | openstack, kvm, micro, test | | micro-bosh-stemcell-vsphere-0.6.4.tgz | vsphere, micro, stable | | micro-bosh-stemcell-vsphere-0.7.0.tgz | vsphere, micro, test | | micro-bosh-stemcell-vsphere-0.8.1.tgz | vsphere, micro, test | +---------------------------------------------+-----------------------------+ To download use `bosh download public stemcell <stemcell_name>'. For full url use --full.
bosh download public stemcell micro-bosh-stemcell-openstack-kvm-0.8.1.tgz
6.2 准备floating ip
通过openstack dashboard 创建一个floating ip，Micro bosh就是要部署在这个ip上。
6.3 配置部署清单
ubuntu@bosh-cli:mkdir -p deployments/microbosh-openstack ubuntu@bosh-cli:vim micro_bosh.yml
清单名称叫做micro_bosh.yml，放在 ～/deployments/microbosh-openstack目录下
name: microbosh-openstack
env:
bosh:
password: $6$fPFEaT8CsB$sFOQhQQ/e9zZqda2a41ThKcA.AxqxiB7L9jxHUorus6cj76qnRll03b4w6DciMLOIZw4MWX23GJzld0n2L1IH1
level: DEBUG
network:
name: default
type: dynamic
label: private
ip: {IPADDRESS} #floating ip
resources:
persistent_disk: 4096 #persistent volume
cloud_properties:
instance_type: test.bosh #flavor
cloud:
plugin: openstack
properties:
openstack:
auth_url: http://{IPADDRESS}:5000/v2.0/tokens # openstack auth_url
username: jae # openstack username
api_key: jae # openstack password
tenant: jae # openstack
default_key_name: test
default_security_groups: ["default"]
private_key: ~/.ssh/test.pem
注意：
1. network下的ip请替换为创建的floating ip
2. auth_url的ip，是openstack提供的
3. test.pem是创建bosh-cil虚拟机时的密钥对文件，因为部署micro bosh时，要ssh上去，执行一些脚本之类的。
4. default_security_groups 一般默认就行。
5. persistent_disk，就是持久化disk，micro bosh在虚拟机之外，单独整出一块空间，用来存储 db，blob的数据，下次recreate micro bosh时，那些数据都还在，比如
bosh 配置，bosh-release的bit。 关于persistent_disk , root_disk 以及
ephemeral disk的区别，当时也比较晕，这是ferdy给的的解答：
- root disk is where the os is installed, - ephemeral disk is where you can store your data, but once the instance is terminated you'll lose all data, because that disk is deleted (it's part of the image). - persistent disk is where you can store your data, and if the instance is terminated, the data is still available as the disk is not deleted, so you can create another instance, attach this persistent disk, and access the same data. So the idea behind having a persistent disk for microbosh is that all db data is stored there, and in case you recreate the microbosh (ie, update the bosh bits), you are not going to lose any data.
stemcell有了，配置清单也有了。接下来才是大块头，很多问题都出现在以下的这个步骤中。具体问题具体分析，多上google group找答案或提问，同时需要了解openstack的运作原理，各组件的功能，以便发现问题，对号入座。
第一步，确定部署目标文件
ubuntu@bosh-cli:~/deployments$ bosh micro deployment microbosh-openstack Deployment set to '/home/ubuntu/deployments/microbosh-openstack/micro_bosh.yml'
第二步，指定stemcell，开始部署
ubuntu@bosh-cli:~/deployments$ bosh micro deploy ~/stemcell/micro-bosh-stemcell-openstack-kvm-0.8.1.tgz Deploying new micro BOSH instance `microbosh-openstack/micro_bosh.yml' to `http://ip:25555' (type 'yes' to continue): yes Verifying stemcell... File exists and readable OK Using cached manifest... Stemcell properties OK Stemcell info ------------- Name: micro-bosh-stemcell Version: 0.8.1 Deploy Micro BOSH unpacking stemcell (00:00:12) uploading stemcell (00:02:38) creating VM from b4fa5bcc-4cd7-4f5a-9064-12d1bfaf14df (00:02:28) waiting for the agent (00:01:24) create disk (00:00:04) mount disk (00:00:15) stopping agent services (00:00:01) applying micro BOSH spec (00:00:20) starting agent services (00:00:00) waiting for the director (00:01:17) Done 11/11 00:08:47 WARNING! Your target has been changed to `http://ip:25555'! Deployment set to '/home/ubuntu/deployments/microbosh-openstack/micro_bosh.yml' Deployed `microbosh-openstack/micro_bosh.yml' to `http://ip:25555', took 00:08:47 to complete
如果顺利，10分钟内完事。主要经过了上传stemcell---->用这个stemcell制作成的镜像创建micro bosh虚拟机--->往虚拟机里植入bosh agent，用于与bosh通讯，执行bosh发的指令----> 创建volume，给db与blobstore用---->将创建的disk mount到micro bosh这个vm上---->启动agent 以及控制器director。
启动完director，bosh-cli就可以target上去了。这里说明一点，所以的操作都是通过bosh-cli完成的，不同的是部署bosh，bosh-cli是target到micro bosh 的director来操作的，部署cf-release，bosh-cli是target到bosh的director来完成后续操作的。
现在我们target到刚部署好的Micro bosh的director上。director是个控制器，所以的指令入口都是通过directro来完成的，类似Java的action ，ruby 的controller。所以bosh-cli只有跟控制器打交道就行了。
看下Micro bosh部署成功了没。
ubuntu@bosh-cli:~/deployments$ bosh micro deployments +---------------------+--------------------------------------+--------------------------------------+ | Name | VM name | Stemcell name | +---------------------+--------------------------------------+--------------------------------------+ | microbosh-openstack | a14b575d-1b2a-40be-8cdc-f9f444135393 | b4fa5bcc-4cd7-4f5a-9064-12d1bfaf14df | +---------------------+--------------------------------------+--------------------------------------+ Deployments total: 1
再target到micro bosh上，看看micro bosh都部署了些什么。
ubuntu@bosh-cli:~/deployments$ bosh target ip:25555 Target set to `microbosh-openstack' File gems/bosh_deployer-1.4.0/lib/bosh/cli/commands/micro.rb has been loaded as plugin but it didn't contain any commands. Make sure this plugin is updated to be compatible with BOSH CLI 1.0. Your username: admin Enter password: ***** Logged in as `admin' ubuntu@bosh-cli:~$ bosh status Updating director data... done Director Name microbosh-openstack URL http://ip:25555 Version 0.7 (release:b240cdfb bosh:57b31b3f) User admin UUID a61a1aa0-0f53-43a8-99bf-e2b30dca6174 CPI openstack dns enabled Deployment not set
micro bosh director默认端口是25555，默认用户 密码是admin admin，可以看到micro bosh 除了自身信息，还没有部署组件。部署完bosh回头看下就有了。
若想登上Mico bosh，可以直接 ssh vcap@ip，密码是 micro_bosh.yml中设置的那个hashcode。micro bosh 的组件应该在/var/vcap目录下。
hashcode生成方法：
mkpasswd -m sha-512 yourpasswd
安装过程中可能会报mkpasswd安装不上，手工安装：
wget http://mirrors.kernel.org/ubuntu/pool/universe/w/whois/mkpasswd_5.0.0ubuntu3_amd64.deb dpkg -i mkpasswd_5.0.0ubuntu3_amd64.deb
七 遇到的问题
主要碰到两个比较大的问题：
1.
BOSH CLI Error: Expected(201) <=> Actual(300 Multiple Choices)
upload micri stemcell时出现以上错误。大概意思是说我调用服务上传stemcell，返回结果却是这样的：
response => #<Excon::Response:0x00000000992b90 @body="{\"versions\": [{\"status\": \"EXPERIMENTAL\", \"id\": \"v2\", \"links\": [{\"href\": \"http://ip:9292/v2/\", \"rel\": \"self\"}]}, {\"status\": \"CURRENT\", \"id\": \"v1.1\", \"links\": [{\"href\": \"http://ip:9292/v1/\", \"rel\": \"self\"}]}, {\"status\": \"SUPPORTED\", \"id\": \"v1.0\", \"links\": [{\"href\": \"http://ip:9292/v1/\", \"rel\": \"self\"}]}]}", @headers={"Content-Type"=>"application/json", "Content-Length"=>"340", "Date"=>"Wed, 22 Aug 2012 16:38:30 GMT"}, @status=300>
期望是201，却返回了一个服务列表，什么意思。原因是openstack没有指定glance endpoint版本，有三个版本（v1.0，v1.1，v2.0），client调用的时候openstack就晕了。openstack image服务是glance。重新create glance endpoint，指定版本问题解决。如果操作详见： http://docs.openstack.org/trunk/openstack-compute/install/apt/content/osfolubuntu-identityservice.html
keystone endpoint-create \
--region RegionOne \
--service_id $GLANCE_SERVICE \
--publicurl "http://$SERVICE_HOST:9292/v1.0" \
--adminurl "http://$SERVICE_HOST:9292/v1.0" \
--internalurl "http://$SERVICE_HOST:9292/v1.0"
有说fog（iaas api的ruby封装）不支持v2.0，这个没试过，只用了1.0
2.
The server could not comply with the request since it is either malformed or otherwise incorrect
deploy micro bosh，create vm的时候出现这个错误，看openstack dashboard，vm实际上已经创建成功了，但注意vm的log，没有任何信息。说明创建时
出现错误了。
这个错误信息没有多大帮助，只是说了服务有问题，没说具体是什么。看trace如下：
Deploy Micro BOSH
unpacking stemcell (00:00:10)
uploading stemcell (00:02:40)
creating VM from 3012d886-e75a-4acc-9040-4bd1d61f8b46 (00:02:10)
Waiting for the agent |oooooo | 3/11 00:06:28 ETA: 00:11:58/usr/local/rvm/gems/ruby-1.9.3-p374/gems/excon-0.16.10/lib/excon/connection.rb:292:in `request_kernel': Expected([200, 202]) <=> Actual(400 Bad Request) (Excon::Errors::BadRequest)
request => {:chunk_size=>1048576, :connect_timeout=>60, :headers=>{"User-Agent"=>"fog/1.9.0", "Content-Type"=>"application/json", "Accept"=>"application/json", "X-Auth-Token"=>"b6a85bea4ad9400bb78d09063873a053", "Host"=>"xxx:8774", "Content-Length"=>108}, :instrumentor_name=>"excon", :mock=>false, :nonblock=>true, :read_timeout=>60, :retry_limit=>4, :ssl_ca_file=>"/usr/local/rvm/gems/ruby-1.9.3-p374/gems/excon-0.16.10/data/cacert.pem", :ssl_verify_peer=>true, :write_timeout=>60, :host=>"xxx", :host_port=>"xxx:8774", :path=>"/v2/69816bacecd749f9ba1d68b3c8bae1f1/os-volumes", :port=>"8774", :query=>"ignore_awful_caching1359608037", :scheme=>"http", :body=>"{\"volume\":{\"display_name\":\"volume-a047594f-1639-408e-9738-3ac86bd67e6e\",\"display_description\":\"\",\"size\":10}}", :expects=>[200, 202], :method=>"POST"}
response => #<Excon::Response:0x00000003cd8f90 @body="{\"badRequest\": {\"message\": \"The server could not comply with the request since it is either malformed or otherwise incorrect.\", \"code\": 400}}", @headers={"Content-Length"=>"141", "Content-Type"=>"application/json; charset=UTF-8", "X-Compute-Request-Id"=>"req-a96db7a9-de91-40e3-b576-788e66be63df", "Date"=>"Thu, 31 Jan 2013 04:54:06 GMT"}, @status=400>
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/excon-0.16.10/lib/excon/connection.rb:103:in `request'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/fog-1.9.0/lib/fog/core/connection.rb:21:in `request'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/fog-1.9.0/lib/fog/openstack/compute.rb:327:in `request'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/fog-1.9.0/lib/fog/openstack/requests/compute/create_volume.rb:23:in `create_volume'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/fog-1.9.0/lib/fog/openstack/models/compute/volume.rb:31:in `save'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/fog-1.9.0/lib/fog/core/collection.rb:52:in `create'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_openstack_cpi-0.0.7/lib/cloud/openstack/cloud.rb:400:in `block in create_disk'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_common-0.5.4/lib/common/thread_formatter.rb:46:in `with_thread_name'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_openstack_cpi-0.0.7/lib/cloud/openstack/cloud.rb:373:in `create_disk'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:258:in `block in create_disk'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:84:in `step'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:256:in `create_disk'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:322:in `update_persistent_disk'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:149:in `block in create'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:84:in `step'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:148:in `create'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:104:in `block in create_deployment'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:97:in `with_lifecycle'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/deployer/instance_manager.rb:103:in `create_deployment'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_deployer-1.4.0/lib/bosh/cli/commands/micro.rb:171:in `perform'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_cli-1.0.3/lib/cli/command_handler.rb:57:in `run'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_cli-1.0.3/lib/cli/runner.rb:61:in `run'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_cli-1.0.3/lib/cli/runner.rb:18:in `run'
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_cli-1.0.3/bin/bosh:16:in `<top (required)>'
from /usr/local/rvm/gems/ruby-1.9.3-p374/bin/bosh:19:in `load'
from /usr/local/rvm/gems/ruby-1.9.3-p374/bin/bosh:19:in `<main>'
from /usr/local/rvm/gems/ruby-1.9.3-p374/bin/ruby_noexec_wrapper:14:in `eval'
from /usr/local/rvm/gems/ruby-1.9.3-p374/bin/ruby_noexec_wrapper:14:in `<main>'
是调用openstack CPI时返回的错误，我们重点看下openstack CPI，看这一行：
from /usr/local/rvm/gems/ruby-1.9.3-p374/gems/bosh_openstack_cpi-0.0.7/lib/cloud/openstack/cloud.rb:373:in `create_disk'
大概是create_disk时出错了，再看到上面几行有“create volume”，说明create_disk就是create_volume。创建volume出错。openstack的块存储服务在Folsom版中叫cinder。在openstack上调入：
cinder help
命令不存在，再ps也没有，哇，这个服务没有安装啊。大概找到问题了，安装过cinder 后问题解决。
=============================================
本文到此结束，欢迎指出不足之足，共同交流。微博：
http://weibo.com/kingjames3
下篇讲解如何用Micro bosh部署 bosh。
原文链接：http://blog.csdn.net/k_james/article/details/8562666
