[Feature] Allow bootstrap of a chef server

[claco]

This is mostly to get a discussion started. Many times with tools like Vagrant, or SpiceWeasel, etc, etc... the first thing I have to do is spin up a chef server. (Think testing a cluster setup, testing books done by QE/Support, or from scratch testing, or Jenkins), etc.

Now, installing Chef Server is easy, but firing up an instance, installing chef server and doing all that jazz takes me outside of the tools used for the rest of the process. Aka, I have to do all of that first, then fire up spice weasel. Yes, I can create Rakefiles, and knife plugins, and more glue, and even more glue. But this gets dirty/complex.

If would be nice to be able to have it so I could use spiceweasel for the entire process.

[mattray]

The intent of chef-solo is to use to bootstrap a chef-server, but there is a spiceweasel cookbook if you want an intermediary https://github.com/mattray/spiceweasel-cookbook

Lots of folks use Hosted Chef to bootstrap their open source Chef servers with the chef-server cookbook: https://github.com/opscode-cookbooks/chef-server

There's also knife-server: http://fnichol.github.io/knife-server/

And chef-zero may be packaged with the Chef fullstack install at some point: https://github.com/opscode/chef-zero

Lots of choices

[claco]

Yes. I guess "lots of choices" is my complaint. knife-vagrant is dead at this point because of the gem vs omnibus vagrant, json conflicts, etc, so I can't just weasel->vagrant my way to nodes. I can do all cloud nodes. And I can do it all from Vagrant -> berk/omnibus plugins. You could even skip knife plugins and go for all vagrant plugins (for the clouds) instead. See what I mean?

Just seems like ever option has a blocker, or things still aren't terribly easy for a "clone this repo and run this one command" perspective. Thanks for the links!

[spheromak]

I moved to pure vagrant with vagrant-chef-zero and berkshelf for full repo testing for the same reason. What u might want is a vagrant-spiceweasl plugin. That would be a bit of work to do, but I think it makes sense.

[claco]

@spheromak The workflow issue I have is, I'm on OSX. I do Vagrant. Some co workers, and Jenkins, don't. So trying to find a sane simple way to go from zero to everything-up seems to be a bit of a challenge. What to create nodes with. If they get pushed and controlled by knife or pull and controlled via chef-client, etc. I'm struggling to find the path that works best where not everyone is using Vagrant, or where everyone knows/wants to spin up a chef server, etc.

Maybe I'm just being dense. This is also not out of the question.

[spheromak]

@claco yea this is the exact same for me. I just took the repo and put a Vagrantfile that reads machine configs based on an environment variable. Using vagrant-chef-zero I don't have to worry about the server provisioning. And vagrant-berkshelf gets all the cooks in there. This is for testing and development on oxs/linux Jenkins runs this stack. I hacked on knife-vagrant a bit till vagrant moved to the new model same as you, but ditched it all for vagrant + berks + vagrant-chef-zero. If you want to have jenkins use a cloud provider you could use a vagrant driver for that provider.

If you want to be able to describe vagrants in spiceweasel I think what you end up with is a vagrant-spiceweasel plugin that would magic up nodes and their run-lists internally based on the infra.yml. Totally doable, but just doesn't exist today.

I don't want to be advocating for a tool change. That's not what I am trying to get you to do. Just saying I moved away from spice cause I needed the vagrant development, and turns out I could get similar results with this stack, and I think what would make sense to do it with spicewasel is a vagrant plugin that understands spiceweasel config and builds/provisions machines.

[claco]

@spheromak So, here's the gap for me. I may be off my rocker, but the first thing I do after installing chef server is make it the first node I register. At that point, I run my ::base recipes using chef-client to configure sys admins, os level stuff, firewall, etc. And chef-client instead of chef-solo because sometimes, it's just not the same. 🎱

It's easy enough to have vagrant up use the omnibus plugin to install chef, run solo to install chef server using the chef server cookbook. It's at this point that you have to manually go into the server and configure a knife.rb, and/or download the validation key, etc and run chef-client to make the chef server a client node of itself.

Once that's done, you're free to move on with the normal chef-client workflow. So I'll probably just make a recipe that turns the chef-server into it's first chef-client node and be fine.

[mattray]

I haven't forgotten about this, and I think I've got it now. Here's the workflow you want?

1. Spiceweasel to push the cookbooks/etc. into Chef Zero
2. knife bootstrap/ec2/etc. against Chef Zero to provision a new Chef Server with the chef-server cookbook
3. get the validation key and knife.rb from the Chef Server
4. Spiceweasel push the cookbooks/etc. into Chef Server
5. re-bootstrap the Chef Server against itself with another run list (like a base role)
6. Spiceweasel continues on with the rest of the machines against the Chef Server

I think I know how to get there with Spiceweasel, but it may be a better use case to tackle with the new Chef Metal project by @jkeiser
https://github.com/jkeiser/chef-metal