The Whys and Hows of the Oslo Namespace Change

During the Kilo cycle the Oslo team has been busy deprecating the oslo. namespace in all of our projects. The first question this probably raises for most people is: why? Ben Nemec has the answer in his blog post and also explains how the Common Libraries team has fixed the issue.

The Road to Vancouver

• Call For Speakers open until February 9
• Accepting applications for OpenStack Travel Support Program

Relevant Conversations

• Will the Big Tent “break” OpenStack?
• OpenStack Product Management Kilo Midcycle Topics Finalized
• The OpenStack Elections – Another Look
• The OpenStack Foundation lumbers toward a stronger IP policy
• EC2 API standalone service
• Vancouver Design Summit format changes

Deadlines and Development Priorities

• [Keystone] Spec proposal deadline Feb 5
• [Cinder] Cutoff deadlines for cinder drivers: volume drivers need to have a CI by end of K-3, March 19th 2015
• [nova] Kilo FeatureProposalFreeze 22nd Jan and FeatureFreeze 5th Feb
• Etherpad for the Nova midcycle meetup
• Etherpad for the Keystone midcycle meetup
• Etherpad for the Cinder midcycle meetup
• [neutron] BPs in Launchpad for Kilo-2
• [Ironic] Bay Area Midcycle
• [barbican] Kilo Mid-Cycle Sprint
• Infrastructure Puppet module split sprint, Jan 28

Reports From Previous Events

• OpenStack Bootstrapping Hour – oslo.messaging anti-patterns – Friday Jan 16th 17:00 UTC
• Linux Conf Australia 2015

Security Advisories and Notices

• [OSSA 2015-002.1] Glance v2 API unrestricted path traversal through filesystem:// scheme (CVE-2015-1195) ERRATA 1

Tips ‘n Tricks

• By Alessandro Pilotti: SaltStack states for OpenStack
• By Sébastien Han: OpenStack Cinder with Ceph under the hood
• By James Page: Extreme OpenStack: Scale testing OpenStack Messaging
• By Jason Baker: Building a successful OpenStack group
• By Sébastien Han: OpenStack: disable a compute node during its first bootstrap
• By Shaifali Agrawal: Two Merge so far…
• By Lars Kellogg-Stedman: Running nova-libvirt and nova-docker on the same host

Upcoming Events

2015 OpenStack Foundation Events Plan and Calls for Papers Available. The 2015 events plan is now available on the Global Events Calendar wiki. The detailed spreadsheet covers industry events, Summits (dates and locations), as well as regional OpenStack Days organized by the community.  If you’re interested in sponsoring or speaking at a planned OpenStack Day, please contact the contact listed in the spreadsheet.

• Jan 26 – 27, 2015 Product Management Group Meeting Palo Alto, CA, US
• Jan 26, 2015 OpenStack Production-Grade Doha, QA
• Jan 27 – 29, 2015 Cinder Kilo Midcycle Meetup Austin, Texas, US
• Jan 31 – Feb 01, 2015 FOSDEM Brussels, BE
• Feb 02 – 03, 2015 Upstream Training in Japan Tokyo, JP
• Feb 02, 2015 OpenStack Seattle Meetup Seattle, WA, US
• Feb 03 – 04, 2015 OpenStack Days Tokyo 2015 Tokyo, JP
• Feb 05, 2015 OpenStack Day in Korea Seoul, Seoul, KR
• Feb 11, 2015 HandsOn Prescriptive Topology Mgr Mountain View, CA, US
• Feb 11, 2015 CloudCamp Bangladesh @Digital World Dhaka, BD
• Feb 18, 2015 First OpenStack BW Meetup Stuttgart, Baden-Württemberg, DE
• Feb 19 – 20, 2015 World Techies Forum Mumbai, Maharashtra, IN
• Mar 04, 2015 OpenStack Finland meetup Helsinki, Uusimaa, FI
• Mar 11 – 12, 2015 Cloud Expo Europe London, GB
• Mar 26, 2015 PDX OpenStack Hackathon Portland, OR, US
• Apr 08 – 16, 2015 PyCon 2015 Montreal, Quebec, CA
• Apr 13 – 14, 2015 OpenStack Live Santa Clara, CA, US
• Apr 21 – 22, 2015 CONNECT 2015 Melbourne, Victoria, AU
• Apr 22 – 23, 2015 China SDNNFV Conference Beijing, CN
• May 05 – 07, 2015 CeBIT AU 2015 Sydney, NSW, AU
• May 18 – 22, 2015 OpenStack Summit May 2015 Vancouver, BC
• Jun 11, 2015 OpenStack DACH Day 2015 Berlin, DE
• Jul 20 – 24, 2015 OSCON 2015 Portland, OR, US
• Aug 10 – 13, 2015 Gartner Catalyst Conference San Diego, CA, US
• Sep 17, 2015 OpenStack Benelux Conference 2015 Bussum, NL
• Oct 04 – 08, 2015 Gartner SymposiumITxpo Orlando, FL, US
• Nov 15 – 20, 2015 Supercomputing 15 Austin, TX, US

Other News

• Group Based Policy – Announcing Release
• Call for Testing: Ubuntu OpenStack Kilo-1 development milestone
• Neutron versus Chaos Monkey: Is Neutron Reliable At Scale?
• Announcing git-deps: commit dependency analysis / visualization tool
• gabbi: A tool for declarative testing of APIs
• What’s Up Doc? Jan 23 2015 [heat] [cinder]
• Announcing Magnum’s First Release
• OpenStack Crossproject Meeting: (skipped).

Got Answers?

Ask OpenStack is the go-to destination for OpenStack users. Interesting questions waiting for answers:

• Migrate VMs from oVirt to OpenStack
• Why openvswitch logs appear in a different timezone?
• openstack_dashboard wsgi gives “m2crypto.so: wrong ELF class: ELFCLASS32″ error
• Red Hat Enterprise Linux OpenStack Platform 5 reference architecture or tutorial for HA installation
• mirroring traffic between vm

OpenStack Reactions

Waiting for something to happen in my long list of reviews queue

The weekly newsletter is a way for the community to learn about all the various activities occurring on a weekly basis. If you would like to add content to a weekly update or have an idea about this newsletter, please leave a comment.

Cloudbase Solutions has partnered with Spirent to build and release a suite of SaltStack states for deploying OpenStack, supporting Icehouse, Juno and Kilo releases, on multiple target platforms. These states are being enhanced and extended to also support OPNFV scenarios. Anyone wishing to collaborate on this parallel effort is welcome to join.
It is recognized that there are many choices available for:

• hardware (number of NICs, CPUs, RAM, storage)
• operating system distribution
• devops tools
• openstack release
• networking solution (Open vSwitch, OpenDaylight, OpenContrail, etc)

Regarding the devops tools, there are certainly many tools available each with their own pros and cons. Each organization will use the one best suited for their needs, though many times based on personal experience of the sysadmins. Changing from one tool to another is no easy task as the library of scripts can contain years of investment from hundreds of contributors. More and more of these devops tools have moved beyond just the ability to update operating system configuration files. Now there are solutions to deploy fully working, highly secure and available, multi-node OpenStack platforms. Some organizations might use Fuel, some native Puppet, or Packstack (RDO), others SaltStack, Ansible, Chef, while some hardcore sysadmins might still rely purely on bash scripts.

With this in mind we started last year contributing to a project to build a flexible deployment framework for OpenStack. The goal is to allow defining profiles for any particular OpenStack deployment configuration, all the way to defining networks, flavors and available Glance images, and ensure easy (re)deployment of these profiles, in a reliable and repeatable manner.

This allows to tear down and rebuild machines multiple times a day as part of automated testing solutions.

We are working closely with the OpenDaylight and OpenStack developers to bring more components into this testing framework.

Since the OPNFV Getting Started project is not using SaltStack, it is our intention to follow along as closely as possible to provide a platform with similar capabilities. The end result should be the same:

• Centos 6.5 / 7, Ubuntu 12.04 / 14.04, Microsoft Hyper-V Server OS
• kernel parameters for CPU and RAM constraints
• 1, 2, 4, 6, 8, 12 NICS (1g or 10g)
• OpenStack Icehouse or Juno (Kilo will be added once it is released)

These salt states are available on GitHub.

The post SaltStack states for OpenStack appeared first on Cloudbase Solutions.

Welcome to the Short Stack, our weekly feature where we search for the most intriguing OpenStack links to share with you. These links may come from traditional publications or company blogs, but if it's about OpenStack, we'll find the best links we can to share with you every week.

If you like what you see, please consider subscribing.

Here we go with this week's links:

We are happy to announce the first informal meetup of the OpenStack user group "Baden-Wuerttemberg" in Stuttgart, located in the South-West of Germany, at 18th February 2015 @ 19:00 in the burger restaurant "Hans im Glueck" in the new Milaneo Center near to the main station.

If you want to join us please visit our group on meetup.com for more details.

What’s happening under the hood while playing with Cinder and Ceph? Answer table :-).

As fast as OpenStack moves, sometimes it’s hard to remember that it’s still a relatively young project — until something needs to be fixed.  First, we moved from “core” projects (theoretically those without which you can’t run OpenStack) to “integrated” (defined as those projects that are part of the semi-annual release cadence), with projects applying for “incubation” in order to get into those categories.

But over the last few months, it became apparent that even this model had problems; while a project may be “blessed” for a particular purpose, it may not really reflect the reality of what is out in the marketplace. To solve this problem, the Technical Committee is implementing a new project restructuring known as the “Big Tent” initiative. The premise here is that projects that wish to live in the 0penstack code namespace should have a clear, objective, set of requirements for entry, even if there’s already another project that partially addresses the same problem domain.

This, of course, is a huge change, and we spoke to Mirantis Principal Technical Architect and OpenStack Technical Committee member Jay Pipes to find out what the implications actually are.

The problems with the current system

“The goal was to open the code namespaces to new projects and introduce some competition where there may be some overlap between project functionalities,” Jay said.

For example, he explained, Ceilometer is the “official” OpenStack project, but the reality is that there are two other projects that also do telemetry-related things:

• Stacktach predates Ceilometer, and is also Python-based. It doesn’t have a REST-ful API, but it does process notifications. It’s focused on processing an API session from start to finish, so it can be used as a debugger in addition to doing telemetry.

• Monasca is based on Apache Storm, and Kafka, and uses Scala and Java to do some similar things to Ceilometer.

All 3 — Ceilometer, Stacktach and Monasca — are viable solutions for similar problems. “We were at a crossroads,” Jay explained. “We had to say, ‘if you want to work on telemetry, work on Ceilometer.’  And that’s not conducive to growing an ecosystem.”

Another example is in the area of deployment.  TripleO is the official OpenStack deployment program, but when it was “blessed”, the TC disregarded the fact that most of the world deploys OpenStack using Puppet, Chef, Ansible, Saltstack, and so on, rather than the image-based approach that TripleO uses. “They’re all are perfectly good ways to do it.  What makes TripleO so much better and more deserving of being in the openstack code namespace compared to the Chef cookbooks or Puppet modules that live in Stackforge?”

Reaching the breaking point

Jay sees two things wrong with this approach. In addition to forcing the TC to bless one project over others when there wasn’t any reason to, the process created a situation where graduating incubation was the only way to “be” OpenStack, because even Stackforge is generally seen as “not really OpenStack”.  Only “integrated” makes the cut. “Projects had to do all this stuff, and ‘Hawk their wares’, so to speak, in front of the TC — the OpenStack High Court, as I call it,” Jay said, “and the TC would then pass judgement.”

What brought the situation to a head was the incubation process for the OpenStack Queuing project, Zaqar (formerly known as Marconi).  The project underwent no less than three incubation rounds, during which the TC would change what was being asked of the project. “We were holding them to a standard no other project had to uphold,” Jay said.  ”Basically we got bogged down in a bunch of email threads and IRC conversations that went round and round and never got anywhere.”

And that led to the real issue. “Projects had no objective set of requirements for entering the openstack code namespace and being ‘part of OpenStack’.”

The Big Tent

The new approach involves enabling virtually all projects to “be” OpenStack, giving them access to the same resources as everyone else, including the mailing lists, the shared infrastructure team and systems, the shared documentation team, and the same release management team.

Rather than having a binary approach where a project is either integrated or it’s not, projects will have multiple “tags” that provide information about them. Some may be quite coarse-grained, such as “integrated-release” (a temporary tag to be used for the Kilo release cycle to indicate projects that are in the legacy integrated release) or “compute”, “networking”, or “storage”. Others will be more fine-grained and focused such as “docs-api-partial” or “has-rest-api”.

Of course there are still SOME barriers to entry, Jay points out.  The resolution provides four examples of requirements for projects:

• They align with the OpenStack Mission: the project should help further the OpenStack mission, by providing a cloud infrastructure service, or directly building on an existing OpenStack infrastructure service

• They follow the OpenStack way: open source (licensing), open community (leadership chosen by the contributors to the project), open development (public reviews on Gerrit, core reviewers, gate, assigned liaisons), and open design (direction discussed at Design Summit and/or on public forums)

• They ensure basic interoperability (API services should support at least Keystone)

• They submit to the OpenStack Technical Committee oversight

But with a change this big, there are bound to be concerns.

Downsides to the Big Tent approach

Concerns about the Big Tent approach fall into several categories

Cross project overloading

One concern, voiced by Doug Hellman, was about overloading cross project teams such as documentation, infra, and so on.  These teams have already been struggling to keep up with the mass of new integrated projects; what would happen when every project was on equal footing?  To solve this problem, the TC has decided to restructure those teams to enable new projects to do things for themselves, rather than do everything for them. Every project must have documented “cross project liasions,” and must work to set up its own CI systems, write its own docs, and so on. The current cross project teams will then focus on helping these teams be self-sufficient.

Testing

Another major — and justifiable — concern was that this would make OpenStack incredibly complicated to test. “But then,” Jay pointed out, “it is already. Right now, all integrated projects are tested against all other integrated projects, but that doesn’t make sense.  Nova consumes Keystone, not the other way around.  This process doesn’t need to be transitive.  The gate needs to be restructured around these dependencies, but a technical problem shouldn’t affect policy.”

Release cadence

But what about the (in)famous six month release cadence? How can this possibly be maintained without defining the projects that are part of it?  Swift was never on the same cadence as OpenStack and nobody died, but surely that’s the exception, isn’t it?

It turns out that this is less of a concern than one might think.  Generally, users fall into one of two categories: release-based, and time-based.

A large contingent of OpenStack users is release-based, which means that they deploy the integrated release, and all of the pieces involved. “For them what matters is actually dependencies, but it’s not what you think,” Jay says.  ”You’d think that nova-server depends on keystone-server, but it doesn’t.  nova-server depends on the python keystone client, which then interacts with Keystone.  That client works with a specific Keystone server API version, which requires a specific library.  So it’s all about dependencies.” Those dependencies can be managed via the micro-tagging that all projects will get in this process, along with the normal dependency-tracking functionality that package management systems enable.

Other users are time-based; they are always X weeks behind trunk, and making their own packages.  For those people, the release cadence has never mattered; they tracked the dependencies on their own.

User overload

For some, OpenStack already has too many choices.  Do you deploy Swift for Object Storage? Ceph? What will happen when there are multiple projects that can serve a specific need?

“There will be a set of tags specific to operations and deployment,” Jay says.  ”For example, maybe you’d filter by ‘mature’ vs ‘experimental’ vs ‘stable’, or by the number of successful releases each project in a particular problem space has had.”

“Ultimately, it’s all a documentation issue.”

The bottom line

Of course the main concern about adding multiple projects in a problem space is the potential for diluting the pool of potential developers.  How will projects get critical mass, if everyone’s doing their own thing?  

The post Will the Big Tent “break” OpenStack? appeared first on Mirantis | The #1 Pure Play OpenStack Company.

At the 2014 Fall OpenStack Conference, Solinea's CEO Francesco Paola, and VP of Operations, Seth Fox spoke about utilizing a Platform as a Service on top of OpenStack:  

As OpenStack gains mainstream adoption, enterprises are looking for ways to unlock the value that a distributed, orchestrated IT infrastructure provides. Many are starting to introduce value-add platforms on top of OpenStack, in particular Platform as a Service (PaaS) solutions, that promise to turn the staid legacy development processes and methodologies into an agile software factory, abstracting developers from infrastructure so they can focus on what they do best - code awesome applications that bring value to the enterprise and their customers. But where should enterprises start? What are the dependencies and pre-requisites? What technology stack fits the current environment? Do skills need to be upgraded? Does the enterprise first have to transform processes and governance to take full advantage of the powerful combination of PaaS and OpenStack? And where do I start?

At the 2014 Fall OpenStack Conference, Solinea's CEO Francesco Paola, and VP of Operations, Seth Fox spoke about utilizing a Platform as a Service on top of OpenStack:  

As OpenStack gains mainstream adoption, enterprises are looking for ways to unlock the value that a distributed, orchestrated IT infrastructure provides. Many are starting to introduce value-add platforms on top of OpenStack, in particular Platform as a Service (PaaS) solutions, that promise to turn the staid legacy development processes and methodologies into an agile software factory, abstracting developers from infrastructure so they can focus on what they do best - code awesome applications that bring value to the enterprise and their customers. But where should enterprises start? What are the dependencies and pre-requisites? What technology stack fits the current environment? Do skills need to be upgraded? Does the enterprise first have to transform processes and governance to take full advantage of the powerful combination of PaaS and OpenStack? And where do I start?

The Board of directors and the bylaws were approved. Summary posts can be found here (2015 Individual Director Election results) and here (Bylaws amendments approved).

Individual Directors

• Tim Bell
• Russell Bryant
• Alex Freedland
• Rob Hirschfeld
• Vishvananda Ishaya
• Kavit Munshi
• Egle Sigler
• Monty Taylor

The voting numbers can be found here.

Congratulations to all the new and re-elected board members. Well deserved!

I have a few things I would like to add about the data that was presented – and my thoughts.

1. Tim Bell 

Tim received the highest number of votes. CERN is a huge OpenStack user and was showcased at the summit in Paris. He is one of only 3 the board members that are not officially affiliated with a vendor directly involved in OpenStack. I see this as a big vote of confidence by the members of the foundation – that are interested in seeing more representation from non-affiliated members. Something I personally would like to see as well.

2. Participation Numbers

Only just over 16% of the members voted in the election. That to me seems to be very low. and I would like to address the OpenStack community to ask why this is so? This is actually something the Board and Foundation should also be actively looking into as well (perhaps they already are).

• Is it because people are not interested in participating?
• Is it because the importance of the process was not made clear enough?
• People did not find the candidates suitable?

More people actually voted for the amendment changes that in the election itself – which I find quite strange. They were already on the page and did not bother to vote for any of the candidates.

3. Operators were not chosen

Neither Jesse Proudman and Randy Bias were voted in – both seemed to me that they were very vocal in their campaign as to why they wanted to get on the board – and that was to bring a change into the way OpenStack is currently “run”. I do personally think this is pity – because I would have liked to see more representation from the Operator standpoint, something which I still feel is still lacking in the OpenStack community.

4. Participation in Amendment changes

The quorum was achieved – but only by 315 votes. My previous post The OpenStack Foundation – 2015 Individual Director Election – spoke about how I see this change as a problematic one. I personally do not think that the quorum would have been reached – if it was not for the “aggressive” marketing campaign that the Foundation embarked upon in order to reach this quorum. Without the countless number of posts from Board members, Foundation members (myself included) and anyone that cared about this election - on their blogs, social media and everywhere that was possible. Jonathan Bryce even promised to remove his beard to achieve this goal..

If we hit quorum in time, @jbryce will shave his beard! If you haven't voted, vote NOW! pic.twitter.com/ys0g3Q60XS

— OpenStack (@OpenStack) January 16, 2015

@jbryce's beard is gone! If you haven't voted, you have 5 MINUTES & soon we'll post the 2015 election results pic.twitter.com/ATFdOe5pQX

— OpenStack (@OpenStack) January 16, 2015

A huge effort indeed, from the foundation and the community at large – but my question is..

How long will it be until it is needed again?

Yes the quorum needed is now officially only 10% (instead of 25%) but I do foresee the day that even that will not be reached (and that will not be too far in the future). That is why I think the change should have been to remove the quorum all together.

5. People were not happy with the change in the quorum

On both the first two amendments, the approval rate was 93-95% – almost everyone agreed. Changing the quorum – only had 80% of the people that approved. Of course that is still a majority and perfectly valid and acceptable as a decision, but still it is interesting to see that more people were not happy with the change.

It would be interesting to know if it was the lowering of the percentage to 10% or was it that the proposed change should have been to remove the quorum altogether. I personally voted against this change because I think that the quorum should be removed completely and not lowered to 10%.

I am very pleased that the changes were made – because it allows OpenStack to continue to grow, but I do think that planning should start now – for when the changes made in these amendments will not suffice and need to be changed again.

If you are willing to share - I would be interested in hearing your thoughts about the points above. Please feel free to leave them in the comments below.

Just prior to the Paris OpenStack Summit in November, the Ubuntu Server team had the opportunity to repeat and expand on the scale testing of OpenStack Icehouse that we did in the first quarter of last year with AMD and SeaMicro. HP where kind enough to grant us access to a few hundred servers in their Discovery Lab; specifically three chassis of HP ProLiant Moonshot m350 cartridges (540 in total): The m350 is an 8-core Intel Atom based server with 16GB of RAM and 64GB of SSD based direct attached storage. They are designed for scale out workloads, so not an immediately obvious choice for an OpenStack Cloud, but for the purposes of stretching OpenStack to the limit, having lots of servers is great as it puts load on central components in Neutron and Nova by having a large number of hypervisor edges to manage. We had a few additional objectives for this round of scale testing over and above re-validating the previous scale test we did on Icehouse on the new Juno release of OpenStack:

• Messaging: The default messaging solution for OpenStack on Ubuntu is RabbitMQ; alternative messaging solutions have been supported for some time – we wanted to specifically look at how ZeroMQ, a broker-less messaging option, scales in a large OpenStack deployment.
• Hypervisor: The testing done previously was based on the libvirt/kvm stack with Nova; The LXC driver was available in an early alpha release so poking at this looked like it might be fun.

As you would expect, we used the majority of the same tooling that we used in the previous scale test:

• MAAS (Metal-as-a-Service) for deployment of physical server resources
• Juju: installation and configuration of OpenStack on Ubuntu

in addition, we also decided to switch over to OpenStack Rally to complete the actual testing and benchmarking activities. During our previous scale test this project was still in its infancy but its grown a lot of features in the last 9 months including better support for configuring Neutron network resources as part of test context set-up.

Messaging Scale

The first comparison we wanted to test was between RabbitMQ and ZeroMQ; RabbitMQ has been the messaging workhorse for Ubuntu OpenStack deployments since our first release, but larger clouds do make high demands on a single message broker both in terms of connection concurrency and message throughput. ZeroMQ removes the central broker from the messaging topology, switching to a more directly connected edge topology.

The ZeroMQ driver in Oslo Messaging has been a little unloved over the last year or so, however some general stability improvements have been made – so it felt like a good time to take a look and see how it scales. For this part of the test we deployed a cloud of:

• 8 Nova Controller units, configured as a cluster
• 4 Neutron Controller units, configured as a cluster
• Single MySQL, Keystone and Glance units
• 300 Nova Compute units
• Ganglia for monitoring

In order to push the physical servers as hard as possible, we also increased the default workers (cores x 4 vs cores x 2) and the cpu and ram allocation ratios for the Nova scheduler. We then completed an initial 5000 instance boot/delete benchmark with a single RabbitMQ broker with a concurrency level of 150.  Rally takes this as configuration options for the test runner – in this test Rally executed 150 boot-delete tests in parallel, with 5000 iterations:

Having established a baseline for RabbitMQ, we then redeployed and repeated the same test for ZeroMQ; we immediately hit issues with concurrent instance creation.  After some investigation and re-testing, the cause was found to be Neutron’s use of fanout messages for communicating with hypervisor edges; the ZeroMQ driver in Oslo Messaging has an inefficiency in that it creates a new TCP connection for every message it sends – when Neutron attempted to send fanout messages to all hypervisors edges with a concurrency level of anything over 10, the overhead in creating so many TCP connections causes the workers on the Neutron control nodes to back up, and Nova starts to timeout instance creation on network setup.

So the verdict on ZeroMQ scalability with OpenStack? Lots of promise but not there yet….

We introduced a new feature to the OpenStack Charms for Juju in the last charm release to allow use of different RabbitMQ brokers for Nova and Neutron, so we completed one last messaging test to look at this:

unfortunately we had some networking problems in the lab which caused some slowdown and errors for instance creation, so this specific test proved a little in-conclusive. However, by running split brokers, we were able to determine that:

• Neutron peaked at ~10,000 messages/sec
• Nova peaked at ~600 messages/sec

It’s also worth noting that the SSDs that the m350 cartridges use do make a huge difference, as the servers don’t suffer from the normal iowait times associated with spinning disks.

So in summary, RabbitMQ still remains the de facto choice for messaging in an Ubuntu OpenStack Cloud; it scales vertically very well – add more CPU and memory to your server and you can deal with a larger cloud – and benefits from fast storage.

ZeroMQ has a promising architecture but needs more work in the Oslo Messaging driver layer before it can be considered useful across all OpenStack components.

In my next post we’ll look at how hypervisor choice stacks up…

With the introduction of OpenStack Heat, you can now deploy an entire cloud application, defined as a template, in a single shot. Before Heat, OpenStack cloud resources needed to be provisioned individually and combined together to build the application. For example, you would request two compute resources separately, and then you would need a network resource to connect them.

In the OpenStack Dashboard, a Heat template can be provisioned by passing a file, a URL or direct input.

But how can you provide governance over the templates that are being used? Meet IBM UrbanCode Deploy with Patterns, where an application developer can design a Heat template using a graphical or text editor, as shown in the following image:

UrbanCode Deploy with Patterns then can persist and catalog all the templates that have been defined.

Now, how can you expose these templates to a user in a self-service interface?

IBM Cloud Orchestrator provides a platform where cloud applications can be deployed as a sequence of orchestration steps and exposed in a self-service user interface, as you can see in the following picture:

This combination enables a powerful DevOps environment in which the application developer utilizes UrbanCode Deploy with Patterns to design and persist the Heat templates, and the user requests them in Cloud Orchestrator. This allows the governance of the Heat templates on the development side, and a clear delimitation on the applications a user can provision on the operational side.

What are your thoughts? Please leave a comment below or get in touch with me on Twitter at @patrocinio

The post Achieve DevOps with UrbanCode and IBM Cloud Orchestrator appeared first on Thoughts on Cloud.

The Ubuntu Server Team is pleased to announce the general availability of the first development milestone of the OpenStack Kilo release in Ubuntu 15.04 development and for Ubuntu 14.04 LTS via the Ubuntu Cloud Archive.

Ubuntu 14.04 LTS

For now, you can enable the Ubuntu Cloud Archive for OpenStack Kilo on Ubuntu 14.04 installations by running the following commands:

echo "deb http://ubuntu-cloud.archive.canonical.com/ubuntu trusty-updates/kilo main" \
    | sudo tee /etc/apt/sources.list.d/cloud-archive.list
sudo apt-get -y install ubuntu-cloud-keyring
sudo apt-get update

The Ubuntu Cloud Archive for Kilo includes updates for Nova, Glance, Keystone, Neutron, Cinder, Horizon, Swift, Ceilometer and Heat; Ceph (0.87), RabbitMQ (3.4.2), QEMU (2.1), libvirt (1.2.8) and Open vSwitch (2.3.1) back-ports from 15.04 development have also been provided.

Ubuntu 15.04 development

No extra steps required; just start installing OpenStack!  Keystone is still pending update due to review of new dependencies by the Ubuntu MIR team, but that should happen in the next few weeks.

New OpenStack components

This cycle we’ve also provided packages for Trove, Sahara and Ironic – these projects are part of the integrated OpenStack release but remain in Ubuntu universe for this development cycle, which means they won’t get point release updates or security updates as part of ongoing stable release maintenance once Ubuntu 15.04 and the Kilo Cloud Archive for Ubuntu 14.04 LTS release.

NOTE: that if you use the Neutron FWaaS driver, you will need to install the ‘python-neutron-fwaas’ package to continue using this functionality; we will improve this situation in the packaging prior to final release.

Reporting bugs

Let’s face it, as the first development milestone there are bound to be a few bugs so please use the ‘ubuntu-bug’ tool to report any bugs that you find – for example:

sudo ubuntu-bug nova-conductor

this will ensure that bugs get logged in the right place in Launchpad.

Thanks and have fun!

by @jd

What would happen if you took a perfectly good OpenStack pod and then injected a heavy dose of network chaos?  

A joint team from Big Switch Networks and Mirantis ran a Hadoop performance benchmark while forcing over 650 different network failures in a 30-minute period in a “chaos monkey”-style stress test inspired by the Netflix engineering team to help answer the question “Is Neutron Reliable Enough?”

This blog post is about what happened next.

For context, OpenStack’s Neutron network stack has traveled a long road, and the stability at scale of today’s implementations have been a source of friction. Below is a picture of the audience that assembled for a talk from Aaron Rosen and Salvatore Orlando’s on this topic at the Paris OpenStack Summit (slides here). I’ve included the picture only to point out that the crowd had gathered to hear not about any new features, but purely about reliability improvements. Wow.

Why has Neutron reliability brought on so much debate? The answer is, unfortunately, complicated. For readers unfamiliar with the stack, keep in mind that the name in practice spans three very distinct areas – vendor-provided plug-ins for production use, a framework codebase that hosts those plug-ins, and a reference plug-in. There are over 20 different Neutron plug-ins to choose from, representing a wide range of effort ranging from “certified for production” to “hacked together as a side project.” There is widespread use of the reference plug-in that, as many of its authors say often, was built for API reference and not for production. Even after choosing a production-grade Neutron plug-in, there are secondary design decisions about redundancy of Neutron and plug-in servers, placement of L3 agents, etc. In short, there is no easy way to characterize the reliability of Neutron in general, but we can take specific production-grade designs as examples, beat them up, and see what happens next.  

For this exercise, a joint Mirantis/Big Switch team used Mirantis’ OpenStack distribution with Big Switch’s Big Cloud Fabric SDN networking software stack and corresponding bare metal switch hardware. Borrowing a large-scale Big Switch testbed with a 32 leaf switches/6 spine fabric, the team loaded three racks of compute nodes and then another 13 racks worth of traffic generators. The Fuel installer was used first to lay out the OpenStack distribution, followed by the Big Cloud Fabric plug-in installer scripts that both install the Neutron plug-in and optimize Neutron configurations as documented in the joint deployment guide.

The team needed a workload that would stress the network, and whose performance lent itself to network-sensitive measurement. We chose Hadoop Terasort, a Big Data benchmark. We needed scale, along with heavy amounts of network background chatter beyond what Terasort could produce. This was achieved with a combination of three racks of worker compute nodes and another 1024 ports from a massive scale traffic generator built from Ixia generators and bare metal switch hardware/software being used as port amplifiers.

Last, we needed chaos. The team added another 42,000 MAC addresses to the network (using the traffic generator) to make sure that network state across leaf switches, spine switches, and Big Cloud Fabric SDN Controllers was heavily loaded. We then added another 750 OpenStack projects, each with their own Neutron network to ensure both the Neutron servers and control channel to the Big Cloud Fabric Controllers were heavily loaded. We then forced the Big Cloud Fabric Controllers to reboot every 30 seconds, forced a random switch reboot every 8 seconds, and a random link failure every 4 seconds. This came out to a total of more than 650 network failures over the 30-minute duration of the test runs.

The result? Over the course of four baseline test runs and four stress test runs, there was no measurable difference in Terasort performance despite the failures and background traffic.  This “no difference” was the exciting result.

For the networking/SDN heads out there, you’ll immediately notice that a traditional spanning tree network can’t converge this quickly, which means we’re already in the “modern era” of data center networking technologies. None of the non-SDN L2 fabrics can maintain stability under these conditions, and very few (if any) of the non-SDN L3 fabrics can either.

For OpenStack aficionados, this result shows that while the Neutron name spans a vast number of different plug-in codebases and configurations with many dead ends, there are Neutron options that are very stable at large scale. With tests like this, “Is Neutron stable enough?” really isn’t the right question to ask anymore, but will be replaced by “Which Neutron designs are stable enough?”  

This is an exciting time to be involved in OpenStack networking.

We’ll be putting together a presentation on this test for the Vancouver summit, so please vote for us.  You can also get more details by emailing the team at info@bigswitch.com,  or downloading the white paper.

Software Version Used for Testing:

Hadoop Version : 1.2.1 vanilla hadoop
Mirantis Openstack Version: Icehouse on Ubuntu 12.04.4(2014.1.1-5.1)
Big Cloud Fabric SDN Platform  : BCF-2.0.1 (#43)

Authors (in alpha order by last name)

Kevin Benton, OpenStack Neutron Core Reviewer and Member of Technical Staff, Big Switch Networks

Kevin Benton is a member of the technical staff at Big Switch Networks. He is an active developer in the OpenStack community and a core reviewer of the Neutron project. He spends most of his effort improving the reliability and performance of Neutron to ensure OpenStack deployments are backed by a resilient networking infrastructure. Kevin is concurrently pursuing a PhD at Indiana University researching the security of next generation network protocols.

Kyle Forster, Founder, Big Switch Networks

Kyle Forster is the founder of Big Switch, and has led various teams at the company during its growth including Product, Sales, Business Development and Marketing. Prior to Big Switch, Kyle spent most of his career in Product Management at Cisco.  Kyle holds a BSE in Electrical Engineering from Princeton, a MS in Computer Science and an MBA from the Stanford Graduate School of Business.

Kanzhe Jiang, Member Technical Staff, Big Switch Networks

Kanzhe Jiang is a member of the technical staff at Big Switch Networks, responsible for integration of Big Cloud Fabric with various cloud orchestration platforms. Prior to Big Switch, Kanzhe spent many years in startups in developing WAN optimization technologies. He holds a MS in Electrical Engineering from Stanford.

Prashanth Padubidry, Member Technical Staff, Big Switch Networks

Prashanth Padubidry is a member of the technical staff at Big Switch Networks, responsible for large scale fabric testing systems and system QA for the integration of Big Cloud Fabric and and various orchestration technologies.  He has 15 years of networking industry experience at Juniper, Aruba, Extreme and Nortel with various networking technologies spanning switching, routing, datacenter (various) and wireless.

Jason Venner, Chief Architect, Mirantis

Jason provides guidance, advice and implementation services for strategic enterprise projects, with a specific focus on using and implementing IaaS and PaaS services for customer experience excellence, operational excellence and high velocity development practices.

Jason has more than twenty years experience as an architect, engineer, and author. His depth of experience includes creative solutions for high performance and highly available systems.

The post Neutron versus Chaos Monkey: Is Neutron Reliable At Scale? appeared first on Mirantis | The #1 Pure Play OpenStack Company.

Patent troll defense is costly and stifles innovation, and the license fees trolls demand can drive companies out of business. It’s not a matter of if, or even when patent trolls—companies that make money through litigation, rather than by making products—will target OpenStack. They already have. The question now is how will the OpenStack ecosystem protect itself against patent litigation? Should the foundation lead an effort to protect members, or should individual members and companies take that on themselves?

“Openstack has significant vulnerabilities, as a platform, to patent trolls,” said Kevin Jakel, CEO of Unified Patents. “There is a need for OpenStack to have some way to help protect the developers and others working on the platform.”

Since at least 2012, the OpenStack Foundation’s legal affairs committee has discussed creating a stronger intellectual property (IP) policy to protect OpenStack from patent litigation that could harm or destroy the ecosystem. In a June 2013 update to the foundation board, Alice King, Rackspace Vice President & Associate General Counsel at the time, presented three options: Make no changes to the current policy, which relies on the Apache 2.0 license terms for protection; adopt the Google Open Patent Non Assertion Pledge; or adopt an Open Invention Network (OIN)-style patent cross license.

By default, the foundation has chosen option #1: Make no changes. When developers contribute, they sign a contributor license agreement (CLA) based on the Apache 2.0 license . “If I contribute code to this project, I also contribute a patent license covering what I have contributed,” Van Lindberg, VP of Technology at Rackspace and a member of the legal affairs committee, told Datacenter Dynamics. In essence, developers’ innovations and contributions to the project share the protection the license affords. They lose the license if they sue.

Level Up: Promise Not To Sue

The next options King presented were stronger than a license; they added the element of a pledge. The Google Open Patent Non-assertion (OPN) Pledge is a promise to not sue users, developers, or distributors of open source software. The OPN Pledge, a predecessor to LOTnet, included just 10 patents related to MapReduce at the time. It has since been expanded to include many other patents. King lauded its ease of adoption and “broad defensive termination”—if anyone who has taken the pledge sues Google or its affiliates over any patent, not just those in the agreement, they are no longer protected by the pledge, according to King’s presentation. If LOTnet had existed at the time of the presentation, King might have recommended that instead.

Finally, the board discussed adopting a cross license agreement like that of the Open Invention Network (OIN). OIN is a defensive patent pool that protects the Linux ecosystem and now counts more than 1200 members. An OpenStack Foundation patent pool could cover not just Apache code contributors and licensors, but other OpenStack members, participants, and their patents as well. The license could be limited to only some, or expanded to include all elements of the OpenStack code, King said. The OpenStack license could be structured to survive if transferred, and be terminated if any licensee sued another over patents based on OpenStack code or over a distributed product.

Linux is now the platform most of the cloud is built upon, but ten years ago, its future was threatened by legal disputes. In 2003, SCO began legal maneuvers to counter the use of “tainted” products containing alleged SCO-owned code. At issue were copyrights, not patents, but over the years, legal battles continued. SCO wanted competitors and users both to pay them hefty license fees.

The Importance of Banding Together

OIN was formed in 2005 to protect Linux from being destroyed by litigation. The OIN is “essentially a lay down your arms, patent neutralization strategy,” said Keith Bergelt, CEO of OIN, in a recent interview. 80% of its more than 1200 licensees don’t own patents, but all members get a royalty-free license to use patents other members have included, as well as patents OIN purchases.

The original founding members, which fund OIN’s efforts, include three very active OpenStack Foundation members—IBM, Red Hat, and SUSE (originally Novell), plus NEC, Philips, and Sony. OpenStack LLC actually joined OIN in 2011, along with Rackspace, which takes a hard line against patent trolls, as did HP and Symantec. Oracle joined the OIN in 2007. In 2014, Oracle became a corporate sponsor of the OpenStack Foundation and announced plans to integrate OpenStack capabilities into several of its products.

Some members are worried that lawsuits may continue, though not necessarily by trolls. Three OpenStack companies—HP, Oracle, and Symantec—have essentially withdrawn from OIN by adopting what’s called a “limitation election,” withdrawing future patents from the protection of the OIN patent pool.

“Essentially it gives them more access to information they can funnel back to support their legitimate business activities, and also their intellectual property business,” Bergelt said. “It’s inconsistent with the tenets that underlie community and project participation.”

Bergelt has lobbied the foundation board since 2012 to encourage members to join the OIN, and to create its own license that would protect OpenStack ecosystem members. “The only reason not to join is to reserve rights to sue on the Linux system,” Bergelt said.

In December 2013, OIN followed though and broadened its organization’s defense to include OpenStack packages. The foundation applauded the move. Since then a few other OpenStack companies have joined, including, Aptira and UnitedStack. Just last week, the OIN board voted to include Icehouse and some Juno packages to its Linux definition.

But that protection is weakened by the limitation elections. HP, Symantec, and Oracle can pursue patent litigation on technology not covered by the window of time specified in their OIN agreements. Other prominent OpenStack members, such as EMC, VMware, and Cisco have not joined OIN.

In August 2013, Bergelt met with the OpenStack Foundation board. According to unofficial meeting minutes, he proposed a “a more formalized relationship with the Foundation, that members would be encouraged to join the OIN, that we’d do joint press releases and that there would be an ongoing direct relationship with the TC [technical committee] to ensure the system definition is regularly updated to include all OpenStack projects.”

Bergelt renewed his campaign again late last year, warning that OpenStack will be a target of trolls. “OpenStack should be meeting us halfway,” he said, “promulgating their own code of behavior, codified in some patent protection elements, whether cross licensing, defensive, or more something more comprehensive.”

After King’s presentation, Van Lindberg, who had just been appointed to the legal affairs committee, proposed another option: that OpenStack members get behind the OIN. “He went on to describe how a difficulty with this approach is that some members of OpenStack will never be members of the OIN because the OIN fundamentally see those companies as a threat to Linux,” according to the unofficial minutes. “He didn’t name these companies.”

Lindberg also proposed a patent indemnity program, in which OpenStack members and/or the foundation would contribute to a legal defense fund. “Van feels that patent trolls often go after some weaker companies and obtain a small settlement because that company feels it’s cheaper to settle than to defend, and that settlement sets a problematic precedent that the troll can then build upon,” according to the unofficial minutes. “An indemnity program would encourage members to fight patent trolls and avoid precedents being established which would set the entire community up for trouble.”

The board has discussed transferring OpenStack LLC’s license to the foundation, according to Lindberg. That doesn’t accomplish the goal, Bergelt said, because they don’t have the ability to convey the license to members. “Irrespective of any progress in OpenStack’s adoption of a formal patent policy, OIN will continue to include core OpenStack project functionality into its Linux System Definition and encourage any and all OpenStack member companies to sign OIN’s free license to insulate themselves from patent aggression in these essential cloud technologies,” Bergelt said.

Is OpenStack at Risk?

OpenStack code covers a lot of different functionality—compute orchestration, storage orchestration, and networking, patents for which are broad and difficult for patent judges to understand. In OpenStack, just as in the rest of the cloud computing world, the field is ripe for trolls to exploit.

Many large publicly traded organizations (which the trolls see as deep pockets) are major players in OpenStack. Many of these large organizations also happen to command vast patent portfolios. Some worry that beyond the patent trolls, OpenStack participants themselves may target each other. Just because they’re involved in an open source project doesn’t negate the fact that these companies are hardcore competitors, some of whom have sued each other over patents before.  As mentioned above, Symantec, HP, and Oracle have opted out of OIN’s patent cross-license agreement.

And finally, OpenStack code is open and there is no legal review that takes place prior to code commits. It might be easy for trolls to find areas that can be classified as “infringement.” Member companies could do the same.

Proposed Changes to OpenStack License Agreements

Today OpenStack has a code contribution process that offers some level of protection. Every developer must sign a code contributor license agreement (CLA), but there is an ongoing discussion to change that to a Developer Certificate of Origin (DCO) approach, where nothing needs to be explicitly signed by a developer.

In July, the OpenStack Foundation Board of Directors discussed the pros and cons of changing to the DCO model. According to the Foundation, the Technical Committee agreed to make a recommendation to the board to adopt the DCO model. “This is ultimately a Board decision since it has legal ramifications, but the consensus on the TC is that the DCO will streamline contributions and offers advantages over the CLA,” Vishvananda Ishaya wrote in a Technical Committee update last October. Before the community votes whether to change to the DCO approach, which would require a change to the bylaws, the foundation board hopes to revise the current bylaws so that changes like this can be voted in with a majority of 10% of members voting, down from the 25% currently required. (Update: This revision was passed in last week’s election.)

Some wonder if the license change would make OpenStack more vulnerable to patent trolls.  There has also been talk about the potential for unscrupulous organizations to simply observe OpenStack discussions and then file patents. This is particularly problematic since, under the Leahy-Smith America Invents Act,  the U.S. has changed from “first to invent” to “first to file.” In essence it’s more important than ever to document inventions.

“There is no one silver bullet to solve this,” Jakel said. “The question is, can you buy your way out of this problem? There are too many patents that cover cloud technology. It would take an unbelieveable amount of money.”

The post The OpenStack Foundation lumbers toward a stronger IP policy appeared first on Mirantis | The #1 Pure Play OpenStack Company.

Linux Conf Australia (LCA) is one of my favorite conferences in the year. It's full of strong technical content, interesting people, stimulating ideas, and overall a very relaxing and energizing event. One of the hallmarks of LCA is that it is volunteer-run and keynotes are invited, not purchased. Each time I've gone to LCA, I've come back home excited about what's going on in the FOSS community.

I've been fortunate to attend the past three LCA conferences. This year, in Auckland New Zealand, I spoke on my experiences in building and leading open-source community projects. I'm quite used to giving technical talks (mostly on OpenStack Swift, of course), but this talk was new for me. Instead of focusing on the technical particulars of a feature we implemented in Swift, I shared my perspective on how we, as a community, actually built the feature--the tools, processes, and coordination that we used to get people around the world organized to work together.

There were quite a few other fascinating talks at the conference. Below are some of the other great talks I attended while in Auckland.

Introduction to Humanitarian Response: I had no idea there was so much required of the response efforts we see on the news when disaster strikes. This is a fascinating look at the huge challenges in humanitarian response and where free software can help.

Tales from the Trenches: Battling Browser Bugs for "Fun" and (Non-)Profit: Roan Kattouwc, the developer of Wikimedia's visual editor, shares his experience in developing a web application that works across all modern browsers for one of the world's most-visited websites.

Keynote from Eben Moglen: An inspiring and challenging call to arms for free software

Coccinelle: A program matching and transformation tool: building a tool to automatically find and fix different classes of bugs in the Linux kernel. This was a very technical and complex talk, but pretty amazing as to what's possible. (Is there a Python version?)

Hacking 3D Printers: 3D printers started and intended to be for "hacking" (in the most noble sense of the word). This talk is all about modifying 3D printers to do new and different things.

Checking Your Privilege: A How-To for Hard Things: A passionate and well-articulated call to the community to be more inclusive

Open Source Protocols and Architectures to Fix the Internet of Things: The "Internet of Things" is one of the current buzzwords, but as an industry we're still struggling with making connected "smart" devices useful and actually smart. Alasdair's talk addresses these issues and proposes some solutions.

Deploying OpenStack Using Containers: A great take on containers as a reliable and dependable replacement for distort packages. I enjoyed this because of the ops focus over the "rah rah! go containers!" hype that is everywhere.

Of course, there were a ton of other great talks too. I'm still watching some of the ones I missed (astronomy miniconf, security challenges, flying drones, building new programming languages...). All of the LCA 2015 videos are available at https://www.youtube.com/user/linuxconfau2015

Trove is moving ahead at a rapid clip. In just a few short weeks, several of the contributors to the Trove project will be gathering in Seattle for the mid-cycle meetup. A draft agenda has been published at here. 

At the meetup we’ll be reviewing the progress made in Kilo and some of the significant new features that will be available in the release.

In addition to a number of bug fixes, Kilo is planned to increase many new capabilities. Tesora has been working on v2 of replication which will feature GTID based replication for MySQL and manual failover.

A significant usability improvement is the ability to retrieve log files from the guest instances via an API call (similar to nova console-log).

Trove depends heavily on modules from OSLO and with rapid developments in OSLO, some of the modules that Trove depended on were either deprecated, or graduated from the oslo-incubator. Tesora contributed code to address many of these changes, with the exception of oslo.messaging which was a significant effort all by itself. Thanks are due to the team at RedHat who took on and delivered that significant change.

A significant amount of work was also done in improving the testing process that is followed as part of the gate. Other projects that are under way and are candidates for Kilo would improve clustering support in Trove.

Tesora is looking forward to making some exciting new announcements of new capabilities in the DBaaS Platform in the same time period. Stay tuned for those announcements.

For operationnal reasons, you might not want to automatically make your compute node available. With the following flag, during its first bootstrap the compute node will register itself to the service list. However it will be disabled, so virtual machines can not be scheduled on it:

enable_new_services=False

I’m happy to announce a new tool called git-deps which performs automatic analysis and visualization of dependencies between commits in a git repository. Here’s a screencast demonstration!

<iframe allowfullscreen="allowfullscreen" frameborder="0" height="360" src="http://www.youtube.com/embed/irQ5gMMz-gE" width="640"></iframe>

Back in 2013 I blogged about some tools I wrote which harness the notes feature of git to help with the process of porting commits from one branch to another. These are mostly useful in the cases where porting is more complex than just cherry-picking a small number of commits.

However, even in the case where there are a small number of desired commits, sometimes those commits have hidden dependencies on other commits which you didn’t particularly want, but need to pull in anyway, e.g. in order to avoid conflicts during cherry-picking. Of course those secondary commits may in turn require other commits, and before you know it, you’re in dependency hell, which is only supposed to happen if you’re trying to install Linux packages and it’s still 1998 … but in fact that’s exactly what happened to me at SUSEcon 2013, when I attempted to help a colleague backport a bugfix in OpenStack Nova from the master branch to a stable release branch.

At first sight it looked like it would only require a trivial git cherry-pick, but that immediately revealed conflicts due to related code having changed in master since the release was made. I manually found the underlying commit which the bugfix required by using git blame, and tried another cherry-pick. The same thing happened again. Very soon I found myself in a quagmire of dependencies between commits, with no idea whether the end was in sight.

So wouldn’t it be nice if you could see the dependency tree ahead of time, rather than spending a whole bunch of time resolving unexpected conflicts due to missing dependencies, only to realise that the tree’s way deeper than you expected, and that actually a totally different approach is needed? Well, I thought it would, and so git-deps was born!

In coffee breaks during the ensuing openSUSE conference at the same venue, I feverishly hacked together a prototype and it seemed to work. Then normal life intervened, and no progress was made for another year.

However thanks to SUSE’s generous Hack Week policy, I have had the luxury of being able to spending some of early January 2015 working to bring this tool to the next level. I submitted a Hack Week project page, announced my intentions on the git mailing list, started hacking, missed quite a bit of sleep, and finally recorded the above screencast.

The tool is available here: https://github.com/aspiers/git-deps

Please give it a go and let me know what you think! I’m particularly interested in hearing ideas for use cases I didn’t think of yet, and proposals for integration with other git web front-ends.

The post Announcing git-deps: commit dependency analysis / visualization tool appeared first on Structured Procrastination.

The OpenStack Foundation has 13,490 eligible voters and our 25% quorum line to allow us to make changes to our outdated bylaws is 3,373. Today's number is that 2000 votes have been cast with about 1400 to go.

I know many of my LinkedIn connections are Foundation members and are eligible to vote. Please vote now to help our project bylaws move forward and while you are voting, please consider casting your 8 votes for Kavit Munshi. http://www.openstack.org/community/members/profile/139 #Kavit2015

I regularly use OpenStack on my laptop with libvirt as my hypervisor. I was interested in experimenting with recent versions of the nova-docker driver, but I didn't have a spare system available on which to run the driver, and I use my regular nova-compute service often enough that I didn't want to simply disable it temporarily in favor of nova-docker.

I guess the simplest solution would be to spin up a vm on which to run nova-docker, but why use a simple solution when there are things to be learned? I wanted to know if it were possible (and if so, how) to run both hypervisors on the same physical host.

The naive solution would be to start up another instance of nova-compute configured to use the Docker driver. Unfortunately, Nova only permits a single service instance per "host", so starting up the second instance of nova-compute would effectively "mask" the original one.

Fortunately, Nova's definition of what constitutes a "host" is somewhat flexible. Nova supports a host configuration key in nova.conf that will cause Nova to identify the host on which it is running using your explicitly configured value, rather than your system hostname. We can take advantage of this to get a second nova-compute instance running on the same system.

Install nova-docker

We'll start by installing the nova-docker driver from https://github.com/stackforge/nova-docker. If you're running the Juno release of OpenStack (which I am), you're going to want to use the stable/juno branch of the nova-docker repository. So:

$ git clone https://github.com/stackforge/nova-docker
$ cd nova-docker
$ git checkout stable/juno
$ sudo python setup.py install

You'll want to read the project's README for complete installation instructions.

Configure nova-docker

Now, rather than configuring /etc/nova/nova.conf, we're going to create a new configuration file, /etc/nova/nova-docker.conf, with only the configuration keys that differ from our primary Nova configuration:

[DEFAULT]
host=nova-docker
compute_driver=novadocker.virt.docker.DockerDriver
log_file=/var/log/nova/nova-docker.log
state_path=/var/lib/nova-docker

You can see that we've set the value of host to nova-docker, to differentiate this nova-compute service from the libvirt-backed one that is already running. We've provided the service with a dedicated log file and state directory to prevent conflicts with the already-running nova-compute service.

To use this configuration file, we'll launch a new instance of the nova-compute service pointing at both the original configuration file, /etc/nova/nova.conf, as well as this nova-docker configuration file. The command line would look something like:

nova-compute --config-file /etc/nova/nova.conf \
  --config-file /etc/nova/nova-docker.conf

The ordering of configuration files on the command line is significant: later configuration files will override values from earlier files.

I'm running Fedora 21 on my laptop, which uses systemd, so I created a modified version of the openstack-nova-compute.service unit on my system, and saved it as /etc/systemd/system/openstack-nova-docker.service:

[Unit]
Description=OpenStack Nova Compute Server (Docker)
After=syslog.target network.target

[Service]
Environment=LIBGUESTFS_ATTACH_METHOD=appliance
Type=notify
Restart=always
User=nova
ExecStart=/usr/bin/nova-compute --config-file /etc/nova/nova.conf --config-file /etc/nova/nova-docker.conf

[Install]
WantedBy=multi-user.target

And then activated the service;

# systemctl enable openstack-nova-docker
# systemctl start openstack-nova-docker

Now, if I run nova service-list with administrator credentials, I can see both nova-compute instances:

+----+------------------+------------------+----------+---------+-------...
| Id | Binary           | Host             | Zone     | Status  | State ...
+----+------------------+------------------+----------+---------+-------...
| 1  | nova-consoleauth | host.example.com | internal | enabled | up    ...
| 2  | nova-scheduler   | host.example.com | internal | enabled | up    ...
| 3  | nova-conductor   | host.example.com | internal | enabled | up    ...
| 5  | nova-cert        | host.example.com | internal | enabled | up    ...
| 6  | nova-compute     | host.example.com | nova     | enabled | up    ...
| 7  | nova-compute     | nova-docker      | nova     | enabled | up    ...
+----+------------------+------------------+----------+---------+-------...

Booting a Docker container (take 1)

Let's try starting a Docker container using the new nova-compute service. We'll first need to load a Docker image into Glance (you followed the nova-docker instructions for configuring Glance, right?). We'll use my larsks/thttpd image, because it's very small and doesn't require any configuration:

$ docker pull larsks/thttpd
$ docker save larsks/thttpd |
  glance image-create --is-public True --container-format docker \
    --disk-format raw --name larsks/thttpd

(Note that you will probably require administrative credentials to load this image into Glance.)

Now that we have an appropriate image available we can try booting a container:

$ nova boot --image larsks/thttpd --flavor m1.small test1

If we wait a moment and then run nova list, we see:

| 9a783952-a888-4fcd-8f5d-cd9291ed1969 | test1   | ERROR  | spawning   ...

What happened? Looking at the appropriate log file (/var/log/nova/nova-docker.log), we find:

Cannot setup network: Unexpected vif_type=binding_failed
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/novadocker/virt/docker/driver.py", line 367, in _start_container
    self.plug_vifs(instance, network_info)
  File "/usr/lib/python2.7/site-packages/novadocker/virt/docker/driver.py", line 187, in plug_vifs
    self.vif_driver.plug(instance, vif)
  File "/usr/lib/python2.7/site-packages/novadocker/virt/docker/vifs.py", line 63, in plug
    _("Unexpected vif_type=%s") % vif_type)
NovaException: Unexpected vif_type=binding_failed

The message vif_type=binding_failed is Nova's way of saying "I have no idea what happened, go ask Neutron". Looking in Neutron's /var/log/neutron/server.log, we find:

Failed to bind port 82c07caa-b2c2-45e9-955d-e8b35112437c on host
nova-docker

And this tells us our problem: we have told our nova-docker service that it is running on a host called "nova-docker", and Neutron doesn't know anything about that host.

NB

If you were to try to delete this failed instance, you would find that it is un-deletable. In the end, I was only able to delete it by directly editing the nova database using this sql script.

Adding a Neutron agent

We're going to need to set up an instance of neutron-openvswitch-agent to service network requests on our "nova-docker" host. Like Nova, Neutron also supports a host configuration key, so we're going to pursue a solution similar to what we used with Nova by creating a new configuration file, /etc/neutron/ovs-docker.conf, with the following content:

[DEFAULT]
host = nova-docker

And then we'll set up the corresponding service by dropping the following into /etc/systemd/system/docker-openvswitch-agent.service:

[Unit]
Description=OpenStack Neutron Open vSwitch Agent (Docker)
After=syslog.target network.target

[Service]
Type=simple
User=neutron
ExecStart=/usr/bin/neutron-openvswitch-agent --config-file /usr/share/neutron/neutron-dist.conf --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini --config-file /etc/neutron/ovs-docker.conf --log-file /var/log/neutron/docker-openvswitch-agent.log
PrivateTmp=true
KillMode=process

[Install]
WantedBy=multi-user.target

NB

While working on this configuration I ran into an undesirable interaction between Docker and systemd's PrivateTmp directive.

This directive causes the service to run with a private mount namespace such that /tmp for the service is not the same as /tmp for other services. This is a great idea from a security perspective, but can cause problems in the following scenario:

1. Start a Docker container with nova boot ...
2. Restart any service that uses the PrivateTmp directive
3. Attempt to delete the Docker container with nova delete ...

Docker will fail to destroy the container because the private namespace created by the PrivateTmp directive preserves a reference to the Docker devicemapper mount in /var/lib/docker/devicemapper/mnt/... that was active at the time the service was restarted. To recover from this situation, you will need to restart whichever service is still holding a reference to the Docker mounts.

I have posted to the systemd-devel mailing list to see if there are any solutions to this behavior. As I note in that email, this behavior appears to be identical to that described in Fedora bug 851970, which was closed two years ago.

Update I wrote a separate post about this issue, which includes some discussion about what's going on and a solution.

If we activate this service...

# systemctl enable docker-openvswitch-agent
# systemctl start docker-openvswitch-agent

...and then run neutron agent-list with administrative credentials, we'll see the new agent:

$ neutron agent-list
+--------------------------------------+--------------------+------------------+-------+...
| id                                   | agent_type         | host             | alive |...
+--------------------------------------+--------------------+------------------+-------+...
| 2e40062a-1c30-46a3-8719-3ce93a56b4ce | Open vSwitch agent | nova-docker      | :-)   |...
| 63edb2a4-f980-4f88-b9c0-9610a1b20f13 | L3 agent           | host.example.com | :-)   |...
| 8482c5c3-208c-4145-9f7d-606be3da11ed | Loadbalancer agent | host.example.com | :-)   |...
| 9922ed54-00fa-41d4-96e8-ac8af8c291fd | Open vSwitch agent | host.example.com | :-)   |...
| b8becb9c-7290-42be-9faf-fd3baeea3dcf | Metadata agent     | host.example.com | :-)   |...
| c46be41b-e93a-40ab-a37e-4d67b770a3df | DHCP agent         | host.example.com | :-)   |...
+--------------------------------------+--------------------+------------------+-------+...

Booting a Docker container (take 2)

Now that we have both the nova-docker service running and a corresponding neutron-openvswitch-agent available, let's try starting our container one more time:

$ nova boot --image larsks/thttpd --flavor m1.small test1
$ nova list
+--------------------------------------+---------+--------+...
| ID                                   | Name    | Status |...
+--------------------------------------+---------+--------+...
| 642b7d61-9189-40ea-86f5-2424c3c86028 | test1   | ACTIVE |...
+--------------------------------------+---------+--------+...

If we assign a floating IP address:

$ nova floating-ip-create ext-nat
+-----------------+-----------+----------+---------+
| Ip              | Server Id | Fixed Ip | Pool    |
+-----------------+-----------+----------+---------+
| 192.168.200.211 | -         | -        | ext-nat |
+-----------------+-----------+----------+---------+
$ nova floating-ip-associate test1 192.168.200.211

We can then browse to http://192.168.200.211 and see the sample page:

$ curl http://192.168.200.211/
.
.
.
  ____                            _         _       _   _                 
 / ___|___  _ __   __ _ _ __ __ _| |_ _   _| | __ _| |_(_) ___  _ __  ___ 
| |   / _ \| '_ \ / _` | '__/ _` | __| | | | |/ _` | __| |/ _ \| '_ \/ __|
| |__| (_) | | | | (_| | | | (_| | |_| |_| | | (_| | |_| | (_) | | | \__ \
 \____\___/|_| |_|\__, |_|  \__,_|\__|\__,_|_|\__,_|\__|_|\___/|_| |_|___/
                  |___/                                                   
.
.
.

Booting a libvirt instance

To show that we really are running two hypervisors on the same host, we can launch a traditional libvirt instance alongside our Docker container:

$ nova boot --image cirros --flavor m1.small --key-name lars test2

Wait a bit, then:

$ nova list
+--------------------------------------+-------+--------+...
| ID                                   | Name  | Status |...
+--------------------------------------+-------+--------+...
| 642b7d61-9189-40ea-86f5-2424c3c86028 | test1 | ACTIVE |...
| 7fec33c9-d50f-477e-957c-a05ee9bd0b0b | test2 | ACTIVE |...
+--------------------------------------+-------+--------+...

2015 Election Results

Thank you to everyone who voted in this year’s election. We reached quorum and passed the three bylaws amendments. We also reached quorum in electing eight Individual Directors to the 2015 OpenStack Board of Directors.

And the winner is….

Individual Directors

• Tim Bell
• Russell Bryant
• Alex Freedland
• Rob Hirschfeld
• Vishvananda Ishaya
• Kavit Munshi
• Egle Sigler
• Monty Taylor

Platinum Directors

• Alan Clark
• Eileen Evans
• Toby Ford
• Van Lindberg
• Mark McLoughlin
• Todd Moore
• Imad Sousou
• John Zannos

Gold Directors

• Simon Anderson
• Robert Esker
• Tristan Goode
• Steven Hallett
• Chris Kemp
• Boris Renski
• Sean Roberts
• Lew Tucker

Kilo Nova deploy recommendations

What would a Nova developer tell a deployer to think about before their first OpenStack install? Michael Still shares his thought process to consider your options and make informed decisions.

OpenStack Ambassadors Come Together

Ambassadors to help other user groups through communicating. meeting alternating Weekly on first, third, fifth Tuesdays at 08:00 GMT and second and fourth Fridays at 18:00 GMT on the chat.freenode.net IRC channel: #openstack-meeting-alt.

Small openstack

An interesting conversation on the Operators mailing list discussing request for small installation of openstack (about 3-5 nodes). The messages started in December, continued in January and drifted to discuss distributed glance.

User self registration and management

Adrian Turjak asked on the Development mailing list if there is any interest or need for an open-source user registration and management service for people using OpenStack. There seems to be a basic need a way for users to sign up themselves, choose their own password, and request new users to be added to their project. David Chadwick pointed to a blueprint and a demo of the system for University of Kent.Enrique Garcia also expressed interest in helping out in the effort.

The Road to Vancouver

• Call For Speakers

Relevant Conversations

• Requirements for future “OpenStack projects”
• The Future of OpenStack is Now, 2015
• Infrastructure Puppet module split sprint, Jan 28
• Vancouver Design Summit format changes
• Another Nova spec update

Deadlines and Development Priorities

• [Keystone] Spec proposal deadline Feb 5
• [Cinder] Cutoff deadlines for cinder drivers: volume drivers need to have a CI by end of K-3, March 19th 2015
• [nova] Kilo FeatureProposalFreeze 22nd Jan and FeatureFreeze 5th Feb
• Etherpad for the Nova midcycle meetup
• Etherpad for the Keystone midcycle meetup
• Etherpad for the Cinder midcycle meetup
• [Nova] Kilo Release Status – just passed spec freeze
• [neutron] BPs in Launchpad for Kilo-2
• [Ironic] Bay Area Midcycle
• [barbican] Kilo Mid-Cycle Sprint

Security Advisories and Notices

• [OSSA 2015-002] Glance v2 API unrestricted path traversal through filesystem:// scheme

Tips ‘n Tricks

• By Sébastien Han: Fix nova-scheduler issue with RBD and UUID not found

Upcoming Events

2015 OpenStack Foundation Events Plan and Calls for Papers Available. The 2015 events plan is now available on the Global Events Calendar wiki. The detailed spreadsheet covers industry events, Summits (dates and locations), as well as regional OpenStack Days organized by the community.  If you’re interested in sponsoring or speaking at a planned OpenStack Day, please contact the contact listed in the spreadsheet.

• OpenStack Bootstrapping Hour – oslo.messaging anti-patterns – Friday Jan 16th 17:00 UTC
• Jan 19, 2015 OpenStack Meetup Nuremberg Nuremberg, Bavaria, DE
• Jan 22, 2015 OpenStack L.A. January Meetup Pasadena, CA, US
• Jan 27 – 29, 2015 Cinder Kilo Midcycle Meetup Austin, Texas, US
• Feb 02 – 03, 2015 Upstream Training in Japan Tokyo, JP
• Feb 03 – 04, 2015 OpenStack Days Tokyo 2015 Tokyo, JP
• Feb 05, 2015 OpenStack Day in Korea Seoul, Seoul, KR
• Feb 19 – 20, 2015 World Techies Forum Mumbai, Maharashtra, IN
• Mar 26, 2015 PDX OpenStack Hackathon Portland, OR, US
• Apr 13 – 14, 2015 OpenStack Live Santa Clara, CA, US
• Apr 22 – 23, 2015 China SDNNFV Conference Beijing, CN
• May 18 – 22, 2015 OpenStack Summit May 2015 Vancouver, BC
• Jun 11, 2015 OpenStack DACH Day 2015 Berlin, DE
• Sep 17, 2015 OpenStack Benelux Conference 2015 Bussum, NL

Other News

• OpenStack Product Management Kilo Midcycle Meeting
• OpenStack Crossproject Meeting: Summary and full logs.

Got Answers?

Ask OpenStack is the go-to destination for OpenStack users. Interesting questions waiting for answers:

• Heat AutoScalingGroup with different user data
• How to retrieve updated topology after an autoscaling has been done?
• How to configure OCCI API with OpenStack/DevStack?

OpenStack Reactions

Vote for me!

Our CEO, Ken Rugg, visits the Austin OpenStack Meetup to do an introduction and deep dive into the OpenStack Trove project, Database as a Service.  Check out his presentation below!

<iframe allowfullscreen="" frameborder="0" height="355" marginheight="0" marginwidth="0" scrolling="no" src="http://www.slideshare.net/slideshow/embed_code/43588709" style="border:1px solid #CCC; border-width:1px; margin-bottom:5px; max-width: 100%;" width="425"></iframe>

Welcome to the Short Stack, our weekly feature where we search for the most intriguing OpenStack links to share with you. These links may come from traditional publications or company blogs, but if it's about OpenStack, we'll find the best links we can to share with you every week.

If you like what you see, please consider subscribing.

Check out this week's articles:

Pass the Mic User Spotlight: Doug Soltesz, Budd Van Lines | Superuser Blog

Doug Soltesz, VP and CIO of Budd Van Lines, a national trucking company shares his OpenStack experience.  OpenStack technology has allowed him to drastically reduce storage costs for the company, providing the ability to invest in new initiatives and better equipment that will differentiate them from competitors. 

The Future of OpenStack is Now, 2015 | Cloudscaling

Randy Bias, OpenStack foundation director, explains how 2015 is an important year for OpenStack, as it becomes at risk of collapsing under its own weight.  He believes that the vision, project structure, integrated release cycle, and the board all need to be reviewed.  He outlines a way forward and how the community can help. 

DBaaS: Don't Blink or You'll Miss that Process | Tesora Blog

As big data, social networking, the Internet of Things, and other data-intensive workloads are on the rise, database functionality is likely to become slower, and as the saying goes "time is money".  Arthur Cole explains how Database as a Service is one way to help enterprises leverage their growing cloud presence and lower the costs of their database infrastructure. 

OpenStack is a Social Contract | Cloud Architect Musings

Kenneth Hui explains how the OpenStack project is reliant on the community by a social contract, in which all members agree and actively come together for the good of the entire community.  He discusses ways in which the community can live up to their obligations as part of the OpenStack social contract. 

Hybrid Cloud, Open Source Lead the Cloud Pack | Information Age

In this article, new IDC research shows open hybrid cloud IT infrastructure is driving change across business operations, development, and management solutions.  IDC conducted an online survey that looked at public and hybrid cloud use, OpenStack, open source, workload placement choice, and management priorities.  

Welcome to the Short Stack, our weekly feature where we search for the most intriguing OpenStack links to share with you. These links may come from traditional publications or company blogs, but if it's about OpenStack, we'll find the best links we can to share with you every week.

If you like what you see, please consider subscribing.

Check out this week's articles:

Pass the Mic User Spotlight: Doug Soltesz, Budd Van Lines | Superuser Blog

Doug Soltesz, VP and CIO of Budd Van Lines, a national trucking company shares his OpenStack experience.  OpenStack technology has allowed him to drastically reduce storage costs for the company, providing the ability to invest in new initiatives and better equipment that will differentiate them from competitors. 

The Future of OpenStack is Now, 2015 | Cloudscaling

Randy Bias, OpenStack foundation director, explains how 2015 is an important year for OpenStack, as it becomes at risk of collapsing under its own weight.  He believes that the vision, project structure, integrated release cycle, and the board all need to be reviewed.  He outlines a way forward and how the community can help. 

DBaaS: Don't Blink or You'll Miss that Process | Tesora Blog

As big data, social networking, the Internet of Things, and other data-intensive workloads are on the rise, database functionality is likely to become slower, and as the saying goes "time is money".  Arthur Cole explains how Database as a Service is one way to help enterprises leverage their growing cloud presence and lower the costs of their database infrastructure. 

OpenStack is a Social Contract | Cloud Architect Musings

Kenneth Hui explains how the OpenStack project is reliant on the community by a social contract, in which all members agree and actively come together for the good of the entire community.  He discusses ways in which the community can live up to their obligations as part of the OpenStack social contract. 

Hybrid Cloud, Open Source Lead the Cloud Pack | Information Age

In this article, new IDC research shows open hybrid cloud IT infrastructure is driving change across business operations, development, and management solutions.  IDC conducted an online survey that looked at public and hybrid cloud use, OpenStack, open source, workload placement choice, and management priorities.  

• Add more detailed network information to the metadata server: review 85673 (approved).
• Add separated policy rule for each v2.1 api: review 127863 (requested a spec exception).
• Add user limits to the limits API (as well as project limits): review 127094.
• Allow all printable characters in resource names: review 126696 (approved).
• Consolidate all console access APIs into one: review 141065 (approved).
• Expose the lock status of an instance as a queryable item: review 127139 (abandoned); review 85928 (approved).
• Extend api to allow specifying vnic_type: review 138808 (requested a spec exception).
• Implement instance tagging: review 127281 (fast tracked, approved).
• Implement the v2.1 API: review 126452 (fast tracked, approved).
• Improve the return codes for the instance lock APIs: review 135506.
• Microversion support: review 127127 (approved).
• Move policy validation to just the API layer: review 127160 (approved).
• Nova Server Count API Extension: review 134279 (fast tracked).
• Provide a policy statement on the goals of our API policies: review 128560 (abandoned).
• Sorting enhancements: review 131868 (fast tracked, approved, implemented).
• Support JSON-Home for API extension discovery: review 130715 (requested a spec exception).
• Support X509 keypairs: review 105034 (approved).

• Expand support for volume filtering in the EC2 API: review 104450.
• Implement tags for volumes and snapshots with the EC2 API: review 126553 (fast tracked, approved).

• Actively hunt for orphan instances and remove them: review 137996 (abandoned); review 138627.
• Add totalSecurityGroupRulesUsed to the quota limits: review 145689.
• Check that a service isn't running before deleting it: review 131633.
• Enable the nova metadata cache to be a shared resource to improve the hit rate: review 126705 (abandoned).
• Implement a daemon version of rootwrap: review 105404 (requested a spec exception).
• Log request id mappings: review 132819 (fast tracked).
• Monitor the health of hypervisor hosts: review 137768.
• Remove the assumption that there is a single endpoint for services that nova talks to: review 132623.

• Allow direct access to LVM volumes if supported by Cinder: review 127318.
• Cache data from volumes on local disk: review 138292 (abandoned); review 138619.
• Enhance iSCSI volume multipath support: review 134299 (requested a spec exception).
• Failover to alternative iSCSI portals on login failure: review 137468 (requested a spec exception).
• Give additional info in BDM when source type is "blank": review 140133.
• Implement support for a DRBD driver for Cinder block device access: review 134153 (requested a spec exception).
• Poll volume status: review 142828 (abandoned).
• Refactor ISCSIDriver to support other iSCSI transports besides TCP: review 130721 (approved).
• StorPool volume attachment support: review 115716 (approved, requested a spec exception).
• Support Cinder Volume Multi-attach: review 139580 (approved).
• Support iSCSI live migration for different iSCSI target: review 132323 (approved).

• Cells Scheduling: review 141486.
• Create an instance mapping database: review 135644 (approved).
• Flexible cell selection: review 140031.
• Implement instance mapping: review 135424 (approved).
• Populate the instance mapping database: review 136490 (requested a spec exception).

• Initial specification: review 114044 (abandoned).

• Develop and implement a profiler for SQL requests: review 142078 (abandoned).
• Enforce instance uuid uniqueness in the SQL database: review 128097 (fast tracked, approved, implemented).
• Nova db purge utility: review 132656.
• Online schema change options: review 102545 (approved).
• Support DB2 as a SQL database: review 141097 (fast tracked, approved).
• Validate database migrations and model': review 134984 (approved).

• Migrate the Docker Driver into Nova: review 128753.

• Implement support for FreeBSD networking in nova-network: review 127827.

• Allow volumes to be stored on SMB shares instead of just iSCSI: review 102190 (approved, implemented).
• Instance hot resize: review 141219.

• Add config drive support: review 98930 (approved).
• Pass through flavor capabilities to ironic: review 136104 (approved).

• Add ephemeral disk support to the VMware driver: review 126527 (fast tracked, approved).
• Add support for the HTML5 console: review 127283 (requested a spec exception).
• Allow Nova to access a VMWare image store over NFS: review 126866.
• Enable administrators and tenants to take advantage of backend storage policies: review 126547 (fast tracked, approved).
• Enable the mapping of raw cinder devices to instances: review 128697.
• Implement vSAN support: review 128600 (fast tracked, approved).
• Support multiple disks inside a single OVA file: review 128691.
• Support the OVA image format: review 127054 (fast tracked, approved).

• Add Quobyte USP support: review 138372 (abandoned); review 138373 (approved).
• Add VIF_VHOSTUSER vif type: review 138736 (approved).
• Add a Quobyte Volume Driver: review 138375 (abandoned).
• Add finetunable configuration settings for virtio-scsi: review 103797 (abandoned).
• Add large page support: review 129608 (approved).
• Add support for SMBFS as a image storage backend: review 103203 (approved, implemented).
• Allow scheduling of instances such that PCI passthrough devices are co-located on the same NUMA node as other instance resources: review 128344 (fast tracked, approved).
• Allow specification of the device boot order for instances: review 133254.
• Allow the administrator to explicitly set the version of the qemu emulator to use: review 138731 (abandoned).
• Consider PCI offload capabilities when scheduling instances: review 135331.
• Convert to using built in libvirt disk copy mechanisms for cold migrations on non-shared storage: review 126979 (fast tracked).
• Derive hardware policy from libosinfo: review 133945 (approved).
• Implement COW volumes via VMThunder to allow fast boot of large numbers of instances: review 128810 (abandoned); review 128813 (abandoned); review 128830 (abandoned); review 128845 (abandoned); review 129093 (abandoned); review 129108 (abandoned); review 129110 (abandoned); review 129113 (abandoned); review 129116; review 137617.
• Implement configurable policy over where virtual CPUs should be placed on physical CPUs: review 129606 (approved).
• Implement support for Parallels Cloud Server: review 111335 (approved); review 128990 (abandoned).
• Implement support for zkvm as a libvirt hypervisor: review 130447 (approved).
• Improve total network throughput by supporting virtio-net multiqueue: review 128825 (requested a spec exception).
• Improvements to the cinder integration for snapshots: review 134517.
• Quiesce instance disks during snapshot: review 128112; review 131587 (abandoned); review 131597.
• Real time instances: review 139688.
• Stop dm-crypt device when an encrypted instance is suspended or stopped: review 140847 (approved).
• Support SR-IOV interface attach and detach: review 139910 (requested a spec exception).
• Support StorPool as a storage backend: review 137830.
• Support for live block device IO tuning: review 136704.
• Support libvirt storage pools: review 126978 (fast tracked, approved).
• Support live migration with macvtap SR-IOV: review 136077.
• Support quiesce filesystems during snapshot: review 126966 (fast tracked, approved).
• Support using qemu's built in iSCSI initiator: review 133048 (approved).
• Volume driver for Huawei SDSHypervisor: review 130919.

• Allow portions of an instance's uuid to be configurable: review 130451.
• Allow the resize of ephemeral disks during resize: review 145736.
• Attempt to schedule cinder volumes "close" to instances: review 130851; review 131050 (abandoned); review 131051 (abandoned); review 131151 (abandoned).
• Dynamic server groups: review 130005 (abandoned).
• Improve the performance of unshelve for those using shared storage for instance disks: review 135387 (requested a spec exception).

• A lock-free quota implementation: review 135296 (approved).
• Automate the documentation of the virtual machine state transition graph: review 94835.
• Fake Libvirt driver for simulating HW testing: review 139927 (abandoned).
• Flatten Aggregate Metadata in the DB: review 134573 (abandoned).
• Flatten Instance Metadata in the DB: review 134945 (abandoned).
• Implement a new code coverage API extension: review 130855.
• Move flavor data out of the system_metadata table in the SQL database: review 126620 (approved).
• Move to polling for cinder operations: review 135367.
• PCI test cases for third party CI: review 141270.
• Transition Nova to using the Glance v2 API: review 84887 (abandoned).
• Transition to using glanceclient instead of our own home grown wrapper: review 133485 (approved).

• Enable lazy translations of strings: review 126717 (fast tracked, approved).

• Add a new linuxbridge VIF type, macvtap: review 117465 (abandoned).
• Add a plugin mechanism for VIF drivers: review 136827 (abandoned).
• Add support for InfiniBand SR-IOV VIF Driver: review 131729 (requested a spec exception).
• Neutron DNS Using Nova Hostname: review 90150 (abandoned).
• New VIF type to allow routing VM data instead of bridging it: review 130732 (approved, requested a spec exception).
• Nova Plugin for OpenContrail: review 126446 (approved).
• Refactor of the Neutron network adapter to be more maintainable: review 131413.
• Use the Nova hostname in Neutron DNS: review 137669.
• Wrap the Python NeutronClient: review 141108.

• Dynamically alter the interval nova polls components at based on load and expected time for an operation to complete: review 122705.

• A nested quota driver API: review 129420.
• Add a filter to take into account hypervisor type and version when scheduling: review 137714.
• Add an IOPS weigher: review 127123 (approved, implemented); review 132614.
• Add instance count on the hypervisor as a weight: review 127871 (abandoned).
• Add soft affinity support for server group: review 140017 (approved).
• Allow extra spec to match all values in a list by adding the ALL-IN operator: review 138698 (fast tracked, approved).
• Allow limiting the flavors that can be scheduled on certain host aggregates: review 122530 (abandoned).
• Allow the remove of servers from server groups: review 136487.
• Cache aggregate metadata: review 141846.
• Convert get_available_resources to use an object instead of dict: review 133728 (abandoned).
• Convert the resource tracker to objects: review 128964 (fast tracked, approved).
• Create an object model to represent a request to boot an instance: review 127610 (approved).
• Decouple services and compute nodes in the SQL database: review 126895 (approved).
• Distribute PCI Requests Across Multiple Devices: review 142094.
• Enable adding new scheduler hints to already booted instances: review 134746.
• Fix the race conditions when migration with server-group: review 135527 (abandoned).
• Implement resource objects in the resource tracker: review 127609 (approved, requested a spec exception).
• Improve the ComputeCapabilities filter: review 133534 (requested a spec exception).
• Isolate Scheduler DB for Filters: review 138444 (requested a spec exception).
• Isolate the scheduler's use of the Nova SQL database: review 89893 (approved).
• Let schedulers reuse filter and weigher objects: review 134506 (abandoned).
• Move select_destinations() to using a request object: review 127612 (approved).
• Persist scheduler hints: review 88983.
• Refactor allocate_for_instance: review 141129.
• Stop direct lookup for host aggregates in the Nova database: review 132065 (abandoned).
• Stop direct lookup for instance groups in the Nova database: review 131553 (abandoned).
• Support scheduling based on more image properties: review 138937.
• Trusted computing support: review 133106.

• Dynamic Management of Server Groups: review 139272.

• Make key manager interface interoperable with Barbican: review 140144 (fast tracked, approved).
• Provide a reference implementation for console proxies that uses TLS: review 126958 (fast tracked, approved).
• Strongly validate the tenant and user for quota consuming requests with keystone: review 92507 (approved).

• Pacemaker service group driver: review 139991.
• Transition service groups to using the new oslo Tooz library: review 138607.

The OpenStack Foundation has 13,490 eligible voters and our 25% quorum line to allow us to make changes to our outdated bylaws is 3,373. We’ve had a little over 900 voters complete their ballots in the first day. Good progress, but we still have a long way to go.

I know many of my LinkedIn connections are Foundation members and are eligible to vote. Please vote now to help our project bylaws move forward and while you are voting, please consider casting your 8 votes for Kavit Munshi. http://www.openstack.org/community/members/profile/139 #Kavit2015

There are still a LOT (more than 75%) of OpenStack community members who have not voted.  If you are in that number, please cast your ballot.

[NOTE 1/16: We made it!  OpenStack reach quorum at 7pm on Thursday – 20 hours before the polls close]

Have you voted and tired of these messages?  Your bylaws vote (yes or no) means nothing if we do not reach quorum.  So, contact your OpenStack friends and make sure they voted.

If you think you should have gotten a ballot, but can’t find it.  Please contact ‘secretary@openstack.org’ to check it out.

While inactive members have been removed from the system, there are some people whose ballots did not arrive (like mine) and had to ask for it to be resent.

PS: Yes, that is Niki Acosta with me in Boston.

Monday next week – on the 19th of January -, there will be two OpenStack community events hosted at SUSE’s offices in Nürnberg, Germany: The OpenStack Bootcamp starts at 14:30 and is followed by the OpenStack Meetup at 19:00. If you like to join, please use the links for details and register! Anybody interested in …

+read more

The post OpenStack Meetup in Nürnberg appeared first on Conversations. Andreas Jaeger

Well I am happy to say that the election is underway and I have already voted.

Kenneth Hui had an interesting post about how OpenStack Is A Social Contract and how people in the OpenStack community should be rushing to vote.

I had a few thoughts about the change of quorum that is proposed.

• Current bylaws: A majority of the Individual Members voting (but only if at least 25% of the Individual Members vote at an annual or special meeting). This 25% threshold was selected rather arbitrarily at the time the bylaws were drafted and did not anticipate the number of Individual Members which have joined the Foundation. The 2014 election for Individual Directors had participation of less than 15% of the Individual Members.
• Revised bylaws: A majority of the Individual Members voting (but only if at least 10% of the Individual Members vote at an annual or special meeting).

Elections are never easy, but it is the only way to get the opinion of the public and your constituents. OpenStack as a community is no different.

I have seen a growing number of posts or tweets about requesting people to cast their vote. There is a concern in the air (at least that is my personal feeling) that a quorum will not be met. This will have a number of implications on the changes in the bylaws, which could be a problematic for the future growth of the community. I am not going to go into the subject of, if I am in favor of the changes or not – we are all entitled to our own opinions.

Accepting results if only a minimum of 10% of the members of the community participate.

(I would like to clarify – that results of a vote are valid – even if the number of participants are low).

According to Stefano Maffulli’s post – the people who can vote are those who are “active members of the OpenStack Foundation”.  The number of people who vote for the TC elections – is going down. The bylaws define “active members” is defined as a member who has participated in one of the last two board elections.

The number of people who have joined the OpenStack foundation has grown so large that it will be very difficult to achieve a quorum going forward.

If you are interested in participating OpenStack, take the time and make the effort to sign up, then you should also make an effort to exercise your right and vote.

I am just worried about the decreasing numbers and what that says about the community. If the OpenStack community accepts or expects that only having 10% of people will participate in elections and vote on important issues then What does that say about the OpenStack community as a whole?

Why is quorum needed? Who is to say that 10% is acceptable? Why not 15%? Why not 20%? Or for that matter – why not 2%? Who says that 10% is a quorum?

Take a national election for example. What would happen that if for the parliament or president – you needed a quorum? That does not happen. It cannot happen. An election is a democratic process, where each eligible is asked to participate. If they choose not to – then that is their right – but you cannot nullify the whole election – when not enough people vote.

I think that a quorum is the number of people who vote. There should be no minimum.

This does not change the fact that today this has to be changed – because the bylaws today state a number of 25%.

It just seems strange to me.

Lowering the threshold – the question is how low do you go? Instead of having to go through this again in the future – it better to remove that obstacle now than have to beg for people to vote in order to achieve a quorum.

Your thoughts and comments are always welcome.

While playing with Ceph on DevStack I noticed that after several rebuild I ended up with the following error from nova-scheduler:

Secret not found: rbd no secret matches uuid '3092b632-4e9f-40ca-9430-bbf60cefae36'

Actually this error is reported by libvirt itself which somehow keeps the secret in-memory (I believe) even when a new virsh secret is applied. The only solution I have found so far to this issue is to restart libvirt:

1

$ sudo service libvirt-bin restart

The OpenStack Individual Director election is on right now from the 12th to the 16th of January. This is who I’d like to see seated at the table at the first 2015 meeting on January 29:

Tim Bell - If the board ever decides to create permanent Board seats, Tim is the reason for it. Say no more.

Randy Bias - Randy gets cloud, gets business and gets everything and he isn’t afraid to share his wisdom. His observation that OpenStack has a mission but lacks vision that has led to the forming of the Product Workgroup is astute and I predict this Workgroup to become a critical focal point for the community in 2015.

Kavit Munshi - The Indian OpenStack community is MASSIVE and in need of representation. The 3000 people in the meetup group is just a fraction of the actual number working on OpenStack country wide. This makes India's voice crucial to bringing much needed diversity.

Rob Hirschfeld - Rob works hard for OpenStack, especially so for DefCore this past year or so. Rob’s a details guy, and the Board needs him as their details guy.

Monty Taylor - I love the way I can emphatically agree and disagree with Monty, all at once. Monty holds the flag for the dyed in the (organic of course) wool FOSS community, and we’d all dearly miss his theatrical interactions around the Board table.

Alex Freedland - Hard to not have a guy that anchors the most successful OpenStack business ever on the Board. Alex is a consummate businessman and that is always needed on any Board.

Haiying Wang - As well as bringing a significant voice in the telco industry, Haiying adds to the diversity of the board representing the vastness of Asia alongside Kavit.

????????? - I’ll leave the 8th spot open because there are several good candidates that can fill it! GO Vish, Jesse, Egle, Peter, John, Kyle, Richard, Andrew, Kyle, Mark and more!

Cheers,

Tristan

HOW TO VOTE: If you are an eligible voter, you should have received an email with the subject “OpenStack Foundation – 2015 Individual Director Election”from secretary@openstack.org. This email includes your unique voting link. If you did not receive an email, please contact secretary@openstack.org.

If you are an active member of the OpenStack Foundation, you should have received an email in your inbox with a link to vote for renewal of the Board of Directors and to change the Foundation’s bylaws. The changes to the bylaws make this election different from others and require your vote.

During 2014, the OpenStack community created several revisions of the Bylaws as part of the process to define what makes up core OpenStack functionality. The proposed revisions to the Bylaws are meant to provide flexibility in determining the software required for use of the OpenStack trademarks. The revisions also permit the adoption of the “DefCore” procedures and any changes to those procedures in the future. These amendments have already gone through fifteen revisions with comments from the Technical Committee, DefCore Committee, OpenStack Foundation management, Legal Affairs Committee and the Board of Directors.

Now it’s your turn: the change to the bylaws require that least 25% of the eligible voters cast a vote. If the quorum is not reached, the bylaws won’t change and the work of the Foundation will be more complicated.

You have a duty to fulfill and it won’t take much of your time. You can learn more about the amendments. We need your click to reach the quorum to improve the Foundation.

There’s been much talk in recent years about the merits of having a benevolent dictator guide the OpenStack Project a la Linux Torvalds and the Linux Kernel.  While I don’t want to get into the merits of such an approach in this blog post, I do want to call out that in the absence of a benevolent dictator, the health of the OpenStack project relies on the willingness of OpenStack Foundation members to live up to the social contract that implicitly under-girds the project and the community.  A social contract dictates that members of a communities agree to actively come together for the good of the entire community because that ultimately works for the mutual benefit of each member.

That commitment to work together for the common good is what allows users and vendors to contribute code to a project despite sometimes competing priorities and interests.  What is good for OpenStack ultimately benefits all the members and users even if that means our competition benefits as well.  A community built on the premise of a social contract starts to break down if it’s members primarily sees the benefits derived from the project as being a zero-sum game where having winners must require there to be losers.

So how do we begin live up to our obligations as part of the OpenStack social contract?  Clearly, contributions in the form of code, documentation, money, and time are all valuable.  But another way to get involved is by exercising our rights as members to vote on laws to govern the project and to elect representatives.  Doing both enable us to implement processes and to give voice to community members that can strengthen the social contract and ultimately the project and the community.

We are in the midst of the latest voting/election cycle for the Foundation which ends this Friday, 1/16th, 2015.  If you are a member of the OpenStack Foundation have been so prior to 7/20/2014, you have the right and, I would argue, the obligation to vote on some important bylaw changes as well as some new individual members to the OpenStack Board of Directors.  The proposed bylaw changes can be found here and I encourage you to think carefully about the implications of voting yes, no, or abstaining.  In this case, by not voting at all you may be implicitly voting “no” since not having a 25% quorum will effectively defeat any proposed changes.

Another key obligation is the election of representatives to the Board of Directors; details on that election can be found here.  It is of utmost importance that members recognize their obligation to elect board members that can best guide the community towards delivering benefits to all its members.  If members choose to abstain here, at best we may end up with a project that does not accurately represent the goals and needs of its members.  At worst, we may end up allowing small interest groups to gain control of the project who may have their own interests in mind and not that of the wider community.  I don’t believe the latter has happened to date but history tells us that as a community grows and its members becomes less engaged, the possibility of factionalism increases and polarization occurs.  Unless the members take seriously their obligations as part of fulfilling the social contract.

So please make sure you vote.  I am a candidate this year but so are many other worthy candidates.  You can find them all listed here so take a few minutes to look over each candidate’s profile and carefully consider who should represent you.  Please also remember that each member has up to 8 votes that can be dispersed between 1 to 8 candidates, with each candidate receiving some number of votes, determined by the voter, as long as the aggregate amount for all candidates does not exceed 8 votes per voter.

I mentioned that I am running as a candidate this time around.  Below is the profile I submitted to the Foundation.

Kenneth is a candidate in the 2015 Board Election .

1/17 Update: We did it!  We reached quorum and approved all the changes!  Also, I am honored to have been re-elected to the Board.  Thank you for the support.

I saw the latest report and we’ve still got a LONG WAY TO GO to get to the quorum that we need.  Don’t let your co-worker or co-contributor be the one missing vote!

Note: If you thing you should have gotten a ballot email but did not.  Contact the OpenStack Election Secretary for assistance.  OpenStack voting is via YOUR PERSONALIZED EMAIL only – you cannot use someone else’s ballot.

Here’s the official request that we’ve been forwarding in the community

OpenStack Individual Members we need your help – Please Vote!

Included on the upcoming individual elections ballot is set of proposed bylaw changes [note: I am also seeking re-election]. To be enacted, these changes require approval by the individual members. At least 25% of the Individual Members must participate in this election in order for the vote to take effect which is why we are reaching out to you. The election will start Monday January 12, 2015 and run thru Friday January 16, 2015.

The unprecedented growth, community size and active nature of the OpenStack community have precipitated the need for OpenStack Bylaw updates. The updates will enable our community to adapt to our continued rapid growth, change and diversity, while reflecting our success and market leadership. Although the proposed changes only effect a small set of verbiage in the bylaws, the changes eliminate some of the hard coded values and naive initial assumptions that found their way into the bylaws when they were initially created in 2013. Those initial assumptions did not anticipate that by 2015 we would have such a large, active community of over 17,000 individual members, over 430 corporate members, and a large diverse set of OpenStack based products and services.

Through many months of community iterative discussion and debate, the DefCore team and board have unanimously accepted a set of changes that are now placed before you for your approval. The changes replace the original hard coded “core” definition with a process for determining the software elements required for use of the OpenStack commercial trademark. Processes which will also account for future revisions and determinations for Core and Trademark Policy.

Note: Another change sets the quorum level at a more reasonable 10%, so these PSAs should not be required in the future.

Complete details on the proposed changes are located at:
https://wiki.openstack.org/wiki/Governance/Foundation/2014ProposedBylawsAmendment

Complete details on the 2015 Board Election are located at:
http://www.openstack.org/election/2015-individual-director-election/