August 23, 2017

OpenStack Superuser

OpenTracing project profile

As part of our ongoing series spotlighting open-source projects, today we’re talking about OpenTracing.

OpenTracing offers vendor-neutral open standard for distributed tracing. Superuser talks to Priyanka Sharma, OpenTracing contributor and head of product marketing, partnerships and open source at LightStep, about how it got started and how you can get involved.

What’s the origin story of the project?

  • Tracing has been around since the 70s
  • In the industry, Google was one of the pioneers of the practice with Dapper
  • Tracing is the only way see through a distributed system and know where the bottlenecks are
  • Despite that, it’s not widely used because the way to get data, the instrumentation, is arduous.
  • The creator of Dapper, Ben Sigelman, and other industry experts such as Yuri Shkuroof Uber (Jaeger tracing system author), Adrian Cole of Zipkin, got together and built the OpenTracing spec to make instrumentation easier.
  • It became the third hosted project to join the Cloud Native Computing Foundation in October 2016.

Resources to learn more:
https://medium.com/opentracing/towards-turnkey-distributed-tracing-5f4297d1736
https://docs.google.com/presentation/d/13QhsINIXglNTP6jdf_0B0YOSrlRNhwgOMNEp8CQpICc/edit#slide=id.g213efb1672_0_60 (slide 1 – 8)

What’s the user profile? Who benefits most?

Great questions! Overall, this is for people who run distributed systems.

There are three audiences:
Application developers — People working on software systems can use OT to instrument their code base without fear of vendor lock-in and with support from the OSS community because of instrumented projects
OSS project maintainers — Now maintainers can offer their users (the app devs) a way to achieve visibility through their frameworks and libraries without a. Having to pick a vendor, b. Without having to built a tracing solution.
Tracing/performance vendors — Vendors can now onboard customers easier because what was before a hard, intractable task with dubious results is now a simple integration process where customers can switch out different vendors with ease.

What kind of contributions or contributors do you need most?

Anything that expands the footprint of OpenTracing is huge. So people who are willing to instrument various frameworks and libraries are very useful. One piece of advice is: tracing systems are more useful when they are cross languages. So rather than build out your own special snowflake tracer, contribute to the existing ones like Jaeger and Zipkin.

How does your project contribute to the open infrastructure landscape?

This project is critical to anyone who is modernizing their architecture. It brings control and visibility to microservices architectures. As people go from monoliths to services, the operational burden increases exponentially. If the tooling isn’t right from when you have one service, the whole structure can come crashing down. OpenTracing is here to help. And it does so with openness, working with all the open source solutions people might use, in conjunction with the closed source things people use. This means people at any stage of openness can benefit from it.

How does your community collaborate? How can people get involved?

We work through two tools primarily – GitHub and Gitter. We also have a twitter handle for the fun stuff and see each other at conferences and meetups. Best way to get involved is to check out opentracing-contrib and opentracing orgs on GitHub to see coverage and then join Gitter with specific questions and ideas. You can also email us at hello@opentracing.io.

Choose your own adventure: anything we didn’t ask that our readers should know?

Some myths about tracing are:

  • I don’t need it if I have structured logs
  • It’s useless unless my whole system is instrumented
  • I have logging and metrics, so I don’t need tracing
  • I need to be an advanced guru to trace
  • It’s only useful if I build my own special system

Want to understand more? Join our Gitter channel!

The post OpenTracing project profile appeared first on OpenStack Superuser.

by Superuser at August 23, 2017 12:26 PM

OpenStack @ NetApp

Cinder Cheesecake & iSCSI Secondary Devices

By Chad Morgenstern and Jenny Yang Overview In the blog post Cinder Cheesecake: Things to Consider, we wrote that we would be looking at various failure scenarios. Scenarios that build on one another to give a clear view of what happening in the environment.  This blog focuses on scenario 1, namely: Your environment encounters an issue ... Read more

The post Cinder Cheesecake & iSCSI Secondary Devices appeared first on thePub.

by Chad Morgenstern at August 23, 2017 12:00 PM

Opensource.com

Why open source should be the first choice for cloud-native environments

Let's take a trip back in time to the 1990s, when proprietary software reigned, but open source was starting to come into its own. What caused this switch, and more importantly, what can we learn from it today as we shift into cloud-native environments?

by pleia2 at August 23, 2017 07:01 AM

August 22, 2017

Chris Dent

TC Report 34

Once again, a slow week in the world of TC-related activity. Plenty of people are on holiday or still engaged with release-related activity.

So really the only thing to talk about is that everyone who will be attending the PTG or who has concerns they wish to see addressed there should be aware of the canonical list of PTG etherpads. Those are used by the various projects, working groups, SIGs, and committees to build agendas and prepare. If you have concerns, add them. The TC will be sharing a room on Monday and Tuesday with the Stewardship Working Group. There may be topics of interest on that etherpad.

There's also a handy set of PTG quick links.

by Chris Dent at August 22, 2017 07:00 PM

OpenStack Superuser

Looking to the future of edge computing

Edge computing has been called the next big thing. Two upcoming events will get you in front of the industry experts who are already making it happen.

The recently launched schedule for the OpenStack Summit Sydney offers over 100 sessions across 12 tracks that range from telecom and Network Functions Virtualization (NFV) to university and government sessions. Edge computing crops up as a popular topic across multiple tracks at the November 6-8 event.

This emerging cloud use case first took center stage at the OpenStack Summit Boston when Verizon’s Beth Cohen performed a demo showing how telecom giant pushes its network to the edge to serve millions of customers worldwide. In Sydney, Cohen and Daniel Fouladi will push the Verizon edge computing story forward with a follow-up session titled “Massively Distributed OpenStack—Thinking outside the Data Center.”

In addition to Verizon’s use case, a number of service providers, users and industry experts will take the stage at the Sydney Summit:

Interested pushing the knowledge boundaries with experts to advance edge computing use cases? Before Sydney, head to OpenDev, where you can collaborate in working sessions to discuss how to assemble and integrate the components needed to meet edge requirements.

Cover Photo // CC BY NC

The post Looking to the future of edge computing appeared first on OpenStack Superuser.

by Allison Price at August 22, 2017 06:01 PM

Mirantis

Prelude To Operational Simplicity – A Two Act Play

Instead of large integrated releases after every OpenStack Foundation release, MCP DriveTrain enables you to consume some or all of the latest innovation without downtime.

by Nick Chase at August 22, 2017 05:46 PM

RDO

Video interviews at the Denver PTG (Sign up now!)

TL;DR: Sign up here for the video interviews at the PTG in Denver next month.

Earlier this year, at the PTG in Atlanta I did video interviews with some of the Red Hat engineering who were there.

You can see these videos on the RDO YouTube channel

Or you can see the teaser video here:

This year, I'll be expanding that to everyone - not just Red Hat - to emphasize the awesome cooperation and collaboration that happens across projects, and across companies.

If you'll be at the PTG, please consider signing up to talk to me about your project. I'll be conducting interviews starting on Tuesday morning, and you can sign up here

Please see the "planning for your interview" tab of that spreadsheet for the answers to all of your questions about the interviews. Or contact me directly at rbowen AT red hat DOT com if you have more questions.

by Rich Bowen at August 22, 2017 05:41 PM

August 21, 2017

Ed Leafe

Handling Unstructured Data

There have been a lot of changes to the Scheduler in OpenStack Nova in the last cycle. If you aren’t interested in the Nova Scheduler, well, you can skip this post. I’ll explain the problem briefly, as most people interested in this discussion already know these details.

The first, and more significant change, was the addition of AllocationCandidates, which represent the specific allocation that would need to be made for a given ResourceProvider (in this case, a compute host) to claim the resources. Before this, the scheduler would simply determine the “best” host for a given request, and return that. Now, it also claims the resources in Placement to ensure that there will be no race for these resources from a similar request, using these AllocationCandidates. An AllocationCandidate is a fairly complex dictionary of allocations and resource provider summaries, with the allocations being a list of dictionaries, and the resource provider summaries being another list of dictionaries.

The second change is the result of a request by operators: to return not just the selected host, but also a number of alternate hosts. The thinking is that if the build fails on the selected host for whatever reason, the local cell conductor can retry the requested build on one of the alternates instead of just failing, and having to start the whole scheduling process all over again.

Neither of these changes is problematic on their own, but together they create a potential headache in terms of the data that needs to be passed around. Why? Because of the information required for these retries.

When a build fails, the local cell conductor cannot simply pass the build request to one of the alternates. First, it must unclaim the resources that have already been claimed on the failed host. Then it must attempt to claim the resources on the alternate host, since another request may have already used up what was available in the interim. So the cell conductor must have the allocation information for both the original selected host, as well as every alternate host.

What will this mean for the scheduler? It means that for every request, it must return a 2-tuple of lists, with the first element representing the hosts, and the second the AllocationCandidates corresponding to the hosts. So in the case of a request for 3 instances on a cloud configured for 4 retries, the scheduler currently returns:

Inst1SelHostDict, Inst2SelHostDict, Inst3SelHostDict

In other words, a dictionary containing some basic info about the hosts selected for each instance. Now this is going to change to this:

(
    [
        [Inst1SelHostDict1, Inst1AltHostDict2, Inst1AltHostDict3, Inst1AltHostDict4],
        [Inst2SelHostDict1, Inst2AltHostDict2, Inst2AltHostDict3, Inst2AltHostDict4],
        [Inst3SelHostDict1, Inst3AltHostDict2, Inst3AltHostDict3, Inst3AltHostDict4],
    ],
    [
        [Inst1SelAllocation1, Inst1AltAllocation2, Inst1AltAllocation3, Inst1AltAllocation4],
        [Inst2SelAllocation1, Inst2AltAllocation2, Inst2AltAllocation3, Inst2AltAllocation4],
        [Inst3SelAllocation1, Inst3AltAllocation2, Inst3AltAllocation3, Inst3AltAllocation4],
    ]
)

OK, that doesn’t look too bad, does it? Keep in mind, though, that each one of those allocation entries will look something like this:

{
    "allocations": [
        {
            "resource_provider": {
                "uuid": "9cf544dd-f0d7-4152-a9b8-02a65804df09"
            },
            "resources": {
                "VCPU": 2,
                "MEMORY_MB": 8096
            }
        },
        {
            "resource_provider": {
                "uuid": 79f78999-e5a7-4e48-8383-e168f307d098
            },
            "resources": {
                "DISK_GB": 100
            }
        },
    ],
}

So if you’re keeping score at home, we’re now going to send a 2-tuple, with the first element a list of lists of dictionaries, and the second element being a list of lists of dictionaries of lists of dictionaries. Imagine now that you are a newcomer to the code, and you see data like this being passed around from one system to another. Do you think it would be clear? Do you think you’d feel safe proposing changing this as needs arise? Or do you see yourself running away as fast as possible?

I don’t have the answer to this figured out. But last week as I was putting together the patches to make these changes, the code smell was awful. So I’m writing this to help spur a discussion that might lead to a better design. I’ll throw out one alternate design, even knowing it will be shot down before being considered: give each AllocationCandidate that Placement creates a UUID, and have Placement store the values keyed by that UUID. An in-memory store should be fine. Then in the case where a retry is required, the cell conductor can send these UUIDs for claiming instead of the entire AllocationCandidate. There can be a periodic dumping of old data, or some other means of keeping the size of this reasonable.

Another design idea: create a new object that is similar to the AllocationCandidates object, but which just contains the selected/alternate host, along with the matching set of allocations for it. The sheer amount of data being passed around won’t be reduced, but it will make the interfaces for handling this data much cleaner.

Got any other ideas?

by ed at August 21, 2017 08:55 PM

Mirantis

Cloud as the disruptor for organisational change

Cloud and agile are tools used to build a technology business.

by Shaun O'Meara at August 21, 2017 06:00 PM

OpenStack Superuser

How to deploy mariadb on Kubernetes with TripleO

I’ve spent quite some time researching how we can migrate TripleO from deploying OpenStack on baremetal to Kubernetes. This work has been going on for around a year already and it started with a migration from baremetal to Docker. Now that this first migration is almost done, I’ve moved to research how we can do the final migration to Kubernetes.

As in most of our works, we’re striving the least possible, backwards compatible, changes. To do this, I’ve focused on three main areas for now:

  • Unified configuration management: Migrate out of puppet for configuration management and adopt a solution that can be shared across different projects in OpenStack.
  • Re-use of existing data: Don’t require greenfield deployments but be able to consume the existing data – hiera files, basically.
  • Re-use existing templates and libraries: Avoid rewriting all the templates that have been written already for the first, docker based, migration. There are libraries, CLI tools, and API’s that were developed for the first phase that can be re-used in the second one to reduce the amount of work needed.

I’m not planning to go into great detail in this post on what has been done in each area – I’ll do that in future posts – but rather show a small screencast that features the TripleO undercloud command deploying mariadb on Kubernetes.

The code used in this screencast includes ansible-role-l8s-mariadb, and ansible-role-l8s-tripleo. The changes to tripleo-heat-template have not been published yet. I’ll work on that and update this post (you can see the mysql.yaml file in the video, that’s all you need to change).

Percoco currently works at Red Hat in the office of technology from where he serves the OpenStack community as a full-time upstream developer. His interests involve community work and deployment tools. He’s been an elected member of OpenStack’s Technical Committee for the past three years.

Percoco uses the @flaper87 handle on Twitter and writes posts about technology, communities, culture and philosophy on his blog, where this post first appeared: https://blog.flaper87.com

 

 

Cover Photo // CC BY NC

The post How to deploy mariadb on Kubernetes with TripleO appeared first on OpenStack Superuser.

by Flavio Percoco at August 21, 2017 03:30 PM

NFVPE @ Red Hat

Deploying and upgrading TripleO with local mirrors

Continued from http://teknoarticles.blogspot.com.es/2017/08/automating-local-mirrors-creation-in.html In the previous blogpost, I explained how to automate the RHEL mirror creation using https://github.com/redhat-nfvpe/rhel-local-mirrors. Now we are going to learn how to deploy and upgrade TripleO using those. Deploying TripleO Undercloud To use local mirrors in the undercloud, you simply need to get the generated osp<version>.repo that you generated with the rhel-local-mirrors playbook, and copy it to /etc/yum.repos.d/ , in the undercloud host: sudo curl http://<local_mirror_ip>/osp<version>_repo/osp<version>.repo \-o /etc/yum.repos.d/osp.repo Then proceed with the standard instructions for deploy. Overcloud Each node from the overcloud (controllers, computes, etc…) needs to have a copy of the repository file from our…

by Yolanda Robla Mota at August 21, 2017 03:23 PM

Dragonflow Team

Distributed SNAT - examining alternatives

Source NAT (SNAT) is a basic cloud network functionality that allows traffic from the private network to go out to the Internet. 

At the time of writing this blog, SNAT still has no agreed-upon mainstream distributed solution for OpenStack Neutron.  While Distributed Virtual Router (DVR) provides a distributed and decentralized solution to local connectivity for floating IP and simplified east-west VM communication, SNAT still needs to be deployed at a network node and remains a traffic bottleneck.


Figure 1

Figure 1 above has two deployed tenants, 'green' and 'orange'. SNAT is performed centrally at the network node.


Possible solutions

There are a number of proposed solutions to decentralize SNAT. Each solution has its own benefits and drawbacks. I am going to dive into two possible solutions that have been recently implemented in Dragonflow SDN Controller.


1. SNAT per {tenant, router} pair

The most straightforward solution is to perform SNAT at compute node router instance. 

However, while DVR deployment can easily copy the internal subnet router address across compute nodes, the router's IP address on the external network can not follow this scheme. Such a deployment will consume extra external address per {tenant, router} pair.


Figure 2

Maximum address consumption equals to:
   [# of compute nodes] x [# of tenants]

This problem may be somewhat mitigated by allocating the external IP in a lazy manner - only when the first VM of the requested tenant is deployed on a compute node that is scheduled by Nova.  In figure 2 external address 172.24.4.2 was allocated only when VM2 of 'orange' tenant was deployed .

This model may be appealing for cloud deployments that have a very large pool of external addresses, or deployments going through an additional NAT beyond the cloud edge. 

However, the Neutron database would need to track all additional gateway router ports and external addresses would need to be assigned implicitly from a separate external address pool. 

A proof of concept implementation for this SNAT model base on Neutron stable/mitaka branch can be found here and is being discussed in this post. This implementation makes a few assumptions that would need to be removed in a future enhancement round, such as:
  • Explicit external address allocation per {router, compute node} pair that required client API modification instead of automated allocation.

2. SNAT per compute node

This second SNAT model we discuss reduces the number of external addresses to a single one per compute node and significantly optimizes network resources consumption, while improving latency and bandwidth of internet-bound traffic.

This model has at least one caveat - When several tenant VMs go out to the internet via the same IP, one tenant abusing an external service (e.g. gmail, fb) may cause blacklisting of the shared external IP, thus affecting the other tenants who share this IP. 

(This problem can be somewhat mitigated by only allowing this capability for "trusted" tenants, while leaving "untrusted" tenants to go via the SNAT node using their pre-assigned external IP).

Figure 3

In figure 3 we can see that the SNAT rule implemented by both tenants is masquerading multiple tenant VMs behind a single external address. On the returning traffic, reverse NAT restores the tenant IP/MAC information and ensures packets return to their rightful owners. 

In order to simplify the visualization of flow, figure 3 shows two routing entities.  In reality this translation could (and probably would) be performed by different routing tables within the same routing function.


Dragonflow implementation

SNAT per compute node model was proposed and implemented in quite elegant manner within the Dragonflow service plugin of Neutron. 

For those of you who are not familiar with its design, Dragonflow runs a tiny local controller on every compute node, that manages the local OVS and feeds off of a central database and a pub/sub mechanism.  The basic control flow is shown on figure 4 below.

Dragonflow uses OVS (Open Virtual Switchas its dataplane implementation, and controls its bridges using OpenFlow. The OVS bridges replace the native Linux forwarding stack. 
Figure 4
This is what happens when Nova requests Neutron to allocate a network port for a VM:
  1. Neutron server writes the new port information in its database and passes port allocation request to the ML2 Plugin
  2. The Dragonflow ML2 Driver writes the newly-created Neutron port information into its separate Dragonflow database (not the Neutron DB)
  3. The Dragonflow ML2 Driver then publishes a port update to the relevant compute nodes, where Dragonflow local controllers are running, using its own pub/sub mechanism 
  4. The Dragonflow local controller running on the compute node where the VM is scheduled for creation fetches the port information from the Dragonflow database or the published port update event, and passes it to the Dragonflow applications
  5. Every application that is registered for this specific event (local neutron port created) may insert/update OVS flows. For instance, the L2 application adds an OVS flow to detect the port's MAC address, and marks the packet to be sent to the relevant port attached to the OVS bridge.
  6. The new SNAT application installs a flow that uses OVS's NAT and connection tracking component to NAT, unNAT, and track the NATed connection. These features are available starting from OVS version 2.6.

SNAT application configuration

When the new SNAT application is enabled, Dragonflow's configuration has to be modified to reflect the host's 'external IP', i.e. the masquerading address of NATed traffic.

figure 5

In figure 5 we see the minimal configuration required to enable the SNAT application in Dragonflow:
  • Add ChassisSNATApp to Dragonflow application list (apps_list)
  • Configure proper external_host_ip address


Questions and comments are welcome.

Useful resources:


by Igor Shafran (noreply@blogger.com) at August 21, 2017 12:38 PM

Policy based routing with SFC in DragonFlow


One of the coolest new Pike release features in Dragonflow is support for Service Function Chaining. In this post I'll give a short intro on the topic and share some details on how we implemented it, and what it's good for.

A quick intro

A network service function is a resource that (as the name suggests) provides a service, which could be an IDS, a Firewall, or even a cache server (or anything else that works on the network data path).

In a traditional network environment, packets are forwarded according to their destination, i.e. when Server A wants to send a packet to Server B, it puts Server B's address on packet destination field.  That way, all the switches between the servers know how to forward the packet correctly.




Now consider that you want to steer this traffic through an IDS and a Firewall. Server A will still put Server B as the destination for its packets.  
One popular way to accomplish this is to place A and B within different subnets. This will allow us to use a router to route the packets through our IDS and firewall.




However, such an approach complicates the network quite a bit, requiring each 2 communicating servers to be placed within separate subnets, and causing all their traffic to go through routers (slower and more expensive).  Moreover, all packets will be routed the same way, even if you want to only apply IDS on HTTP traffic.  This headache scales with the number of servers, and it can quickly become a configuration hell.


In the SDN world we should be able to do better


SFC introduces the concept of service chains. 

There are two aspects to a service chain:


Classification

What traffic should be served by the chain.  For example, outbound TCP connections with destination port 80 from subnet 10.0.0.0/16


Service path

Which service functions (and in what order) should be applied to the packet.  For example, a firewall, then IDS, then local HTTP cache

    With this in mind, a forwarding element is enhanced to handle service function chains, so everything can be deployed in a more intuitive way:





    SFC in OpenStack

    OpenStack Neutron supports SFC through the networking-sfc extension. This extension provides a vendor-neutral API for defining service function chains. 

    The basic model is composed of 4 object types:

    PortChain

    Represents the whole service function chain.  It is composed of FlowClassifiers, and PortPairGroups where the former specifies the subset of traffic for which this port chain applies, and the latter specifies what service functions need to be applied.

    FlowClassifier

    Specifies what packets should enter the specific chain.  The classification is done by matching against packet's fields.  Some of the fields that can be specified are:

    • Source/destination logical ports
    • IP(or IPv6) source and dest CIDRs
    • Protocol types and port numbers
    • L7 URLs

    PortPairGroup

    Represents a step in the service function chain.  The model aggregates all port pairs that can be used to perform this specific step.

    PortPair

    Represents a service function instance.  It specifies what port we need to forward our packet into to apply the service and what port the resulting packet will emerge at.


      TCP/80 egress traffic of Server A will go through the port chain above. Blue arrow shows possible path of classified traffic, red shows path of not classified traffic.


      What goes on the wire

      We solved all our issues a few paragraphs above by adding a mysterious SFC forwarder element.  How does it make sure that packets traverse the correct path? 
      Usually, packets that need to be serviced by a service function chain are encapsulated and a service header is added to the packet:




      The service header is used to store information needed to steer the packet along the service chain (usually, what chain is performed, and how far along the chain are we). Two of the trending choices for service protocols are MPLS and NSH.

      With this metadata on the packet, the forwarder can easily decide where packet should be sent next. The service function themselves will receive the packet with the service header and operate on the encapsulated packet.


      A packet classifier at SFC forwarder. If the service function supports service headers, the packet is sent in encapsulated form (right). If the service function does on support service headers, a proxy must be used.

      The left side of the above figure depicts service protocol unaware function, a function that expects ingress packets to be without any encapsulation. Dragonflow's forwarding element will act as proxy when the function is service unaware.

      Dragonflow drivers

      In Pike release we have added SFC drivers to Dragonflow, the drivers aim to implement the classification and forwarding elements. The initial version supports:
      • MPLS service chains (the only protocol supported by networking-sfc API)
      • both MPLS aware and unaware service functions

      In Dragonflow, we manage our own integration bridge to provide various services in a distributed manner. We implemented service function chaining in a as such. Each Dragonflow controller is a fully capable SFC forwarding element, so a packet does not need to travel elsewhere, unless the service function itself is not present on the current node.

      Take SFC for a spin

       Easiest way to get a working environment with Dragonflow + SFC is to deploy it in a devstack.  this is the local.conf I used to deploy it:


      [[local|localrc]]
      DATABASE_PASSWORD=password
      RABBIT_PASSWORD=password
      SERVICE_PASSWORD=password
      SERVICE_TOKEN=password
      ADMIN_PASSWORD=password

      enable_plugin dragonflow https://github.com/openstack/dragonflow
      enable_service q-svc
      enable_service df-controller
      enable_service df-redis
      enable_service df-redis-server
      enable_service df-metadata

      disable_service n-net
      disable_service q-l3
      disable_service df-l3-agent
      disable_service q-agt
      disable_service q-dhcp

      Q_ENABLE_DRAGONFLOW_LOCAL_CONTROLLER=True
      DF_SELECTIVE_TOPO_DIST=False
      DF_REDIS_PUBSUB=True
      Q_USE_PROVIDERNET_FOR_PUBLIC=True
      Q_FLOATING_ALLOCATION_POOL=start=172.24.4.10,end=172.24.4.200
      PUBLIC_NETWORK_NAME=public
      PUBLIC_NETWORK_GATEWAY=172.24.4.1

      ENABLED_SERVICES+=,heat,h-api,h-api-cfn,h-api-cw,h-eng

      ENABLE_DF_SFC=True
      enable_plugin networking-sfc git://git.openstack.org/openstack/networking-sfc

      IMAGE_URL_SITE="http://download.fedoraproject.org"
      IMAGE_URL_PATH="/pub/fedora/linux/releases/25/CloudImages/x86_64/images/"
      IMAGE_URL_FILE="Fedora-Cloud-Base-25-1.3.x86_64.qcow2"
      IMAGE_URLS+=","$IMAGE_URL_SITE$IMAGE_URL_PATH$IMAGE_URL_FILE


      Once stacking is done, Neutron will have networking-sfc extension enabled with Dragonflow drivers, and Dragonflow controller will have SFC apps loaded.

      The above config also enables Heat, so the following stack template can be deployed: https://review.openstack.org/#/c/488987/

      This stack brings up 2 VMs, one containing a UDP echo server and another containing a service function that intercepts return traffic and doctors it:






      To be continued...

      We have done some, but we certainly haven't done all, some of the things that can make Dragonflow's SFC even better:
      • Support for Network Service Header protocol - NSH is a new protocol designed specifically for service function chaining. It features fields designed for SFC and several kinds of metadata that can be used to pass information between forwarders.
      • Allow pluggable protocol drivers - While MPLS and NSH are great, many other protocols might fit the bill. We will make driver loading dynamic so anyone can implement their own protocol driver independently.
      •  Graph API - a new extension to networking-sfc's API. Chains are nice, but what if you want to have branching in the middle? The new Graph API allows specification of what chain to follow.
      • Move to Common Classification Framework - A Neutron wide effort to use uniform classification objects among different extensions.

      by Dima Kuznetsov (noreply@blogger.com) at August 21, 2017 12:35 PM

      August 18, 2017

      RDO

      Ada Lee: OpenStack Security, Barbican, Novajoin, TLS Everywhere in Ocata

      Ada Lee talks about OpenStack Security, Barbican, Novajoin, and TLS Everywhere in Ocata, at the OpenStack PTG in Atlanta, 2017.

      by Rich Bowen at August 18, 2017 07:54 PM

      SUSE Conversations

      Zapraszamy na Letnią Akademię SUSE – Od zera do bohatera

      Zwykle w czasie wakacji w dziale IT jest nieco mniej pracy. Warto więc to wykorzystać i poszerzyć swoją wiedzę o najważniejszych obecnie rozwiązaniach open source podczas drugiej edycji Letniej Akademii SUSE.  W tym roku będziemy na niej mówić o konteneryzacji (Kubernetes/MicroOS/Salt), wdrażaniu chmur (OpenStack) i pamięci masowej zdefiniowanej programowo (Ceph), zapewnieniu ciągłości działania (Live Patching) …

      +read more

      The post Zapraszamy na Letnią Akademię SUSE – Od zera do bohatera appeared first on SUSE Blog. Rafal Kruschewski

      by Rafal Kruschewski at August 18, 2017 06:46 PM

      OpenStack Superuser

      Updates from the OpenStack Ops Midcycle

      MEXICO CITY — It was great spending time with the OpenStack community in Mexico City. Our host venue — Centro de Cultura Digital — did not disappoint with an interactive light wall, multi-purpose spaces and a theater seating up to 120 where we held our sessions.

      As is normally the case with the Ops Midcycles, we had a good mix of users and developers which made for very interesting and interactive discussions. A full list of the Etherpads used for the MidCycle can be found here: https://etherpad.openstack.org/p/MEX-ops-meetup.

      We asked some folks who moderated sessions to comment on what stood out during the discussions:

      Chris Morgan of Bloomberg

      Documentation of older versions of OpenStack
      I run an older version of OpenStack (nine production-quality Liberty clusters) and I still need the documents for it, I wanted to share our problems and see who else is in the same boat. We have a full-time contractor working for Bloomberg from Canonical who investigated this and it turns out that the commonly given answer for old documents (“just build from tag”) doesn’t work well at all. I explained that we’re pushing for a better solution for the entire community and that work is still in flight. For example, there’s some acceptance of keeping documentation longer in future, but distinguishing it better from the latest via use of watermarks, for example. We may be able to keep the Mitaka onwards documents longer  — they haven’t been taken down yet.

      Sampath Priyankara of NTT Communications

      Extreme testing

      Since this is a quite new project, it took me most of the session to explain why and what we are doing in extreme testing. We had a good discussion on what we should include in the testing and what other aspects to consider. I started a follow-up ML thread — http://lists.openstack.org/pipermail/openstack-dev/2017-August/121104.html — to continue this discussion and getting good response.

      Robert Starmer of Kumulus Technologies

      OpenStack on containers:
      Folks are interested in seeing a Kubernetes solution, but in the group, most had only dabbled with some container ops, and saw the value in at least containerization. Some in the group were still happy with, and continuing with the LXC based approach (OpenStack Ansible, or similar), and others are moving on with Kolla (a subset have tried it, a smaller subset have it in production). All saw the value of this mechanism, and those that aren’t on containers would like better guidance on how to migrate from bare metal deploy to container deploy (which was another session)

      Containers on OpenStack:
      Still massive confusion on what networking to use, what works, how the pieces are all to come together. Most are not aware if their customers are deploying containers _on_ their OpenStack systems (i.e., there’s no integration between the two domains). Some are investigating how one might manage container deployments (specifically Kubernetes) on OpenStack, and what integration makes sense (without RBAC in K8S that maps to Keystone, that doesn’t integrate well), Cinder -> K8S PVs works, LBaaS->k8s loadbalancer works, but again, what’s the integration model (is there even LBaaS available, etc.) Magnum is a no-op for the group, Zun is outside of most of the groups scope/interest. Almost no one had even heard of Kuryr (and the “right” network model didn’t seem to include Neutron anyway, or so I understood from a couple folks in the group).

      Shintaro Mizuno of NTT Communications

      Telco/NFV

      NTT and AT&T shared their experiences on operating VNF in production. Most of the challenges come from the fact that the VNF(i.e. vEPCs) are monster applications, which were brought straight from legacy hardware, and requires full hardware performance. Technical challenges are around SR-IOV, OVS-DPDK, CPU pinning and NUMA placement which is to get performance VNF requires. If we have enough NFV operators in the future Meetups, we can discuss these issues in more detail. NTT is working on upstreaming NFV related features into Blazar (non-official project). Ops Telco/NFV WG is discussing to merge with LCOO WG.

      Sean McGinnis of Huawei

      TC feedback

      I moderated to talk about the role of the Technical Committee and stimulate discussion around needs and issues. We answered some questions about what the TC does and does not do. The session ended up being mostly a discussion about having LTS releases. I really pushed that this is something that Red Hat, Canonical, Suse and Huawei need to collaborate to support. The biggest asks were to help skip level upgrades and longer support for releases.


      These are just a few of the sessions we had during the MidCycle. Overall, we had a great time and look forward to bringing these discussions to the community and furthering the work of OpenStack.

      We greatly appreciate our sponsors Bloomberg, Centro de Cultura Digital, Nearsoft, Nearshore, Suse, Meltsan and Sentinel.

      We look forward to seeing as many of you as we can during the next midcycle — March 7–8 2018 in Tokyo, Japan. Please remember to visit #openstack-operators IRC channel on Tuesdays @ 1400UTC and/or join the OpenStack Operators mailing list — http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators — to stay updated.

      About the author

      Melvin Hillsman is a self-taught tech enthusiast currently living in Houston, Texas. He has spent a number of years working as a Linux system administrator and now enjoys learning more about and working with open source communities. As a member of the OpenStack User Committee, he spends most of his days looking for opportunities to grow the user base of the OpenStack community. His love for technology, culture, and people has afforded him the opportunity to travel to many countries speaking about OpenStack and striving to understand and break down barriers to collaboration between those who develop open source software and those who use it.

       

      Superuser wants to hear from operators for our ongoing interview series, get in touch at editorATopenstack.org.

      The post Updates from the OpenStack Ops Midcycle appeared first on OpenStack Superuser.

      by Melvin Hillsman at August 18, 2017 01:39 PM

      Assaf Muller

      Octavia Developer Wanted

      I’m looking for a Software Engineer to join the Red Hat OpenStack Networking team. I am presently looking to hire in Europe, Israel and US East. The candidate may work from home or from one of the Red Hat offices. The team is globally distributed and comprised of talented, autonomous, empowered and passionate individuals with a healthy work/life balance. The candidate will work on OpenStack Octavia and LBaaS. The candidate will write and review code while working with upstream community members and fellow Red Hatters. If you want to do open source, Red Hat is objectively where it’s at. We have an institutional culture of open source at all levels and this has a ripple effect on your day to day and your career at the company.

      Please email me CVs at assaf@redhat.com.

      The ideal candidate is familiar with some or all of the following subjects:

      • Networking knowledge and terms such as L2, L3, ARP and IP
      • Cloud networking knowledge. For example VXLAN tunneling, network namespaces, OVS and haproxy
      • Familiarity with virtualization technology, cloud and infrastructure as a service and OpenStack in particular
      • Python
      • Bonus points for familiarity with Octavia

      Responsibilities:

      • Write a bunch o’ code
      • Review code
      • Resolve bugs
      • Draft design documents
      • Implement features
      • Lead and participate in design discussions
      • Attend conferences
      • Improve our testing infrastructure
      • RPM packaging
      • Resolve customer issues

      Required skills:

      • Bachelors Degree in Computer Science or equivalent
      • 3 years of significant software development experience
      • Excellent English verbal and written communication skills
      • Comfortable communicating and collaborating with upstream community members outside of the team and company

      by assafmuller at August 18, 2017 11:33 AM

      August 17, 2017

      NFVPE @ Red Hat

      Ratchet CNI — Using VXLAN for network isolation for pods in Kubernetes

      In today’s episode we’re looking at Ratchet CNI, an implementation of Koko – but in CNI, the container networking interface that is used by Kubernetes for creating network interfaces. The idea being that the network interface creation can be performed by Kubernetes via CNI. Specifically we’re going to create some network isolation of network links between containers to demonstrate a series of “cloud routers”. We can use the capabilities of Koko to both create vEth connections between containers when they’re local to the same host, and then VXLAN tunnels to containers when they’re across hosts. Our goal today will be to install & configure Ratchet CNI on an existing cluster, we’ll verify it’s working, and then we’ll install a cloud router setup based on zebra pen (a cloud router demo).

      by Doug Smith at August 17, 2017 07:30 PM

      OpenStack Superuser

      Helping global community members participate, wherever they are

      Sometimes things just don’t work out, particularly when the “thing” is juggling schedules and international travel. For the OpenStack Cinder team, this meant the palpable absence of a team member at a PTG or Forum session, until the Cinder AV Club rolled into town to save the day!

      Many of us remember our AV Club from high school–maybe you were even on it! If you don’t, the audiovisual club was the group of geniuses who could wire anything, anywhere, anytime. Need a sound system? Call the AV Club! Broken projector? AV Club!

      Here the Cinder AV Club shares their audiovisual setup for keeping their team connected when a team member can’t make a PTG, Forum or working session:

      The hardware

      It’s a pretty simple setup. We started with just one webcam and a bluetooth speaker we passed around, but now we have a tripod for the webcam and two bluetooth speakers to capture the conversation from more angles. At the last PTG, we figured out how to make a composite of all the sound inputs to go with the one video input. Mac’s operating system actually had a tool- Audio MIDI Setup- to take multiple audio inputs and make them into a single output. This ensures that most people in the room are covered by one of the microphones and that folks listening in should be able to hear everyone.

      The video

      We use both Google Hangouts and YouTube. From the webcam we are connected to a Google Hangout that is live streamed to YouTube. This way people that want to speak up and more actively participate can join the Hangout, which we post links to in the #openstack-cinder IRC channel and link to in our Etherpad. If people would rather just follow along with the discussion, they can watch the YouTube live stream.

      Setting up the live stream can be tricky due to the constant changes to the YouTube Creator Studio, but don’t be intimidated! The Creator Studio offers an option to live stream. From there you can customize the thumbnail and video description. After that you need to set up an encoder before you can stream––Mobile Live, airServer, or something similar. Once the encoder is started your live stream is ready to go! You can get the URL to the Hangouts and your stream on YouTube to share with virtual attendees.

      A few times we have had to work around people not being able to participate with Hangouts. On those occasions, we have had them call in via Zoom or something similar and then put the microphone close to that laptop––it’s almost like they are in the room, too.

      The social media

      We try to remember to tweet about what the next topics are with the links to the live stream every time we get started. The idea is that people can join or start watching the stream if they find the topics interesting. All you have to do is follow @knelson92 or @jungleboyj––we’re both pretty active Twitter users when it comes to OpenStack events!

      The Etherpads

      As with many projects, we brainstorm and plan discussion beforehand in an Etherpad. Within a day or so of discussions starting, it gets more organized into an agenda. We take into account who is leading the conversation, if they are remote or not, what timezone they are in if they are remote, if we need to coordinate with another project for larger multi-project discussions, etc.

      More important than the brainstorming Etherpad are the note taking Etherpads. Each day we start a new Etherpad to take notes in. This way people can more easily follow along and add their thoughts if they can’t join the Hangout or don’t feel comfortable speaking up. We make sure to display the notes and comments people are making during discussions so that it’s easier to rope people into the conversation and make sure everyone is heard.

      The way we take notes is also important. Towards the end of discussions we try to formulate an opinion or action plan. Seeing it written down often helps everyone get on the same page while giving people the chance to ask questions about things they don’t quite agree with or might not understand. By writing down action items at the end of conversations, it’s easier to recap after the event is over and follow up with people and their to-do lists.

      The follow up

      A few days after the event we circle back to the Etherpads to collect all the to-dos into a single place and post it to the mailing list. Everyone knows what they have to do and if there is something they are interested in tracking they know who’s working on it [1].

      The last thing we do is update the titles and descriptions of our YouTube videos. We add the Etherpad links to the descriptions so people can see the accompanying notes. We also update the titles to cover the topics discussed in the video.

      Continuous improvement

      With each event, we improve. Whether it’s adding one more speaker/microphone or taking better notes, we’re constantly looking for ways to make things more accessible to those that can’t attend. There are still many hurdles to overcome, but hopefully by teaching people what we do they can help us improve as well––a true community effort!

      If you’re interested in doing this for your remote team and get stuck somewhere along the line, feel free to tweet at either of us (or ping us on IRC– diablo_rojo or jungleboyj) and we’ll try to help!

      [1] https://wiki.openstack.org/wiki/CinderPikePTGSummary

      About the authors

      Kendall Nelson is an upstream developer advocate at the OpenStack Foundation based in Seattle, WA. She started working on Cinder and os-brick in the Liberty release and has since gotten involved in StoryBoard, the Women of OpenStack (WoO), WoO Mentoring, and OpenStack Upstream Institute.

      Jay Bryant is the cloud storage lead for the Lenovo Cloud Technology Center and has been an active member of the OpenStack community (primarily Cinder, Upstream Institute and documentation) since 2013.

      Superuser is always interested in community content, get in touch at editorATopenstack.org

      Cover Photo // CC BY NC

      The post Helping global community members participate, wherever they are appeared first on OpenStack Superuser.

      by Kendall Nelson and Jay Bryant at August 17, 2017 05:01 PM

      StackHPC Team Blog

      John Garbutt Joins our Team

      We are excited to announce our star summer signing: John Garbutt comes to us from Rackspace and the OpenStack Innovation Center (OSIC). John is also active as a Nova core and sits on the OpenStack Technical Committee. We are thrilled to have John's input from both of these vital areas.

      John adds "I'm really excited to join the talented StackHPC team. Looking forward to the part we will play in shaping how the Scientific Computing community does their computing."

      Aside from drawing on John's depth of technical input and experience, our new team member will be helping us to drive forward our ambitious plans for the development of Scientific OpenStack. In the coming months we hope to provide further details on what we will be getting up to together.

      Follow John on Twitter @johnthetubaguy.

      John Garbutt

      by Stig Telfer at August 17, 2017 09:20 AM

      OpenStack Blog

      User Group Newsletter August 2017

      Sydney Summit

      Don’t miss out on your early bird tickets, sales end September 8 at 11:59pm PT . Find all you need to know in this Summit guide. 

      It includes information about where to stay, featured speakers, a Summit timeline and much more.

      An important note regarding travel. All non-Australian residents will need a visa to travel to Australia (including United States citizens). Click here for more information

      Final chance to complete the User Survey!

      If you haven’t had a chance to complete the OpenStack User Survey, the deadline has been extended to this Friday, August 18. If you’re running OpenStack, this is your chance to anonymously share feedback.

      Fill out the survey here. 

      OpenDev

      Interested in Edge Computing? Join us for OpenDev, a new collaborative working event at DogPatch Studios in San Francisco, September 7-8. Tickets are just $100 for the two-day event, get yourself registered here!

       

      The event will feature real world Edge use case demos from Verizon, AT&T and eBay in addition to updates from industry pioneers including Satya from Carnegie Mellon who’s doing cutting “edge” research. Program committee includes Beth Cohen (Verizon) Kandan Kathirvel (AT&T), Chris Price (Ericsson), Andrew Mitry (Walmart) and Gregory Katsaros (Inmarsat). Participating open tech groups include OpenStack, Open Edge Computing, ETSI MEC, OPNFV, Kubernetes and others.

       

      As cloud architectures continue to evolve, compute is being pushed to the Edge in telecom, retail, research and other industries. This represents a huge opportunity for open infrastructure, so we’re meeting to talk about how to assemble and integrate the components needed to meet Edge requirements. The event is designed to facilitate collaboration at a technical level with productive, working group discussions, in addition to lightning talk presentations on Edge use cases. There will be a range of sessions that are collaborative, led by moderators who dive into specific topics. View the schedule line-up here.

       

      Questions? Contact events [at] openstack.org.

      OpenStack Days

      Check out the upcoming OpenStack Days around the globe! See the full calendar here.

      Great news! The OpenStack Upstream Institute will be running at the OpenStack Days UK and OpenStack Nordic events. 

       

      CFP closing soon!

       

      • CloudNativeCon/KubeCon
      • OpenStack Days Canada

       

       

      Marketing Portal Content

      There is some fantastic OpenStack Foundation content available on the Marketing Portal.

      This includes materials like

      • OpenStack 101 slide deck
      • 2017 OpenStack Highlights & Stats presentation
      • Collateral for events (Sticker and T-Shirt designs)

       

      Latest from Superuser

       

      Some fantastic articles have been published in the last few weeks, with some featuring you, the user groups! Check them out below:

      OpenStack’s 7th Birthday

       

      OpenStack User Group spotlight: Phoenix

       

      OpenStack User Group spotlight: Canada

       

      How to navigate new cultures in an open, diverse community

       

      OpenStack User spotlight: CERN

       

      Learn OpenStack: A new community site

       

      Contributing to the User Group Newsletter.

      If you’d like to contribute a news item for next edition, please submit to this etherpad.

      Items submitted may be edited down for length, style and suitability.

      by Sonia Ramza at August 17, 2017 08:34 AM

      August 16, 2017

      OpenStack Superuser

      How to navigate new cultures in an open, diverse community

      Gender and race are not the only indicators of how diverse a community is. There are other areas where communities should also strive for diversity that will bring tons of benefits to it. A good example of this is culture.

      Without going into details on what cultures are, let’s say, for the sake of brevity, that culture is the way people do things. We react in different ways to different scenarios. The way we react is driven by our culture and our personality.

      Getting to know new cultures is one of the reasons why I love (and do) traveling so much. You can read about cultures and learn a lot without traveling, but I believe the best way to know how to be respectful and how to interact with other cultures is by going into their environment. Whether the trip is for a conference, customer meeting, team meeting or just leisure, it is always a great opportunity to learn more from other cultures and bound with them.

      I believe being familiar and including other cultures is critical for any community and for our own growth as professionals. Being familiar with other cultures is not just about knowing how to interact with other people, it is about being able to create an environment where people feel comfortable. Without such environment, you (and your community) lose the power that a cultural-diverse team has.

      Cultural diversity brings in an enormous set of skills that can be combined depending on the task at hand. It brings the power to explore different roads when implementing new features or addressing issues, it brings different perspectives to the prioritization of tasks, it brings in a more controllable space where conversations can be had. Let’s go through some examples.

      Be aware of cultures that prefer top-down management

      Some cultures are used to a top-down kind of management. If you look at how members of these cultures interact in the community, they tend to follow the lead of others and do things that are considered important by the majority of the members. These members would normally not aim for leadership positions unless someone encourages them.

      Some cultures are individualist and others are collectivist

      Not everyone is used to working in teams. Some folks are used to working by themselves and this is, more often than not, dictated by their own culture. This doesn’t mean individualist cultures won’t work towards a common goal. They will but they won’t necessarily do it with the rest of the team.

      Some cultures don’t like confrontation

      I think we’d all prefer to have less confrontation in our lives but there are cultures that simply avoid it altogether. For some cultures, even simple arguments on technical topics can be considered confrontations.

      You’ll find folks from these cultures to be often agreeable even when they disagree. What motivates this may be respect, fear, or other cultural barriers like language. What really matters is that we might have to try other ways of communication if we want folks from these groups to speak up.

      Emotions don’t mean the same thing in every culture

      Everyone reacts to emotions in different ways but sometimes it’s also driven by culture. Frustration, failure, or making simple mistakes could mean something completely different depending on the culture, which causes different reactions.

      Cultures, however, are just one more thing we should strive to be diverse on but there’s really a lot more. There are also languages, geographic locations, time zones, etc. We should strive for anything that brings a different perspective to our community.

      Ultimately, we have to accept that we’re ignorant on many things. We have to listen to folks from these other groups for us to be able to make our community a nice environment for them. Furthermore, we have to be open to, sometimes, trade our own values. Want to join?

      To learn more:

      About the author

      Flavio Percoco holds multiple passports, speaks three language (outside programming and dialects), has visited more than 40 countries, stayed for >=1 month in at least 10 of the countries he has visited and lived for years in two out of these 10 countries. His passion for cultures guides his travels, relationships and even his tattoos.
      Percoco currently works at Red Hat in the office of technology from where he serves the OpenStack community as a full-time upstream developer. His interests involve community work and deployment tools. He’s been an elected member of OpenStack’s Technical Committee for the past three years.

      Percoco uses the @flaper87 handle on Twitter and writes posts about technology, communities, culture and philosophy on his blog: https://blog.flaper87.com

       

      Cover Photo // CC BY NC

      The post How to navigate new cultures in an open, diverse community appeared first on OpenStack Superuser.

      by Flavio Percoco at August 16, 2017 02:06 PM

      David Moreau Simard

      What's coming in ARA 1.0 ?

      Not long ago, I wrote that ARA: Ansible Run Analysis had it’s first birthday. It was an important milestone and it was a great opportunity to reflect back on where the project was coming from and think about what we needed to do in the future. Just for fun, let’s look at what I had written back in May to summarize what was probably coming: Python 3 compatibility This is done and was shipped in ARA 0.

      August 16, 2017 12:00 AM

      August 15, 2017

      OpenStack Superuser

      Growing allies in the OpenStack community

      The Women of OpenStack (WOO) recently launched a call to grow the number of allies. One of WOOs most active members Amy Marrich considers herself a jack-(or should that be jill?) of all trades: she’s currently an instructor at the Linux Academy and acted as director of operations as well as senior dev ops engineer. You can find out more about her on LinkedIn, Twitter or find her on IRC at spotz. Superuser talked to her over email about who should become an ally and what it entails.

      What’s your role (roles) in the OpenStack community?

      I’m kind of a jack-of-all trades. I’m a core reviewer for OpenStack Ansible, one of the Upstream Institute mentors and instructors at the Boston Summit, I’ve been one of the Women of OpenStack mentors, was a member of the Diversity Working Group, I’ve led the Git and Gerrit lunch and learns for the past five summits and currently I’m a track chair and serving as one of the officials for the User Committee elections.

      What inspired you to become so active in WOO?

      The Women of OpenStack is a great group — it’s a mix of all aspects of the OpenStack community. There’s people from different projects as well as different skill sets all with a love for OpenStack. I feel it’s important to cultivate diversity in any technical community and the Women of OpenStack works hard to bring new people on board and not just into our group but into the community as a whole. When I first joined the group we had some really strong women who have since retired or moved on from the project and I’m just doing my part to fill their shoes.

      Let’s talk about the WOO recruitment efforts — who makes a good ally and why?

      A good ally is anyone who is supportive and believes women are just as capable as men with technology. I know everyone has heard about that internal memo from Google on gender differences. This is exactly why it’s important to find supportive allies that can help dispel that sort of closed-minded thinking.

      A good ally can be of any sex or orientation who believes in technology equality for everyone. They are positive and supportive with good critical feedback and mentoring to help a person who, on their own, feels reluctant or afraid to contribute.

      Allies help people grow and become confident in their own skills and in the community. It’s important for the community to have diverse ideas and approaches to problems that can improve OpenStack as a whole.

      Is there a minimum of hours that people should plan to dedicate?

      The Women of OpenStack meet every other week for an hour, but how involved someone wants to be is totally up to them and whether they want to be involved in other projects or activities we might have going on. I know the Mentoring folks have their own meetings and do a lot of work pairing people up.

      You bring a lot of training experience to OpenStack – any advice about getting started for newcomers?

      I think the most important thing is not to be afraid to ask questions whether it’s on IRC or a mailing lists. Take advantage of any of the community provided trainings whether its Upstream Institute, the Git and Gerrit Lunch and Learn or the new onboarding sessions they held in Boston and I hope are repeating for Sydney. Also I’m really excited about the new portal Mike Perez is working on.

      There are always a lot of interesting debates in the OpenStack community – which one is the most important, now?

      One of the important debate this cycle has been around documentation. There was a significant change and reduction of contributors mostly due to companies re-organizing their strategy with OpenStack and open-source contributions.  The document team has done such a great job the last few years building out enterprise quality open-source documentation around OpenStack. Keeping high-quality documentation lowers the barrier for new people to understand the project, how to use it and contribute to it.

      Anything else you’d like to add:

      Just that the Women of OpenStack is a great group to be part of and the opportunities are endless!

       

      Superuser is always interested in community content, get in touch at editorATopenstack.org

      The post Growing allies in the OpenStack community appeared first on OpenStack Superuser.

      by Nicole Martinelli at August 15, 2017 06:56 PM

      Chris Dent

      TC Report 33

      Some are still in peak release cycle time and others have left for their summer holidays, so once again: a bit of a slow week. No major spectacles or controversies, so the best I can do is give some summaries of the conversations in the #openstack-tc IRC channel.

      Continuous Deployment Continued

      The Wednesday morning (at least in UTC) office hour continued the discussion about whether or not Continuous Deployment is an integral part of the OpenStack development experience. There was some disagreement on who had made what kind of commitment and since when, but there's agreement in principle that:

      We need a "Declare plainly the current state of CD in OpenStack" 3RAY

      Maintenance Status

      The Thursday afternoon office hour started a few hours early with discussion about those projects that do not have PTLs. This evolved to some discussion about the implications of single PTLs volunteering leading to projects not needing elections. I suggested that perhaps the job was too hard, or too difficult to manage without sustained commitment from an employer and that such commitment was harder to come by these days. Anyone care to dissuade me of this opinion?

      Board Meeting Agendas

      As we rolled around to actual office hour time the discussion returned to building agendas for the upcoming meetings with the board. This was mentioned last week. My idea for a generic retrospective was considered too open-ended and too much of an invitation for unproductive complaining, so we discussed some ideas on how to narrow things down.

      less "it's hard" and more "it's hard because X" 2yZa

      This migrated to the common topic of "how do we get corporate engagement to be strong enough and focused on the right things" which naturally led to (because it was rc1 week) "ow, we could do with some more CI nodes".

      And finally, on today's office hour we came back to the agenda question and after discussion came to two relatively concrete topics:

      • What else should go on the top-5 list? Is what's there already correct?
      • What is the status of the strategic goals? Are there areas that we are missing or need revision?

      These were added to the meeting agenda.

      by Chris Dent at August 15, 2017 06:45 PM

      RDO

      Introducing opstools-ansible

      Introducing Opstools-ansible

      Ansible

      Ansible is an agentless, declarative configuration management tool. Ansible can be used to install and configure packages on a wide variety of targets. Targets are defined in the inventory file for Ansible to apply the predefined actions. Actions are defined as playbooks or sometime roles in the form of YAML files. Details of Ansible can be found here.

      Opstools-ansible

      The project opstools-ansible hosted in Github is to use Ansible to configure an environment that provides the support of opstools, namely centralized logging and analysis, availability monitoring, and performance monitoring.

      One prerequisite to run opstools-ansible is that the servers have to be running with CentOS 7 or RHEL 7 (or a compatible distribution).

      Inventory file

      These servers are to be defined in the inventory file with reference structure to this file that defines 3 high level host groups:

      • am_hosts
      • pm_hosts
      • logging_host

      There are lower level host groups but documentation stated that they are not tested.

      Configuration File

      Once the inventory file is defined, the Ansible configuration files can be used to tailor to individual needs. The READM.rst file for opstools-ansible suggests the following as an example:

      fluentd_use_ssl: true

      fluentd_shared_key: secret

      fluentd_ca_cert:

      —–BEGIN CERTIFICATE—–

      —–END CERTIFICATE—–

      fluentd_private_key:

      —–BEGIN RSA PRIVATE KEY—–

      —–END RSA PRIVATE KEY—–

      If there is no Ansible configuration file to tune the system, the default settings/options are applied.

      Playbooks and roles

      The playbook specifies what packages are to be installed in for the opstools environment by Ansible. Basically, the packages to be installed are:

      Besides the above packages, opstools-ansible playbook also applies these additional roles

      • Firewall – this role manages the firewall rules for the servers.
      • Prereqs – this role checks and installs all the dependency packages such as python-netaddr or libselinux-python … etc. for the successful installation of opstools.
      • Repos - this is a collection of roles for configuring additional package repositories.
      • Chrony – this role installs and configures the NTP client to make sure the time in each server is in sync with each other.

      opstools environment

      Once these are done, we can simply apply the following command to create the opstools environment:

          ansible-playbook playbook.yml -e @config.yml
      

      TripleO Integration

      TripleO (OpenStack on OpenStack) has the concept of Undercloud and Overcloud

      • Undercloud : for deployment, configuration and management of OpenStack nodes.
      • Overcloud : the actual OpenStack cluster that is consumed by user.

      RedHat has an in-depth blog post on TripleO and OpenStack has this document on contributing and installing TripleO

      When opstools is installed at the TripleO Undercloud, the OpenStack instances running on the Overcloud can be configured to run the opstools service when it deployed. For example:

      openstack overcloud deploy … \

      -e /usr/share/openstack-tripleo-heat-templates/environments/monitoring-environment.yaml \

      -e /usr/share/openstack-tripleo-heat-templates/environments/logging-environment.yaml \

      -e params.yaml

      There are only 3 steps to integrate opstools with TripleO with opstools-ansible. Detail of the steps can be found here.

      1. Use opstools-ansible to create the opstools environment at the Undercloud.
      2. Create the params.yaml for TripleO to points to the Sensu and Fluentd agents at the opstools hosts.
      3. Deploy with the "openstack overcloud deploy …" command.

      by atc at August 15, 2017 04:02 PM

      Daniel P. Berrangé

      ANNOUNCE: libosinfo 1.1.0 release

      I am happy to announce a new release of libosinfo version 1.1.0 is now available, signed with key DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF (4096R). All historical releases are available from the project download page.

      Changes in this release include:

      • Force UTF-8 locale for new glib-mkenums
      • Avoid python warnings in example program
      • Misc test suite updates
      • Fix typo in error messages
      • Remove ISO header string padding
      • Disable bogus gcc warning about unsafe loop optimizations
      • Remove reference to fedorahosted.org
      • Don’t hardcode /usr/bin/perl, use /usr/bin/env
      • Support eject-after-install parameter in OsinfoMedia
      • Fix misc warnings in docs
      • Fix error propagation when loading DB
      • Add usb.ids / pci.ids locations for FreeBSD
      • Don’t include private headers in gir/vapi generation

      Thanks to everyone who contributed towards this release.

      by Daniel Berrange at August 15, 2017 11:09 AM

      Opensource.com

      3 new OpenStack guides

      Trying to keep up with the latest on OpenStack? Check out these newly published guides and tutorials

      by Jason Baker at August 15, 2017 07:00 AM

      August 14, 2017

      OpenStack Superuser

      OpenStack Days Taiwan: The intersection of software and hardware

      TAIPEI, Taiwan — Similar to the popular Taiwanese pineapple cake that expertly pairs a sturdy crust with gooey jam, traditional IT hardware companies here are integrating software capabilities into their offerings. In July, over 1,000 attendees met at OpenStack Days Taiwan to discuss this integration, its impact on the cloud landscape as well as emerging OpenStack use cases like Internet of Things (IoT) and edge computing.


      In this cloud era, software-hardware integration is the strength that Taiwan’s industry must build on to seize the opportunities created by the global growth of OpenStack.

      Emergence of IoT and edge computing

      Thor Chin, chief architect at WinStack delivered a keynote highlighting the expected growth in the IoT over the next eight years and the impact this will have on edge computing, an emerging use case that Verizon shared at the OpenStack Summit Boston and will be the focus of the upcoming OpenDev event in September where industry experts will gather to develop the open source requirements to deliver compute at the edge. Business Insider Intelligence estimates that 5.6 billion IoT devices owned by enterprises and governments will use edge computing for data collection and processing in 2020 and global revenue for artificial intelligence products and $4.8 trillion in aggregate IoT investment between 2016 and 2021.

      Chin explains the impact that edge computing will have on IoT devices.

      AI is currently on the road map for Gemini Open Computing Inc, a Taiwan-based cloud provider. They’ve made inroads in Japan with an in-house-developed OpenStack monitoring cloud solution, now deployed by Tokyu Railway, one of Tokyo’s six major subway companies, to monitor level crossings. Through the city’s 4G network, imaging from the company’s wireless monitoring devices is transferred back to the cloud engine room at headquarters, where real-time observation of screen displays of the crossings has already improved incident handling and traffic safety. The next stage will be integrating AI into this system to identify and analyze traffic situations and issue real-time alerts.
      CloudCube Co., Ltd. is another Taiwanese organization working with OpenStack. Since June 2016, CloudCube has worked with the provincial government of Hubei to roll out the Chutian Cloud Data Center Project, part of the Smart Hubei program. The Chutian data center will collect and store all of the province’s demographic, economic, geographical and additional information in a database, which will feed into 394 application systems and 52 departments to ensure seamless information sharing throughout the province’s public sector.

      In just six months, CloudCube has successfully built an OpenStack architecture for six regions, covering 1,000 physical servers, the largest such deployment by a Taiwanese cloud service provider ever.

      “In order for businesses to survive, they have to integrate IT trends on a common infrastructure,” said Alan Clark, the director of the OpenStack Foundation board of directors. “The way to do that is to use OpenStack where you can use these new technologies including virtualization, containers and bare metal.”

      Cover Photo // CC BY NC

      The post OpenStack Days Taiwan: The intersection of software and hardware appeared first on OpenStack Superuser.

      by Allison Price at August 14, 2017 06:45 PM

      Julio Villarreal Pelegrino

      Building an Open Cloud Strategy

      Building an Open Cloud Strategy

      Building an Open Cloud Strategy

      “Choices made, whether bad or good, follow you forever and affect everyone in their path one way or another.” --J.E.B. Spredemann, An Unforgivable Secret

      For several years now, one of the most used words in the last couple years in every IT organization has been “cloud”. Why? Because using or providing “cloud” services is one of the main objectives for CIOs and CTOs across the globe.

      In 2017, the “cloud” word is not new anymore, but still relevant and a big part of the IT transformation process. Here are some numbers that highlight the importance of cloud adoption:

      • According to an IDC study (The Salesforce Economy: Enabling 1.9 Million New Jobs and $389 Billion in New Revenue Over the Next Five Years, IDC), cloud computing is growing at 4.5 times the rate of IT spending since 2009, they also expect to grow at better than 6 times the rate of IT spending from 2015 to 2020.
      • Gartner predicts the worldwide public cloud services market will grow 18% in 2017 to $246.8B, up from $209.2B in 2016.
      • Gartner also predicts that just Infrastructure-as-a-Service (IaaS) is projected to grow 36.8% in 2017 and reach $34.6B.

      The importance of cloud was reaffirmed when the big tech companies like Amazon, SalesForce, Google, Microsoft, IBM, VMware, Dell, Red Hat, Oracle, and HP joined the race to become “cloud” providers and get a piece of the market share. Enterprises across the world also knew that in order to compete and survive in a technology driven world, IT transformation was imperative: one must join the cloud revolution.

      The Problem

      One of the main problems with Cloud adoption is, without a doubt, the lack of a Cloud Strategy. Gartner estimates that “less than one-third of enterprises have a documented Cloud Strategy”. In my opinion, having a cloud strategy will provide multiple benefits to an enterprise, including:

      • Maximize the cloud business impact and benefits such as: accelerate time to market, increase agility and efficiency, and cut costs.
      • Translate business objectives and enterprise requirements into technology.
      • Be able to prepare for the cloud infrastructure needs. Have a detailed and well defined roadmap for a cloud adoption framework.

      At the end of the day, having a Cloud Strategy should enable the IT leadership to plan and be effective using cloud technologies as the base for IT modernization and digital transformation.

      Going beyond a Cloud Strategy: Adopting an Open Cloud Strategy

      Open Cloud is not just using Open Source software to build private or hybrid clouds. In my opinion, it is also the adoption of the open source culture and best practices as the cornerstone of an Open Cloud Strategy.

      For example, when talking about Open Cloud, let’s not forget that most public cloud providers use open source software as their foundation, and making their offerings able to have interoperability with open source software workloads has been a priority for large vendors like Amazon, Google, and Microsoft. The reason behind this strategy is simple: open source is great for business!

      Here are some of the benefits of adopting an Open Cloud Strategy:

      • Expand the software support ecosystem by being part of open source projects and their communities.
      • Reduce costs.
      • Avoid vendor lock in.
      • Improve code quality.
      • Improve security.
      • Increase interoperability.

      Some of the benefits outlined before could be exemplified by the following open source project examples:

      • Linux, which is no longer just for hobbyists and amateurs, runs deep in the enterprise powering critical applications in all industries. Linux not only is the best example of open source and community collaboration, but also the flagship product for several companies like Red Hat (my current employer).
      • OpenStack, a project where community work and best practices are providing an IaaS (Infrastructure-as-a-Service) alternative, 100% open source and with a vibrant community.

      Here are some of the things to consider while adopting an Open Cloud Strategy:

      1. Business impact.
      2. Cultural impact
        a) Evaluate the cultural impact of adopting an open cloud approach in the organization.
        b) Evaluate the benefits of the open source community model and how that could drive collaboration and innovation in the organization.
      3. Workload impact
        a) What will it entail, from a technology point of view, to adopt open source? For example, which workloads will need to be migrated or re-architected.
      4. Learning curve
        a) The level of effort required from the employees to efficiently manage the new technology.
        b) Is there internal talent inside the organization with expertise in the technologies to be adopted that could accelerate the learning curve process.
      5. Software assessment
        a) While adopting an Open Source project to be part of the cloud strategy, there are several questions that should be asked to determine complexity and impact of the implementation and maintenance :
        • License type
        • Age of the project and maturity
        • Public references of success (enterprise usage)
        • Number of contributors
        • IT experts opinions about it
        • Enterprise support availability
        • Change rate: commits, frequency, number of releases
        • Size of community

      Bottom line, adopting an open cloud strategy at the end of the day is a business decision. A decision that now more than ever is easier to make because of the increased benefits and popularity of open source projects and their communities, the impact of their use in the enterprise, and the amount of quality Cloud computing open source projects available (OpenStack, Kubernetes, Docker, LXC, KVM, Ansible, etc.).

      by Julio Villarreal Pelegrino at August 14, 2017 04:00 PM

      Mirantis

      Intro to CI/CD: How to make Jenkins build on Github changes

      Jenkins is crucial for CI/CD. But how do you get it to build the app when changes are pushed to your GitHub repo? Here's a simple explanation.

      by Nick Chase at August 14, 2017 05:21 AM

      August 11, 2017

      Ed Leafe

      Fanatical Support

      “Fanatical Support®” – that’s the slogan for my former employer, Rackspace. It meant that they would do whatever it took to make their customers successful. From their own website:

      Fanatical Support® Happens Anytime, Anywhere, and Any Way Imaginable at Rackspace

      It’s the no excuses, no exceptions, can-do way of thinking that Rackers (our employees) bring to work every day. Your complete satisfaction is our sole ambition. Anything less is unacceptable.

      Sounds great, right? This sort of approach to customer service is something I have always believed in. And it was my philosophy when I ran my own companies, too. Conversely, nothing annoys me more than a company that won’t give good service to their customers. So when I joined Rackspace, I felt right at home.

      Back in 2012 I was asked to create an SDK in Python for the Rackspace Cloud, which was based on OpenStack. This would allow our customers to more easily develop applications that used the cloud, as the SDK would handle the minutiae of dealing with the API, and allow developers to focus on the tasks they needed to carry out. This SDK, called pyrax, was very popular, and when I eventually left Rackspace in 2014, it was quite stable, with maybe a few outstanding small bugs.

      Our team at Rackspace promoted pyrax, as well as our SDKs for other languages, as “officially supported” products. Prior to the development of official SDKs, some people within the company had developed some quick and dirty toolkits in their spare time that customers began using, only to find out some time later when they had an issue that the original developer had moved on, and no one knew how to correct problems. So we told developers to use these official SDKs, and they would always be supported.

      However, a few years later there was a movement within the OpenStack community to build a brand-new SDK for Python, so being good community citizens, we planned on supporting that tool, and helping our customers transition from pyrax to the OpenStackSDK for Python. That was in January of 2014. Three and a half years later, this has still not been done. The OpenStackSDK has still not reached a 1.0 release, which in itself is not that big a deal to me. What is a big deal is that the promise for transitioning customers from pyrax to this new tool was never kept. A few years ago the maintainers began replying to issues and pull requests stating that pyrax was deprecated in favor of the OpenStackSDK, but no tools or documentation to help move to the new tool have been released.

      What’s worse, is that Rackspace now actively refuses to make even the smallest of fixes to pyrax, even though they would require no significant developer time to verify. At this point, I take this personally. For years I went to conference after conference promoting this tool, and personally promising people that we would always support it. I fought internally at Rackspace to have upper management commit to supporting these tools with guaranteed headcount backing them before we would publish them as officially supported tools. And now I’m extremely sad to see Rackspace abandon these people who trusted my words.

      So here’s what I will do: I have a fork of pyax on my GitHub account. While my current job doesn’t afford me the time to actively contribute much to pyrax, I will review and accept pull requests, and try to answer support questions.

      Rackspace may have broken its promises and abandoned its customers, but I cannot do that. These may not be my customers, but they are my community.

      by ed at August 11, 2017 06:17 PM

      OpenStack Superuser

      OpenStack User Groups spotlight: Phoenix

      In this series of interviews, Superuser takes you around the world to meet our newest user groups. These busy groups act as liaisons between the Foundation and the general community in their regions. You can find out more about a group in your area — or how to start one — at the User Groups portal.

      Here we talk to Ben Silverman, principle cloud architect at OnX Enterprise Solutions, about the User Group in Phoenix, whose Aug 24 event features Kubernetes-as-a-service on Plaform 9.

      What is the most important OpenStack debate in your region right now?

      In general we’re seeing less discussion about the maturity and stability of the platform and more discussion about operations and application development tooling.

      The most heated of the latest debate topics is centered around Kubernetes, Docker and how they fit into application development as well as the services underlying OpenStack. Users are also having discussions around hybrid and multi-cloud operational topics such as cloud management platforms and aligning development pipelines of both OpenStack and public cloud platforms to minimize process differences. Debates around tooling, monitoring and governance are also very common.

      What’s the key to closing the talent gap in the OpenStack community?

      I really think it’s simply getting additional talented technical resources exposure to production OpenStack clouds. I am encouraged by the number of new COA certifications I see and I have heard that OpenStack training remains very popular. The customers I work with, which include most of the large service providers, are all currently looking for experienced talent. Lately I’ve seen a lot of requests with Kubernetes, Docker and intimate knowledge of the core services that underlay OpenStack as well as an increase in demand for OpenStack Engineers that know Ansible, Chef, Puppet and/or Python who can create automation for post-configuration of OpenStack infrastructure as well as operational site reliability engineering.

      What trends have you seen in your user group and within your region? And how do these compare to the trends from the global OpenStack Summit?

      We now have a community of some very experienced operators from companies like PayPal, American Express, Wells Fargo Bank, CSAA (AAA) and others. At the same time, we get a lot of new faces from small companies to large enterprises at the meetups who are simply coming to learn the basics. I think we’re seeing the evolution of the early adopters into savvy operators and new users who see the early adopters as a testament to the stability and maturity of the platform. I find that attendees from the global summit are more equally dispersed than we see here in Phoenix. One reason may be the amount of early adopters we have locally versus the distribution nationally.

      What drives cloud adoption in your region?

      There is a lot of development happening in The Silicon Desert (as Phoenix has been called) and OpenStack is the logical choice when companies are looking for open solutions for private clouds. Additionally, some of the larger enterprises in the valley are well into their journey of agile development and have either decided that a multi-cloud approach fits them best, or have embarked on a single public cloud strategy only to realize the cost was greater than expected and have adopted OpenStack to help balance their strategic and financial goals.

      What’s the biggest obstacle to adoption?

      OpenStack still has the stigma of being complicated to install, operate and requires lots of hard to find resources. I find that this unfair stigma drives companies to an all-in public cloud strategy who are then faced with the reality that migrating applications to public cloud is not as easy nor as inexpensive as it appears.

      What types of clouds/organizations are most active in the community and at local events, including meetups and OpenStack Days?

      We have a lot of operators like the ones I mentioned above and more at our events. These companies run large and small clouds that service use cases like web scale, internal IT needs, development and NFV. We also get some great support and sponsorship from companies like NetApp, Red Hat and OnX Enterprise Services. We’ve had speakers and collaboration from RackSpace, Juniper, Talligent, Zabbix and countless others.

      Which users have come forward in your local community to share their story and get involved?

      Our members have been very involved in our local user group. It’s always great to see them time and time again and hear their updates and challenges. We also have a surprising large SMB group of attendees from local hospitals, casinos and other local businesses who are simply interested in the technology.

       

      Superuser is always interested in hearing from the community. Want to tell us about your user group? Email editorATopenstack.org

       

      Cover Photo // CC BY NC

      The post OpenStack User Groups spotlight: Phoenix appeared first on OpenStack Superuser.

      by Sonia Ramza at August 11, 2017 04:30 PM

      NFVPE @ Red Hat

      Automating local mirrors creation in RHEL

      Sometimes there is a need to consume RHEL mirrors locally, not using the Red Hat content delivery network. It may be needed to speed up some deployment, or due to network constraints. I create an ansible playbook, rhel-local-mirrors (https://github.com/redhat-nfvpe/rhel-local-mirrors), that can help with that. What does rhel-local-mirrors do? It is basically a tool that connects to the Red Hat CDN, and syncs the repositories locally, allowing to populate the desired mirrors, that can be accessed by other systems via HTTP. The playbook is performing several tasks, that can be run together or independently: register a system on the Red Hat…

      by Yolanda Robla Mota at August 11, 2017 11:19 AM

      August 10, 2017

      NFVPE @ Red Hat

      Be a hyper spaz about a hyperconverged GlusterFS setup with dynamically provisioned Kubernetes persistent volumes

      I’d recently brought up my GlusterFS for persistent volumes in Kubernetes setup and I was noticing something errant. I had to REALLY baby the persistent volumes. That didn’t sit right with me, so I refactored the setup to use gluster-kubernetes to hook up a hyperconverged setup. This setup improves on the previous setup by both having the Gluster daemon running in Kubernetes pods, which is just feeling so fresh and so clean. Difference being that OutKast is like smooth and cool – and I’m an excited spaz about technology with this. Gluster-Kubernetes also implements heketi which is an API for GlusterFS volume management – that Kube can also use to allow us dynamic provisioning. Our goal today is to spin up Kube (using kube-centos-ansible) with gluster-kubernetes for dynamic provisioning, and then we’ll validate it with master-slave replication in MySQL, to one-up our simple MySQL from the last article.

      by Doug Smith at August 10, 2017 09:00 PM

      OpenStack Superuser

      OpenStack operator spotlight: CERN

      We’re spotlighting users and operators who are on the front lines deploying OpenStack in their organizations to drive business success. These users are taking risks, contributing back to the community and working to secure the success of their organization in today’s software-defined economy.

      Today we talk to Belmiro Moreira, cloud architect, who tells Superuser about how the research center went from a few hundred nodes to 280,000 cores.

      Describe how are you using OpenStack now. What kinds of applications or
 workloads are you currently running on OpenStack?

      CERN provides several facilities and resources to scientists all around the world, including compute and storage resources, for their fundamental research.

      Back in 2012, the CERN IT department decided to deploy a private cloud based on OpenStack and other open source tools to improve the utilization and reduce the compute resources provision time. The Cloud Infrastructure Service was moved into production with only few hundred compute nodes in July 2013. Since then, it has been growing rapidly to several thousand compute nodes. This scaling represented a huge architectural challenge.

      In order to manage the large number of compute nodes, we split the infrastructure in several Nova cells. This allow us to scale OpenStack Nova and reduce the failure domains into only few hundred compute nodes. Currently, we have more than 280,000 cores available that are split between 60 Nova cells.

      Users can increase their application availability by creating resources in different availability zones that are mapped to several Nova cells.

      The infrastructure runs in two CERN data centers (Geneva and Budapest) that are separated by 22ms, however only one region is defined. Users only have one endpoint to access the CERN Cloud.

      When we started with the cloud setup, we offered only a few OpenStack projects (Nova, Glance, Cinder, Keystone). Today we run more than 12 different OpenStack projects and continue to evaluate new ones.

      We have a very wide, diverse and geographically dispersed user community. Currently, more than 3,000 users use the CERN Cloud for their research projects. Because of that, the set of applications that run in our Cloud is extremely heterogeneous. They range from IT services to scientific data-processing applications to remote desktops. As an example, projects like “CMS Data Acquisition,” “ATLAS Tile Calorimeter“, “IT Windows Terminal Service” and “Personal Belmiro” share the same compute nodes. In the past they would have had different dedicated servers. This makes the CERN Cloud a very dynamic and challenging environment for our engineering team because of all the different use cases, service expectations and requirements.

      The biggest use case, however, is the data processing from the Large Hadron Collider (LHC) data.

      The data analysis entails the execution of a large number of loosely coupled jobs responsible for the data crunch of the 200PB stored in our tape libraries. These jobs are executed in resources provisioned directly by the LHC experiments using the OpenStack APIs or in the batch system that runs on top of the CERN Cloud.
      Recently, we added OpenStack Magnum to our service offering. This allows us to provision Kubernetes, Docker Swarm and Mesos clusters. Several users are testing the possibilities of containers to run data analysis and specially to run legacy code from previous experiments and data sets.

      What results has CERN seen from using OpenStack? How are you measuring 
the impact?

      Before the introduction of the Cloud Infrastructure Service, all compute resources were manually allocated to all of the different use cases and projects. Managing this process was complicated and the allocation could take up to several weeks. Also, after the resources were provisioned it was extremely difficult to move sporadic idle compute capacity to other projects.

      Moving into a self-service resource allocation model based on predefined quotas allowed us to decrease the provision time drastically and since the resources are only created when needed, globally it resulted in more compute capacity available to our scientific communities.

      We currently only provision compute resources as virtual machines and containers, however we are working to offer bare metal-as-a-service in the near future. This will not only improve the efficiency of physical nodes allocation but most importantly it will allow us to consolidate all resources management and provisioning within the CERN data centers through an OpenStack interface. A common workflow for all resources provisioned will offer further productivity gains.

      For updates from the CERN cloud team, check out the OpenStack in Production blog.

      Superuser wants to hear more from operators like you, get in touch at editorATopenstack.org

       

      Cover Photo by Marceline Smith // CC BY NC

      The post OpenStack operator spotlight: CERN appeared first on OpenStack Superuser.

      by Nicole Martinelli at August 10, 2017 05:59 PM

      RDO

      EasyFix: Getting started contributing to RDO

      It can be intimidating trying to get involved in an open source project. Particularly one as huge and complicated as OpenStack. But we want you to join us on the RDO project, so we're trying to make it as easy as possible to get started.

      To that end, we've started the EasyFix initiative. EasyFix is a collection of "easy" tickets that you should be able to get started on without having a deep knowledge of the RDO project. And in the process, you'll learn what you need to know to move on to more complex things.

      These tickets cover everything from documentation, to packaging, to helping with events, to writing tools to make things easier. And each ticket comes with a mentor - someone who has volunteered to help you get through that first commit.

      There's also a general mentors list, categorized by area of expertise, if you just have a question and you're not sure who to ping on IRC.

      We've been running the EasyFix program for about 3 weeks now, and in that time, four new contributors have started on the RDO project.

      We're very pleased to welcome these new contributors, and hope they'll be around for a long time to come.

      Christopher "snecklifter" Brown - I'm Christopher Brown, a HPC Cloud Engineer based in Sheffield in the UK. I've worked on OpenStack since June 2015. I used to work on packaging for the Fedora project so I am transferring and updating those skills to help out with RDO when I can.

      Anthony Chow - I am a software developer for legacy networking equipment wanting to be a Developer Advocate. I have been venturing into other technologies such as cloud, container, and configuration management tools. Passionate in learning and sharing technologies related topics.

      Treva Williams - T. Nichole Williams is an RHCSA 7 Certified Linux and OpenStack engineer and content author for LinuxAcademy.com, & an OpenStack active technical contributor in the Magnum project. She is actively involved in several OpenStack, OpenShift, RDO, and Ceph communities and groups. When not OpenStacking or Cephing, she enjoys doggos, candy, cartoons, and playing "So You Think You're a Marine Biologist" on Google.

      Ricardo Arguello - I am an Open Source enthusiast that tries to collaborate as much as I can. I have helped in the past with patches for WildFly, and as a Fedora packager. The RDO project is very complex an intimidating, but collaborators are welcome and there are easy to fix bugs for newbies to make their first contributions! That makes RDO a great community if you are interested in helping the project while learning about Open Stack internals in the process.

      If you want to participate in the RDO project, we encourage you to find something on the EasyFix list and get started. And please consider attending EasyFix office hours, Tuesdays at 13:30 UTC.

      by Rich Bowen at August 10, 2017 03:10 PM

      Thierry Carrez

      Introducing OpenStack SIGs

      Back in March in Boston, the OpenStack Board of Directors, Technical Committee, User Committee and Foundation staff members met for a strategic workshop. The goal of the workshop was to come up with a list of key issues needing attention from OpenStack leadership. One of the strategic areas that emerged from that workshop is the need to improve the feedback loop between users and developers of the software. Melvin Hillsman volunteered to lead that area.

      Why SIGs ?

      OpenStack was quite successful in raising an organized, vocal, and engaged user community. However the developer and user communities are still mostly acting as separate communities. Improving the feedback loop starts with putting everyone caring about the same problem space in the same rooms and work groups. The Forum (removing the artificial line between the Design Summit and the Ops Summit) was a first step in that direction. SIGs are another step in addressing that problem.

      Currently in OpenStack we have various forms of workgroups, all attached to a specific OpenStack governance body: User Committee workgroups (like the Scientific WG or the Large Deployment WG), upstream workgroups (like the API WG or the Deployment WG), or Board workgroups. Some of those are very focused on a specific segment of the community, so it makes sense to attach them to a specific governance body. But most are just a group of humans interested in tackling a specific problem space together, and establishing those groups in a specific community corner sends the wrong message and discourages participation from everyone in the community.

      As a result (and despite our efforts to communicate that everyone is welcome), most TC-governed workgroups lack operator participants, and most UC-governed workgroups lack developer participants. It's clearly not because the scope of the group is one-sided (developers are interested in scaling issues, operators are interested in deployment issues). It's because developers assume that a user committee workgroup about "large deployments" is meant to gather operator feedback rather than implementing solutions. It's because operators assume that an upstream-born workgroup about "deployment" is only to explore development commonalities between the various deployment strategies. Or they just fly below the other group's usual radar. SIGs are about breaking the artificial barriers and making it clear(er) that workgroups are for everyone, by disconnecting them from the governance domains and the useless upstream/downstream division.

      SIGs in practice

      SIGs are neither operator-focused nor developer-focused. They are open groups, with documented guidance on how to get involved. They have a scope, a clear description of the problem space they are working to address, or of the use case they want to better support in OpenStack. Their membership includes affected users that can discuss the pain points and the needs, as well as development resources that can pool their efforts to achieve the groups goals. Ideally everyone in the group straddles the artificial line between operators and developers and identifies as a little of both.

      In practice, SIGs are not really different from the various forms of workgroups we already have. You can continue to use the same meetings, git repositories, and group outputs that you used to have. To avoid systematic cross-posting between the openstack-dev and the openstack-operators mailing-lists, SIG discussions can use the new openstack-sigs mailing-list, SIG members can take advantage of our various events (PTG, Ops meetups, Summits) to meet in person.

      Next steps

      We are only getting started. So far we only have one SIG: the "Meta" SIG, to discuss advancement of the SIG concept. Several existing workgroups have expressed their willingness to become early adopters of the new concept, so we'll have more soon. If your workgroup is interested in being branded as a SIG, let Melvin or myself know, we'll guide you through the process (which at this point only involves being listed on a wiki page). Over time we expect SIGs to become the default: most community-specific workgroups would become cross-community SIGs, and the remaining workgroups would become more like subteams of their associated governance body.

      And if you have early comments or ideas on SIGs, please join the Meta discussion on the openstack-sigs mailing-list, (using the [meta] subject prefix)!

      by Thierry Carrez at August 10, 2017 12:12 PM

      August 09, 2017

      OpenStack @ NetApp

      Cinder Cheesecake: Things to Consider

      By Chad Morgenstern and Jenny Yang In the blog post An Introduction to Cheesecake in OpenStack, we spoke to Cinder Cheesecake replication, but we didn’t go into detail about how it works. In summary: Cinder automatically sets up a replication relationship between backends. If a primary storage backend fails, the OpenStack administrator issues the cinder ... Read more

      The post Cinder Cheesecake: Things to Consider appeared first on thePub.

      by Chad Morgenstern at August 09, 2017 07:00 PM

      OpenStack Superuser

      Learn OpenStack: A new community site

      One of the great things about open source is the large community of people sharing their knowledge. Take Geoff Rich, who created a site called Learn OpenStack.  Right now you can use it to get started for free with OpenStack to deploy a single VM based on the Ocata release. He’s made all of the code available over at GitHub and set up a Trello board so you can see what he’s working on next.

      Superuser spoke to Rich over email about the challenges he faced and what’s next for the site.

      It’s an impressive amount of work: what inspired you to do it?

      I was initially learning how to deploy OpenStack in my home lab environment as I work in a large organization that owns a lot of our own servers and I wanted to see how OpenStack works. Our levels of automation for provisioning network infrastructure still need work and OpenStack seems to solve these problems well.  (My day job is managing teams of software engineers.)

      As I was learning, I found a lot of the online material somewhat lacking in places. More as a hobby I decided to make a tutorial website out of my learnings and to also create a set of easily repeatable steps that anyone with a few technical skills could follow. I also wanted a set of steps that anyone can run, regardless of if they run Mac, Windows or Linux.

      I think the biggest challenge at the moment is the learning curve and getting an OpenStack environment up and running. From a capability perspective, OpenStack is great. But I could imagine a lot of people find aspects of it very daunting. I personally have found the networking to have a very steep learning curve as a lot of my previous experience is in building software systems.

      What’s next for the site? What are you looking for from the community?

      I’m currently working on finishing the steps to perform a multi-node deployment. Once I have that done, I will probably do a section on exploring OpenStack and showing people how to use it for real-world tasks.

      In terms of feedback, I’m interesting in what people are finding hard to learn when they start with OpenStack. I’m keen to incorporate other tutorials for areas people are having difficulties with.

      Superuser is always interested in community content, get in touch: editorATopenstack.org

      The post Learn OpenStack: A new community site appeared first on OpenStack Superuser.

      by Nicole Martinelli at August 09, 2017 06:35 PM

      Stackmasters team

      Private cloud: not just for the big boys

      Private cloud. “This is not for me”. “It is too complicated and expensive”. “It’s only for large or corporate enterprises”. Right? Wrong.

      The fact is that from the various types of organizations out there, it’s the small and medium-sized firms stand to reap the most benefits by switching to cloud. So what’s the problem? Why don’t they? Basically. it comes down to knowledge. Very few (and we mean very) understand how it can help their businesses.

      Private cloud: not just for the big boys

      Open source removing cloud barriers

      Open source software is removing barriers to starting a private cloud project, with the most prominent example of OpenStack. A vast community of larger or smaller enterprises and individuals, are working closely to produce cloud management software, for all types of environment sizes.

      Arguably, the standout misconception about private cloud is that it’s all on-premise, hosted in a company’s data centre. While this can sometimes be the case, an ever-growing part is hosted in third-party hosting providers or provided as a managed service.

      There are many benefits attached to this hosted cloud approach, apart from the obvious cost-saving and reduction in complexity.

      The benefits of private cloud for SME

      • PERSONALIZED SERVICE: moving your IT infrastructure to a private cloud makes it more flexible and customizable in order to meet individual business requirements. And gain full control over a most critical part of your business chain.
      • ENHANCED SECURITY: you know exactly where the data centres storing your data are located and the security measures in place. As a private cloud consumer you control how data encryption, backup and disaster recovery policies are set and enforced.

      Compact, not complex

      If you stop to think about it, the most enthusiastic adopters of cloud computing – and the easiest businesses to sell it to – should be SME’s. That’s because they don’t come with the baggage associated with larger corporations. Things like legacy infrastructures and/or specialized systems which they can’t or simply don’t want to move into the cloud. In contrast, most smaller companies don’t have such deep-rooted and complicated IT. By nature, they should be much more open to the concept of cloud.

      The cloud grows with you

      Because the cloud is completely scalable, a typical SME can respond to higher demand coming from their users base without having to redesign and implement everything from scratch. As you grow, you can effortlessly add more on the resources size your cloud manages. In essence, the cloud grows with you.

      Also, you have the perfect opportunity to start slowly and try a “see how it goes” approach. For example, why not start with simple, not mission-critical applications to establish a process around your own cloud tight with your business and people. Then you can evolve your cloud strategy to include all elements of the business, and thus achieve further efficiencies.

      Going back to the cost factor, a major point to remember is that you don’t need to worry about
      price drops, special offers and the pace your cloud bills grow, while your needs for IT resources grow, as you can control the amount of IT resources allocated for your business at standard operation and scale when needed either on ad-hoc basis with public cloud products or permanently scale up with your private cloud solution.

      Challenges of private cloud for SME

      Is all of the above too good to be true? Well, in a word, no. For an SME, moving to a private cloud is essentially easy. But it’s not without challenges (that would be too easy). And for the record, those are the following:

      • Expertise: lack of a technical team to carry out the project. As we are talking about cutting edge technology, it’s hard to find and attract the right people for the job. On the other hand, if you manage to engage with the right people then you essentially extend your human resources with highly-skilled and innovative voices.
      • Entry cost: the initial outlay may be higher than with public cloud. But in the long run you’ll see that this more than balances out and actually becomes more cost-effective.
      • On-site or hosted: build everything from scratch on your own or reach out to a managed service provider to jump start it. The latter, hosted private cloud, comes with an established team of experts to deal with all the technical details and a significant cost proposition, as you will be partnering with a professional offering that combines engineering and industry landscape awareness.

      Your cloud. Your way.

      In conclusion, at Stackmasters we all agree that the above challenges pale in comparison to the actual benefits for SME’s switching to private cloud. The most important thing is to find the right partner who will provide you with a high quality consultation. Then, in a very short time, you can get yourself a customized, private (or even multi) cloud solution designed to grow with your business.

      And without wanting to blow our own trumpet, this is something we can help you with. A lot. Our managed cloud service offering will help you build, manage and scale your Stack.

      Contact us for a free consultation and we’ll provide an assessment of your required infrastructure and a pricing plan to suit your needs.

      Private cloud: not just for the big boys was last modified: August 9th, 2017 by Stackmasters

      The post Private cloud: not just for the big boys appeared first on Stackmasters.

      by Stackmasters at August 09, 2017 09:19 AM

      August 08, 2017

      Chris Dent

      TC Report 32

      As we are still in the heady peak of the release cycle, there's not a great deal of Technical Committee activity to report, but I've dredged through some IRC logs to find a couple of bits that might be relevant to the slicke of governance activity that might actually be interesting to a broader audience.

      Continuous Deployment

      In a thread on Backwards incompatible changes based on config, Joshua Harlow chose to ask one of those questions it's worth asking every now and again: Who are these continuous deployers?. If they still exist, where are they, how do we get them to step up and participate? If nobody is doing it any more, can we stop and make between-release adjustments more easily?

      The mailing list thread petered out, as they are wont to do, but the topic carried over to IRC. In the #openstack-tc channel the conversation went long and wide:

      • Even if there aren't people who do real deploys from trunk, the constraints imposed by behaving as if they do may be beneficial.
      • If we stop the behavior, it is very hard to go back.
      • Some people say we historically supported CD. Other people say that sure, some people say that, but only some people.
      • Or maybe the divisions are between projects?
      • APIs and database migrations are in a different class from everything else and they themselves are different from one another.
      • Whatever the behaviors have been, they've never been official (for all projects?).
      • Whatever the behavior should be, it needs to be official (for all projects?).

      For each of these strongly asserted opinions, there was at least one person who disagreed, at least in part. Fun times.

      As far as I could tell, there was no resolution or forward plan. The status quo of benign (?) ambiguity will maintain for the time being. When it hurts, something will happen.

      Forthcoming Meetings With the Board

      A Foundation list posting asked for agenda items for the combined Board and Leadership (TC and UC) meeting happening before the PTG. Details about a similar meeting in Sydney are also starting to cohere.

      Yet as far as I can tell, very little has yet been gathered in terms of substantive agenda items. Given how often different members of the community state that we need to be more activist with the board, and the board with member companies -- especially with regard to making sure corporate engagement is not just strong but focused in the right place -- I'm surprised there's not more. I fished around a bit in IRC but I think there's more to be done here.

      As a community we've become over adapted to the idea that people who wish to discuss problems should come prepared with solutions, or if not that, at least a detailed description of the failure scenario. This is incredibly limiting. Community interactions aren't software. We can't always create a minimal test case and iterate towards a solution. Sometimes what we need is to sit around as peers and reflect. In person is the ideal time for this; we don't get the opportunity all that much.

      What I'd like to see with the board, the TC, the UC, and anyone else who wants to participate is a calm retrospective of the last three, six or twelve months. So we can see where we need to go from here. We can share some accolades and, if necessary, air some grievances. Someone can say "there's a rough edge here" so someone else with a lot of spare sandpaper they thought was useless can say "I can help with that". We might even sing Kum ba yah.

      If you're not going to be at the PTG, or you don't feel comfortable raising an issue, feel free to contact me with anything you think is important and relevant to the ongoing health of OpenStack and I will try to bring it up at one of the meetings.

      by Chris Dent at August 08, 2017 08:15 PM

      OpenStack Superuser

      How to build clusters for scientific applications-as-a-service

      How can we make a workload easier on cloud? In a previous article we presented the lay of the land for HPC workload management in an OpenStack environment. A substantial part of the work done to date focuses on automating the creation of a software-defined workload management environment – SLURM-as-a-Service.

      SLURM is only one narrow (but widely used) use case in a broad ecosystem of multi-node scientific application clusters: let’s not over-specialize. It raises the question of what is needed to make a generally useful, flexible system for creating Cluster-as-a-Service?

      What do users really want?

      A user of the system will not care how elegantly the infrastructure is orchestrated:

      • Users will want support for the science tools they need, and when new tools are needed, the users will want support for those too.
      • Users will want to get started with minimal effort. The learning curve they must climb to deploy tools needs to be shallow.
      • Users will want easy access to the datasets upon which their research is based.

      Scientists certainly do not want to be given clusters which in truth are just replicated infrastructure. We must provide immediately useful environments that don’t require scientists to be sysadmins. The time to science (or more bluntly, the time to paper) is pretty much the foremost consideration. Being able to use automation to reliably reproduce research findings comes a close second.

      Application building blocks

      A really useful application cluster service is built with several key design criteria taken into account:

        • The right model for sharing. Do we provide a globally-available shared infrastructure, project-level multi-user infrastructure or per-user isolation? Per-user isolation might work until the user decides they prefer to collaborate. But can a user trust every other user in their project? Per-project sharing might work unless the users don’t actually trust one another (which might in itself be a bigger problem).
        • Users on the cluster. Are the members of the project also to be the users of the cluster? In this case, we should do what we can to offer a cluster deployment that is tightly integrated with the OpenStack environment. Why not authenticate with their OpenStack credentials, for example? In a different service model, the OpenStack project members are the cluster admins, not its users. If the cluster is being provided as a service to others who have no connection with the infrastructure, an external mechanism is required to list the users, and to authenticate them. Some flexibility should be supported in an effective solution.
        • Easy data access. Copying user files into a cluster introduces a boundary to cross, which adds inconvenience for using that resource. Furthermore, copying data in requires the same data to be stored in two (or more) places. Where a cluster requires a shared filesystem, creating its own ad-hoc filesystem is unlikely to be the best solution. In the same manner as provider networks, a cluster should support “provider filesystems” – site production filesystems that are exported from other infrastructure in the data center. Large scientific datasets may also be required, and are often mediated using platform data services such as iRODS, or HDF5 (note, that’s a 5). Object storage (S3 in particular) is seen as the long-term solution for connecting applications with datasets, and does appear to be the direction of travel for many. However, sharing read-only filesystems, at either file or block level, are simple and universal approaches that work perfectly well. Both are also well-established choices.
        • Scaling up and down. The cloud access model does not entail queuing and waiting for resources like a conventional HPC batch queuing system. Equally, cloud resources are assumed to grow and shrink dynamically, as required. Perhaps this could even happen automatically, triggered by peaks or troughs in demand. To maximize overall utilization, cluster resizing is actually quite important, and managing all the resources of a cluster together enables us to do it well.
        • Self-service creation – for some definition of self. Users shouldn’t need to learn sysadmin skills in order to create a resource for doing their science. For some, the Horizon web interface might be too complex to bother learning. Enter the ResOps role – for example described in a recent SuperUser article on the Scientific WG – a specialist embedded within the project team, trained on working with cloud infrastructure to deliver the best outcomes for that team. To help the task of cluster creation, it should also be automated to the fullest extent possible.

      Bringing it all together: OpenHPC-as-a-Service

      The OpenHPC project is an initiative to build a community package distribution and ecosystem around a common software framework for HPC.

      OpenHPC clusters are built around the popular SLURM workload manager.

      A team from Intel including Sunil Mahawar, Yih Leong Sun and Jeff Adams have already presented their work at the latest OpenStack summit in Boston:

      Independently, we have been working on our own OpenHPC clusters as one of our scientific cluster applications on the SKA performance prototype system, and I’m going to share some of the components we have used to make this project happen.

      SKA Alaska: Performance Prototype Platform

      Software base image: An image with major components baked in saves network bandwidth and deployment time for large clusters. We use our os-images role, available on Ansible Galaxy. We use this role with our custom elements written for Diskimage-builder

      Here’s an example of the configuration to provide for the os-images:
      os_images_list:
      # Build of OpenHPC image on a CentOS base
      - name: "CentOS7-OpenHPC"
      elements:
      - "centos7"
      - "epel"
      - "openhpc"
      - "selinux-permissive"
      - "dhcp-all-interfaces"
      - "vm"
      env:
      DIB_OPENHPC_GRPLIST: "ohpc-base-compute ohpc-slurm-client 'InfiniBand Support'"
      DIB_OPENHPC_PKGLIST: "lmod-ohpc mrsh-ohpc lustre-client-ohpc ntp"
      DIB_OPENHPC_DELETE_REPO: "n"
      properties:
      os_distro: "centos"
      os_version: 7

      os_images_elements: ../stackhpc-image-elements

      Flexible infrastructure definition: A flexible definition is needed because it soon becomes apparent that application clusters do not all conform to a template of 1 master, n workers. We use a Heat template in which the instances to create are parameterized as a list of groups. This takes Heat close to the limits of its expressiveness – and requires the Newton release of Heat as a minimum. Some users of Heat take an alternative approach of code-generated Heat templates.Our flexible Heat template is encapsulated within an Ansible role available on Ansible Galaxy. This role includes a task to generate a static inventory file, extended with user-supplied node groupings, which is suitable for higher-level configuration.

      The definition of the compute node components cluster takes a simple form:

      cluster_groups:
      - "{{ slurm_login }}"
      - "{{ slurm_compute }}"

      slurm_login:
      name: “login”
      flavor: “compute-B”
      image: “CentOS7-OpenHPC”
      num_nodes: 1

      slurm_compute:
      name: “compute”
      flavor: “compute-A”
      image: “CentOS7-OpenHPC”
      num_nodes: 8

      The invocation of the infrastructure role becomes equally simple:

      ---
      # This playbook uses the Ansible OpenStack modules to create a cluster
      # using a number of baremetal compute node instances, and configure it
      # for a SLURM partition
      - hosts: openstack

      roles:
      – role: stackhpc.cluster-infra
      cluster_name: “{{ cluster_name }}”
      cluster_params:
      cluster_prefix: “{{ cluster_name }}”
      cluster_keypair: “{{ cluster_keypair }}”
      cluster_groups: “{{ cluster_groups }}”
      cluster_net: “{{ cluster_net }}”
      cluster_roles: “{{ cluster_roles }}”

      Authenticating our users: On this prototype system, we currently have a small number of users, and these users are locally defined within Keystone. In a larger production environment, a more likely scenario would be that the users of an OpenStack cloud are stored within external authentication infrastructure, such as LDAP.
      Equivalent user accounts must be created on our OpenHPC cluster. Users need to be able to login on the externally-facing login node. The users should be defined on the batch compute nodes, but they should not be able to login on these instances.

      Our solution is to enable our users to authenticate using Keystone on the login node. This is done using two projects, PAM-Python and PAM-Keystone – a minimal PAM module that performs auth requests using the Keystone API. Using this, our users benefit from common authentication on OpenStack and all the resources created on it.

      Access to cluster filesystems: OpenHPC clusters require a common filesystem mounted across all nodes in a workload manager. One possible solution here would be to use Manila, but our bare metal infrastructure may complicate its usage. It is an area for future exploration for this project.
      We are using CephFS, exported from our local Ceph cluster, with an all-SSD pool for metadata and a journaled pool for file data. Our solution defines a CephX key, shared between project users, which enables access to the CephFS storage pools and metadata server. This CephX key is stored in Barbican. This appears to be an area where support in Shade and Ansible’s own OpenStack modules is limited. We have written an Ansible role for retrieving secrets from Barbican and storing them as facts, and we’ll be working to package it and publish on Galaxy in due course.

      Converting infrastructure into platform: Once we have built upon the infrastructure to add the support we need, the next phase is to configure and start the platform services. In this case, we build a SLURM configuration that draws from the infrastructure inventory to define the workers and controllers in the SLURM configuration.

      Adding value in a cloud context

      In the first instance, cloud admins recreate application environments, defined by software and deployed on demand. These environments meet user requirements. The convenience of their creation is probably offset by a slight overhead in performance. On balance, an indifferent user might not see compelling benefit to working this way. Our OpenHPC-as-a-Service example described here largely falls into this category.

      Don’t stop here.

      Software-defined cloud methodologies enable us to do some more imaginative things in order to make our clusters the best they possibly can be. We can introduce infrastructure services for consuming and processing syslog streams, simplifying the administrative workload of cluster operation. We can automate monitoring services for ensuring smooth cluster operation, and application performance telemetry as standard to assist users with optimzation. We can help admins secure the cluster.

      All of these things are attainable, because we have moved from managing a deployment to developing the automation of that deployment.

      Reducing the time to science

      Our users have scientific work to do and our OpenStack projects exist to support that.

      We believe that OpenStack infrastructure can go beyond simply recreating conventional scientific application clusters to generate Cluster-as-a-Service deployments that integrate cloud technologies to be even better.

      Further reading

      The post How to build clusters for scientific applications-as-a-service appeared first on OpenStack Superuser.

      by Stig Telfer at August 08, 2017 11:38 AM

      August 07, 2017

      Cisco Cloud Blog

      Cloud Unfiltered Podcast, Episode 14: OpenDaylight, Kolla, and Containers with Vikram Hosakote

      Like many of you out there, Vikram Hosakote was one of those kids who was always taking apart the remote controls and other electronic gadgets in his home to see how they worked. By the time he was 11 he had gotten his hands on an x86 box, and soon he was learning COBOL and […]

      by Ali Amagasu at August 07, 2017 06:38 PM

      OpenStack Superuser

      OpenStack’s seventh birthday: A world of cake!

      There were cakes — cupcakes, sheet cakes, towering wedding-type concoctions — shared by OpenStack users around the world this month from Cote D’Ivoire to Korea. (The Koreans also made a sweet tribute to Stackers everywhere: a tower of Red Bull.)

      We could just call this stack of great pics “Cakes of OpenStack” but we’re already hungry. (Is it snack time yet?)

      Check out some of the festivities for OpenStack’s seventh birthday below — and remember that there are still a few celebrations happening from Puerto Alegre Brazil to Bangkok.

      The post OpenStack’s seventh birthday: A world of cake! appeared first on OpenStack Superuser.

      by Superuser at August 07, 2017 12:32 PM

      Opensource.com

      New report highlights OpenStack's gender diversity and retention

      New OpenStack report recommends ways to increase women's contributions to open source development.

      by uoduckswtd at August 07, 2017 07:00 AM

      August 04, 2017

      OpenStack Superuser

      Using NFS for OpenStack with Security-Enhanced Linux

      As announced already, I have been (between other things) playing with Openstack/RDO and deployed some small openstack setups in the CentOS Infra. Then I had to look at our existing DevCloud setup. This setup was based on Opennebula running on CentOS 6,and also using Gluster as backend for the VM store.

      That’s when I found out that Gluster isn’t a valid option anymore: Gluster was deprecated and even removed from Cinder. Sad, as one advantage of Gluster is that you could (you had to!) user libgfapi so that qemu-kvm process could talk directly to Gluster through ligbfapi and not accessing VM images over locally mounted Gluster volumes (please, don’t even try to do that, through Fuse).

      So what could be a replacement for Gluster from an OpenStack side? I still have some dedicated nodes for storage –backends– but not enough to even just think about Ceph. So it seems my only option was to consider NFS. (Technically speaking, the driver was removed from Cinder, but I could have only tried to use it for Glance and Nova, as I have no need for cinder for DevCloud project, but clearly it would be dangerous for potential upgrades.)

      It’s not that I’m a fan of storing qcow2 images on top of NFS, but it seems it was my only option and at least the most transparent/less intrusive path, would I need to migrate to something else later. So let’s test this before then using NFS through Infiniband (using IPoIB), and so at “good speed” (still have theInfiniBand hardware in place running for Gluster that will be replaced.)

      It’s easy to mount the nfs exported dir under /var/lib/glance/images for glance, and then on every compute node also a nfs export under /var/lib/nova/instances/.

      That’s where you have to see what would be blocked by SELinux, as it seems the current policy shipped with openstack-selinux-0.8.6-0 (from Ocata) doesn’t seem to allow that.

      I initially tested services one and one and decided to open Pull Request for this, but in the mean time I rebuilt a custom SELinux policy that seems to do the job in my RDO playground.

      Here it is the .te that you can compile into usable .pp policy file :

      module os-local-nfs 0.2;
      
      require {
          type glance_api_t;
          type virtlogd_t;
          type nfs_t;
          class file { append getattr open read write unlink create };
          class dir { search getattr write remove_name create add_name };
      }
      
      #============= glance_api_t ==============
      allow glance_api_t nfs_t:dir { search getattr write remove_name create add_name };
      allow glance_api_t nfs_t:file { write getattr unlink open create read};
      
      #============= virtlogd_t ==============
      allow virtlogd_t nfs_t:dir search;
      allow virtlogd_t nfs_t:file { append getattr open };

      Of course you also need to enable some Booleans. Some are already loaded by openstack-selinux (and you can see that from the enabled Booleans by looking at /etc/selinux/targeted/active/booleans.local) but you also now need virt_use_nfs=1

      Now that it works, I can replay that (all that coming from Puppet) on the DevCloud nodes.

      This tutorial first appeared on Fabian Arrotin’s blog.

       

      Superuser is always interested in community content — get in touch: editorATopenstack.org

      Cover Photo // CC BY NC

       

       

      The post Using NFS for OpenStack with Security-Enhanced Linux appeared first on OpenStack Superuser.

      by Fabian Arrotin at August 04, 2017 05:13 PM

      August 03, 2017

      NFVPE @ Red Hat

      Chainmail of NFV (+1 Dexterity) — Service Chaining in Containers using Koko & Koro

      In this episode – we’re going to do some “service chaining” in containers, with some work facilitated by Tomofumi Hayashi in his creation of koko and koro. Koko (the “container connector”) gives us the ability to connect a network between containers (with veth, vxlan or vlan interfaces) in an isolated way (and it creates multiple interfaces for our containers too, which will allow us to chain them), and then we can use the functionality of Koro (the “container routing” tool) to manipulate those network interfaces, and specifically their routing in order to chain them together, and then further manipulate routing and ip addressing to facilitate the changing of this chain. Our goal today will be to connect four containers in a chain of services going from a http client, to a firewall, through a router, and terminating at a web server. Once we have that chain together, we’ll intentionally cause a failure of a service and then repair it using koro.

      by Doug Smith at August 03, 2017 10:00 PM

      Using Koko to create vxlan interfaces for cross-host container network isolation — and cross-connecting them with VPP!

      I’ve blogged about koko in the past – the container connector. Due to the awesome work put forward by my associate Tomofumi Hayashi – today we can run it and connect to FD.io VPP (vector packet processing), which is used for a fast data path, something we’re quite interested with in the NFV space. We’re going to setup vxlan links between containers (on separate hosts) back to a VPP forwarding host, where we’ll create cross-connects to forward packets between those containers. As a bonus, we’ll also compile koro, an auxillary utility to use with koko for “container routing”, which we’ll using in a following companion article. Put your gloves on start up your terminals, we’re going to put our hands right on it and have it all up and running.

      by Doug Smith at August 03, 2017 09:00 PM

      OpenStack Superuser

      Pay it forward: Become an OpenStack mentor

      If you’ve been involved with open source projects for more than 29 seconds, you know it takes a village.

      Now’s the time to step up and help out with your hard-won knowledge by speed mentoring at upcoming Summits, long-term mentoring and the Upstream Institute.

      Emily Hugenbruch, a software engineer at IBM and Women of OpenStack member who has spearheaded these initiatives since they started, recently put out the call to action for upcoming events.  If you’re a potential mentor or mentee, here’s what you need to know.

      Mentors

      We can set you up as a technical mentor based on your areas of expertise within the community or we can match you with a mentee looking for more career advice.Technical mentoring doesn’t just mean coding, we’re looking for people who are a specialist in any area of OpenStack – for example docs or marketing. Technical mentors and mentees should expect to spend about an hour a week talking about projects to get involved in, people to meet, anything to help their mentees get to their next step.
      Career mentors could be in a completely different area of OpenStack from their mentees. They should expect to spend about an hour a month with their mentees offering more general career guidance, maybe helping their mentees make connections in the community.

      Mentees

      There are a variety of Stackers interested in giving you both technical and career advice. We have mentors from a wide range of projects and working groups willing to help you get involved in different aspects of the community.

      “We would like to make as many matches before the Denver PTG  — September 11-15 —  so you can meet with your mentee/mentor there if you’re both attending,”  Hugenbruch says.

      If you’ve been involved with previous rounds of mentoring, we would love to have you back!

      Please fill out the form below, since we are starting from a clean slate on mentors and mentees this time.

      Sign up to get involved here:

      https://openstackfoundation.formstack.com/forms/mentoring_pike_queens

      The post Pay it forward: Become an OpenStack mentor appeared first on OpenStack Superuser.

      by Superuser at August 03, 2017 05:28 PM

      StackHPC Team Blog

      Clusters for Scientific Applications: as-a-Service

      How can we make a workload easier on cloud? In a previous article we presented the lay of the land for HPC workload management in an OpenStack environment. A substantial part of the work done to date focuses on automating the creation of a software-defined workload management environment - SLURM-as-a-Service.

      SLURM logo

      SLURM is only one narrow (but widely-used) use case in a broad ecosystem of multi-node scientific application clusters: let's not over-specialise to that. It raises the question of what is needed to make a generally useful, flexible system for creating Cluster-as-a-Service?

      What do Users Really Want?

      A user of the system will not care how elegantly the infrastructure is orchestrated:

      • Users will want support for the science tools they need, and when new tools are needed, the users will want support for those too.
      • Users will want to get started with minimal effort. The learning curve they must climb to deploy tools needs to be shallow.
      • Users will want easy access to the datasets upon which their research is based.

      Scientists certainly do not want to be given clusters which in truth are just replicated infrastructure. We must provide immediately useful environments that don’t require scientists to be sysadmins. The time to science (or more bluntly, the time to paper) is pretty much the foremost consideration. Being able to use automation to reliably reproduce research findings comes a close second.

      Application Building Blocks

      A really useful application cluster service is built with several key design criteria taken into account:

      • The right model for sharing. Do we provide a globally-available shared infrastructure, project-level multi-user infrastructure or per-user isolation? Per-user isolation might work until the user decides they prefer to collaborate. But can a user trust every other user in their project? Per-project sharing might work unless the users don't actually trust one another (which might in itself be a bigger problem).

      • Users on the cluster. Are the members of the project also to be the users of the cluster? In this case, we should do what we can to offer a cluster deployment that is tightly integrated with the OpenStack environment. Why not authenticate with their OpenStack credentials, for example?

        In a different service model, the OpenStack project members are the cluster admins, not its users. If the cluster is being provided as a service to others who have no connection with the infrastructure, an external mechanism is required to list the users, and to authenticate them. Some flexibility should be supported in an effective solution.

      • Easy data access. Copying user files into a cluster introduces a boundary to cross, which adds inconvenience for using that resource. Furthermore, copying data in requires the same data to be stored in two (or more) places.

        Where a cluster requires a shared filesystem, creating its own ad-hoc filesystem is unlikely to be the best solution. In the same manner as provider networks, a cluster should support "provider filesystems" - site production filesystems that are exported from other infrastructure in the data centre.

        Large scientific datasets may also be required, and are often mediated using platform data services such as iRODS, or HDF5 (note, that's a 5).

        Object storage (S3 in particular) is seen as the long-term solution for connecting applications with datasets, and does appear to be the direction of travel for many. However, sharing read-only filesystems, at either file or block level, are simple and universal approaches that work perfectly well. Both are also well-established choices.

      • Scaling up and down. The cloud access model does not entail queuing and waiting for resources like a conventional HPC batch queuing system. Equally, cloud resources are assumed to grow and shrink dynamically, as required. Perhaps this could even happen automatically, triggered by peaks or troughs in demand. To maximise overall utilisation, cluster resizing is actually quite important, and managing all the resources of a cluster together enables us to do it well.

      • Self-service creation - for some definition of self. Users shouldn't need to learn sysadmin skills in order to create a resource for doing their science. For some, the Horizon web interface might be too complex to bother learning. Enter the ResOps role - for example described in a recent SuperUser article on the Scientific WG - a specialist embedded within the project team, trained on working with cloud infrastructure to deliver the best outcomes for that team. To help the task of cluster creation, it should also be automated to the fullest extent possible.

      Bringing it all together: OpenHPC-as-a-Service

      The OpenHPC project is an initiative to build a community package distribution and ecosystem around a common software framework for HPC.

      OpenHPC logo

      OpenHPC clusters are built around the popular SLURM workload manager.

      A team from Intel including Sunil Mahawar, Yih Leong Sun and Jeff Adams have already presented their work at the latest OpenStack summit in Boston:

      Independently, we have been working on our own OpenHPC clusters as one of our scientific cluster applications on the SKA performance prototype system, and I'm going to share some of the components we have used to make this project happen.

      SKA Alaska: Performance Prototype Platform
      • Software base image: An image with major components baked in saves network bandwidth and deployment time for large clusters. We use our os-images role, available on Ansible Galaxy. We use this role with our custom elements written for Diskimage-builder

        Here's an example of the configuration to provide for the os-images role:

      os_images_list:
        # Build of OpenHPC image on a CentOS base
        - name: "CentOS7-OpenHPC"
          elements:
            - "centos7"
            - "epel"
            - "openhpc"
            - "selinux-permissive"
            - "dhcp-all-interfaces"
            - "vm"
          env:
            DIB_OPENHPC_GRPLIST: "ohpc-base-compute ohpc-slurm-client 'InfiniBand Support'"
            DIB_OPENHPC_PKGLIST: "lmod-ohpc mrsh-ohpc lustre-client-ohpc ntp"
            DIB_OPENHPC_DELETE_REPO: "n"
          properties:
            os_distro: "centos"
            os_version: 7
      
      os_images_elements: ../stackhpc-image-elements
      
      • Flexible infrastructure definition: A flexible definition is needed because it soon becomes apparent that application clusters do not all conform to a template of 1 master, n workers. We use a Heat template in which the instances to create are parameterised as a list of groups. This takes Heat close to the limits of its expressiveness - and requires the Newton release of Heat as a minimum. Some users of Heat take an alternative approach of code-generated Heat templates.

        Our flexible Heat template is encapsulated within an Ansible role available on Ansible Galaxy. This role includes a task to generate a static inventory file, extended with user-supplied node groupings, which is suitable for higher-level configuration.

        The definition of the compute node components cluster takes a simple form:

      cluster_groups:
        - "{{ slurm_login }}"
        - "{{ slurm_compute }}"
      
      slurm_login:
        name: "login"
        flavor: "compute-B"
        image: "CentOS7-OpenHPC"
        num_nodes: 1
      
      slurm_compute:
        name: "compute"
        flavor: "compute-A"
        image: "CentOS7-OpenHPC"
        num_nodes: 8
      

      The invocation of the infrastructure role becomes equally simple:

      ---
      # This playbook uses the Ansible OpenStack modules to create a cluster
      # using a number of baremetal compute node instances, and configure it
      # for a SLURM partition
      - hosts: openstack
      
        roles:
          - role: stackhpc.cluster-infra
            cluster_name: "{{ cluster_name }}"
            cluster_params:
              cluster_prefix: "{{ cluster_name }}"
              cluster_keypair: "{{ cluster_keypair }}"
              cluster_groups: "{{ cluster_groups }}"
              cluster_net: "{{ cluster_net }}"
            cluster_roles: "{{ cluster_roles }}"
      
      • Authenticating our users: On this prototype system, we currently have a small number of users, and these users are locally defined within Keystone. In a larger production environment, a more likely scenario would be that the users of an OpenStack cloud are stored within external authentication infrastructure, such as LDAP.

        Equivalent user accounts must be created on our OpenHPC cluster. Users need to be able to login on the externally-facing login node. The users should be defined on the batch compute nodes, but they should not be able to login on these instances.

        Our solution is to enable our users to authenticate using Keystone on the login node. This is done using two projects, PAM-Python and PAM-Keystone - a minimal PAM module that performs auth requests using the Keystone API. Using this, our users benefit from common authentication on OpenStack and all the resources created on it.

      • Access to cluster filesystems: OpenHPC clusters require a common filesystem mounted across all nodes in a workload manager. One possible solution here would be to use Manila, but our bare metal infrastructure may complicate its usage. It is an area for future exploration for this project.

        We are using CephFS, exported from our local Ceph cluster, with an all-SSD pool for metadata and a journaled pool for file data. Our solution defines a CephX key, shared between project users, which enables access to the CephFS storage pools and metadata server. This CephX key is stored in Barbican. This appears to be an area where support in Shade and Ansible's own OpenStack modules is limited. We have written an Ansible role for retrieving secrets from Barbican and storing them as facts, and we'll be working to package it and publish on Galaxy in due course.

      • Converting infrastructure into platform: Once we have built upon the infrastructure to add the support we need, the next phase is to configure and start the platform services. In this case, we build a SLURM configuration that draws from the infrastructure inventory to define the workers and controllers in the SLURM configuration.

      Adding Value in a Cloud Context

      In the first instance, cloud admins recreate application environments, defined by software and deployed on demand. These environments meet user requirements. The convenience of their creation is probably offset by a slight overhead in performance. On balance, an indifferent user might not see compelling benefit to working this way. Our OpenHPC-as-a-Service example described here largely falls into this category.

      Don't stop here.

      Software-defined cloud methodologies enable us to do some more imaginative things in order to make our clusters the best they possibly can be. We can introduce infrastructure services for consuming and processing syslog streams, simplifying the administrative workload of cluster operation. We can automate monitoring services for ensuring smooth cluster operation, and application performance telemetry as standard to assist users with optimsation. We can help admins secure the cluster.

      All of these things are attainable, because we have moved from managing a deployment to developing the automation of that deployment.

      Reducing the Time to Science

      Our users have scientific work to do, and our OpenStack projects exist to support that.

      We believe that OpenStack infrastructure can go beyond simply recreating conventional scientific application clusters to generate Cluster-as-a-Service deployments that integrate cloud technologies to be even better.

      by Stig Telfer at August 03, 2017 04:00 PM

      Aptira

      OpenStack Summit Sydney – Community Voting

      The first ever Sydney OpenStack Summit is getting closer, and as one of the headline sponsors, we’re gearing up for a great event!
      Session voting is open until Thursday, August 3 at 11:59pm Pacific Time (August 4 at 6:59 UTC). We’ve submitted a few sessions, so if you’d like to hear more from us at the Summit, please vote for the following sessions:

      View full details of the session selection process here, or visit https://www.openstack.org/summit/sydney-2017/ for all Summit-related information. We look forward to seeing you there!

      The post OpenStack Summit Sydney – Community Voting appeared first on Aptira Cloud Solutions.

      by Aptira at August 03, 2017 03:28 AM

      hastexo

      More recommendations for Ceph and OpenStack

      A few months ago, we shared our Dos and Don'ts, as they relate to Ceph and OpenStack. Since that post has proved quite popular, here are a few additional considerations for your Ceph-backed OpenStack cluster.

      Do configure your images for VirtIO-SCSI

      By default, RBD-backed Nova instances use the virtio-blk driver to expose RBD images to the guest -- either as ephemeral drives, or as persistent volumes. In this default configuration, VirtIO presents a virtual PCI device to the guest that represents the paravirtual I/O bus, and devices are named /dev/vda, /dev/vdb, and so forth. VirtIO block devices are lightweight and efficient, but they come with a drawback: they don't support the discard operation.

      Not being able to use discard means the guest cannot mount a filesystem with mount -o discard, and it also cannot clean up freed blocks on a filesystem with fstrim. This can be a security concern for your users, who might want to be able to really, actually delete data from within the guest (after overwriting it, presumably). It can also be an operational concern for you as the cluster operator.

      This is because not supporting discard also means that RADOS objects owned by the corresponding RBD image and never removed during the image's lifetime -- they persist until the whole image is deleted. So your cluster may carry the overhead of perhaps tens of thousands of RADOS objects that no-one actually cares about.

      Thankfully, there is an alternative VirtIO disk driver that does support discard: the paravirtualized VirtIO SCSI controller, virtio-scsi.

      Enabling the VirtIO SCSI controller is something you do by setting a couple of Glance image properties, namely hw_scsi_model and hw_disk_bus. You do so by running the following openstack commands on your image:

      openstack image set \
        --property hw_scsi_model=virtio-scsi \
        --property hw_disk_bus=scsi \
        <name or ID of your image>
      

      Then, if you boot an instance from this image, you'll see that its block device names switch from /dev/vdX to /dev/sdX, and you also get everything else you expect from a SCSI stack. For example, there's /proc/scsi/scsi, you can extract information about your bus, controller, and LUs with lsscsi command, and so on.

      It's important to note that this image property is inherited by the instance booted from that image, which also passes it on to all volumes that you may subsequently attach to that instance. Thus, openstack server add volume will now add /dev/sdb, not /dev/vdb, and you will automatically get the benefits of discard on your volumes, as well.

      Do set disk I/O limits on your Nova flavors

      In a Ceph cluster that acts as backing storage for OpenStack, naturally many OpenStack VMs share the bandwidth and IOPS of your whole cluster. When that happens, occasionally you may have a VM that’s very busy (meaning it produces a lot of I/O), which the Ceph cluster will attempt to process to the best of its abilities. In doing so, since RBD has no built-in QoS guarantees (yet), it might cause other VMs to suffer from reduced throughput, increased latency, or both.

      The trouble with this is that it’s almost impossible for your users to calculate and reckon with. They’ll see a VM that sustains, say, 10,000 IOPS at times, and then drop to 2,000 with no warning or explanation. It is much smarter to pre-emptively limit Ceph RBD performance from the hypervisor, and luckily, OpenStack Nova absolutely allows you to do that. This concept is known as instance resource quotas, and you set them via flavor properties. For example, an you may want to limit a specific flavor to 1,500 IOPS and a maximum throughput of 100 MB/s:

      openstack flavor set \
        --property quota:disk_total_bytes_sec=$((100<<20))
        --property quota:disk_total_iops_sec=1500
        m1.medium
      

      In the background, these settings are handed through to libvirt and ultimately fed into cgroup limitations for Qemu/KVM, when a VM with this flavor spins up. So these limits aren’t specific to RBD, but they come in particularly handy when dealing with RBD.

      Obviously, since flavors can be public, but can also be limited to specific tenants, you can set relatively low instance resource quotas in public flavors, and then make flavors with higher resource quotas available to select tenants only.

      Do differentiate Cinder volume types by disk I/O limits

      In addition to setting I/O limits on flavors for VMs, you can also influence the I/O characteristics of volumes. You do so by specifying distinct Cinder volume types. Volume types are frequently used to enable users to select a specific Cinder backend — say, to stick volumes either on a NetApp box or on RBD, but it’s perfectly OK if you define multiple volume types using the same backend. You can then set characteristics like maximum IOPS or maximum throughput via Cinder QoS specifications. A QoS specification akin to the Nova flavor decribed above — limiting throughput to 100 MB/s and 1,500 IOPS would be created like this:

      openstack volume qos create \
        --consumer front-end
        --property total_bytes_sec=$((100<<20)) \
        --property total_iops_sec=1500 \
        "100MB/s-1500iops"
      

      You would then create a corresponding volume type, and associate the QoS spec with it:

      openstack volume type create \
        --public \
        "100MB/s-1500iops"
      openstack volume qos associate \
        "100MB/s-1500iops" \
        "100MB/s-1500iops"
      

      Again, as with Nova flavors, you can make volume types public, but you can also limit them to specific tenants.

      Don't forget about suspend files

      When you suspend a Nova/libvirt/KVM instance, what really happens is what libvirt calls a managed save: the instance's entire memory is written to a file, and then KVM process shuts down. This is actually quite neat because it means that the VM does not consume any CPU cycles nor memory until it restarts, and it will continue right where it left off, even if the compute node is rebooted in the interim.

      You should understand that these savefiles are not compressed in any way: if your instance has 16GB of RAM, that's a 16GB file that instance suspension drops into /var/lib/nova/save. This can add up pretty quickly: if a single compute node hosts something like 10 suspended instances, their combined save file size can easily exceed 100 GB. Obviously, this can put you in a really bad spot if this fills up your /var (or worse, /) filesystem.

      Of course, if you already have a Ceph cluster, you can put it to good use here too: just deep-mount a CephFS file system into that spot. Here's an Ansible playbook snippet that you may use as inspiration:

      ---
      - hosts:
        - compute-nodes
      
        vars:
          ceph_mons:
            - ceph-mon01
            - ceph-mon02
            - ceph-mon03
          cephfs_client: cephfs
          cephfs_secret: "{{ vaulted_cephfs_secret }}"
      
        tasks:
      
        - name: "install ceph-fs-common package"
          apt:
            name: ceph-fs-common
            state: installed
      
        - name: "create ceph directory"
          file:
            dest: /etc/ceph
            owner: root
            group: root
            mode: '0755'
            state: directory
      
        - name: "create cephfs secretfile"
          copy:
            dest: /etc/ceph/cephfs.secret
            owner: root
            group: root
            mode: '0600'
            content: '{{ cephfs_secret }}'
      
        - name: "mount savefile directory"
          mount:
            fstype: ceph
            path: /var/lib/nova/save
            src: "{{ ceph_mons | join(',') }}:/nova/save/{{ ansible_hostname }}"
            opts: "name={{ cephfs_client }},secretfile=/etc/ceph/cephfs.secret"
            state: mounted
      
        - name: "fix savefile directory ownership"
          file:
            path: /var/lib/nova/save
            owner: libvirt-qemu
            group: kvm
            state: directory
      

      Got more?

      Do you have Ceph/OpenStack hints of your own? Leave them in the comments below and we’ll include them in the next installment.

      by hastexo at August 03, 2017 12:00 AM

      August 02, 2017

      Ben Nemec

      Creating OpenStack Client Instances in Python

      So you're writing a Python application and you want to talk to an OpenStack service. Or maybe even multiple services. A quick Google turns up something like python-novaclient, which provides a way to create an instance of a Nova client in Python. Great! We're done here, right? Not so fast, my friend.

      While it is true that you can directly use python-novaclient to talk to Nova, it's not necessarily your best move. Why? Well, there's a dirty little secret about the python-*client projects for OpenStack: They were mostly written for the services to talk to each other, not for users to talk to the services. True, they can be used for the latter, but as we'll discuss here there are some pretty serious usability issues with doing that.

      The Problems

      The biggest problem is Keystone v3. With Keystone v2 you could get away with directly instantiating client instances. There were only a few required auth parameters and it was pretty easy to figure out which ones you needed to provide (username, password, auth_url, and tenant, essentially). Not so bad, at least for novaclient. Other clients were not so nice, but more on that later.

      Enter Keystone v3. Now you have domains to worry about in addition to all the above auth parameters. But that's okay, we just need to make sure we provide the extra parameters and everything will be fine, right? Nope. This was the approach I tried to take when adding Keystone v3 support to OpenStack Virtual Baremetal, and it went badly enough to prompt this blog post.

      The problem is that apparently not every cloud needs exactly the same Keystone v3 auth parameters. I don't actually understand why that is the case, but in my experience it is. And on top of that there's no apparent sanity in why some parameters are needed but others are not.

      For example, against my local Devstack-based cloud I needed to provide the following auth parameters: username, password, auth_url, project, user_domain, and project_domain. Fair enough. I added those configuration options and thought my work was done. It was not. Against RDO Cloud (a cloud being run by the RDO admins to provide development resources for RDO and its related projects), those auth parameters did not work. I don't have the exact error message anymore, but it was something about expecting to find the user in the domain. Or maybe the domain in the user. I can't remember and it's not relevant because the point of this blog post is to help you avoid it in the first place. :-)

      Ultimately I discovered that RDO cloud needed the following parameters: username, password, auth_url, project_name, project_id (yes, both), and user_domain_name. But strangely, not project_domain_name. In fact, the rc file provided did not even set project_domain_name. But it did set both project_name and project_id, and if I tried to remove the id (because previously I had been passing it by name) it caused my client calls to fail.

      I haven't the faintest clue why these different configurations are needed, and I have even less idea how to predict what a particular cloud is going to require and include logic that can just do the right thing. If I've convinced you that you don't want to deal with this yourself, you can skip below to The Solution. But I'm going to continue and discuss another drawback of creating your own clients from scratch, for historical reference if nothing else.

      The other drawback relates to my earlier point about the clients being primarily used to talk between services. Within a service, you probably have a number of things in place already. Keystone sessions and tokens, for example. When starting from scratch in your own application you don't. Some clients require a Keystone session or token to be passed in on client creation, which makes your simple "instantiate a client" call turn into multiple calls to different services.

      One particularly fun example is the Heat client. It requires a Keystone token and the Heat endpoint in order to construct a client instance. So you get to retrieve a token and then pull the id and endpoint out of that data. But wait, there's more! The token data format changed from v2 to v3, so I had to special-case the endpoint retrieval logic based on the version. There may have been a better way to do that, but if so I wasn't aware of it at the time. And the data was in the token content, so why wouldn't I just read it from there?

      Heat is hardly the only client that requires excessive effort just to create a client instance though. Novaclient gets a special mention here too for breaking compatibility from v6 to v7. It was necessary to fix problems with its own Keystone v3 support, but it was still a headache for users of the project.

      If you're not convinced by now that you don't want to deal with all this hassle, then more power to you. I threw up my hands and made it someone else's problem. Read on for details.

      The Solution

      TLDR: Use os-client-config.

      Or Shade. I went with os-client-config because it was a drop-in replacement since I was already calling the clients directly. YMMV.

      Honestly, the only reason I didn't do this in the first place is because my initial introduction to os-client-config was as a way to add clouds.yaml support to OVB. But I only work with a few clouds, so I don't use clouds.yaml. I just have an rc file for each cloud I use.

      Good news! os-client-config supports that too! There's a special value that you can pass in to the 'cloud' parameter in its make_client function that will cause it to read the authentication data from the environment, just like I was doing manually when calling the clients directly. That special value is cleverly named "envvars". I actually found that passing cloud=None also resulted in the correct behavior, but that may vary depending on whether you have a clouds.yaml or not. I believe that makes it use the "default" cloud, which apparently is "envvars" if you have no clouds.yaml at all.

      Here's an example of creating a client with os-client-config:

      nova = os_client_config.make_client('compute', cloud='envvars')

      The OVB build-nodes-json command needs to create 3 different OpenStack clients: nova, neutron, and glance. The code I used to have for that was around 30 lines and those 30 lines called into dozens of additional lines of code in authentication helper functions. The os-client-config version is three lines, the one above and two more for neutron and glance. That's it.

      Conclusion

      The change to os-client-config has made OVB much less complex and more maintainable. I'm hopeful that documenting my struggles with this may help someone else avoid them altogether since my mistakes were almost entirely due to ignorance. Had I known all of this ahead of time I'd have made much different choices. Better choices. I guess the main takeaway from this post is "make better choices than I did." :-)

      by bnemec at August 02, 2017 10:31 PM

      RDO

      Upcoming events

      There's a number of upcoming events where RDO enthusiasts will be present. Mark your calendar!

      Join us for Test Day!

      Milestone 3 of the Pike cycle was released last week, and so it's time to test the RDO packages. Join us on Thursday and Friday of next week (August 10th and 11th) for the Pike M3 test day. We'll be on the #RDO channel on the Freenode IRC network to test, help, answer questions, and propose solutions.

      Meetups

      We encourage you to attend your local OpenStack meetup. There's hundreds of them, all over the world, every day. We post a list of upcoming meetups to the RDO mailing list each Monday, so that you can mark your calendar. Or you can search meetup.com for events near you.

      If you're going to be speaking at an OpenStack meetup group, and you'd like to have some RDO swag to take along with you, please contact me - rbowen@redhat.com - or Rain - rain@redhat.com - with your request, at least two weeks ahead of time.

      And if you're a meetup organizer who is looking for speakers, we can sometimes help you with that, too.

      OpenStack Days

      The following OpenStack days are coming up, and each of them will have RDO enthusiasts in attendance:

      If you're speaking at any of these events, please get in touch with me, so that I can help promote your presence there.

      Other Events

      The week of September 11 - 15th, the OpenStack PTG will be held in Denver, Colorado. At this event, project teams will meet to determine what features will be worked on for the upcoming Queens release. Rich Bowen will be there conducting video interviews, like last time, where OpenStack developers will be talking about what they worked on in Pike, and what to expect for Queens. If you'll be there, watch the announcements from the OpenStack foundation for how to sign up for an interview slot.

      On October 20th, we'll be joining up with the CentOS community at the CentOS Dojo at CERN. Details of that event may be found on the CERN website. CERN is located just north of Geneva. The event is expected to have a number of RDO developers in attendance, as CERN has one of the largest OpenStack deployments in the world, running on RDO.

      Other RDO events, including the many OpenStack meetups around the world, are always listed on the RDO events page. If you have an RDO-related event, please feel free to add it by submitting a pull request on Github.

      by Rich Bowen at August 02, 2017 06:30 PM

      OpenStack Superuser

      Creating an OpenStack Ironic deploy image with Buildroot

      Ironic is an OpenStack project which provisions bare metal machines (as opposed to virtual).

      A tool called Ironic Python Agent (IPA) is used to control and provision these physical nodes, performing tasks such as wiping the machine and writing an image to disk. This is done by booting a custom Linux kernel and initramfs image which runs IPA and connects back to the Ironic Conductor.

      The Ironic project supports a couple of different image builders, including CoreOS, TinyCore and others via Disk Image Builder.

      These have their limitations, however, for example they require root privileges to be built and, with the exception of TinyCore, are all hundreds of megabytes in size. One of the downsides of TinyCore is limited hardware support and although it’s not used in production, it is used in the OpenStack gating tests (where it’s booted in virtual machines with ~300MB RAM).

      Large deployment images means a longer delay in the provisioning of nodes and so I set out to create a small, customizable image that solves the problems of the other existing images.

      Buildroot

      I chose to use Buildroot, a well regarded, simple to use tool for building embedded Linux images.

      So far it has been quite successful as a proof of concept.

      Customization can be done via the menuconfig system, similar to the Linux kernel.

      Buildroot menuconfig
      Buildroot menuconfig

      Source code

      All of the source code for building the image is up on my GitHub account in the ipa-buildroot repository. I have also written up documentation which should walk you through the whole build and customisation process.

      The ipa-buildroot repository contains the IPA specific Buildroot configurations and tracks upstream Buildroot in a Git submodule. By using upstream Buildroot and our external repository, the IPA Buildroot configuration comes up as an option for regular Buildroot build.

      IPA in list of Buildroot default configs
      IPA in list of Buildroot default configs

      Buildroot will compile the kernel and initramfs, then post build scripts clone the Ironic Python Agent repository and creates Python wheels for the target.

      The following finalised images will be found under ./output/images:

      • bzImage (kernel)
      • rootfs.cpio.xz (ramdisk)
      • rootfs.iso9660 (ISO image)

      These files can be uploaded to Glance for use with Ironic.

      Customization

      Buildroot lets you customize the build parameters, the target initramfs and the Linux kernel. Doing so also is pretty easy and each time you run make, it’ll pick up where you left off and re-create your images.

      make menuconfig
      # do buildroot changes, e.g. change IPA Git URL
      make linux-menuconfig
      # do kernel changes
      make

      The post build scripts fetch Ironic Python Agent directly so that the build is highly flexible, getting the exact release you want to use. You can specify the location and branch of the ironic-python-agent and requirements repositories by customizing settings in Buildroot’s menuconfig.

      Set Ironic Python Agent and Requirements location and Git version
      Set Ironic Python Agent and Requirements location and Git version

      I created the kernel config from scratch (using tinyconfig) and deliberately tried to balance size and functionality. It should boot on most Intel based machines (BIOS and UEFI), however hardware support like hard disk and ethernet controllers is deliberately limited. The goal was to start small and add more support as needed so it may need to be modified to work on your hardware.

      Automating builds with imagebuild scripts

      Doing a build manually is good because it helps to understand how it’s pieced together, however it is slightly more involved.

      The Ironic Python Agent (IPA) repo has some imagebuild scripts which make building the CoreOS and TinyCore images pretty trivial. I now have some patches which add support for creating the Buildroot images, too.

      The patches consist of a few scripts which wrap the manual build method and a Makefile to tie it all together. Only the install-deps.sh script requires root privileges, if it detects missing dependencies, all other Buildroot tasks are run as a non-privileged user. It’s one of the great things about the Buildroot method!

      Again, I have included documentation in the repo, so please see there for more details on how to build and customise the image. However in short, it is as simple as:

      git clone https://github.com/csmart/ironic-python-agent.git
      cd ironic-python-agent/imagebuild/buildroot
      make
      # or, alternatively:
      ./build-buildroot.sh --all

      These actions will perform the following tasks automatically:

      • Fetch the Buildroot Git repositories
      • Load the default IPA Buildroot configuration
      • Download and verify all source code
      • Build the toolchain
      • Use the toolchain to build:
        • System libraries and packages
        • Linux kernel
        • Python Wheels for IPA and dependencies
      • Create the kernel, initramfs and ISO images

      The default configuration points to the upstream IPA Git repository, however you can change this to point to any repo and commit you like. For example, if you’re working on IPA itself, you can point Buildroot to your local Git repo and then build and boot that image to test it!

      Help

      To see available Makefile targets, simply run the help target:

      make help

      Help is also available for the shell scripts if you pass the –help option:

      ./build-buildroot.sh --help
      ./clean-buildroot.sh --help
      ./customise-buildroot.sh --help

      If this interests you, please check it out! Any suggestions are welcome.

      Chris Smart describes himself as “just another Linux geek.” This post first appeared on his blog in two parts.

      Cover Photo // CC BY NC

      The post Creating an OpenStack Ironic deploy image with Buildroot appeared first on OpenStack Superuser.

      by Chris Smart at August 02, 2017 05:47 PM

      Lars Kellogg-Stedman

      Ansible for Infrastructure Testing

      At $JOB we often find ourselves at customer sites where we see the same set of basic problems that we have previously encountered elsewhere ("your clocks aren't in sync" or "your filesystem is full" or "you haven't installed a critical update", etc). We would like a simple tool that could …

      by Lars Kellogg-Stedman at August 02, 2017 04:00 AM

      OpenStack Blog

      OpenStack Developer Mailing List Digest July 22-28

      Summaries

      • Nova placement/resource providers update 30 1
      • TC Report 30 2
      • POST /api-wg/news 3
      • Release countdown for week R-4, July 28 – Aug 4 4
      • Technical Committee Status update, July 28 5

      Project Team Gathering Planning

      • Nova 6
      • Keystone 7
      • Sahara 8
      • Cinder 9
      • Oslo 10
      • Neutron 11
      • Documentation 12

      Oslo DB Network Database Base namespace throughout OpenStack Projects

      • Mike Bayer has been working with Octave Oregon on adding the network database storage engine for mysql to oslo.db module so other projects can take advantage of it. Mike Bayer notes:
        • The code review 13
        • Support in Nova 14
        • Support in Neutron 15
      • New data types that map to types from the NDB namespace.
        • oslo_db.sqlalchemy.types.SmallString
        • oslo_db.sqlalchemy.types.String
      • Full thread: 16
      1. http://lists.openstack.org/pipermail/openstack-dev/2017-July/120290.html
      2. http://lists.openstack.org/pipermail/openstack-dev/2017-July/120112.html
      3. http://lists.openstack.org/pipermail/openstack-dev/2017-July/120245.html
      4. http://lists.openstack.org/pipermail/openstack-dev/2017-July/120304.html
      5. http://lists.openstack.org/pipermail/openstack-dev/2017-July/120280.html
      6. http://lists.openstack.org/pipermail/openstack-dev/2017-July/120020.html
      7. http://lists.openstack.org/pipermail/openstack-dev/2017-July/119299.html
      8. http://lists.openstack.org/pipermail/openstack-dev/2017-July/119352.html
      9. http://lists.openstack.org/pipermail/openstack-dev/2017-July/119358.html
      10. http://lists.openstack.org/pipermail/openstack-dev/2017-July/119462.html
      11. http://lists.openstack.org/pipermail/openstack-dev/2017-July/120270.html
      12. http://lists.openstack.org/pipermail/openstack-dev/2017-July/119990.html
      13. https://review.openstack.org/#/c/427970/
      14. https://review.openstack.org/#/c/446643
      15. https://review.openstack.org/#/c/446136/
      16. http://lists.openstack.org/pipermail/openstack-dev/2017-July/thread.html#120037

      by Mike Perez at August 02, 2017 01:57 AM

      August 01, 2017

      Chris Dent

      TC Report 31

      Writing these reports is somewhat difficult when there hasn't been a meeting and we're in a crunch time in the release cycle. The things that have happened that are related to TC activity are disjointed, a bit hard to track, and hard to summarize. There have, however, been a few blips in IRC traffic in #openstack-tc and in the governance repo, so I'll attempt to apply my entirely subjective eye to those things.

      Tags, Projects, Deliverables, Oh My

      The review to add the api interop assert tag to ironic and nova has foundered a bit on the diversity of opinions on not just what that specific tag means, but what tags in general mean, and who uses them. There's plenty of conversation on that review, as well as most recently some chatter in this morning's office hour.

      One outcome of the discussion (in its many places) is that the primary use of the tags is to engineer a technical solution to a marketing problem, in the form of the Project Navigator. As such it tends to get messy at the interface and difference between the boundaries and identifiers that matter to the people developing the projects and the boundaries and identifiers that matter to the people promoting or using the project navigator. It can also mean that anyone who is aware of the marketing value of the tags may experince conflicts when considering any of the tags which may be voluntarily asserted.

      The current case in point is that if asserting the api interoperability tag is presented as a sign of maturity in the navigator, any project that considers itself mature would like to not be penalized in the event that they are mature in a way that is not in alignment with the tag.

      The Top 5 List, Barbican, Designate, Castellan

      Another topic from this morning's office hour was discussing whether it might make sense to put contribution to the barbican and designate projects on the Top 5 List and/or make them base services (there's something of a chicken and egg problem there).

      Later in the day the topic was picked back up, with dims suggesting that castellan should be the base requirement, not barbican. Castellan, however, does not yet satisfy all the needs designate would have, if doing DNSSEC.

      Further discussion required, as far as I could tell.

      Increasing Engagement From And With People In China

      Last Thursday's office hour started with a brief report about China where ttx and many others had been at OpenStack Days China. This led to a discussion about the many difficulties present in synchronous communication technologies including all forms of chat, audio and video. Each presents difficulties either in terms of comprehension, limited access, or issues with latency.

      yeh, we need to get people less al[l]ergic to doing real conversations in email :) sdague

      (Words to live by.)

      by Chris Dent at August 01, 2017 07:00 PM

      OpenStack Superuser

      How to make successful contributions to open source projects

      What’s the secret sauce to making successful contributions to open source projects?  Superuser gets the scoop from two OpenStack veterans on why it takes a village and the pleasure in tossing out old code.

      Kendall Nelson, upstream developer advocate:

      Contributing to, maintaining and using open source code is a tall order if you have no experience with the people that surround the project. To effectively do all three, you need to understand the community. There is a history and an etiquette you should know to effectively use the code and get the help you need if you hit a wall.  There is a structure and set of expectations the community has that is vital to becoming a member; however, once you are a member, contributions to code, bug fixes, documentation additions, etc.  are infinitely easier and quicker than if you had made no effort to join the community. OpenStack is like a family in that once you make a connection, people will have your back and you’re in for life.

      Ildiko Vancsa, ecosystem team lead:

      Maintaining open source code also means maintaining the team and community around it; always be open to new people, new perspectives and new ideas.
      Invest time and effort into mentoring people joining your team and community; while you gain new members who understand the code and technology better, you will also refresh your knowledge and learn new things and viewpoints.
      When you start contributing upstream it can be challenging to join a large and diverse community for the first time, but it also gives you invaluable experience and a lot to learn.
      Don’t be afraid to make your voice heard, but never forget to listen to others either at the same time.
      In an open source environment, communication skills are just as important as coding and technical knowledge, even if many don’t admit that.
      I often hear from developers that they like deleting code. I think it’s very important in an open source project to remove the old code paths that are not used anymore and keep the code as clean as possible. It increases readability and maintainability and often more improvements come out of the process than the original plans outlined.

      What tips you would add? Tweet @superuser or add in the comments below!

      The post How to make successful contributions to open source projects appeared first on OpenStack Superuser.

      by Superuser at August 01, 2017 05:00 PM

      About

      Planet OpenStack is a collection of thoughts from the developers and other key players of the OpenStack projects. If you are working on OpenStack technology you should add your OpenStack blog.

      Subscriptions

      Last updated:
      August 23, 2017 01:35 PM
      All times are UTC.

      Powered by:
      Planet