Block Device Mapping? Block Device Mapping(s)(BDMs) define how block devices are exposed to an instance by Nova. At present Nova accepts and stores data relating to these mappings in various ways across the codebase often leading to the five stages of grief being experienced by anyone unlucky enough to come into contact with them. The user facing format while awkward and unforgiving is pretty well documented both in the API reference guide and project user documentation.

VEXXHOST is a Canadian cloud computing provider with an impressive global presence and record. As a former Superuser Awards winner and a major contributor to the open source community, VEXXHOST’s growth trajectory is of keen interest to any member of the community. 

From small businesses to governments, VEXXHOST’s operations are spread over 150 countries. The company started as a web hosting provider in 2006 and later transitioned into cloud solutions. They adopted OpenStack for their cloud operations in 2011, coinciding with the platform’s second release, Bexar. VEXXHOST now offers OpenStack-based public cloud, private cloud, consulting, and other enterprise-grade cloud solutions. 

VEXXHOST recently announced a massive revamp in its public cloud offerings during its keynote address at the Open Infrastructure Summit 2020. There are some new and exciting developments, including a brand new Data Center in Amsterdam. The company says that the revamp is part of their new growth strategy, powered by their association with various open source foundations and communities. They are a reputed member of the Open Infrastructure Foundation, Linux Foundation, CNCF, and the Ceph Foundation, to name a few. 

With this revamp, VEXXHOST joins the range of community-driven companies evolving and growing strong even during these testing times (i.e., pandemic time). Without further delay, here’s what they have included in the cloud revamp. 

Amsterdam – New Region

The Amsterdam data center is VEXXHOST’s third region to date, the other two being Montreal and Santa Clara. VEXXHOST also manages several private cloud operations for different enterprises and organizations in various other parts of the world, in association with local data centers. The launch of the new region not only puts them physically in a different continent but also cements their global footprint.

The state-of-the-art data center used in Amsterdam provides unparalleled service in terms of connectivity, efficiency, server protection mechanisms,  security, and reliability. The facility is built to Uptime Institute Tier 3 standards, and has  PCI/SOC, and AMS-IX, and multiple ISO certifications. The Amsterdam data center also boasts of a multi-tier security system, 24×7 on-site support, high energy efficiency with green standards, advanced global connectivity via being linked to AMS-IX, among other features. 

Modified Public Cloud Pricing 

Another change that is part of the revamp is VEXXHOST’s new pricing strategy. The modified pricing is available to all new users as well as existing customers who make the switch to their latest solutions. 

According to VEXXHOST’s new introductory and limited-time pricing model, the hourly rate for a standard 2 core, 8 GB offering is just $0.055, compared to the market rate, which stands at $0.086. The difference between the current market rate and the new VEXXHOST rate is 36%. Higher core/memory offerings such as 4 core – 16 GB, 8 core – 32 GB, and 16 core – GB, will also carry proportionate price reduction.

This new pricing strategy couldn’t have come at a better time. Many businesses are facing a lot of challenges due to the pandemic situation and there is also a need for companies to find remote and cloud solutions to adapt to the rapid changes around. Getting users enterprise-grade cloud solutions at great pricing is beneficial not only for the parties involved but also for the overall community.

Montreal Region – 2nd Gen AMD EPYC-powered Servers 

Next in VEXXHOST’s revamp is the addition of new servers equipped with 2nd Gen AMD EPYC processors for their Montreal region. Driven by AMD Infinity Architecture, these processors are the first x86 processor of their kind for servers based on 7nm process technology. Also boasting of a hybrid, multi-die architecture, they offer up to 64 high-performance cores per SOC and are equipped with AMD Infinity Guard and PCIe® Gen 4 I/O security features. 

The new AMD processors bring about significant change to VEXXHOST’s servers in better performance and secure computing, improving workload acceleration, data protection, and overall infrastructure to the users.

Montreal Region – Upgraded Storage from SSD to NVMe

VEXXHOST has upgraded storage from SSD to NVMe in their Montreal region, offering its public cloud users the fastest hard drive in the market. A major benefit available with this storage type is faster parallel read and write capabilities.  

NVMe drives can work at speeds of more than 2,000MB/s compared to the typical SATA III SSD running under 600MB/s. This is possible because of the interaction with flash memory, made through a PCIe interface. This interface is bi-directional and runs at a stellar speed. NVMe storage is also better in power-efficiency, reducing consumption by significant percentages in standby mode. NVMe is also a scalable alternative going beyond the four lanes found in most conventional PCIe SSDs. 

Moving forward, VEXXHOST has a lot planned in terms of cloud offerings to customers. If you would like to know more about the company and its extensive cloud solutions, check out VEXXHOST’s website.

The post New in OpenInfra: VEXXHOST’s Public Cloud Revamp Includes a Lot More Than the New Data Center in Amsterdam appeared first on Superuser.

What changed with CentOS?

What’s the current situation in RDO?

What’s our plan for RDO roadmap?

• RDO Wallaby (ETA is end of April 2021) will be built, tested and released only on CentOS 8 Stream.
• RDO CloudSIG repos for Victoria and Ussuri will be updated and tested for both CentOS Stream and CentOS Linux 8 until end of 2021 and then continue on CentOS Stream.
• We will create and test new RDO CloudSIG repos for Victoria and Ussuri on CentOS Stream 8.
• The RDO Trunk repositories (aka DLRN repos) will be built and tested using CentOS 8 Stream buildroot for all releases currently using CentOS Linux 8 (since Train on)

How do we plan to implement these changes?

• We will keep building packages just once. We will move buildroots for both DLRN and CloudSIG to use CentOS Stream 8 in the near future.
• For Ussuri and Victoria CloudSIG repos, while we are supporting both C8 and C8S, we will be utilizing separated CBS Tags. This will allow us to have separated repositories, promotion jobs and package versions for each OS.
• In order to reduce the impact of potential issues and discover issues related to C8S as soon as possible, we will put more focus on periodic jobs on C8S.
• At a later stage, we will move the CI jobs used to gate changes in distgits to use C8S instead of C8 for all RDO releases where we use CentOS 8.
• The CentOS/RHEL team has made public their interest in applying Continuous Delivery approach to CentOS Stream to provide a stable CentOS Stream using gating integration jobs. Our intent is to collaborate with the CentOS team on any initiatives that will help to validate RDO as early in the delivery pipeline as possible and reduce the impact on potential issues.

What’s next?

Codership is pleased to announce a new Generally Available (GA) release of the multi-master Galera Cluster for MySQL 5.6, 5.7 and 8.0, consisting of MySQL-wsrep 5.6.50 (release notes, download), 5.7.32 (release notes, download) and 8.0.22 (release notes, download) with Galera replication library 3.32 (release notes, download) implementing wsrep API version 25, and Galera replication library 4.7 (release notes, download) implementing wsrep API version 26 for 8.0. This release incorporates all changes to MySQL 5.6.50, 5.7.32 and 8.0.22 respectively, adding a synchronous option for your MySQL High Availability solutions.

This release is notable because there have been improvements with CRC32 hardware support compatibility on ARM based architectures for the Galera replication library 3.32 and 4.7. GCache locking has also been improved.

From a lower level, the build system has changed from using SCons to CMake, so while SCons scripts are still included they will not be actively maintained any longer.

In MySQL 5.6.50, we improved the XtraBackup v2 SST script for FreeBSD users.

In MySQL 5.7.32, we added a new variable wsrep_mode. The first application of the variable is to allow ignoring native replication filter rules if configured with replicate-do-db. We also improved LOAD DATA splitting to generate Xid events for binary log events with intermediate commits.

In MySQL 8.0.22, we now have a new SST method based on the CLONE plugin. Similar improvements around wsrep_mode exist too. Improvements around foreign key handling are also present in this release: load data with foreign keys had intermittent failures that have been fixed, as well as BF-BF conflicts between OPTIMIZE/REPAIR/ALTER TABLE and DML in presence of foreign key constraints were suppressed. We would also like to note that Percona XtraBackup version 8.0.22-15.0 and greater is required to perform XtraBackup based SSTs.

For now, MySQL 5.6.50 and 5.7.32 are the last official builds for CentOS 6 and Red Hat Enterprise Linux 6. It is also important to remember that MySQL 5.6 is nearing End Of Life (EOL) so we do recommend that you upgrade to MySQL-wsrep 5.7.

Fleio 2021.01.1 was released today, 14th of January. The latest version is marked as stable and can be used in production environment (read more about our release model here). File uploads in memory stream As we have mentioned in the beta release blogpost, we have changed the way that file uploads (image uploads and object […]

Cyborg is an accelerator resource (GPU, vGPU, FPGA, NVMe SSD, QAT, DPDK, SmartNIC etc.) management project, and it uses a micro-service architecture to support distributing deployment, which contains cyborg-api, cyborg-conductor and cyborg-agent services. Cyborg-agent collects accelerator resources information and reports it to the cyborg-conductor. Cyborg-conductor stores accelerator resources information to the database, and reports it to Placement resource management service. Placement stores the resources information and provides available resources for Nova scheduling when the server is creating. Cyborg-api service provides interfaces to request accelerator resources information. Diagram 0 is the architecture of Cyborg.

With Cyborg, we can boot server with accelerators by interacting with Nova and Placement, and support batch boot servers with scheduling accelerators (this is an enhancement in Inspur InCloud OpenStack Enterprise edition that we plan to contribute to the Nova and Cyborg community.). Then users can use these devices in server to program with FPGA, progress images via GPU and so on. We can also bind and unbind accelerators to exist server by hot-plug and non-hot plug devices, which guarantees the convenient usage of accelerators. Diagram 1 is the interaction flow between Nova and Cyborg when booting a server.

On the side of enhancing server operations with accelerators, we have supported most operations for servers, such as creation and deletion, reboots (soft and hard), pause and unpause, stop and start, take a snapshot, backup, rescue and unrescue, rebuild, evacuate. And there are other operations going on, such as shelve and unshelve, suspend and resume that are close to merge. We plan to support migration (live and cold) and resize soon. With these enhancements, operators can use accelerators more flexibly. Diagram 2 is the sequence flow about booting a server with vGPU devices.

And in Inspur InCloud OpenStack Enterprise edition we have made enhancements to some features, such as batch boot servers, bind and unbind accelerator devices based on hot-plug which are mentioned above. Cyborg was used to managing virtual GPU and the utilization rate was improved by 80%. Data synchronization strategy makes Cyborg and Placement data transportation increased by 30% on efficiency.

From diagram 3, the main components of N3000 include the Intel Arria® 10 FPGA, Dual Intel Ethernet Converged Network Adapter XL710, Intel MAX® 10 FPGA Baseboard Management Controller (BMC), 9 GB DDR4, 144 Mb QDR-IV. It can support High-speed network with 10Gbps/25Gbps interface and High-speed host interface with PCIe* Gen 3×16.

Intel® FPGA Programmable Acceleration Card N3000 (Intel FPGA PAC N3000) is a highly customized platform that enables high-throughput, lower latency, and high-bandwidth applications. It allows the optimization of data plane performance to achieve lower costs while maintaining a high degree of flexibility. End-to-end industry-standard and open-source tool support allow users to quickly adapt to evolving workloads and industry standards. Intel is accelerating 5G and network functions virtualization (NFV) adoption for ecosystem partners, such as telecommunications equipment manufacturers (TEMs), virtual network functions (VNF) vendors, system integrators, and telcos, to bring scalable and high-performance solutions to market. Diagram 4 is the sequence flow for program in Cyborg.

For SmartNIC, we can program it with OVS image as NFVI function in OpenStack, e.g. N3000, Mellanox CX5 and BF2. Diagram 5 is the sequence for enabling accelerators for SmartNIC.

In Cyborg, we support several new features:

• Asynchronous program API.
• Program process API.
• Resource report dynamically.

Users can start a new program process request for N3000. It is an async API, and it is safe to detect intelligently whether there is a conflict in resource usage and decide on accepting or rejecting the request. We also support a friendly program process query API. Users can use it to check the stage of the process and what percentage is completed at any time. When the program is finished, the resource typed and quantified, then Cyborg can discover and report the change dynamically.

In the whole OpenStack, we have also made some new improvements.

• Cyborg supports new drivers for new SRIOV NIC. 
• Neutron supports a new option to create a port with accelerator device profile.
• Improvements on Nova, to retrieve the new VF NIC and pass-through to server as an interface.

After these improvements, the OpenStack can support SmartNIC more flexibly and conveniently, and the improvements include SmartNIC cards (as mentioned above), even other SRIOV card. 

All the new features and improvements on OpenStack will be upstream.

On November 25, Inspur InCloud OpenStack (ICOS) completed the 1000-nodes “The world’s largest single-cluster practice of Cloud, Bigdata and AI”, a practice for convergence of Cloud, Bigdata and Artificial Intelligence. It is the largest SPEC Cloud test and the first large-scale multi-dimensional fusion test in the industry. It has achieved a comprehensive breakthrough in scale, scene and performance, and completed the upgrade from 500 nodes to 1000 nodes, and achieved the sublimation from quantitative change to qualitative change. Inspur is working on the white paper for this large-scale test, which will be released soon. This will certainly be a reference for products in large-scale environments.

Get Involved

This article is a summary of the Open Infrastructure Summit session, Enhancement of new heterogeneous accelerators based on Cyborg.

Watch more Summit session videos like this on the Open Infrastructure Foundation YouTube channel. Don’t forget to join the global Open Infrastructure community, and share your own personal open source stories using the hashtag, #WeAreOpenInfra, on Twitter and Facebook.

Thanks to the 2020 Open Infrastructure Summit sponsors for making the event possible:

Headline: Canonical (ubuntu), Huawei, VEXXHOST

Premier: Cisco, Tencent Cloud

Exhibitor: InMotion Hosting, Mirantis, Red Hat, Trilio, VanillaStack, ZTE

The post Enhancement of New Heterogeneous Accelerators Based on Cyborg appeared first on Superuser.

The post An Overview of Cloud File Sharing with OpenStack Manila appeared first on VEXXHOST.

For people who have attended the Open Infrastructure Summits in recent years, you have probably heard of the first winner of the Superuser Awards, CERN. CERN is the European Organization for Nuclear Research. It has a laboratory that is located at the border between France and Switzerland, and the main mission of CERN is to uncover what the universe is made of and how it works. 

For a few years, all physical servers deployed in the CERN IT data centers have been provisioned as part of the OpenStack private cloud offering, leveraging Nova and Ironic. What started with a few test nodes has now grown to more than 5,000 physical machines, with the aim to move to around 10,000 nodes.

CERN’s Nova and Ironic Deployment

CERN and CERN IT rely heavily on OpenStack, and we have deployed OpenStack in production since 2013 with around 8,500 compute nodes with 300k cores and 35k instances in around 80 cells. In addition, we deploy three regions mainly for scalability and to ease the rollout of the new features.

Issues and Solutions when Scaling Bare Metal with Nova and Ironic

When we scaled our bare metal deployment with Nova and Ironic to several thousand nodes, we encountered three main issues so far:

• Controller Crashes
• API Responsiveness
• Resource Discovery

Scaling Issue 1: Controller Crashes

The CERN Cloud Infrastructure Team uses the iscsi deploy interface for Ironic. It means that a node upon deployment is exporting an iscsi device to the controller, and the controller is then dumping the image downloaded from Glance onto it. (Note that this deploy interface will be deprecated in the future, so direct deploy interface should be used instead.) 

For deployment, since the images are tunneled through the controller, many parallel deployments will drive the conductor into out-of-memory (OOM) situations. In consequence, the controller would crash and leave the nodes in an error state. 

Solutions

To address this issue we horizontally scaled the controllers and introduced the “wing” controllers to have more controllers to handle the requests. Another solution would be to use a scalable deploy interface, such as direct or ansible which makes the node download the image directly.

Scaling Issue 2: API Responsiveness

When scaling up the infrastructure, we noticed that all requests involving the database were slow. Looking at the request logs, we realized that the inspector as the other component running on the controller gets a list of all nodes to clean up its database every 60 seconds. In addition, we had disabled pagination when we reached 1,000 nodes, which means that every request that went to the API assembled all the nodes in one giant request and tried to give that back to the requester. 

Solutions

To solve this issue, we re-enabled pagination and changed the sync interval from 60 seconds to one hour. You can see how this solution affected the response time from the graph below. For a more scalable solution, we introduced “Inspector Leader Election”, which we developed together with upstream and deployed now in production. Inspector Leader Election will be available in the Victoria release, and you can see more details about how it works here.

Scaling Issue 3: Resource Discovery

Another issue that we faced when increasing the number of physical nodes managed by Ironic is the time that the Resource Tracker (RT), which runs in each nova-compute, takes to report all the available resources to Placement. From the graph below, you can see the original OpenStack Nova and Ironic setup for the dedicated bare metal cell at CERN. 

There is a dedicated Nova cell for Ironic which is the standard way that we partition and manage the infrastructure. In the dedicated Nova cell, we only have the cell control plane, nova-conductor, RabbitMQ, and nova-compute. This single nova-compute is responsible for all the communication between Nova and Ironic using the Ironic API. It is possible to run several nova-computes in parallel based on a hash ring to manage the nodes, but when we were testing this functionality, several issues were observed. 

Since there is only one nova-compute that interacts with Ironic and the RT needs to report all the Ironic resources, the RT cycle takes a long time when the number of resources is increased in Ironic. In fact, it took more than three hours to complete our deployment with about 5,000 nodes. During the RT cycle, all the users’ actions are queued until all the resources are updated. This created a bad user experience with the new resources creation taking a few hours. 

Conductor Groups

In order to have failure domains in Ironic and to allow to split the infrastructure, a new feature was introduced in the Stein release of Ironic and Nova. It’s called “conductor groups”. A conductor group is an association between a set of physical nodes and a set of (one or more) Ironic conductors which manage these physical nodes. This association reduces the number of nodes a conductor is looking after, which is key in scaling the deployment. 

Conductor Groups Configuration

The conductor group an Ironic conductor is taking care of is configured in Ironic’s configuration file “ironic.conf” on each Ironic conductor:

[conductor]
conductor_group = MY_CONDUCTOR_GROUP

Next, each Ironic resource needs to be mapped to the conductor group that is selected, and this can be done in the Ironic API. 

openstack baremetal node set --conductor-group "MY_CONDUCTOR_GROUP" 
<node_uuid>

Finally, each group of nova-compute nodes needs to be configured to manage only a conductor group. This is done in Nova’s configuration file “nova.conf” for each “nova-compute” node:

[ironic]
partition_key = MY_CONDUCTOR_GROUP
peer_list = LIST_OF_HOSTNAMES

Now, there is one nova-compute per conductor group, and you can see how the deployment looks like in the graph below.

The Transition Steps

How did we deploy the conductor groups? The transition steps below are summarized from a CERN Tech Blog, Scaling Ironic with Conductor Groups. 

• Nova-compute and Ironic services stopped since the databases will be manually updated
• Updated the Ironic and Nova configuration files with the conductor groups’ names 
• Updated Ironic “nodes” tables to map conductor groups to nodes
• Updated Nova “compute_nodes” and “instances” tables to set the new nova-compute dedicated to the conductor groups
• Finally, start again all the services 

Impact on the Resource Tacker (RT) Cycle Time

In the graph below, you can see the number of Placement requests per conductor group. As a result, the RT cycle time now only takes about 15 minutes to complete instead of three hours previously since it’s now divided through the number of conductor groups.

Number of Resources Per Conductor Group

How did we decide the number of resources per conductor groups? The RT cycle time increases linearly with the number of resources. The compromise between management and RT cycle time is around 500 nodes per conductor group in our deployment. The deployment scales horizontally as we add conductor groups

Conclusion and Outlook

Scaling infrastructure is a constant challenge! By introducing conductor groups, the CERN Cloud Infrastructure Team was able to split this locking and reduce the effective lock time from three hours to 15 minutes (and even shorter in the “leading” group)

Although we have addressed various issues, some issues are still open and new ones will arise. Good monitoring is key to see and understand issues.

Get Involved

This article is a summary of the Open Infrastructure Summit session, Scaling Bare Metal Provisioning with Nova and Ironic at CERN, and the CERN Tech Blog, Scaling Ironic with Conductor Groups. 

Watch more Summit session videos like this on the Open Infrastructure Foundation YouTube channel. Don’t forget to join the global Open Infrastructure community, and share your own personal open source stories using the hashtag, #WeAreOpenInfra, on Twitter and Facebook.

The post Scaling Bare Metal Provisioning with Nova and Ironic at CERN: Challenges & Solutions appeared first on Superuser.

The post Adding Linux Images to OpenStack Using IaC  appeared first on VEXXHOST.

Fleio version 2021.01.0 is now available in beta and you can test it in your lab environment, since this release is not recommended for production. You can read more about our release model here. With the latest release we would like to announce that we have added the following changes and improvements to Fleio. File […]

The post How Resource Pooling Works in an OpenStack Private Cloud   appeared first on VEXXHOST.

I want to be able to see the level of change between OpenStack releases. However, there are a relatively small number of changes with simply huge amounts of delta in them — they’re generally large refactors or the delete which happens when part of a repository is spun out into its own project.

I therefore wanted to explore what was a reasonable size for a change in OpenStack so that I could decide what maximum size to filter away as likely to be a refactor. After playing with a couple of approaches, including just randomly picking a number, it seems the logical way to decide is to simply plot a histogram of the various sizes, and then pick a reasonable place on the curve as the cutoff. Due to the large range of values (from zero lines of change to over a million!), I ended up deciding a logarithmic axis was the way to go.

For the projects listed in the OpenStack compute starter kit reference set, that produces the following histogram:I feel that filtering out commits over 10,000 lines of delta feels justified based on that graph. For reference, the raw histogram buckets are:

Commit size	Count
< 2	25747
< 11	237436
< 101	326314
< 1001	148865
< 10001	16928
< 100001	3277
< 1000001	522
< 10000001	13

I wanted a quick summary of OpenStack git release tags for a talk I am working on, and it turned out to be way more complicated than I expected. I ended up having to compile a table, and then turn that into a code snippet. In case its useful to anyone else, here it is:

Release	Release date	Final release tag
Austin	October 2010	2010.1
Bexar	February 2011	2011.1
Cactus	April 2011	2011.2
Diablo	September 2011	2011.3
Essex	April 2012	2012.1.3
Folsom	September 2012	2012.2.4
Grizzly	April 2013	2013.1.5
Havana	October 2013	2013.2.4
Icehouse	April 2014	2014.1.5
Juno	October 2014	2014.2.4
Kilo	April 2015	2015.1.4
Liberty	October 2015	Glance: 11.0.2 Keystone: 8.1.2 Neutron: 7.2.0 Nova: 12.0.6
Mitaka	April 2016	Glance: 12.0.0 Keystone: 9.3.0 Neutron: 8.4.0 Nova: 13.1.4
Newton	October 2016	Glance: 13.0.0 Keystone: 10.0.3 Neutron: 9.4.1 Nova: 14.1.0
Ocata	February 2017	Glance: 14.0.1 Keystone: 11.0.4 Neutron: 10.0.7 Nova: 15.1.5
Pike	August 2017	Glance: 15.0.2 Keystone: 12.0.3 Neutron: 11.0.8 Nova: 16.1.8
Queens	February 2018	Glance: 16.0.1 Keystone: 13.0.4 Neutron: 12.1.1 Nova: 17.0.13
Rocky	August 2018	Glance: 17.0.1 Keystone: 14.2.0 Neutron: 13.0.7 Nova: 18.3.0
Stein	April 2019	Glance: 18.0.1 Keystone: 15.0.1 Neutron: 14.4.2 Nova: 19.3.2
Train	October 2019	Glance: 19.0.4 Keystone: 16.0.1 Neutron: 15.3.0 Nova: 20.4.1
Ussuri	May 2020	Glance: 20.0.1 Keystone: 17.0.0 Neutron: 16.2.0 Nova: 21.1.1
Victoria	October 2020	Glance: 21.0.0 Keystone: 18.0.0 Neutron: 17.0.0 Nova: 22.0.1

Or in python form for those so inclined:

RELEASE_TAGS = {
    'austin': {'all': '2010.1'},
    'bexar': {'all': '2011.1'},
    'cactus': {'all': '2011.2'},
    'diablo': {'all': '2011.3'},
    'essex': {'all': '2012.1.3'},
    'folsom': {'all': '2012.2.4'},
    'grizzly': {'all': '2013.1.5'},
    'havana': {'all': '2013.2.4'},
    'icehouse': {'all': '2014.1.5'},
    'juno': {'all': '2014.2.4'},
    'kilo': {'all': '2015.1.4'},
    'liberty': {
        'glance': '11.0.2',
        'keystone': '8.1.2',
        'neutron': '7.2.0',
        'nova': '12.0.6'
    },
    'mitaka': {
        'glance': '12.0.0',
        'keystone': '9.3.0',
        'neutron': '8.4.0',
        'nova': '13.1.4'
    },
    'newton': {
        'glance': '13.0.0',
        'keystone': '10.0.3',
        'neutron': '9.4.1',
        'nova': '14.1.0'
    },
    'ocata': {
        'glance': '14.0.1',
        'keystone': '11.0.4',
        'neutron': '10.0.7',
        'nova': '15.1.5'
    },
    'pike': {
        'glance': '15.0.2',
        'keystone': '12.0.3',
        'neutron': '11.0.8',
        'nova': '16.1.8'
    },
    'queens': {
        'glance': '16.0.1',
        'keystone': '13.0.4',
        'neutron': '12.1.1',
        'nova': '17.0.13'
    },
    'rocky': {
        'glance': '17.0.1',
        'keystone': '14.2.0',
        'neutron': '13.0.7',
        'nova': '18.3.0'
    },
    'stein': {
        'glance': '18.0.1',
        'keystone': '15.0.1',
        'neutron': '14.4.2',
        'nova': '19.3.2'
    },
    'train': {
        'glance': '19.0.4',
        'keystone': '16.0.1',
        'neutron': '15.3.0',
        'nova': '20.4.1'
    },
    'ussuri': {
        'glance': '20.0.1',
        'keystone': '17.0.0',
        'neutron': '16.2.0',
        'nova': '21.1.1'
    },
    'victoria': {
        'glance': '21.0.0',
        'keystone': '18.0.0',
        'neutron': '17.0.0',
        'nova': '22.0.1'
    }
}

This proposal was submitted for FOSDEM 2021. Given that acceptances were meant to be sent out on 25 December and its basically a week later I think we can assume that its been rejected. I’ve recently been writing up my rejected proposals, partially because I’ve put in the effort to write them and they might be useful elsewhere, but also because I think its important to demonstrate that its not unusual for experienced speakers to be rejected from these events.

---

OpenStack today is a complicated beast — not only does it try to perform well for large clusters, but it also embraces a diverse set of possible implementations from hypervisors, storage, networking, and more. This was a deliberate tactical choice made by the OpenStack community years ago, forming a so called “Big Tent” for vendors to collaborate in to build Open Source cloud options. It made a lot of sense at the time to be honest. However, OpenStack today finds itself constrained by the large number of permutations it must support, ten years of software and backwards compatability legacy, and a decreasing investment from those same vendors that OpenStack courted so actively.

Shaken Fist makes a series of simplifying assumptions that allow it to achieve a surprisingly large amount in not a lot of code. For example, it supports only one hypervisor, one hypervisor OS, one networking implementation, and lacks an image service. It tries hard to be respectful of compute resources while idle, and as fast as possible to deploy resources when requested — its entirely possible to deploy a new VM and start it booting in less than a second for example (if the boot image is already held in cache). Shaken Fist is likely a good choice for small deployments such as home labs and telco edge applications. It is unlikely to be a good choice for large scale compute however.

The post An Overview of OpenStack Barbican for Secure Key Management appeared first on VEXXHOST.

The post Santa’s Scalable Sack Storage & Delivery Automation appeared first on Aptira.

The post Why a Managed OpenStack Cloud Is Better For You Than Self-Managed appeared first on VEXXHOST.

In 2019, the term “digital sovereignty” gained momentum in Europe. For the first time in history, several ministries such as the German and French ministries for Economic Affairs pushed forward initiatives to establish digital sovereignty.

At the Open Infrastructure Summit keynotes, Johan Christenson, CEO of City Network, discussed the importance of digital sovereignty in the infrastructure world. Christenson mentioned that on July 16th 2020, the Court of Justice of the European Union invalidated “Privacy Shield”, which means companies seeking to transfer personal data from the European Economic Area to the United States must now use other mechanisms recognized by General Data Protection Regulation (GDPR) to appropriately safeguard personal data.

“As Europe grapples with an ever more centralized cloud-world and lack of true choice in providers, new initiatives are surging like GAIA-X. Europe is looking to have vendors whom can support both its values and laws in the infrastructure layer.” Christenson said. Therefore, digital sovereignty is becoming critically important from both the aspects of achieving digital independence and the long-term innovation for Europe as a whole.

But, what exactly is digital sovereignty? Why is open infrastructure important to achieve digital sovereignty? What is the European initiative GAIA-X project and how is it leveraging open infrastructure for its use case? In this article, we will focus on what digital sovereignty is and why open infrastructure matters, from the talk that Marius Feldmann, Chief Operation Officer of Cloud&Heat Technologies, and Kurt Garloff, leading the Sovereign Cloud Stack (SCS) initiative, delivered—Digital Sovereignty Why Open Infrastructure Matters—at the Open Infrastructure Summit.

“Digital sovereignty is about sovereignty, not about autarky.”

It’s easy to confuse digital sovereignty with digital autarky, and they are completely different concepts. Digital sovereignty is about integrating the rising and existing technologies and innovative ideas into an overall system and adding its own solutions in order to have the freedom of choice among different options that exist. 

“Sovereignty matters – for sovereignty you need control and choice.”

Just like Feldmann discussed in his Summit session, “digital sovereignty is built on freedom of choice.” Consumers should have particularly effective freedom of choice when they become the “active managers” of their own data (Digital Sovereignty Report by the Advisory Council for Consumer Affairs, 2017).

But what are the essential preconditions to achieve having the freedom of choice in digital sovereignty? One of the most important preconditions is that alternatives should be available to achieve the freedom of choice. In addition, the alternatives must differ regarding their functional and non-function properties. It’s also fundamentally important that discovery based on these properties must be rendered possible, which means there should be possibilities and means to discover the digital services that are offered based on the properties.

European Initiative GAIA-X

With the aim to develop common requirements for a European data infrastructure, GAIA-X is a project that has been initiated by the German ministry for economy and energy. Currently, hundreds of companies worldwide work together on GAIA-X to push forward the overall vision of digital sovereignty. 

“A strong open infrastructure ecosystem is the foundation of digital sovereignty” (GAIA-X: Technical Architecture, June 2020).

GAIA-X aims to develop common requirements for a European data infrastructure. Therefore openness, transparency and the ability to connect to other European countries are central to GAIA-X. This project is the cradle of an open, transparent digital ecosystem, where data and services can be made available, collated and shared in an environment of trust. Read more about GAIA-X on GAIA-X: Technical Architecture document. 

Towards Decentralization

To further explain what it needs to achieve digital sovereignty, let’s assume a heterogeneous / distributed ecosystem as the foundation for a digital sovereign future. 

Without having heterogeneous and decentralized infrastructure, there is no freedom of choice. Therefore, Having various providers and decentralized infrastructures is key to digital sovereignty. Feldmann summarized it well that 

“Future digital infrastructures are not monolithic and centralized but highly distributed and heterogeneous regarding their properties.”

However, to avoid a potential drawback of this future digital infrastructure, which is having various APIs to achieve mainly the same things on the infrastructure side, there should be a standardized API for all the cloud providers. Therefore, on one hand, the future digital infrastructures should be heterogeneous in order to have alternatives available due to the different properties. On the other hand, there should be standards for the API and for the platform itself. 

To clarify, when talking about standardized APIs as well as standardized tools to make an infrastructure platform operational, Feldmann was only talking about the software layer for commodity (compute, storage, networking). There is no need for many alternatives on this layer. The commodity layer should leverage open infrastructure projects to render possible collaborative development and to avoid API fragmentation. 

“Open Infrastructure is bridging the chasm on the commodity layer.”

Reinventing the wheel on the commodity layer may be an interesting exercise; however, it slows down innovative and technical progress.

On the path towards digital sovereignty, on one hand, it is crucial to provide a modular open infrastructure solution that renders it possible to enable various actors to set up an operational platform for hosting services quickly. On the other hand, open infrastructure projects should reduce development overhead and should ensure API interoperability in order to avoid vendor-lock-in contradicting the freedom of choice.

Just like what Christenson said, “open source is the only realistic way to a solid choice in vendors. Open source is also the foundation for Europe’s long term innovation and sovereignty in anything data.” 

Get Involved

This article is a summary of the Open Infrastructure Summit session, Digital Sovereignty – Why Open Infrastructure Matters.

Watch more Summit session videos like this on the Open Infrastructure Foundation YouTube channel. Don’t forget to join the global Open Infrastructure community, and share your own personal open source stories using the hashtag, #WeAreOpenInfra, on Twitter and Facebook.

Thanks to the 2020 Open Infrastructure Summit sponsors for making the event possible:=

Headline: Canonical (ubuntu), Huawei, VEXXHOST

Premier: Cisco, Tencent Cloud

Exhibitor: InMotion Hosting, Mirantis, Red Hat, Trilio, VanillaStack, ZTE

The post Digital Sovereignty – Why Open Infrastructure Matters appeared first on Superuser.

When Ironic is chosen to manage bare metal nodes, it is not unusual that there is already an existing in-production fleet of physical servers. This may leave the operator in the sub-optimal situation where tools and workflows need to handle Ironic and pre-Ironic systems. To address this, Ironic supports an “adoption” feature: adoption allows to add nodes which Ironic should regard as in-use, and they can therefore take a slightly different path through the Ironic state machine.

Fleio v2020.12.1 was released today, 16th of December. The latest version is marked as stable and can be used in production environment (read more about our release model here). Swift object store In the last blog post, we have announced that we have added Swift object store to our support Openstack projects. With the latest […]

While reviewing  the comments on the Ironic spec, for Secure RBAC. I had to ask myself if the “project” construct makes sense for Ironic.  I still think it does, but I’ll write this down to see if I can clarify it for me, and maybe for you, too.

Baremetal servers change.  The whole point of Ironic is to control the change of Baremetal servers from inanimate pieces of metal to “really useful engines.”  This needs to happen in a controlled and unsurprising way.

Ironic the server does what it is told. If a new piece of metal starts sending out DHCP requests, Ironic is going to PXE boot it.  This is the start of this new piece of metals journey of self discovery.  At least as far as Ironic is concerned.

But really, someone had to rack and wire said piece of metal.  Likely the person that did this is not the person that is going to run workloads on it in the end.  They might not even work for the same company;  they might be a delivery person from Dell or Supermicro.  So, once they are done with it, they don’t own it any more.

Who does?  Who owns a piece of metal before it is enrolled in the OpenStack baremetal service?

No one.  It does not exist.

Ok, so lets go back to someone pushing the button, booting our server for the first time, and it doing its PXE boot thing.

Or, we get the MAC address and enter that into the ironic database, so that when it does boot, we know about it.

Either way, Ironic is really the playground monitor, just making sure it plays nice.

What if Ironic is a multi-tenant system?  Someone needs to be able to transfer the baremetal server from where ever it lands up front to the people that need to use it.

I suspect that ransferring metal from project to project is going to be one of the main use cases after the sun has set on day one.

So, who should be allowed to say what project a piece of baremetal can go to?

Well, in Keystone, we have the idea of hierarchy.  A Project is owned by a domain, and a project can be nested inside another project.

But this information is not passed down to Ironic.  There is no way to get a token for a project that shows its parent information.  But a remote service could query the project hierarchy from Keystone. 

https://docs.openstack.org/api-ref/identity/v3/?expanded=show-project-details-detail#show-project-details

Say I want to transfer a piece of metal from one project to another.  Should I have a token for the source project or the remote project.  Ok, dump question, I should definitely have a token for the source project.  The smart question is whether I should also have a token for the destination project.

Sure, why not.  Two tokens. One has the “delete” role and one that has the “create” role.

The only problem is that nothing like this exists in Open Stack.  But it should.

We could fake it with hierarchy;  I can pass things up and down the project tree.  But that really does not one bit of good.  People don’t really use the tree like that.  They should.  We built a perfectly nice tree and they ignore it.  Poor, ignored, sad, lonely tree.

Actually, it has no feelings.  Please stop anthropomorphising the tree.

What you could do is create the destination object, kind of a potential piece-of-metal or metal-receiver.  This receiver object gets  a UUID.  You pass this UUID to the “move” API. But you call the MOVE api with a token for the source project.   The move is done atomically. Lets call this thing identified by a UUID a move-request. 

The order of operations could be done in reverse.  The operator could create the move request on the source, and then pass that to the receiver.  This might actually make mores sense, as you need to know about the object before you can even think to move it.

Both workflows seem to have merit.

And…this concept seems to be something that OpenStack needs in general. 

Infact, why should the API not be a generic API. I mean, it would have to be per service, but the same API could be used to transfer VMs between projects in Nova nad between Volumes in Cinder. The API would have two verbs one for creating a new move request, and one for accepting it.

POST /thingy/v3.14/resource?resource_id=abcd&destination=project_id

If this is called with a token, it needs to be scoped. If it is scoped to the project_id in the API, it creates a receiving type request. If it is scoped to the project_id that owns the resource, it is a sending type request. Either way, it returns an URL. Call GET on that URL and you get information about the transfer. Call PATCH on it with the appropriately scoped token, and the resource is transferred. And maybe enough information to prove that you know what you are doing: maybe you have to specify the source and target projects in that patch request.

A foolish consistency is the hobgoblin of little minds.

Edit: OK, this is not a new idea. Cinder went through the same thought process according to Duncan Thomas. The result is this API: https://docs.openstack.org/api-ref/block-storage/v3/index.html#volume-transfer

Which looks like it then morphed to this one:

https://docs.openstack.org/api-ref/block-storage/v3/index.html#volume-transfers-volume-transfers-3-55-or-later

The post How OpenStack Sahara Works As a High-Performance Big Data Platform appeared first on VEXXHOST.

I've had to re-teach myself how to do this so I'm writing my own notes.

Prerequisites:

1. Get a working undercloud (perhaps from tripleo-lab)
2. git clone https://git.openstack.org/openstack/tripleo-ansible.git ; cd tripleo-ansible
3. Determine the test name: ls roles

Once you have your environment ready run a test with the name from step 3.

./scripts/run-local-test tripleo_derived_parameters

Some tests in CI are configured to use `--skip-tags`. You can do this for your local tests too by setting the appropriate environment variables. For example:

 export TRIPLEO_JOB_ANSIBLE_ARGS="--skip-tags run_ceph_ansible,run_uuid_ansible,ceph_client_rsync,clean_fetch_dir"
 ./scripts/run-local-test tripleo_ceph_run_ansible

This last tip should get added to the docs.

Codership is pleased to announce our first ever online training for Galera Cluster, with the launch of the Database Administration for Galera Cluster course. There are two sets of times for our EMEA and American attendees, with the former happening on January 2021, 13th and 14th, starting 10 AM CET and the latter happening on January 2021, 20th and 21st, starting 9 AM PST.

This is a hands on course that spans two days, and you will have about six contact hours per day in this instructor-led course. If you are a DBA or been given a Galera Cluster to manage, this is the course for you, as you’ll learn how to run your Galera Cluster in an optimal fashion, from setup, performance tuning, monitoring, as well as backups and more. We will cover not just Codership’s Galera Cluster distribution but also MariaDB Galera Cluster and Percona XtraDB Cluster in this course. For more information, please read the full content breakdown.

As an added bonus, we will also be covering database administration with Galera Manager our new GUI tool.

What are you waiting for? Sign up now, as the early bird rate ends 31 December 2020. There is also a volume discount available.

The post Ceph Storage Basics and How It Benefits Your OpenStack Private Cloud appeared first on VEXXHOST.

Fleio version 2020.12.0 is now available in beta and you can test it in your lab environment, since this release is not recommended for production. You can read more about our release model here. With the latest release we would like to announce that we have implemented Swift object store user interface. With the 2020.12 […]

Happy 10 years of OpenStack! Millions of cores, 100,000 community members, 10 years of you. Storytelling is one of the most powerful means to influence, teach, and inspire the people around us. To celebrate OpenStack’s 10th anniversary, we are spotlighting stories from the individuals in various roles from the community who have helped to make... Read more »

Storytelling is one of the most powerful means to influence, teach, and inspire the people around us. To celebrate OpenStack’s 10th anniversary, we are spotlighting stories from the individuals in various roles from the community who have helped to make OpenStack and the global Open Infrastructure community successful.  Here, we’re talking to Thomas Goirand from... Read more »

European Centre for Medium range Weather Forecasts (ECMWF), an intergovernmental organisation, was established in 1975. Based in Reading, UK and the data center soon moving to Bologna Italy, ECMWF spans 34 States in Europe. It operates one of the largest supercomputer complexes in Europe and the world’s largest archive of numerical weather prediction data. In terms of its IT infrastructure, ECMWF’s HPC (High-performance computing) facility is one of the largest weather sites globally. With cloud infrastructure for Copernicus Climate Change Service (C3S), Copernicus Atmosphere Monitoring Service (CAMS) and WEkEO, which is a Data and Information Access Service (DIAS) platform, and the European Weather Cloud, teams at ECMWF maintain an archive of climatological data with a size of 250 PB with a daily growth of 250TB.

European Weather Cloud:

The European Weather Cloud started three years ago in a collaboration between ECMWF and The European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) aiming to make it easier to work on weather and climate big data in a cloud-based infrastructure. With the goal of bringing the computation resources (Cloud) closer to their Big data (meteorological archive and satellite data), ECMWF’s pilot infrastructure was with open source software – Ceph and OpenStack using TripleO.

The graph below shows the current state of the European Weather Cloud overall infrastructure comprising two OpenStack clusters: one built with OpenStack Rocky and another one with OpenStack Ussuri. The total hardware of the current configuration comprises around 3,000 vCPUs, 21 TB RAM for both clusters, 1PB of storage and 2×5 NVIDIA Tesla V100 GPUs.

Integration with Ceph:

The graph below shows the cloud infrastructure of the European Weather Cloud. As you can see, Ceph is built and maintained separately from OpenStack which gives the teams at the European Weather Cloud a lot of flexibility in building different clusters on the same Ceph storage. Both of its OpenStack clusters use the same Ceph infrastructure and the same rbd pools. Besides some usual HDD failures, Ceph performs very well, and the teams at the European Weather Cloud are planning to gradually move to CentOS8 (due to partial support of CentOS7) and upgrade to Octopus and cephadm on a live cluster after a lot of testing to their development environment.

OpenStack with Rocky version:

The first OpenStack cluster in the European Weather Cloud, which was built in September 2019, is based on Rocky with TripleO installer. In the meantime, engineers at the European Weather Cloud also created another development environment with OpenStack and Ceph clusters that are similarly configured for testing-experimentation.

Experience and Problems:

Their deployment has about 2,600 VCPUs with 11TB RAM which doesn’t have any significant problem. The external Ceph cluster integration worked with minimum effort by simply configuring the ceph-config.yaml with little modifications. The two external networks (one public facing and another for fast access to their 300PB data archive) were straightforward. 

Most of their VMs are attached to both external networks with no floating IPs, which was a challenging VM routing issue without dynamic routing on the switches. To solve this issue, they used dhcp hooks and configured VM routing before they made the images available to the users.

There were some problems that they encountered with the NIC bond interface configuration and in conjunction with the configuration of their switches at the beginning. Therefore, the engineers decided to not use a Link Aggregation Control Protocol (LACP) configuration, and now they have a single network interface card (NIC) deployment for OpenStack. They also encountered some problems with Load-Balancing-as-a-Service (LBaas) due to Octavia problems with certificates overridden on each deployment. The problem is presented here. 

As soon as they found the solutions to these challenges, the engineers updated their live system and moved the whole cluster from a Single NIC to a Multiple NIC deployment which is transparent to the users with zero downtime. The whole first cluster was redeployed, and the network was re-configured with Distributed Virtual Routing (DVR) configuration for better network performance. 

OpenStack update efforts from Stein-Train-Ussuri:

In March 2020, engineers at the European Weather Cloud added more hardware for their OpenStack and Ceph cluster, and they decided to investigate upgrading to the newest versions of OpenStack. 

Experience and Problems:

First, they converted their Rockey undercloud to a VM for better management and as a safety net for backups and recovery. From March to May 2020, they investigated and tested upgrading to Stein (first undercloud and then overcloud upgrade to a test environment). Updating was possible from Rocky to Stein to Train and finally to Ussuri, but due to the CentOS7 to CentOS8 transition since Ussuri was based on CentOS8, it was considered impractical. Therefore, they made a big jump from Rocky to Ussuri by skipping three updates and decided to directly deploy their new systems on OpenStack Ussuri.

OpenStack Ussuri Cluster:

The second OpenStack cluster, based on Ussuri, was first built in May 2020, 17 days after the release of Ussuri on May 13. This cluster was a plain vanilla configuration which means that although the network was properly configured with OVN and provider networks with 25 nodes they haven’t had any integration with Ceph storage. 

Experience and Problems:

The new building method that was based on Ansible instead of Mistral had some hiccups, such as the switch from stack to heat-admin which is not what the users are used to deploy. In addition, they were trying to quickly understand and master the CentOS8 base operating system for both the host systems and service containers. Engineers at the European Weather Cloud also continued with OVS instead of OVN because of the implications in assigning floating IP addresses. With the help from the OpenStack community, the problems were solved, and the cluster was built again in mid-June 2020.

Regarding GPUs, the configuration of Nvidia GPUs was straightforward. However, since they haven’t implemented IPv6 to their Ussuri cluster when they installed and configured the GPUs drivers to a node, OVS was trying to bind to IPv6 addresses during the booting time which results in a considerable increase in booting time. A workaround was to explicitly remove PIv6 configuration to their GPU nodes. All nodes with a GPU also resolved as normal compute nodes, and they have configured nova.conf with their Ansible playbooks.  

Future Next Steps:

In terms of the European Weather Cloud’s infrastructure, the engineers are planning to integrate the infrastructure with other internal systems for better monitoring and logging. They are also planning to phase out the Rocky cluster and move all the nodes to Ussuri. Trying to follow the latest versions of OpenStack and Ceph, they will continue to operate, maintain and upgrade the Cloud’s infrastructure.

For the federation, the goal is to federate their Cloud infrastructure with infrastructures of their Member states. They have identified and will continue to explore potential good use cases to federate. 

Regarding the integration with other projects, the European Weather Cloud will be interfacing with the Digital Twin Earth which is a part of the Destination Earth Program of the EU. 

Teams at the European Weather Cloud are also planning to contribute code and help other users that are facing the same problems while deploying clusters in the OpenStack community.

Get Involved:

Watch this session video and more on the Open Infrastructure Foundation YouTube channel. Don’t forget to join the global Open Infrastructure community, and share your own personal open source story using #WeAreOpenInfra on social media.

Special thanks to our 2020 Open Infrastructure Summit sponsors for making the event possible:

• Headline: Canonical (ubuntu), Huawei, VEXXHOST
• Premier: Cisco, Tencent Cloud
• Exhibitor: InMotion Hosting, Mirantis, Red Hat, Trilio, VanillaStack, ZTE

The post OpenStack in Production and Integration with Ceph: A European Weather Cloud User Story appeared first on Superuser.

I have two business ideas to explore, and I decided that now is a good time to take the plunge and create a prototype. My hesitation throughout the last year was due to the time and financial investment required. After some inspiration, detailed thought, and self-evaluation, I am ready to go for it. Worst case scenario, this is going to eat up a lot of my time. Even if I lose time, I will learn a lot about cloud infrastructure, cloud networking, and cloud instance provisioning. My first business idea is in the realm of home and small business network cyber security. The second utilizes a private cloud platform to provision labs for IT and cyber security training. A small virtual lab isn’t going to cut it for these ventures.

My Current Setup

Before I can pursue these builds, I need to upgrade my home network and lab and select a platform. I currently have 3 old used servers (2 Dell PowerEdge R510s and an HP Proliant DL360) for the cloud. For networking, I have an ancient Cisco switch. I think I can get by with the old switch for now, but my small private cloud requires more servers. I can use the private cloud to provision networks to test out capabilities, learn, and design. These can also hold prototypes and proof of concepts for demonstrations. For the private cloud, I selected OpenStack as my platform. This will allow me to provision instances using Terraform, and have more flexibility with networking configuration. I can also avoid a large AWS and Azure bill while I experiment with different configurations. The only thing that will suffer is my power bill.

Project Goals

Based on the OpenStack documentation I will need at least 4-5 servers to support my configuration which is a small compute cloud. To use Juju and Metal as a Service (MAAS) to deploy the cloud, I will need 2 more servers, but I could probably use one of my servers and host 2 VMs instead of purchasing another server. I haven’t yet decided whether I am going to use Juju and MAAS to deploy OpenStack, but I do know that I need at least 2 more servers for my project. I also want to separate my private cloud from the rest of my network and still maintain network performance with the added security, so I will need a firewall / IPS appliance. Once complete, my home network will look something like this:

My Private Cloud Budget

I am trying to stay under $2,000 total for this project (including what I already spent). Below is the price I paid for everything I already have.

Device	Qty	Unit Cost	Shipping	Total Cost
HP ProLiant DL360	1	$149.99	$112.89	$262.88
Dell PowerEdge R510	2	$238.99	$75.00	$552.98
Cisco Catalyst 3560	1	$69.00	$17.95	$86.95
			Total Cost	$902.81

Existing devices with costs at the time of purchase

So, based on that I have about $1100 to spend. Although I have plenty of room, I am sticking with used equipment. The only exception I am making is my firewall appliance.

Purchasing New Equipment

I was able to find 2 Dell PowerEdge R610s for $157 each, well within budget. My shipping costs to my location are really high, so I have to keep that in mind. Even with the shipping costs, I still consider these a bargain and they meet my needs. These servers also come from the same vendor as my previous purchases (PC Server and Parts), so I know they will arrive in good condition and operate well.

Next I need a firewall appliance, for this I am going straight to a vendor because their site is a lot cheaper than Amazon. This appliance from Protectli has 4 NICs, a quad core processor, and a small SSD. This is more than enough to run pfsense (and it was already tested for it), so it will easily meet my needs and be a step up from my current options for under $300.

Total Costs

With those 2 purchases, I have all the equipment I will need, and significantly under my max budget! The only other purchase I might make is a rack to store the equipment and a PDU. For now, I just have to wait for them to arrive. I plan to start sometime in December. While I wait, I am going to work on my remote access solutions, determine what IDS/IPS I am going to use (Suricata, Snort, or Bro), and finalize my design of how this will all fit together.

Device	Qty	Unit Cost	Shipping	Total Cost
HP ProLiant DL360	1	$149.99	$112.89	$262.88
Dell PowerEdge R510	2	$238.99	$75.00	$552.98
Cisco Catalyst 3560	1	$69.00	$17.95	$86.95
Protectli FW4B	1	$282.00	$7.00	$289.00
Dell PowerEdge R610	2	$156.99	$111.00	$424.98
			Total Cost	$1616.79

Existing devices with costs at time of purchase

This article was originally posted on mattglass-it.com. See the original article here.

The post Embarking on a New Venture: Creating a Private Cloud with OpenStack for Under $1700 appeared first on Superuser.

The post Why OpenStack Private Cloud for DevOps is the Way Forward appeared first on VEXXHOST.

The post A Brief History of the Open Infrastructure Foundation appeared first on VEXXHOST.

Kata containers are containers that use hardware virtualization technologies for workload isolation almost without performance penalties. Top use cases are untrusted workloads and tenant isolation (for example in a shared Kubernetes cluster). This blog post describes how to run Percona Kubernetes Operator for Percona XtraDB Cluster (PXC Operator) using Kata containers.

Prepare Your Kubernetes Cluster

Setting up Kata containers and Kubernetes is well documented in the official github repo (cri-o, containerd, Kubernetes DaemonSet). We will just cover the most important steps and pitfalls.

Virtualization Support

First of all, remember that Kata containers require hardware virtualization support from the CPU on the nodes. To check if your linux system supports it run on the node.

$ egrep ‘(vmx|svm)’ /proc/cpuinfo

VMX (Virtual Machine Extension) and SVM (Secure Virtual Machine) are Intel and AMD features that add various instructions to allow running a guest OS with full privileges, but still keeping host OS protected.

For example, on AWS only i3.metal and r5.metal instances provide VMX capability.

Containerd

Kata containers are OCI (Open Container Interface) compliant, which means that they work pretty well with CRI (Container Runtime Interface) and hence well supported by Kubernetes. To use Kata containers please make sure your Kubernetes nodes run using CRI-O or containerd runtimes.

The image below describes pretty well how Kubernetes works with Kata.

Hint: GKE or kops allows you to start your cluster with containerd out of the box and skip manual steps.

Setting Up Nodes

To run Kata containers, k8s nodes need to have kata-runtime installed and runtime configured properly. The easiest way is to use DaemonSet which installs required packages on every node and reconfigures containerd. As a first step apply the following yamls to create the DaemonSet:

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-deploy/base/kata-deploy.yaml

DaemonSet reconfigures containerd to support multiple runtimes. It does that by changing /etc/containerd/config.toml. Please note that some tools (ex. kops) keep containerd in a separate configuration file config-kops.toml. You need to copy the configuration created by DaemonSet to the corresponding file and restart containerd.

Create runtimeClasses for Kata. RuntimeClass is a feature that allows you to pick runtime for the container during its creation. It has been available since Kubernetes 1.14 as Beta.

$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/k8s-1.14/kata-qemu-runtimeClass.yaml

Everything is set. Deploy test nginx pod and set the runtime:

$ cat nginx-kata.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx-kata
spec:
  runtimeClassName: kata-qemu
  containers:
    - name: nginx
      image: nginx

$ kubectl apply -f nginx-kata.yaml
$ kubectl describe pod nginx-kata | grep “Container ID”
    Container ID:   containerd://3ba8d62be5ee8cd57a35081359a0c08059cf08d8a53bedef3384d18699d13111

On the node verify if Kata is used for this container through ctr tool:

# ctr --namespace k8s.io containers list | grep 3ba8d62be5ee8cd57a35081359a0c08059cf08d8a53bedef3384d18699d13111
3ba8d62be5ee8cd57a35081359a0c08059cf08d8a53bedef3384d18699d13111    sha256:f35646e83998b844c3f067e5a2cff84cdf0967627031aeda3042d78996b68d35 io.containerd.kata-qemu.v2cat 

Runtime is showing kata-qemu.v2 as requested.

The current latest stable PXC Operator version (1.6) does not support runtimeClassName. It is still possible to run Kata containers by specifying io.kubernetes.cri.untrusted-workload annotation. To ensure containerd supports this annotation add the following into the configuration toml file on the node:

# cat <> /etc/containerd/config.toml
[plugins.cri.containerd.untrusted_workload_runtime]
  runtime_type = "io.containerd.kata-qemu.v2"
EOF

# systemctl restart containerd

Install the Operator

We will install the operator with regular runtime but will put the PXC cluster into Kata containers.

Create the namespace and switch the context:

$ kubectl create namespace pxc-operator
$ kubectl config set-context $(kubectl config current-context) --namespace=pxc-operator

Get the operator from github:

$ git clone -b v1.6.0 https://github.com/percona/percona-xtradb-cluster-operator

Deploy the operator into your Kubernetes cluster:

$ cd percona-xtradb-cluster-operator
$ kubectl apply -f deploy/bundle.yaml

Now let’s deploy the cluster, but before that, we need to explicitly add an annotation to PXC pods and mark them untrusted to enforce Kubernetes to use Kata containers runtime. Edit deploy/cr.yaml:

pxc:
  size: 3
  image: percona/percona-xtradb-cluster:8.0.20-11.1
  …
  annotations:
 
      io.kubernetes.cri.untrusted-workload: "true"

Now, let’s deploy the PXC cluster:

$ kubectl apply -f deploy/cr.yaml

The cluster is up and running (using 1 node for the sake of experiment):

$ kubectl get pods
NAME                                               READY   STATUS    RESTARTS   AGE
pxc-kata-haproxy-0                                 2/2     Running   0          5m32s
pxc-kata-pxc-0                                     1/1     Running   0          8m16s
percona-xtradb-cluster-operator-749b86b678-zcnsp   1/1     Running   0          44m

In crt output you should see percona-xtradb cluster running using Kata runtime:

# ctr --namespace k8s.io containers list | grep percona-xtradb-cluster | grep kata
448a985c82ae45effd678515f6cf8e11a6dfca159c9abf05a906c7090d297cba    docker.io/percona/percona-xtradb-cluster:8.0.20-11.2 io.containerd.kata-qemu.v2

We are working on adding the support for runtimeClassName option for our operators. The support of this feature enables users to freely choose any container runtime.

Conclusions

Running databases in containers is an ongoing trend and keeping data safe is always the top priority for a business. Kata containers provide security isolation through mature and extensively tested qemu virtualization with little-to-none changes to the existing environment.

Deploy Percona XtraDB Cluster with ease in your Kubernetes cluster with our Operator and Kata containers for better isolation without performance penalties.

This article was originally posted on percona.com/blog. See the original article here.

The post Running Percona Kubernetes Operator for Percona XtraDB Cluster with Kata Containers appeared first on Superuser.

OpenStack upstream CI/CD tests the things on defined  LTS or stable distribution versions. OpenStack Technical Committee defines each cycle testing runtime. As per OpenStack Victoria testing runtime, defined versions are:

• Ubuntu 20.04
• CentOS 8
• openSUSE Leap 15

Ubuntu Focal (Ubuntu LTS 20.04) was released on April 23, 2020, and in OpenStack Victoria (released on 14th Oct 2020), we migrated the upstream CI/CD testing on the above-defined testing runtime. This work is done as one of the community-wide goals “Migrate CI/CD jobs to new Ubuntu LTS Focal“.

    What is this migration:

OpenStack CI/CD is implemented with Zuul jobs prepare the node to deploy the OpenStack using Devstack and run tests (Tempest or its plugins, project in-tree tests, rally tests etc). Base OS installed on the node is where OpenStack will be deployed by DevStack.

Till OpenStack Ussuri release, the base OS on the majority of the job’s node was Ubuntu Bionic (18.04). So DevStack used to deploy OpenStack on Ubuntu Bionic and then run tests.

With the new version of Ubuntu Focal (20.04), the node’s base OS has been moved from Ubuntu Bionic -> Ubuntu Focal. On every code change, it will make sure OpenStack work properly on Ubuntu Focal.

NOTE: This migration target only zuulv3 native jobs. Legacy jobs are left to keep running on Bionic and plan to be migrated on Focal while they migrate to zuulv3 native jobs . We had another community-goal to migrate all the legacy jobs to zuulv3 native.

 

    Perform migration testing in advance:

We started the work in June and prepared the devstack, tempest, and tox-based base jobs on Focal so that all projects gate can be tested and fixed in advance before devstack and Tempest base jobs merge. The idea behind the advance testing is to avoid or minimize the gate failure in any of the repo under any projects. This advance testing includes integration as well as tox based unit, functional, doc, pep8, and lower-constraint testing.

    Bugs & fixes:

This migration had more things to fix compared to the previous migration from Ubuntu Xenial to Bionic. One reason for that was Ubuntu Focal and python dependencies dropping python2.7 support and MySQL 8.0 compatibility. OpenStack already dropped the Python2.7 in the Ussuri release but OpenStack dependencies lower constraints were not updated to their python-3 only version because many of them were not python3-only at that time.  So in Ubuntu Focal, those dependencies versions are python3-only which caused many failures in our lower constraints jobs.

A few of the key issues we had to fix for this migrations are:

1. Failing device detachments on Focal: Bug#1882521
2. Few of the lower constraints are not compatible with python3.8: Bug#1886298.
3. Migrations broken on MySQL 8.x: story/2007732
4. Duplicate check constraint name ‘ck_started_before_ended’: story/2008121
5. No implicit user creation with GRANT syntax in MySQL 8.0: Bug#1885825
6.  pyflakes till 2.1.0 not compatible with py3.8: Bug#1886296.

    Actual Migration:

Fixing these bugs took a lot of time for us and that is the reason this migration was late and missed the initial deadlines.

• Tox based job migration happened on Sept 9th.
• devstack and Tempest base jobs migration happened on Sept 24th.

All the work for this migration are tracked on: https://storyboard.openstack.org/#!/story/2007865

All changes are: https://review.opendev.org/q/topic:%2522migrate-to-focal%2522+(status:open+OR+status:merged)

    How to migrate the third-party CI:

If your 3rd party CI jobs are still not migrated to zuulv3 then you need to first migrate legacy jobs to zuulv3 native. Refer to this community-goal for details.

For zuulv3 native jobs, like upstream jobs you need to switch the jobs nodeset from ubuntu Bionic to ubuntu Focal.

Below Diagram gives a quick glance of changing the nodeset to Ubuntu Focal:

If you want to verify the nodeset used in your zuul jobs, you can see the hostname and label in job-output.txt

In the same way, you can migrate your third-party CI also to Focal. If third-party job is using the base job without overriding the ‘nodeset’ then the job is automatically switched to Focal. If the job overrides the ‘nodeset’ then, you need to switch to Focal nodeset like shown above. All the Ubuntu Focal nodeset for a single node to multinode jobs are defined in devstack.

We encourage all the 3rd party jobs to migrate to Focal asap as devstack will not support the Bionic related fixes from Victoria onwards.

    Dependencies compatibility versions for upgrades to OpenStack Victoria:

There are many dependencies constraints that need to be bumped to upgrade the OpenStack cloud to the Victoria release. To know all those compatible versions, check the project-specific patches merged from here or from this Bug#1886298. This can help to have prepared the smooth upgrades.

    Completion Summary Report:

• A lot of work was involved in this. There are many changes required for this migration as compare to previous migration from Xenial to Bionic.
  • Many dependencies with lower constraints compatible with py3.8 and Focal distro had to be updated. Almost in all the repos.
  • Mysql 8.0 caused many incompatible DB issues.
• I missed the original deadline of m-2 to complete this goal due to falling gates and to avoid any gate block for any projects.
• More than 300 repo had been fixed or tested in advance before migration happened. This helped a lot to keep the gate green during this migration.

 

I would like to convey special Thanks to everyone who helped in this goal and made it possible to complete it in the Victoria cycle itself.

The post Why Cloud Networking with OpenStack Neutron Works Great for Enterprises appeared first on VEXXHOST.

Look back at our Pushing Keystone over the Edge presentation from the OpenStack Summit. Many of the points we make are problems faced by any application trying to scale across multiple datacenters. Cassandra is a database designed to deal with this level of scale. So Cassandra may well be a better choice than MySQL or other RDBMS as a datastore to Keystone. What would it take to enable Cassandra support for Keystone?

Lets start with the easy part: defining the tables. Lets look at how we define the Federation back end for SQL. We use SQL Alchemy to handle the migrations: we will need something comparable for Cassandra Query Language (CQL) but we also need to translate the table definitions themselves.

Before we create the tables, we need to create keyspace. I am going to make separate keyspaces for each of the subsystems in Keystone: Identity, Assignment, Federation, and so on. Here’s the Federated one:

CREATE KEYSPACE keystone_federation WITH replication = {'class': 'NetworkTopologyStrategy', 'datacenter1': '3'}  AND durable_writes = true;

The Identity provider table is defined like this:

    idp_table = sql.Table(
        'identity_provider',
        meta,
        sql.Column('id', sql.String(64), primary_key=True),
        sql.Column('enabled', sql.Boolean, nullable=False),
        sql.Column('description', sql.Text(), nullable=True),
        mysql_engine='InnoDB',
        mysql_charset='utf8')
    idp_table.create(migrate_engine, checkfirst=True)

The comparable CQL to create a table would look like this:

CREATE TABLE identity_provider (id text PRIMARY KEY , enables boolean , description text);

However, when I describe the schema to view the table defintion, we see that there are many tuning and configuration parameters that are defaulted:

CREATE TABLE federation.identity_provider (
    id text PRIMARY KEY,
    description text,
    enables boolean
) WITH additional_write_policy = '99p'
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND cdc = false
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
    AND compression = {'chunk_length_in_kb': '16', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
    AND crc_check_chance = 1.0
    AND default_time_to_live = 0
    AND extensions = {}
    AND gc_grace_seconds = 864000
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair = 'BLOCKING'
    AND speculative_retry = '99p';

I don’t know Cassandra well enough to say if these are sane defaults to have in production. I do know that someone, somewhere, is going to want to tweak them, and we are going to have to provide a means to do so without battling the upgrade scripts. I suspect we are going to want to only use the short form (what I typed into the CQL prompt) in the migrations, not the form with all of the options. In addition, we might want an if not exists  clause on the table creation to allow people to make these changes themselves. Then again, that might make things get out of sync. Hmmm.

There are three more entities in this back end:

CREATE TABLE federation_protocol (id text, idp_id text, mapping_id text,  PRIMARY KEY(id, idp_id) );
cqlsh:federation> CREATE TABLE mapping (id text primary key, rules text,    );
CREATE TABLE service_provider ( auth_url text, id text primary key, enabled boolean, description text, sp_url text, RELAY_STATE_PREFIX  text);

One thing that is interesting is that we will not be limiting the ID fields to 32, 64, or 128 characters. There is no performance benefit to doing so in Cassandra, nor is there any way to enforce the length limits. From a Keystone perspective, there is not much value either; we still need to validate the UUIDs in Python code. We could autogenerate the UUIDs in Cassandra, and there might be some benefit to that, but it would diverge from the logic in the Keystone code, and explode the test matrix.

There is only one foreign key in the SQL section; the federation protocol has an idp_id that points to the identity provider table. We’ll have to accept this limitation and ensure the integrity is maintained in code. We can do this by looking up the Identity provider before inserting the protocol entry. Since creating a Federated entity is a rare and administrative task, the risk here is vanishingly small. It will be more significant elsewhere.

For access to the database, we should probably use Flask-CQLAlchemy. Fortunately, Keystone is already a Flask based project, so this makes the two projects align.

For migration support, It looks like the best option out there is cassandra-migrate.

An effort like this would best be started out of tree, with an expectation that it would be merged in once it had shown a degree of maturity. Thus, I would put it into a namespace that would not conflict with the existing keystone project. The python imports would look like:

from keystone.cassandra import migrations
from keystone.cassandra import identity
from keystone.cassandra import federation

This could go in its own git repo and be separately pip installed for development. The entrypoints would be registered such that the configuration file would have entries like:

[application_credential] driver = cassandra

Any tuning of the database could be put under a [cassandra] section of the conf file, or tuning for individual sections could be in keys prefixed with cassanda_ in the appropriate sections, such as application_credentials as shown above.

It might be interesting to implement a Cassandra token backend and use the default_time_to_live value on the table to control the lifespan and automate the cleanup of the tables. This might provide some performance benefit over the fernet approach, as the token data would be cached. However, the drawbacks due to token invalidation upon change of data would far outweigh the benefits unless the TTL was very short, perhaps 5 minutes.

Just making it work is one thing. In a follow on article, I’d like to go through what it would take to stretch a cluster from one datacenter to another, and to make sure that the other considerations that we discussed in that presentation are covered.

Feedback?

For the first time, the November TOP500 list (published to coincide with Supercomputing 2020) includes fully OpenStack-based Software-Defined Supercomputers:

• At #99 is UM6P’s Toubkal
• At #421 is Cambridge University’s Cascade Lake extension of CSD3.

Drawing on experience including from the SKA Telescope Science Data Processor Performance Prototypting Platform and Verne Global's hpcDIRECT project, StackHPC has helped bootstrap and is providing support for these OpenStack deployments. They are deployed and operated using OpenStack Kayobe and OpenStack Kolla-Ansible.

A key part of the solution is being able to deploy an OpenHPC-2.0 Slurm cluster on server infrastructure managed by OpenStack Ironic. The Dell C6420 servers are imaged with CentOS 8, and we use our OpenHPC Ansible role to both configure the system and build images. Updated images are deployed in a non-impacting way through a custom Slurm reboot script.

With OpenStack in control, you can quickly rebalance what workloads are deployed. Users can move capacity between multiple Bare Metal, Virtual Machine and Container based workloads. In particular, OpenStack Magnum provides on demand creation of Kubernetes clusters, an approach popularised by CERN.

In addition to user workloads, the solution interacts with iDRAC and Redfish management interfaces to control server configurations, remediate faults and deliver overall system metrics. This was critical in optimising the data centre environment and resulted in the high efficiency achieved in the TOP500 list.

For more details, please watch our recent presentation from the OpenInfra Summit:

The OpenStack community had its second virtual Project Teams Gathering (PTG) following the Open Infrastructure Summit in October. Over 500 individuals and 46 teams (30+ OpenStack teams) across the globe, met and collaborated at the vPTG. Since the event concluded, several of those teams have posted summaries of the discussions they have had and the... Read more »

RDO Victoria Released

The RDO community is pleased to announce the general availability of the RDO build for OpenStack Victoria for RPM-based distributions, CentOS Linux and Red Hat Enterprise Linux. RDO is suitable for building private, public, and hybrid clouds. Victoria is the 22nd release from the OpenStack project, which is the work of more than 1,000 contributors from around the world.

The release is already available on the CentOS mirror network at http://mirror.centos.org/centos/8/cloud/x86_64/openstack-victoria/.

The RDO community project curates, packages, builds, tests and maintains a complete OpenStack component set for RHEL and CentOS Linux and is a member of the CentOS Cloud Infrastructure SIG. The Cloud Infrastructure SIG focuses on delivering a great user experience for CentOS Linux users looking to build and maintain their own on-premise, public or hybrid clouds.

All work on RDO and on the downstream release, Red Hat OpenStack Platform, is 100% open source, with all code changes going upstream first.

PLEASE NOTE: RDO Victoria provides packages for CentOS8 and python 3 only. Please use the Train release, for CentOS7 and python 2.7.

Interesting things in the Victoria release include:

• With the Victoria release, source tarballs are validated using the upstream GPG signature. This certifies that the source is identical to what is released upstream and ensures the integrity of the packaged source code.
• With the Victoria release, openvswitch/ovn are not shipped as part of RDO. Instead RDO relies on builds from the CentOS NFV SIG.
• Some new packages have been added to RDO during the Victoria release:
  • ansible-collections-openstack: This package includes OpenStack modules and plugins which are supported by the OpenStack community to help with the management of OpenStack infrastructure.
  • ansible-tripleo-ipa-server: This package contains Ansible for configuring the FreeIPA server for TripleO.
  • python-ibmcclient: This package contains the python library to communicate with HUAWEI iBMC based systems.
  • puppet-powerflex: This package contains the puppet module needed to deploy PowerFlex with TripleO.
  • The following packages have been retired from the RDO OpenStack distribution in the Victoria release:
    • The Congress project, an open policy framework for the cloud, has been retired upstream and from the RDO project in the Victoria release.
    • neutron-fwaas, the Firewall as a Service driver for neutron, is no longer maintained and has been removed from RDO.

Other highlights of the broader upstream OpenStack project may be read via https://releases.openstack.org/victoria/highlights.

Contributors
During the Victoria cycle, we saw the following new RDO contributors:

Amy Marrich (spotz)
Daniel Pawlik
Douglas Mendizábal
Lance Bragstad
Martin Chacon Piza
Paul Leimer
Pooja Jadhav
Qianbiao NG
Rajini Karthik
Sandeep Yadav
Sergii Golovatiuk
Steve Baker

Welcome to all of you and Thank You So Much for participating!

But we wouldn’t want to overlook anyone. A super massive Thank You to all 58 contributors who participated in producing this release. This list includes commits to rdo-packages, rdo-infra, and redhat-website repositories:

Adam Kimball
Ade Lee
Alan Pevec
Alex Schultz
Alfredo Moralejo
Amol Kahat
Amy Marrich (spotz)
Arx Cruz
Bhagyashri Shewale
Bogdan Dobrelya
Cédric Jeanneret
Chandan Kumar
Damien Ciabrini
Daniel Pawlik
Dmitry Tantsur
Douglas Mendizábal
Emilien Macchi
Eric Harney
Francesco Pantano
Gabriele Cerami
Gael Chamoulaud
Gorka Eguileor
Grzegorz Grasza
Harald Jensås
Iury Gregory Melo Ferreira
Jakub Libosvar
Javier Pena
Joel Capitao
Jon Schlueter
Lance Bragstad
Lon Hohberger
Luigi Toscano
Marios Andreou
Martin Chacon Piza
Mathieu Bultel
Matthias Runge
Michele Baldessari
Mike Turek
Nicolas Hicher
Paul Leimer
Pooja Jadhav
Qianbiao.NG
Rabi Mishra
Rafael Folco
Rain Leander
Rajini Karthik
Riccardo Pittau
Ronelle Landy
Sagi Shnaidman
Sandeep Yadav
Sergii Golovatiuk
Slawek Kaplonski
Soniya Vyas
Sorin Sbarnea
Steve Baker
Tobias Urdin
Wes Hayutin
Yatin Karel

The Next Release Cycle
At the end of one release, focus shifts immediately to the next release i.e Wallaby.

Get Started
There are three ways to get started with RDO.

To spin up a proof of concept cloud, quickly, and on limited hardware, try an All-In-One Packstack installation. You can run RDO on a single node to get a feel for how it works.

For a production deployment of RDO, use TripleO and you’ll be running a production cloud in short order.

Finally, for those that don’t have any hardware or physical resources, there’s the OpenStack Global Passport Program. This is a collaborative effort between OpenStack public cloud providers to let you experience the freedom, performance and interoperability of open source infrastructure. You can quickly and easily gain access to OpenStack infrastructure via trial programs from participating OpenStack public cloud providers around the world.

Get Help
The RDO Project has our users@lists.rdoproject.org for RDO-specific users and operators. For more developer-oriented content we recommend joining the dev@lists.rdoproject.org mailing list. Remember to post a brief introduction about yourself and your RDO story. The mailing lists archives are all available at https://mail.rdoproject.org. You can also find extensive documentation on RDOproject.org.

The #rdo channel on Freenode IRC is also an excellent place to find and give help.

We also welcome comments and requests on the CentOS devel mailing list and the CentOS and TripleO IRC channels (#centos, #centos-devel, and #tripleo on irc.freenode.net), however we have a more focused audience within the RDO venues.

Get Involved
To get involved in the OpenStack RPM packaging effort, check out the RDO contribute pages, peruse the CentOS Cloud SIG page, and inhale the RDO packaging documentation.

Join us in #rdo and #tripleo on the Freenode IRC network and follow us on Twitter @RDOCommunity. You can also find us on Facebook and YouTube.

If you haven’t already tried SkySQL, it is worth noting that you can now launch a Galera Cluster. SkySQL is an automated Database as a Service (DBaas) solution to launch a Galera Cluster within Google Cloud Platform. Launching a Galera Cluster is currently a tech preview and you are still eligible for USD$500 worth of credit, which should let you evaluate it for quite a bit.

When you choose Transactions (it also supports Analytics, Both (HTAP) and Distributed SQL which is also Galera Cluster), you’ll notice that you can launch the Galera Cluster tech preview in multiple regions: Americas, APAC, or EMEA. Costs per hour for Sky-4×15 which has 4 vCPUs and 15GB of memory is USD$0.6546/hour/node (and when you think about it, you’re getting a minimum of 3 Galera Cluster nodes and one MaxScale node which acts as a load balancer and endpoint for your application). You’ll also pay a little more for storage (100GB SSD storage is USD$0.0698/hour due to the 3 nodes). So overall, expect an estimated total of USD$1.9638/hour for the Sky-4×15 nodes, and $0.0698/hour for the 100GB storage per node, bringing your total to USD$2.0336/hour.

Once launched, you’ll note that the service state will be pending till all the nodes are launched. During this time you will also have to whitelist your IP addresses that are planning to access the endpoint. Doing so is extremely straightforward as it does automatic detection within the browser for you. You’ll probably need to add a few more for your application and so forth, but this is extremely straightforward and very well documented.

You’re then given temporary service login credentials, and again, it is extremely well documented. You also get an SSL certificate to login with, and considering this is using the cloud, it makes absolute sense.

A quick point to note: you may see such an error when you’re trying to connect to the MaxScale endpoint, especially if you’re using the MySQL 8 client: ERROR 1105 (HY000): Authentication plugin 'MariaDBAuth' failed. The easy fix is of course to use the proper client library. You also automatically get connected to one of the three nodes in the cluster.

Overall, when we evaluated it, you end up with a 10.5.5-3-MariaDB-enterprise-log, which means it also comes with a few handy additions not present in the community versions: GCache encryption, BlackBox, and non-blocking DDL (wsrep_osu_method = NBO is an option). When you run a SHOW VARIABLES you will notice a few new additions, some of which include: wsrep_black_box_name, wsrep_black_box_size, and an obviously new wsrep_provider, ibgalera_enterprise_smm.so.

Why not take SkySQL for a spin? It is a really easy way to launch a Galera Cluster, and you also have a $500 credit. Load some data. Send some feedback. And if you’re interested in learning more, why not attend: Achieving uninterrupted availability with clustering and transaction replay on November 17 at 10 a.m. PT and 4 p.m. CET? This and more will be discussed at the webinar.

Today, 11th of November, 2020, we have released v2020.11.1. The latest version is marked as Stable and can be used for production environment. Docker integration If you didn’t read our latest blog post we want to let you know that you can now install Fleio trough docker and docker compose by running a single command. […]

Our friends from https://amperecomputing.com kindly provided the computing resources that we used to check the current state of Windows virtualization on ARM64.

The test lab consisted of 3 Ampere Computing EMAG servers (Lenovo HR330A – https://amperecomputing.com/emag), each with 32 ARM64 processors, 128 GB of RAM and 512 GB SSD.

Toolchain ecosystem on Windows ARM64: building and running Cloudbase-Init

Cloudbase-Init is a provisioning agent designed to initialize and configure guest operating systems on various platforms: OpenStack, Azure, Oracle Cloud, VMware, Kubernetes CAPI, OpenNebula, Equinix Metal (formerly: Packet), and many others.

Building and running Cloudbase-Init requires going through multiple layers of an OS ecosystem, as it needs a proper build environment, C compiler for Python and Python extensions, Win32 and WMI wrappers, a Windows service wrapper and an MSI installer.

This complexity made Cloudbase-Init the perfect candidate for checking the state of the toolchain ecosystem on Windows ARM64.

Install Windows 10 PRO ARM64 on the EMAG ARM servers

EMAG servers come with CentOS 7 preinstalled, so the first step was to have a Windows ARM64 OS installed on them.

Windows Server ARM64 images are unfortunately not publicly available, so the best option consists in using Windows Insider (https://insider.windows.com/), Windows 10 PRO ARM64 images available for download.

As there is no ISO available on the Windows Insiders website, we had to convert the VHDX to a RAW file using qemu-img.exe, boot a Linux Live ISO which had dd binary tool on it (Ubuntu is great for this) on the EMAG server and copy the RAW file content directly on the primary disk.

For the dd step, we needed a Windows machine where to download / convert the Windows 10 PRO ARM64 VHDX and two USB sticks. One USB stick for the Ubuntu Live ISO and one for the Windows 10 PRO ARM64 RAW file.

Rufus was used for creating the Ubuntu Live ISO USB and copying the RAW file to the other USB stick. Note that one USB stick must be at least 32 GB in size to cover for the ~25GB of the Windows RAW file.

Tools used for the dd step:

• qemu-img.exe for VHDX -> RAW conversion: https://cloudbase.it/qemu-img-windows/
• Ubuntu 20.04 Focal Fossa ISO for ARM64: https://cdimage.ubuntu.com/focal/daily-live/current/focal-desktop-arm64.iso
• Rufus for burning ISO and dd-ing the RAW file on USB sticks: https://github.com/pbatard/rufus/releases

After the dd process succeeds, a server reboot was required. The first boot took a while for the Windows device initialization followed by the usual “Out of the box experience”.

The following steps show how we built Cloudbase-Init for ARM64. As a side note, Windows 10 ARM64 has a builtin emulator for x86, but not for x64. Practically, we could run the x86 version of Cloudbase-Init on the system, but it would have run very slow and some features would have been limited by the emulation (starting native processes).

Gather information on the toolchain required to build Cloudbase-Init

The Cloudbase-Init ecosystem consists of these main building blocks:

• Python for Windows ARM64
• Python setuptools
• Python pip
• Python PyWin32
• Cloudbase-Init
• OpenStack Service Wrapper executable

Toolchain required:

• Visual Studio with ARM64 support (2017 or 2019)
• git

Python for Windows ARM64

Python 3.x for ARM64 can be built using Visual Studio 2017 or 2019. In our case, we used the freely available Visual Studio 2019 Community Edition, downloadable from https://visualstudio.microsoft.com/downloads/.

The required toolchain / components for Visual Studio can be installed using this vsconfig.txt. This way, we make sure that the build environment is 100% reproducible.

Python source code can be found here: https://github.com/python/cpython.

To make the build process even easier, we leveraged GitHub Actions to easily build Python for ARM64. An example workflow can be found here: https://github.com/cloudbase/cloudbase-init-arm-scripts/blob/main/.github/workflows/build.yml.

Also, prebuilt archives of Python for Windows ARM64 are available for download here: https://github.com/ader1990/CPython-Windows-ARM64/releases.

Notes:

• We used Python 3.8.x ARM64.
• A requirement for Cloudbase-Init (oslo.config) is not yet supported on Python 3.9.x > 3.9.rc2, otherwise 3.9 is ok as well.
• A CPython patch is required to build the PyWin32 Cloudbase-Init requirement https://github.com/ader1990/cpython/commit/b8c59c9b96a7ad11094224b5631aae3b89323f7a

Python setuptools

Python setuptools is a Python package that handles the “python setup.py install” workflow.

Source code can be found here: https://github.com/pypa/setuptools.

The following patches are required for setuptools to work:

• https://github.com/ader1990/setuptools/commit/3e936d223e78952e86a5501742adfd2f1b154cb0
• https://github.com/ader1990/setuptools/commit/4345aa40888ab15e2789487c5df215fa42504948
• https://github.com/ader1990/setuptools/commit/64c10b9a5facd8e0cdabc57bc5fc56f67b2890fa

Installation steps for setuptools (Python and Visual Studio are required):

set VCVARSALL="C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat"
set CL_PATH="C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX86\ARM64\cl.exe"
set MC_PATH="C:\Program Files (x86)\Windows Kits\10\bin\10.0.17763.0\arm64\mc.exe"

call %VCVARSALL% amd64_arm64 10.0.17763.0 & set

git clone https://github.com/ader1990/setuptools 1>nul
IF %ERRORLEVEL% NEQ 0 EXIT 1

pushd setuptools
    git checkout am_64
    echo "Installing setuptools"
    python.exe bootstrap.py 1>nul 2>nul
    IF %ERRORLEVEL% NEQ 0 EXIT 1

    %CL_PATH% /D "GUI=0" /D "WIN32_LEAN_AND_MEAN" /D _ARM64_WINAPI_PARTITION_DESKTOP_SDK_AVAILABLE launcher.c /O2 /link /MACHINE:ARM64 /SUBSYSTEM:CONSOLE /out:setuptools/cli-arm64.exe
    IF %ERRORLEVEL% NEQ 0 EXIT 1

    python.exe setup.py install 1>nul
    IF %ERRORLEVEL% NEQ 0 EXIT 1
popd

Python pip

Python pip is required for easier management of Cloudbase-Init’s requirements installation and wheels building.

Python’s wheel package is required to build wheels. Wheels are the pre-built versions of Python packages. There is no need to have a compiler to install the package from source on the exact system version the wheel has been built for.

Pip sources can be found here: https://github.com/pypa/pip.

The following pip patch is required: https://github.com/ader1990/pip/commit/0559cd17d81dcee43433d641052088b690b57cdd.

The patch introduces two binaries required for ARM64, which were built from: https://github.com/ader1990/simple_launcher/tree/win_arm64

This patched version of pip can use the wheel to create proper binaries for ARM64 (like setuptools).

Installation steps for wheel (Python is required):

echo "Installing pip"
python.exe -m easy_install https://github.com/ader1990/pip/archive/20.3.dev1.win_arm64.tar.gz 1>nul 2>nul
IF %ERRORLEVEL% NEQ 0 EXIT

Python PyWin32

Python PyWin32 package is a wrapper for (almost) all Win32 APIs from Windows. It is a behemoth from the source code perspective, with Cloudbase-Init using a limited amount of Win32 APIs via PyWin32.

Source code can be found here: https://github.com/mhammond/pywin32.

The following patches are required:

• https://github.com/ader1990/pywin32/commit/e7864cdeb9474eb7d5e672ea5aee989f9457362d
• https://github.com/ader1990/pywin32/commit/f6795ff6cb91c0c613c23e5bb3897f2e7d45b9f5
• https://github.com/ader1990/pywin32/commit/8d02293df764e69fdafd225316922a9a806e25b7

Installation steps for PyWin32 (Python 3.8 and Visual Studio 2019 are required):

echo "Installing pywin32"
git clone https://github.com/ader1990/pywin32 1>nul
IF %ERRORLEVEL% NEQ 0 EXIT 1
pushd pywin32
    git checkout win_arm64
    IF %ERRORLEVEL% NEQ 0 EXIT 1
    pushd "win32\src"
        %MC_PATH% -A PythonServiceMessages.mc -h .
    popd
    pushd "isapi\src"
        %MC_PATH% -A pyISAPI_messages.mc -h .
    popd
    mkdir "build\temp.win-arm64-3.8\Release\scintilla" 1>nul 2>nul
    echo '' > "build\temp.win-arm64-3.8\Release\scintilla\scintilla.dll"
    python.exe setup.py install --skip-verstamp
    IF %ERRORLEVEL% NEQ 0 EXIT 1
popd

The build process takes quite a lot of time, at least half an hour, so we took a(nother) cup of coffee and enjoyed the extra time.

The patches hardcode some compiler quirks for Visual Studio 2019 and remove some unneeded extensions from the build. There is work in progress to prettify and upstream the changes.

Cloudbase-Init

Now, as all the previous steps have been completed, it is time to finally build Cloudbase-Init. Thank you for your patience.

Source code can be found here: https://github.com/cloudbase/cloudbase-init

Installation steps for Cloudbase-Init (Python and Visual Studio are required):

echo "Installing Cloudbase-Init"
git clone https://github.com/cloudbase/cloudbase-init 1>nul
IF %ERRORLEVEL% NEQ 0 EXIT 1
pushd cloudbase-init
  echo "Installing Cloudbase-Init requirements"
  python.exe -m pip install -r requirements.txt 1>nul
  IF %ERRORLEVEL% NEQ 0 EXIT 1
  python.exe -m pip install .  1>nul
  IF %ERRORLEVEL% NEQ 0 EXIT 1
popd

After the installation steps were completed, the cloudbase-init.exe AMR64 executable wrapper will be available.

OpenStack Service Wrapper executable

Cloudbase-Init usually runs as a service at every boot. As cloudbase-init.exe is a normal executable, it needs a service wrapper for Windows. A service wrapper is a small program that implements the hooks for the Windows service actions, like start, stop and restart.

Source code can be found here: https://github.com/cloudbase/OpenStackService

The following patch was required: https://github.com/ader1990/OpenStackService/commit/a48c4e54b3f7db7d4df163a6d7e13aa0ead4a58b

For an easier build process, a GitHub actions workflow file can be found here: https://github.com/ader1990/OpenStackService/blob/arm64/.github/workflows/build.yml

A prebuilt release binary for OpenStackService ARM64 is available for download here: https://github.com/ader1990/OpenStackService/releases/tag/v1.arm64

Epilogue

Now we are ready to use Cloudbase-Init for guest initialization on Windows 10 PRO ARM64.

Main takeaways:

• The main building blocks (Python and Visual Studio) are in great shape to be used for ARM64 applications
• Some of the Python packages required for Cloudbase-Init still need minor tweaks when it comes to the build process on ARM64.

The post Windows on ARM64 with Cloudbase-Init appeared first on Cloudbase Solutions.

Installation of Openshift on the OpenStack cloud During last “Day of Learning” in Red Hat which was great opportunity for me to spent whole day on learning something new to me and I choose to learn a bit about installation and management of the Openshift cluster. This post is mostly note for myself from what I did during that training. I was using Openshift 4.6.1 and I installed it on the OpenStack based cloud.

Fleio version 2020.11.0 is now available in beta and you can test it in your lab environment, since this release is not recommended for production. You can now install Fleio through docker and docker-compose by running a single command. Fleio now works on Ubuntu 20.04 and CentOS 8. Note that we continue to support Fleio […]

Yet another virtual PTG happened between 26th and 30th of October. Neutron team had sessions in each day of the PTG. Below is summary of what we discussed and agreed during that time. Etherpad with notes from the discussions can be found at OpenDev Etherpad. Retrospective of the Victoria cycle From the good things during Victoria cycle team pointed: Complete 8 blueprints including the Metadata over IPv6, Improved feature parity in the OVN driver, Good review velocity From the not so good things we mentioned:

Due to COVID-19 pandemic, OpenStack PTG for Wallaby development planning and discussions are held virtually from Oct 26 – 30th, 2020. Overall we had a good discussion and another successful virtual PTG though most of us missed the face to face interaction. This blog covers the policy popup (consistent RBAC) discussion that happened in the Forum and Monday and Tuesday PTG.

    Forum:

Etherpad

Progress of consistent RBAC:

• Three projects completed till now – Keystone, Nova and, Cyborg
• One in progress – Barbican
• Planning in Wallaby – Neutron, Manila

We discussed the current and possible future challenges while migrating to the new policy. policy file in JSON format is one know and we talked about a current workaround and long term plan.

Deprecation warnings are still an issue and a lot of warnings are logged. This is logged as a nova bug. Also, the HTML version of the policy doc does not have the deprecation rule and reason. Example: https://docs.openstack.org/nova/latest/configuration/policy.html We need to add it in these docs too.

A clear step by step document about how to use system scope in cloud.yaml as well as in general with all migration steps are much needed.

We also asked if any deployment is migrated to new policy but there is none yet.

    PTG:

Etherpad

We carried the Forum sessions discussion in PTG with few extra topics.

    Migrate Default Policy Format from JSON to YAML (All projects)

We talked about it and decided that it will be great to do this in advance before projects start moving towards the new policy. and having this as a community goal in wallaby will help this effort to move faster. I have proposed this as a goal in TC PTG and it was agreed to select it for wallaby (Goal proposal) We also need to update devstack for the neutron policy file which is policy.json.

    Deprecation warning on-demand based on config option?

There is no best solution for deprecation warnings when it is a lot in case of new policy work. We cannot stop logging warnings completely. We discussed an option to provide a config option to disable the warnings (enable by default) and only for default value change, not the policy name change. The policy name change is a little critical even in policy overridden case also that is why we should not make it switchable. This way operators can disable warnings after seeing it for the first time and if it is too noisy.

    New policy in Horizon

It is challenging to adopt new policy in Horizon when some projects have new policies and some not. We left this for now and will continue brainstorming once we do some investigation on the usage of system token for new policy and old one. For now, amotoki proposed a workaround to keep the policy file with the deprecated rule as the overridden rules. –

    What are the expectations from project teams?

In the end, we discussed what all things to be targeted as part of new policy work and what all as a separate effort. Below is the list and I will be documenting it on wiki.

• Things to do as part of new policy effort:
  • Migrating JSON to YAML (in advance though)
  • Policy Tests coverage
  • System scoped administrator policy support and “reader” role support
  • upgrade-check support when changing the default policy
• As separate effort:
  • Remove hard-coded admin checks in the API/DB/”Views”
  • Standardizing policy names (depends on projects)

Due to COVID-19 pandemic, OpenStack PTG for Wallaby development planning and discussions are held virtually from Oct 26 – 30th, 2020. Overall we had a good discussion and another successful virtual PTG though most of us missed the face to face interaction. This blog covers the QA (Quality Assurance) discussion that happened on Monday and Tuesday.

Etherpad

    Victoria Retrospective:

We talked about the retrospective of the last cycle, one main challenge we discussed is the slow review in QA
due to fewer contributors. Below are the action items that came up from improving things:

Action Items:

• microversion need more work also when you try more combination or latest (gmann).
• Triage bugs in the office hour as 30 min -> update agenda
• Speed up the review.

 

    Office hour schedule:

With daylight ending in Nov, we decided to shift the QA office hour by an hour late which is at 14 UTC.

 

    Move back the single horizon test in tempest tree from tempest-horizon:

Tempest and Horizon team decided to bring back the single horizon test to Tempest tree and retire the
tempest-horizon plugins which was too much maintenance for single tests. It is unlikely to happen but if
Tempest team in future plan to remove this test then it needs to be consulted with Horizon team first.

Action items:

• Move test back to Tempest – gmann
• Deprecate and retire tempest-horizon – amotoki
• Check with the release team if any other release needed. (amotoki + gmann)
• modify the ‘horizon-dsvm-tempest-plugin’ job to direct to tests’ new location

 

   Add run_validation as a centralized point for Tempest scenario tests:

For now, validation resources automation and skip ssh part via run_validation is done in API test only and this
proposal is to extend it to scenario tests also. There is no reason not to do that which will help to run tempest
scenario tests on images where ssh is not allowed. There is a most possible situation where complete tests
need to be skipped but we will see case by case and at least skipping automatically will help tester to avoid
explicitly excluding the scenario tests to the skip list.

Action Items:

• Implement it in scenario tests – paras333
• Refactor scenario manager’s create_server method to consume common/compute.py – paras333

 

    Patrole stable release:

We are lacking the maintainer in Patrole project which is a big challenge to release a stable version of it. Sonia
and doug (already helping) will try to spend more bandwidth in Patrole. Another
thing to reduce the Patrole tests execution time. We discussed a few options like policy engine to return the
API based on the flag or use fake driver etc but we did not conclude any option yet as they need more investigation.

Action items:

• Keep eyes on gate and how tests are running and based on that we can make a stable release for Wallaby. – gmann
• Keep exploring the options for making patrole lightweight. – gmann, doug

 

    Grenade: extending the testing for zero downtime upgrade testing:

This is just a thought. There is TC tag ‘assert:supports-zero-downtime-upgrade’ which is not
used by any of the projects and also we do not have any testing framework which can verify this
tag. We talked about it if we can do such testing in grenade or not. Before moving forward on
discussion and investigation we checked if anyone can volunteer for this work. As of now, no
volunteer.

 

    Use different guest image for gate jobs to run tempest tests:

We only test cirros guest image to create the test VM and all proposal from Paras is to try more
images to enhance the different configuration scenarios in upstream testing. This will help
to able to catch more failure/scenario at the upstream gate itself compare to the current situation where
most of them are reported from downstream testing.

Action items:

• Add a job as experimental – paras333

 

    Tempest Cred Manager proposed changes:

‘primary’ and ‘alt_promary’ credentials in Tempest are hardcoded to non-admin and they are assigned
configured ‘tempest_role’. There is no way we can assign a different role to both of these creds. The idea here
is to make ‘primary’ and ‘alt_promary’ credentials configurable so that different deployment can configure
these with different roles in various combinations. We will be adding two config option similar to
‘tempest_role’ and default to an empty list so that we continue the default behavior with what we have currently.
So that there is no backward-incompatible change instead it is an additional new thing provided.

Action Items:

• Doug needs to propose the patch.

 

    Tempest Scenario manager stable version brainstorming:

We talked about a few method changes to make the scenario manager stable. If any method is used among plugins
only and not in Tempest then we do not actually need to move it to Tempest and they can stay on the side of the plugin itself.
We ran out of time here and will continue the brainstorming in office hours or so.
(this etherpad – https://etherpad.opendev.org/p/tempest-scenario-manager)

Action Items:

• Sonia and Martin will continue working on this.

 

    Wallaby Priority & Planning:

We did not prioritize the things as such but listed all working items in etherpad along with Victoria cycle backlogs.

Due to COVID-19 pandemic, OpenStack PTG for Wallaby development planning and discussions are held virtually from Oct 26 – 30th, 2020. Overall we had a good discussion and another successful virtual PTG though most of us missed the face to face interaction. This blog covers the TC discussion that happened on Tuesday and Friday.

 

 Etherpad

    Wallaby Leaderless assignment:

Four projects are still pending for leadership assignment. We discussed the next step. If you would like to help any of these or using in your cloud/distro, this is the right time to step up:

1. Karbor

• Action item: diablo_rojo will start the retirement process.

2. Qinling

• Action item: gmann to start the retirement process.

3. Searchlight

• Action item: gmann to start the retirement process.

4. Placement

•  Action item: gibi will update TC about the final discussion and accordingly mnaser to propose the patch in governance.

 

    Victoria Retrospective:

There are a couple of things TC finished in last cycle, few of them are:

• Reduced size of TC to speed up the things.
• Updated policy to add projects faster.
• Outlined Distributed Project Leadership Model as an alternate leadership solution of PTL. Oslo will be the first project to try this new model.
• Normalized retirement processes. All the retired repos cleanup is finished.
• Tag cleanups & additions This is not complete yet but started with tc-approved release tag in the Victoria cycle.
• Merged UC with TC. This is one of the key things finished and the best strategy to make users-developers work more closely.

 

    TC Meeting time:

TC is going to try (re-try) the weekly meeting every Thursday 15:00 UTC.

Action Items:

• mnaser: propose patch.

 

    TC policy regarding OpenStack client:

Forum Discussion

This is one of the important items and still no good progress on this. From TC perspective, we talked about how to ask or motivate projects to migrate to OSC. TC needs to be very clear on whether it’s a strict policy or just guidelines. After a long discussion on this, we are going with the below strategy: These strategies will be documented as TC resolution.

•  All user docs should have `openstack` client commands.
•  Focus on user docs first and then ops docs later
•  All ci-tooling should have `openstack` client usage
•  Use the process to find what is missing so we can prioritize it.

Action items:

•  diablo_rojo to work on initial resolution proposal

 

    Finalize the Wallaby cycle goal selection:

There are two proposals for the Wallaby community-wide goals. First is ‘oslo.rootwrap to oslo.privsep’ which is already being discussed in the last cycle also and all set to be selected. 2nd proposal came up during PTG itself. During policy-popup PTG, it came up that deprecating the JSON formatted of policy file will be a good advance step before the projects move to new RBAC policies. This will help operators to smoothly migrate to new policies. This does not involve much work and ok to select as 2nd goal for Wallaby cycle,

TC agreed to have the below goals for the Wallaby cycle:

• Migrate from oslo.rootwrap to oslo.privsep 
• Migrate Default Policy Format from JSON to YAML

Action items:

• TC to merge the patches which selects both goals for the Wallaby cycle.

 

    TC tag cleanup:

In the Victoria cycle, we started the process to audit all the TC tags and start cleanup those. We removed the ‘tc:approved-release’ tag in the Victoria cycle. In this PTG we discussed two more tags.

1. assert:supports-zero-downtime-upgrade:

Currently, there is no project who has this tag and also no testing framework available. Testing for zero downtime is not so easy in upstream testing. We decided to remove this tag as it is advertising something we aren’t doing. If anyone interested to spend time on this in the future then we can add it after projects start testing it and document it.

2. assert:supports-api-interoperability:

This tag is whether project API is interoperable also or not and this is important from the interop trademark program also. We only have Nova having this tag. Our goal is to promote more projects to apply for this tag. During the discussion, we found that we need to clarify this tag more clearly. For example, this tag is not about implementing the microversion but any versioning schema which provides the feature (API changes) discoverability. And also it is about how we change the API not how our APIs are currently. As long as services have some versioning mechanism to discover the changes and follow the API SIG guidelines for interoperability, and test it via branchless testing way, that service is applicable to apply for this tag.

TC will document this tag in a more clear way and encourage each project to start working on applying this tag.

Action Items:

• Graham to put up a change to update the wording to allow for discovery mechanisms.
• Should include the release in which they started to assert this tag.
• Documents that it’s about following the guidelines for future changes not existing one. – gmann
• Should socialize this out to the projects after we get the documentation improvements landed. – gmann

 

    k8s Community Bridging :

This is one of the exciting discussion for everyone. We hosted this cross-community meeting with Kubernetes steering committee teams. Bob and dims from the Kubernetes steering committee joined us in this discussion. It was started with a quick introduction from both teams and then started the discussion on the below topics:

• Sharing governance models:

k8s governance hierarchy is LF->CNCF->k8s steering commitee->various SIG/working group. k8s Steering the committee consists of 7 elected members and doesn’t actually influence the direction of the code instead leave it to SIG and arch committee. There is no chair in this committee and host biweekly private as well as public meetings.  Each SIG (repos) team has approver and reviewer roles where the reviewer review the code and the approver are responsible for merging the code. Naser explained the OpenStack governance model.

•  What are your 3 biggest issues that are barriers to growth/sustainability as a community? as a project?
  • Going up the contribution ladder is hard.
  • Stale/emeritus membership in reviewer/approver level groups.
  • Mono repo makes it so that each SIG involved needs to review which can slow things down.
• Area/Domain mapping vs service/repo centered teams
  • mono repo forces people to work together and it’s a little more diverse who is reviewing
  • mono repo is bad because sometimes things need to be in multiple places like documentation
  • mono repo struggles with libraries especially on testing the external one.
• Related Ecosystem projects comparisons

The SIG lead has to sign off and accept that they are willing to take ownership + handle maintenance + releasing etc. It is general consensus that we try to distribute much work as possible to the subgroup and keep it out of k/k. There is general consensus across k8s leadership that work should be delegated out to subgroups. For API interoperability challenge in a distributed model, k8s have conformance tests that exercise the API performance and vendors try and upload conformance results every release or dot release.

• Common FOSS topics

release and CI part was discussed. and also on how COVID things are impacting community health. k8s community almost lost their independents or part-time contributors. also, the k8s community is doing 3 releases per year compared to 4.

• Staying connected going forward

To stay connected it will be a good idea if we extend an invite for K8s to join PTG.

 

    Upstream Investment Opportunities:

We have three upstream opportunities defined for 2020 but there is no help on any of these, even in previous (2018, 2019) upstream opportunities also. We started the discussion if we need to continue this for 2021 years also or just stop defining it and decide the area to help when we have someone interested. Before deciding anything on this mnaser will discuss this with the board of directors and get their opinion.

Action Items:

•  mnaser to find ways to engage with members and talk with them.
•  mnaser to talk to board about infrastructure
•  mnaser to talk to board about platinum members contribution levels + what are they doing to drive engagement.

 

    Pop Up Team Check In:

Currently, we have two active popup teams 1. policy 2. Encryption. TC checked the progress on both teams. The policy team is very active and finished some work in Voctoria cycle (cyborg
finished it and Barbican started) and also hosted forum, PTG sessions, and discussed Wallaby development plan. This team host biweekly meeting to discuss and review the progress.
The encryption team is also active. Josephine explained the progress on this. Glance spec is merged.

Both teams will continue in Wallaby cycle also.

 

    Completing the Gerrit breach audit :

There are still 19 teams pending to finish this audit. Those are listed in etherpad

We encourage all those pending teams to finish the audit, TC members will start following with those projects every week.

Action Items:

• TC to follow up every week on progress

 

    Other topics:

We ran out of time and all the pending (below) topics will be discussed in TC regular meetings. TC will skip this month’s meeting but will have weekly meetings on 12th Nov onwards.

• Better way to test defined testing runtime
• Monitoring in OpenStack: Ceilometer + Telemetry + Gnocchi state
• Stable core team maintainer and process to recruit the new members and adding core team members in stable core list.

 

Last week, thousands of community members participated in the Open Infrastructure Summit. This time, the commute for over 10,000 attendees was short, because just like every other 2020 conference, the Summit was virtual. Hosted by the Open Infrastructure Foundation (previously the OpenStack Foundation (OSF)—if you missed this announcement, check out the news), the Summit gathered people from over 120 countries and 30 different open source communities to collaborate around the next decade of open infrastructure. 

The hallway track and networking activities were missed, but the week was still full of announcements and new (and growing!) users sharing their open source, production use cases. The event was free to participate, and this was only possible with the support of the Summit sponsors: 

Headline: Canonical (ubuntu), Huawei, VEXXHOST
Premier: Cisco, Tencent Cloud
Exhibitor: InMotion Hosting, Mirantis, Red Hat, Trilio, VanillaStack, ZTE

Below is a snapshot of what you missed and what you may want to rewatch. 

Like I mentioned earlier, the OSF opened the Summit with some big news. Jonathan Bryce, executive director, announced the Open Infrastructure Foundation (OIF) as the successor to the OSF during the opening keynotes on Monday, October 19. With support from over 60 founding members and 105,000 community members, the OIF remains focused on building open source communities to build software that runs in production. Bryce was joined by Mark Collier, OIF COO, who announced that the OIF board of directors approved four new Platinum Members (a record number of new Platinum Members approved at one time): Ant Group, Facebook Connectivity, FiberHome, and Wind River. 

The OIF builds open source communities who write software that runs in production.The OpenStack and Kata Containers communities celebrated software releases, and dozens users shared their open infrastructure production use cases. 

Five days before the Summit, the OpenStack community released its 22nd version, Victoria. In Tuesday’s keynote, Kendall Nelson, chair of the First Contact SIG, talked about some of the features that landed including Ironic features for a smaller standalone footprint and supporting more systems at the edge. There were also features around hardware enablement and supporting FPGAs that she says will continue through the Wallaby cycle, which the upstream developers are discussing at the Project Teams Gathering (PTG) this week. 

Right in time for the Summit, the Kata Containers community released its 2.0 version, including a rewrite of the Kata Containers agent to help reduce the attack surface and reduce memory overhead. The agent was rewritten in Rust, and users will see a 10x improvement in size, from 11MB to 300KB. Xu Wang, a member of the Kata Architecture Committee, joined the keynotes on Monday to talk about how Kata 2.0 is already running in production at Ant Group, home of the largest payment processor in the world as well as other financial services. At Ant Group, Kata Containers is running on thousands of nodes and over 10,000 CPU cores. 

Ant Group is one of many users who shared information around their production use cases. Below are some highlights of the users who spoke. You can now watch all of the breakout and keynote sessions, and there will also be some Forum sessions uploaded in the coming days. 

Production use cases: 

The OIF announced its newest open infrastructure pilot project, OpenInfra Labs, a collaboration among universities and vendors to integrate and optimize open source projects in production environments and publish complete, reproducible stacks for existing and emerging workloads. Michael Daitzman, a contributor to the project, delivered a keynote introducing the project, thanking the community for their work with projects like OpenStack, Kubernetes, and Ceph, and inviting new contributors to get involved.

Magma, an open source mobile packet core project initiated by Facebook Connectivity, was front and center at the Summit last week. In the opening keynotes, Amar Padmanabhan, engineer at Facebook, introduced Magma and shared the community’s mission to bridge the digital divide and connect the next billion people to the Internet. The project was further discussed in a production use case from Mariel Triggs, the CEO of MuralNet, who talked about the connectivity issues that indigneous nations face and how her organization leverages Magma for an affordable way to keep them connected. Boris Renski, founder and CEO of FreedomFi, returned to the Summit keynote stage to show that building an LTE network with Magma is so easy, even a goat could learn to do it. And sure enough, the goat successfully deployed the network. I’m pretty sure the looks on these faces sum it all up. 

Announced a few weeks ago, Verizon is running Wind River’s distribution of StarlingX in production for its 5G virtualized RAN. During Tuesday’s keynote, Ildiko Vancsa talked about their use case and why Verizon relies on StarlingX for ultra low latency, high availability, and zero-touch automated management. 

Over 15 million compute cores are managed by OpenStack around the world. Imtiaz Chowdhury, cloud architect at Workday, talked about how their deployment has contributed to that growth with their own 400,000 core OpenStack deployment. 

Additional OpenStack users talking about their production use cases include: 

• • CERN shared their OpenStack journey from 0 to 300,000 cores and how their resource management and deployment model changed completely: from virtualization and server consolidation to a cloud infrastructure and from bare metal to virtual machines to managed bare metal to containers. Belmiro Moreira also shared some of the challenges like leveraging container orchestration to deploy the OpenStack control plane and how they collaborated with the community to get where they are today. 
  • China Mobile showcased the automated hardware integration system its team built for its NFV cloud project and explain how 50,000 servers have been implemented with plug-and-play. 
  • GE Digital presented the tools, described the migration procedure, and highlighted the biggest challenges in upgrading from OpenStack Newton to Queens with minimal downtime. 
  • Société Générale, one of the top three French banks and a founding silver member of the OIF, talked about how they provide cloud services in multiple availability zones using a single OpenStack deployment.
  • The European Weather Cloud, a joint project between the European Centre for Medium-Range Weather Forecasts (ECMWF) and the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT), shared their infrastructure architecture which is based 100% on open source software: OpenStack (Ussuri) and Ceph (Nautilus). 

Volvo Cars shared their Zuul production use case to kick off the second day of Summit keynotes. Johannes Foufas and Albin Vass talked about how premium cars need premium tools, and Zuul is a premium tool. The team uses Zuul to build several software components including autonomous driving software, and Foufas says speculative merge and the prioritized queue system are two Zuul features their team relies on.  

SK Telecom 5GX Labs won the 2020 Superuser Awards for their open infrastructure use case integrating multiple open source components in production, including Airship, Ceph, Kubernetes, and multiple components of OpenStack.  

This was the first year the Superuser Awards ceremony was only held once, and there were eight organizations who shared production open infrastructure use cases that were reviewed by the community and advisors to determine the winner. 

Learn how the 2020 Superuser Awards nominees are powering their organization’s infrastructure with open source in production: 

• Adobe
• China Mobile 
• Leboncoin
• LINE
• SK Telecom 5GX Labs
• StackHPC
• Trendyol Tech 
• Workday

If you missed any of the above sessions or announcements, check out the Open Infrastructure Foundation YouTube channel. Then, join the global Open Infrastructure community, and share your own personal open source story using #WeAreOpenInfra on social media.

The post Virtual Open Infrastructure Summit Recap appeared first on Superuser.

In this blog post we will go back 10 years to understand how OpenStack started at CERN. We will explore the motivation, the first installation, the prototypes, the deployment challenges and the initial production architecture. Let’s start! Before OpenStack 10 years ago CERN IT was working to bring virtualization into the CERN data centre. At that time most applications were still deployed directly in the physical nodes. With the start of the LHC (Large Hadron Collider) it was expected that the resource requests from the users and experiments would increase significantly.

The post OpenStack Victoria is Here! Let’s Get To Know Version 22 appeared first on VEXXHOST.

The post Open Infrastructure Summit 2020 – Recap of the First-Ever Virtual Summit appeared first on VEXXHOST.

We have video recording available for you to learn how you can benefit the New Galera Manager. It includes live demo how to install Galera Manager and deploy easily Galera Cluster on Amazon Web Service for Geo-distributed Multi-master MySQL, Disaster Recovery and fast local reads and writes. Now you can monitor and manage your Galera Cluster with Graphical Interface.

“The presentation was great with lots of valuable information. We will definitely try to implement Galera Manager in our environment very soon” stated attendee of the webinar.

Watch Galera Manager webinar recording

Download Galera Manager

The Ubuntu OpenStack team at Canonical is pleased to announce the general availability of OpenStack Victoria on Ubuntu 20.10 (Groovy Gorilla) and Ubuntu 20.04 LTS (Focal Fossa) via the Ubuntu Cloud Archive. Details of the Victoria release can be found at:  https://www.openstack.org/software/victoria.

To get access to the Ubuntu Victoria packages:

Ubuntu 20.10

OpenStack Victoria is available by default for installation on Ubuntu 20.10.

Ubuntu 20.04 LTS

The Ubuntu Cloud Archive for OpenStack Victoria can be enabled on Ubuntu 20.04 by running the following command:

sudo add-apt-repository cloud-archive:victoria

The Ubuntu Cloud Archive for Victoria includes updates for:

aodh, barbican, ceilometer, cinder, designate, designate-dashboard, glance, gnocchi, heat, heat-dashboard, horizon, ironic, keystone, magnum, manila, manila-ui, masakari, mistral, murano, murano-dashboard, networking-arista, networking-bagpipe, networking-baremetal, networking-bgpvpn, networking-hyperv, networking-l2gw, networking-mlnx, networking-odl, networking-sfc, neutron, neutron-dynamic-routing, neutron-vpnaas, nova, octavia, octavia-dashboard, openstack-trove, trove-dashboard, ovn-octavia-provider, panko, placement, sahara, sahara-dashboard, sahara-plugin-spark, sahara-plugin-vanilla, senlin, swift, vmware-nsx, watcher, watcher-dashboard, and zaqar.

For a full list of packages and versions, please refer to:

http://reqorts.qa.ubuntu.com/reports/ubuntu-server/cloud-archive/victoria_versions.html

Reporting bugs

If you have any issues please report bugs using the ‘ubuntu-bug’ tool to ensure that bugs get logged in the right place in Launchpad:

sudo ubuntu-bug nova-conductor

Thank you to everyone who contributed to OpenStack Victoria. Enjoy and see you in Wallaby!

Corey

(on behalf of the Ubuntu OpenStack Engineering team)

TheJulia was kind enough to update the docs for Ironic to show me how to include IPMI information when creating nodes.

To all delete the old nodes

for UUID in `openstack baremetal node list -f json | jq -r '.[] | .UUID' ` ; do openstack baremetal node delete $UUID; done

nodes definition

I removed the ipmi common data from each definition as there is a password there, and I will set that afterwards on all nodes.

{
  "nodes": [
    {
      "ports": [
        {
          "address": "00:21:9b:93:d0:90"
        }
      ],
      "name": "zygarde",
      "driver": "ipmi",
      "driver_info": {
      		"ipmi_address":  "192.168.123.10"
      }
    },
    {
      "ports": [
        {
          "address": "00:21:9b:9b:c4:21"
        }
      ],
      "name": "umbreon",
      "driver": "ipmi",
      "driver_info": {
	      "ipmi_address": "192.168.123.11"
	}
      },	
    {
      "ports": [
        {
          "address": "00:21:9b:98:a3:1f"
        }
      ],
      "name": "zubat",
      "driver": "ipmi",
       "driver_info": {
	      "ipmi_address": "192.168.123.12"
       }
    }
  ]
}

Create the nodes

openstack baremetal create  ./nodes.ipmi.json 

Check that the nodes are present

$ openstack baremetal node list
+--------------------------------------+---------+---------------+-------------+--------------------+-------------+
| UUID                                 | Name    | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+---------+---------------+-------------+--------------------+-------------+
| 3fa4feae-0d5c-4e38-a012-29258d40651b | zygarde | None          | None        | enroll             | False       |
| 00965ad4-c972-46fa-948a-3ce87aecf5ac | umbreon | None          | None        | enroll             | False       |
| 8702ea0c-aa10-4542-9292-3b464fe72036 | zubat   | None          | None        | enroll             | False       |
+--------------------------------------+---------+---------------+-------------+--------------------+-------------+

Update IPMI common data

for UUID in `openstack baremetal node list -f json | jq -r '.[] | .UUID' ` ; 
do  openstack baremetal node set $UUID --driver-info ipmi_password=`cat ~/ipmi.password`  --driver-info   ipmi_username=admin   ; 
done

EDIT: I had ipmi_user before and it does not work. Needs to be ipmi_username.

Final Check

And if I look in the returned data for the definition, we see the password is not readable:

$ openstack baremetal node show zubat  -f yaml | grep ipmi_password
  ipmi_password: '******'

Power On

for UUID in `openstack baremetal node list -f json | jq -r '.[] | .UUID' ` ; do  openstack baremetal node power on $UUID  ; done

Change “on” to “off” to power off.

Codership is pleased to announce a new Generally Available (GA) release of the multi-master Galera Cluster for MySQL 5.6, 5.7 and 8.0, consisting of MySQL-wsrep 5.6.49 (release notes, download), 5.7.31 (release notes, download), and 8.0.21 (release notes, download) with Galera Replication library 3.31 (release notes, download) implementing wsrep API version 25 for 5.6 and 5.7, and Galera Replication library 4.6 (release notes, download) implementing wsrep API version 26 for 8.0. This release incorporates all changes to MySQL 5.6.49, 5.7.31 , and 8.0.21 respectively, adding a synchronous option for your MySQL High Availability solutions.

It is recommend that one upgrades their Galera Cluster for MySQL 5.6, 5.7 and 8.0 because it releases a fix for security vulnerability CVE-2020-15180. The binary tarball is also compiled with OpenSSL 1.1.1g.

A highlight of this release is that with MySQL 8.0.21, you will now have access to using the Percona audit log plugin, which will help with monitoring and logging connection and query activity that has been performed on specific servers. This implementation is provided as an alternative to the MySQL Enterprise Audit Log Plugin.

In addition to fixing deadlocks that may occur between DDL and applying transactions, in 8.0.21 the write-set replication patch is now optimised to work with the Contention-Aware Transaction Scheduling (CATS) algorithm that is present in InnoDB. You can read more about transaction scheduling in the MySQL manual.

For those that requested the missing binary tarball package, the MySQL 8.0.21 build includes just that. Packages continue to be available for: CentOS 7 & 8, Red Hat Enterprise Linux 7 & 8, Debian 10, SLES 15 SP1, as well as Ubuntu 18.04 LTS and Ubuntu 20.04 LTS. The latest versions are also available in the FreeBSD Ports Collection.

The Galera Replication library has had some notable fixes, one of which improves memory usage tremendously. The in-memory GCache index implementation now uses sorted std::deque instead of std::map, and this leads to an eightfold reduction in memory footprint. Hardware CRC32 is now supported on x86_64 and ARM64 platforms.

There are also three new status variables added: wsrep_flow_control_active (to tell you whether flow cotrol is currently active (replication paused) in the cluster), wsrep_flow_control_requested (to tell you whether the node has requested a replication pause because the received events queue is too long) and wsrep_gmcast_segment (to tell you which cluster segment the node belongs to).

For Galera Replication library 3.31, this is the last release for Debian Jessie and openSUSE 15.0. For Galera Replication library 4.6, this is the last release for openSUSE 15.0. For MySQL-wsrep 5.6 and 5.7, this is also the last release for Debian Jessie. For MySQL-wsrep 5.7 and MySQL-wsrep 8.0, this is the last release for openSUSE 15.0.

SK Telecom Cloud 5GX Labs is the 12th organization to win the Superuser Awards. The news was announced today during the virtual 2020 Open Infrastructure Summit. You can watch the announcement on demand in the Summit platform.

Elected by members of the community, the team that wins the Superuser Awards is lauded for the unique nature of its use case as well as its integration and application of open infrastructure. SK Telecom 5GX Cloud Labs was among eight nominees for the Award this year and is the first to receive the Award for an Airship use case.

In addition to contributing upstream to OpenStack and Airship, an open source project supported by the Open Infrastructure Foundation, SK Telecom developed a containerized OpenStack on Kubernetes solution called SKT All Container Orchestrator (TACO), based on OpenStack-helm and Airship. TACO is a containerized, declarative, cloud infrastructure lifecycle manager that enables them to provide operators the capability to remotely deploy and manage the entire lifecycle of cloud infrastructure and add-on tools and services by treating all infrastructure like cloud native apps. They deployed it to SKT’s core systems including telco mobile network, IPTV services, which currently has 5.5 million subscriptions; also for external customers (next generation broadcasting system, VDI, etc). Additionally, the team strongly engaged in community activity in Korea, sharing all of their technologies and experiences to regional communities (OpenStack, Ceph, Kubernetes, etc).

Just before the big announcement, Jeff Collins and Matt McEuen discussed the upcoming Airship 2.0 release, which is now in beta. Rewatch the announcement now!

 

 

The post SK Telecom 5GX Cloud Labs wins the 2020 Superuser Awards appeared first on Superuser.

Happy 10 years of OpenStack! Millions of cores, 100,000 community members, 10 years of you. Storytelling is one of the most powerful means to influence, teach, and inspire the people around us. To celebrate OpenStack’s 10th anniversary, we are spotlighting stories from the individuals in various roles from the community who have helped to make... Read more »

Open infrastructure will underpin the next decade of transformation for cloud infrastructure. With the virtual Open Infrastructure Summit well underway, the first major announcement has been the formation of a new foundation, the Open Infrastructure Foundation. StackHPC is proud to be a founding member.

StackHPC's CEO, John Taylor, comments "We are extremely pleased to be a part of the new decade of Open Infrastructure and welcome the opportunity to continue to transfer the values of "Open" to our clients."

StackHPC's CTO, Stig Telfer, recorded a short video describing how the concept of open infrastructure is essential to our work, and how as a company we contribute to open infrastructure as a central part of what we do: