Big Data: Cloudera: New SQL Choices in the Apache Hadoop Ecosystem: Why Impala Continues to Lead In our previous post from January 2014, we reported that Impala had achieved query performance over Apache Hadoop equivalent to that of an analytic DBMS over its own proprietary storage system. We believed this was an important milestone because Impala’s objective has been to support a high-quality BI experience on Hadoop data, not to produce a “faster Apache Hive.” An enterprise-quality BI experience requires low latency and high concurrency (among other things), so surpassing a well-known proprietary MPP DBMS in these areas was important evidence of progress. Read more.
Cloudera: Apache Spark 1.0 is Released Congratulations to the Apache Spark community for today’s release of Spark 1.0, which includes contributions from more than 100 people (including Cloudera’s own Diana Carroll, Mark Grover, Ted Malaska, Sean Owen, Sandy Ryza, and Marcelo Vanzin). We think this release is an important milestone in the continuing rapid uptake of Spark by enterprises — which is supported by Cloudera via Cloudera Enterprise 5 — as a modern, general-purpose processing engine for Apache Hadoop. Read more.
Datadog: Filter your Datadog Events Stream to pinpoint issues in your infrastructure Do you run a large or busy service that you monitor with Datadog or do you find yourself struggling to keep track of everything going on in your infrastructure? In my last role at a large software as a service company providing ratings and reviews functionality to brand and retail Web sites, we had a dozen teams all using the same Datadog account to monitor many different services, with thousands of servers and tens of thousands custom metrics. This resulted in hundreds of events each day within our infrastructure, such as hotfixes, configuration changes and servers being powered on or off. Any one person only cared about a small number of these events, and our teams were finding it challenging to pull those few key events to track from hundreds each day. Read more.
OpenStack: Aaron Rosen: Implementing High Availability Instances with Neutron using VRRP In the Havana release we added a new extension called “Allowed-Address-Pairs” that allows one to add additional ips (or cidrs) with their mac-address to a port to allow traffic that matches those values to pass through. This was needed because by default neutron ports only allow traffic through that match the mac-address and fixed-ips fields on a port (which is done to enforce anti-spoofing). Because of this, there was no way to support protocols such as VRRP which require mapping the same ip-address to multiple ports which neutron does not allow by design. Read more.
Adam Young: Kerberos, Keystone Client, and S4U2Proxy Since my eventual goal is to Kerberize Horizon, my next step after getting a CGI solution working was to make use of the Keystone client. Since the Kerberos auth plugin is still a work-in-progress, it required a little tweaking, but not all that much. Read more.
Adam Young: Kerberos, Federation, and Horizon I’ve been looking in to enabling Kerberos for Horizon. Since Horizon passes the Users credentials on to Keystone to get a token, Kerberos requires an additional delegation mechanism. This leads to some questions about how to handle delegation in the case of Federated Identity. Read more.
Elasticsearch: OpenStack Elastic-Recheck: powered by The Elk Stack Every day, the OpenStack project runs hundreds of patches through its continuous integration system to assure code consistency, functionality and smooth integration with other projects in the OpenStack ecosystem. This process works exceptionally well for a majority of patches, but as with any system pushing so much data through development infrastructure, there are times when there are failures unrelated to the patches being tested. It may be that a VM goes down unexpectedly during a test, an external resource is unavailable (nameserver, package repository, etc), a service unrelated to the test locks up or one of many race conditions or other transient bugs pops up.Read more.
ICCLab: Ceilometer Performance issues There have been some criticisms of the implementation of Ceilometer (or Telemetry as of Icehouse) – however, it’s still the main show in town for understanding what’s going on inside your Openstack. We’ve been doing a bit of work with it in multiple projects. In one of our efforts – pulling in energy info via kwapi – we noticed that Ceilometer really crawls to a halt with the API giving a response in 20s when trying to enter just a single energy consumption data point. (Yes, it might make more sense to batch these up…). For our simple scenario, this performance was completely unworkable. Read more.
Julien Danjou: OpenStack Design Summit Juno, from a Ceilometer point of view Last week was the OpenStack Design Summit in Atlanta, GA where we, developers, discussed and designed the new OpenStack release (Juno) coming up. I've been there mainly to discuss Ceilometer upcoming developments. Read more.
Mark Shuttleworth: Canonical’s cloud-init saves you from image soup, on every cloud We run an extensive program to identify issues and features that make a difference to cloud users. One result of that program is that we pioneered dynamic image customisation and wrote cloud-init. I’ll tell the story of cloud-init as an illustration of the focus the Ubuntu team has on making your devops experience fantastic on any given cloud.Read more.
Mirantis: OpenStack Database (Trove): Native Replication, Part 1 Let’s talk about replication a bit. How much do you know about it? Why is it so important? As for me, I can think of plenty of questions about replication, and they generally fall into these three categories: What is it? Why do we need it? (Justification and benefits) How it can be applied? In this two-part article series, we’re going to look at those questions in the context of a proposal to add replication capabilities to Trove, the OpenStack Database project.Read more.
Sean Roberts: My take on OpenStack projects, training, part 2 of 10 This is the second of a ten part series on OpenStack projects. There are critical OpenStack projects that get a bit lost in the noise. I aim to highlight why these projects are important and how you can get involved. Read more.
Sean Roberts; OpenStack community developer training, part 2 of 3 Teaching the OpenStack architecture and services is a good place to begin. That means no installing software until the student is ready to understand what they are installing. The best place to start is going through the basics in the Associates Training Guide and the intermediate topics in the Operators Training Guide first. These Guides are mostly combined from the existing OpenStack documentation. The Training project aims to reuse the existing materials as much as possible. Read more.
SiliconANGLE: Why OpenStack doesn’t need a vendor to lead it The young-but-growing OpenStack ecosystem consists in part of vendors who seem content to work together forever as one big happy, code-sharing family. But the reality is, they’re all lined up at the proverbial starting line, secretly jockeying for position in a race they hope will ultimately prove lucrative. Which vendor will emerge the leader? More importantly, does OpenStack, a series of community-based, open-source projects, even need a leader? Read more.