Showing all posts tagged bladelogic:

What if…

While it may seem obvious to those of us who have been around this market for a while, it was interesting to read that at the recent Puppetconf 2016 event, Puppet still felt the need to state that "In the future code is going to be managed and deployed by other code". If you’re surprised that this sentiment still needs to be articulated explicitly in 2016, you have not been paying attention.

It is certainly true that the leading edge is all "cattle, not pets" and "automate all the things", but there’s a pretty long tail behind that head. Only 2% of workloads are currently running in "the cloud" - although the precise definition is complicated by that nebulous term. Everything else? Still running on premises, or at best in a colo.

The same goes for automation: for every fully-automated containerised full-stack deployment, there are fifty that are not automated.

Nevertheless, Puppet has built a $100M business on automation. I know a bit about this space, having worked at BladeLogic, one of the pioneers of automation. While BladeLogic and Puppet have a history, today I am wondering about whether things might have gone differently.

Luke Kanies, the founder of Puppet, was a BladeLogic employee, although he left before I ever joined. From what I gather, he was a proponent of extending BladeLogic’s foundation in Network Shell, or NSH, into a free open-source platform, on which a commercial product could be built. Instead, BladeLogic’s management preferred to shut down the open-source NSH project and just use the technology inside the commercial BladeLogic product.

For those in the know, NSH was a fantastic tool. Basically it was a shell based on ZSH, but with network awareness on top. What this meant was that you could do things like this:

 host $ cp /etc/hosts //host1/etc/hosts

 host $ cd //host2/home

 host2 $ ps -ef | grep inetd

 host2 $ diff //host3/etc/passwd //host4/etc/passwd

 host2 $ iostat 2 5

 host2 $ vi //nthost/c/AUTOEXEC.BAT

 host2 $ nexec nthost reboot Let's reboot NT

You could copy files between systems, compare them or even edit them in place, and generally do all sorts of good things - including developing scripts to automate those tasks. For me at least, this was the first hint of the new world in which systems are no longer managed one by one, with admins ssh’ing into them individually, but in bulk, deploying a single config to many systems in one action. Best of all, it was multi-platform, abstracting the differences between different UNIX variants, and even working on Windows. ZSH on NT? That’s a major selling point right there!

However, even among BladeLogic employees and users, the interactive mode of NSH was a well-kept secret, with most people working exclusively within the BladeLogic GUI. What might the combination of NSH and BladeLogic have become if it had been allowed to flourish? Could a free NSH have taken the place in sysadmin’s hearts that is currently occupied by Puppet? Would this have prevented the long, quiet death of BladeLogic?

20/20 Hindsight

Of course hindsight is a wonderful thing, and what is a fairly uncontroversial strategy to propose in 2016 was not so obvious fifteen years ago. Back then, there were vanishingly few successful hybrid business models that combined an open-source platform with a commercial component. It would not be fair to criticise BladeLogic’s management at the time for not taking that route - especially since they were outstandingly successful with the strategy that they did choose. The hybrid model would have been a major strategic choice, and there is no guarantee that VCs and other investors would have gone along with it.

I just wonder sometimes - what might have been, in a world where a free download of NSH would have gained mindshare in the data center, at the same time that high-powered, PTC-trained sales people were gaining the trust of the C-suite?

Today, in 2016, Robert Stroud, a Forrester analyst at the Puppet event, is saying the following:

Businesses services now involve infrastructure, middleware, and applications, said Stroud. "Moving forward, to be a complete automation environment, the successful player in the space will have a role in all three," he said.

At BladeLogic, we were saying that ten years ago. Regardless of the commercials, this market of automated server configuration management is arguably ten years behind where it should be. Sure, we can deploy things at scale, but managing them at scale is still a challenge - although the challenge is as much one of process as of tools. The cloud has enabled all sorts of new businesses and even entire new business models, but it is still constrained by the complexity and consequent fragility of the underlying infrastructure.

What might be possible if we had solved that problem ten years ago? What new possibilities might have been enabled, that we will only find out about years from now?

As one chapter ends, another begins

I haven’t blogged in ages - which is a good thing, I hasten to add! It’s just that I have been drinking from the firehose at my new gig. It’s now more than a month since I started at Moogsoft, and I think I can begin to talk about what it all means.

I joined Moogsoft from BMC, but it’s important to note that I did not join BMC, I wound up there as part of the BladeLogic acquisition. BladeLogic was my first startup, and it was a huge amount of fun, a great learning experience, and probably my period of fastest professional development to date. Before BladeLogic I was at Mercury, but I quit to join BladeLogic, due in no small part to the acquisition by HP1.

What is BladeLogic?

Both BladeLogic2 Operations Manager (BLOM) and Incident.MOOG are innovative products in their place and time. BladeLogic, together with Opsware, redefined what server configuration management meant, and both companies went on to be acquired by larger “Big 4" IT vendors: Opsware by HP, and a year or so later, BladeLogic by BMC.

For a while both products thrived in their new environment, but in the last few years, both have been flagging. There are many reasons for this, from internal politics at both BMC and HP acting as distraction, to the rise of open-source configuration management tools such as Chef and Puppet. However, I wonder if those tools were simply the end of an era.

This is a known pattern: technologies reach their peak right before they get displaced by their successor technologies. The speed record for steam engines was set in 1938, but a diesel engine had already exceeded that speed in 1936, and by the 1950s diesel locomotives were well on track to replace steam traction3.

This pattern even agrees with disruption theory: investment continues in the old technology, but it simply becomes overly complex and uneconomical compared to simpler, (initially) low-end competitors.

Pets vs Cattle

This disruption is exactly what I see happening now. The BladeLogic model of painstaking management of single servers is still relevant, but mainly for legacy or specialised systems. Much new-build development is already moving to disposable or even stateless VMs or containers, according to the classic “pets vs cattle" model.

servers_pets_or_cattle.jpg

In this brave new world, there is very little need to worry about the configuration of your “cattle" containers. Something like BladeLogic is arguably overkill, and users should instead focus on their provisioning process.

Of course it’s not quite as simple as that. Cloud zealots have been talking about replaceable short-lived cloud servers for a while, but it hasn’t really happened outside of some rather specific situations. The lifetime of VMs in the cloud has often ended up being much longer than this model would suggest, meaning that there is plenty of time for their configurations to drift and to require management in place. Part of the reason for this is that management processes and techniques that are still based on the paradigm of a persistent physical server. Much of this Weltanschauung has been adopted wholesale for virtual and cloud-based servers without much reconsideration.

There is also the topic of security and policy compliance to be considered. Given long system lifetimes, it is not sufficient to be able to validate that something was deployed correctly, as its configuration may have drifted away from that known-good state. The desired state may also change as vendors release updates and patches, or new security vulnerabilities are disclosed. In all of these cases, some mechanism is needed to check for differences between the current live configuration and the required configuration, and to bring the system into compliance with that desired state.

However, this is now. As Docker and other container-centric infrastructure technologies become more prevalent, and as business functions continue to migrate from legacy to new-build applications, I would expect that that paradigm will evolve to replaceable plug&play infrastructure components, and do so everywhere, not just at the “unicorn" companies.

What does it all mean?

Lots of smart people are working hard to enable infrastructure to be managed as code. One of the characteristics of code is that you don’t change it in production, you develop it offline, then release it and don’t change it until you overwrite with a new version. The big variables that I think will affect the speed of the transition to this new model are firstly, the rate of replacement of legacy applications, and secondly, the evolution of IT management processes and culture to take advantage of new tools.

BladeLogic itself has the opportunity to evolve to have a role in the new model, of course. Regardless, BladeLogic was a huge part of my career development - and just huge fun, if I’m honest - so I will be watching development of the IT infrastructure management market intently, but no longer from the front lines.


  1. I’d say my fears on that score have been amply borne out. 

  2. The Wikipedia entry for BladeLogic now redirects to BMC, which is not especially helpful. 

  3. Sorry - not sorry. 

Spring is a time for new beginnings

(╯°□°)╯︵ ┻━┻

After 7 years, it's time to move on. Today is my last day at BMC, then I get a whole one day off - and even that only because it's a public holiday - before I start my next gig.

Today I hand in my badge and gun - er, I mean, MacBook - and on Monday morning, bright and early, I will be in San Francisco to start my new job at Moogsoft. I could not be more excited!

It's definitely a wrench to leave after so many years, and so many different roles. In particular, I have to admit that I will miss the view from BMC's Milan offices, with the Alps in the background:

Onwards to new adventures!

The Bigger Picture

## Or, Why BladeLogic Isn’t Puppet

***Disclaimer:*** *In case you don’t know already, I work for BMC Software, having joined with the acquisition of BladeLogic. My current role is in marketing for BMC’s cloud computing and data center automation products, including BladeLogic. In other words, if you are looking for a 100% objective take, look elsewhere. The reason this piece is here rather than on a bmc.com site is to make it clear that this is personal opinion, not official corporate communication.*

At the turn of the last century, life as a sysadmin was nasty and brutish. Your choice was either to spend a lot of time doing the same few things by hand, generally at the command line, or to spend a lot of time writing scripts to automate those tasks, and then even more time maintaining the scripts, all the while being interrupted to do those same things.

Then along came two automation frameworks, Opsware and BladeLogic, that promised to make everything better. The two tools had somewhat similar origin stories: both were based around cores that came out of the hosting world, and both then built substantial frameworks around those cores. In the end both were bought by larger players, Opsware by HP, and BladeLogic by BMC.

BladeLogic and Opsware both let sysadmins automate common tasks without scripting, execute actions against multiple servers at once, and assemble sub-tasks together to do full-stack provisioning. There were (and are) any number of differences in approach, many of which can still be traced back to the very beginnings of the two companies, but it’s safe to say that they are the two most similar tools in today’s automation marketplace.

Yes, *marketplace*. Because in the last decade or so a huge number of automation tools have emerged (or matured to the point of usefulness), mainly in the open-source arena. If you are managing tons of OSS Linux boxes, running an OSS application stack, it makes sense to have an OSS config management tool as well.

So far, so good. For a while the OSS tools flew under the radar of the big vendors, since the sorts of people willing to download a free tool and hack Ruby to do anything with it tended not to be the same sorts of people with the six- or seven-figure budgets for the big-vendor tools. As is the way of such things, though, the two markets started to overlap, and people started to ask why one tool was free and the other was expensive. This all came to a head when Puppet Labs published a document entitled ["Puppet Enterprise vs BMC BladeLogic"](https://puppetlabs.com/sites/default/files/Puppet_Enterprise_vs_BladeLogic.pdf). Matthew Zito from BMC responded with [An Open Letter to PuppetLabs](https://communities.bmc.com/community/bmcdn/bmc_service_automation/server_configuration_automation_bladelogic/blog/2014/04/28/an-open-letter-to-puppetlabs) on BMC’s site, which led to an exchange on Twitter, storified [here](https://storify.com/dwellington/puppet-vs-bladelogic "Puppet vs BladeLogic · dwellington · Storify" ).

### This is the longer response I promised Luke Kanies.

When I was at BladeLogic, I was in pre-sales, and one of my jobs (on top of the usual things like demos, proof-of-concept activities, and RfI/RfP responses) was to help the sales people assemble the business case for buying BladeLogic. This usually meant identifying a particular activity, measuring how much time and effort it took to complete without BladeLogic, and then proposing savings through the use of BladeLogic. Because *for some unknown reason* prospective customers don’t take vendors’ word for these sorts of claims, we would then arrange to prove our estimates on some mutually-agreed subset of the measured activities.

We would typically begin by talking to the sysadmins and people in related teams. They had generally spent a lot of time scripting, automating and streamlining, and were eager to tell us that there was no reason to buy what we were selling, because there were no further savings to be made. Any requests could be delivered in minutes.

The interesting thing is, we would then go around the corner to the users and ask them how long they typically had to wait for requested services to be delivered. The answers varied, but generally in the range of, not minutes, but two to eight *weeks*.
### *Weeks*, not minutes

Where is that huge discrepancy coming from? Because, depressingly, it’s still the same today, a decade later.

The delay experienced by users is not caused by the fact that sysadmins are frantically flailing away at the keyboard for a full month to deliver something. Credit us sysadmins (for I am still one at heart) with a bit more sense than that. No, the problem is that there are many many different people, functions and teams involved in delivering something to users. Even if each individual *step* is automated and streamlined and standardised to within epsilon of perfection, the overall *process* is delayed by the hand-offs between the steps, and even more so when the hand-off isn’t clean and something needs to be reworked, or worse, discussed and argued about.

**That** is the difference between Puppet and BladeLogic. Puppet is trying to address one - or, in all fairness, *several* - of those steps, but BladeLogic is trying to address the entire *process*.

In a wider sense, this is what BMC is trying to do for all of enterprise IT. “Consumerisation of IT" has become a cliché, but it’s true that in the same way that Puppet has moved from a hobbyist market to the IT mainstream, Dropbox has moved from a home user market to a corporate one, AWS has eaten the cloud, and so on. We are living in a Cambrian explosion of tools and services.

Enterprise IT departments and the vendors that serve them cannot compete with these tools, and nor should they. The models of the new entrants give them economies - of scale, of attention, of access - that the traditional model cannot touch. The role for enterprise IT is to provide *governance* across the top of this extremely diverse and rapidly evolving ecosystem, and fill in the gaps between those tools so that we deliver the correct end product on time, on spec and on budget[^1].

Sure, use Puppet - and Chef, and Ansible, and SaltStack, and CFEngine, and your home-grown scripts, and maybe even BladeLogic’s BLpackages. Just make sure that you are using them in a way that makes sense, and that meets the users’ needs. At the end of the day, that’s what we are all here for.

[^1]: Yes, all three. The point of automation is to resolve that dilemma.