L1TF

VMware has published new security advisories, knowledge base articles, updates and tools in response to newly disclosed speculative-execution vulnerabilities on Intel CPUs — collectively as “L1 Terminal Fault” — that can occur on Intel processors made from 2009 to 2018.

I’m going to outline our response to this issue, and make an attempt to summarize this complex event as best as I can. I would highly suggest reading through the linked articles as they’ll be more extensive and evolving.

Because this is complex, and evolving, to properly respond to these issues, consider KB55636 as the centralized source of truth from VMware.

Like the previously known Meltdown, Rogue System Register Read, and “Lazy FP state restore” vulnerabilities, the “L1 Terminal Fault” vulnerability can be exploited when affected Intel microprocessors speculate beyond an unpermitted data access.

L1TF – VMM (CVE-2018-3646VMSA-2018-0020)

This is the specific L1TF issue that affects the vSphere/ESXi hypervisor. It has two known attack vectors, both of which need to be mitigated. The first attack vector is mitigated through patches for both vCenter and ESXi.

The second attack vector is mitigated by enabling a new advanced configuration option (hyperthreadingMitigation) included in the updates. However, this advanced configuration option may have a performance impact so we have not be enabled it by default. This will limit operational risk by giving you time to analyze the effects prior to enabling.

There are new updates to both vCenter and ESXi that deliver the mitigation to L1TF:

  • vCenter 6.7.0d, 6.5u2c, 6.0u3h, and 5.5u3j
  • ESXi670-20180840x, ESXi650-20180840x, ESXi600-20180840x, and ESXi550-20180840x

There are also new versions of VMware Workstation (14.1.3) and Fusion (10.1.3) which address this issue.

L1TF – OS (CVE-2018-3620)

This is a local privilege escalation which requires base operating system updates for mitigation. Patches are pending for affected VMware appliances. Make sure you contact your operating system vendor(s) (Microsoft, Oracle, Red Hat, etc) for mitigation instructions in guest virtual machines as well.

L1TF – SGX (CVE-2018-3615)

This does not affect VMware products.

VCIX

I’m pleased to announce that yesterday I passed the VCAP6.5-DCD exam, thus earning the VMware Certified Implementation Engineer – Data Center Virtulization “milestone” after elevating the VCAP5-DCA exam that I earned back in 2014.

The DCD exam has been on my list of things to do since not long after I did the DCA. My first attempt was during the beta cycle for the 6.0 exam. The results for that exam took so long to be returned, and after shifting in job roles since then, I’d not had an oppertunity to sit for it until now. The 6.5 version of the exam differs from the 6.0 in that there are no longer the “Visio” style questions, which I think were problematic for the exam from the beginning. There are 60 questions consisting of multiple-choice, drag-n-drop, and multi-select questions, with 140 minutes to complete the exam. I was able to complete the exam in just under 90 minutes, and I didn’t feel like I was rushing.

In terms of advice I can pass on to others who are interested in taking this exam, make sure that you understand:

  • AMPRS (Availability, Manageability, Performance, Recoverability and Security)
  • RCAR (Requirement, Constraint, Assumption and Risk)
  • The difference between Functional and Non-Functional requirements

If you are hands on with vSphere 6.5, especially working with vCenter HA, PSC/SSO and cluster design, you should have all of the bases covered. I have been removed from much of that in the day-to-day for the last year or two, so that was probably the more challenging part of the exam for me. I think if I’d done more to read up on differences between 5.x/6.0 and 6.5, I’d have come back with a better score. But, pass is pass.

Shine

Once upon a time there was a meeting of minds,
The sun and the moon made a deal with the sky,
One would take the morning and the other the night,
Together they would blanket the world with light,
But the moon had a shadow, he felt like a liar,
The sun was the only one who carried the fire,
The sun saw this, she kept on glowing,
Bound to the moon, never saying, “you owe me”
She said “I’ll shine on you.” Jason Mraz

Who will you shine on today?

VMware TAM

I have accepted a job with VMware, as a Technical Account Manager (TAM).

To say I’m excited about this would be a gross understatement. VMware has been the company I’ve spent the majority of my technical focus on up to this point, and since announcing this change on Twitter last week I’ve been thrilled with the replies like “I’ve been here 4 years and it’s an amazing place to work.” During the interview process, one of the current TAM’s told me point blank: “This is the best job I’ve had in my career.” All of this has maintained a level of anticipation about this career change that I’ve not had for any other.

It’s not as if this is a surprise because I interact with so many great people on a regular basis who work for VMware, who seem to genuinely love the work they’re doing. But it’s been refreshing to get the same messages from people I’d never even met before. I’ve never worked for a vendor before, and reguardless of the company I had reservations before going this route. Would I lose my independent voice? I’ve had opportunities to make the switch arise before, but didn’t always see myself as a fit because the product itself didn’t engage me in any meaningful way.

The last year has been a rebuilding year, for me. In early 2017, I left my role as an data center engineer at a Value Added Reseller, to go back into a customer role. I had been working as a consultant for nearly six years, but prior to that I spent seven years on the customer side. So now I was back working 9-5, at the same desk. It was tough because I loved consulting, and I literally couldn’t wait to get back, but for various reasons I needed the transition. The role I took was intentionally outside my comfort zone, to force myself to do something different and pickup new skills. It was challenging in ways both expected and unexpected. The team I was working on has some great people, and it has been a fun to work with them, even if all the while I knew this wasn’t the place I wanted to stay at for very long.

This year in transition was a change that I needed, being a customer was a place to lay low, reset, and figure out my future and my priorities. There was no travel and no on-call, not even an expectation to even have email on my phone, let alone respond after hours to it.

But now I’m back, and ready to get to work doing what I love, for the company that I’ve spent the last decade focusing on, in the company of all the great people who’ve helped me get to this point.

The value of certification — For the love of the blueberry shirt

Occasionally I’ll wear my “blueberry” VMware certification shirt to work. Some people in the community love these shirts, some people don’t. I, do.

Blue also happens to be my favorite color.

Occasionally someone I work with in my /current/ workplace will comment on it. Before the last year, it was a bit of personal marketing while working as a VAR engineer. When I’d show up on site maybe there was a bit of “you can trust me because hey look it says right here I’m not some rando off the street.” In my current role, it’s not always obvious that I’m engaged in the VMware ecosystem. Since the shirt is, very blue, it gathers comments that range from “oh I didn’t know you were a…” to genuine curiosity of “what does that mean?”

Occasionally though, someone makes the less than flattering comment: “you know no one here cares about certifications, right?”

My usual response? “I do.”

In the moment I might get a little defensive and mention the number of hours required to sit for multiple VCAP exams, the underlying VCP exams, between training classes, time spent doing self guided learning or the process and stress of the actual exam.

The cost of the training, both in currency and time, is sometimes carried by the owner or sometimes their employer. I’ve been fortunate enough in my recent career to have had an employer that would make those investments on our behalf. It wasn’t always that way. Despite being deeply engaged with VMware products since 2007, it took until 2011 to obtain my first VCP. The financial hit for the required class was too much for me to take on at the time.

That VCP was my first industry certification of any kind.

I’m acutely aware that certification doesn’t mean you’re an expert, or that there are plenty of folks running around with certificates for things they have no practical experience with. That’s one reason why I’m such an advocate, and so proud of obtaining two practical/administration VCAP certificates. You can’t just memorize a test dump to walk in and regurgitate against multiple choice questions. You have to demonstrate your competency in a -slow- live environment.

So it’s fine that “nobody” in your organization cares about certifications. They have a value, if sometimes only to the holder.

In the wake of the last comment I got at work, I ordered two new blueberry VCAP shirts. My old one was getting a little rough looking. They’ll come in handy, especially in my next role.

In re, doorbell tweets

I received a lot of feedback from my tweet about ditching a new Ring for Nest Hello.

Rather than tweetstorm it up, I’ll try and summarize it all here as to why I’m switching.

Most of it boils down to already owning a fair amount of devices in the Nest ecosystem (2nd-gen Thermostat, 3 Dropcams, 3 Protects) and wanting to stay in that. All my smart home gear is split between HomeKit and Nest. Since Ring doesn’t play in either of those ecosystems, it’s yet another platform to manage, and especially since Nest and HomeKit have zero integration without flaky hacks (Homebridge) adding a third platform that talks to neither, was already a step backwards.

I’ve toyed with the idea of replacing the Nest equipment, over time, but if I do it’ll have to be into HomeKit compatible devices. Ecobee has a great thermostat alternative, but as it is, Nest makes some of the best cameras, and there’s not an alternative to the Protect that I’m interested in right now. There is a First Alert competitor that looks interesting but I’ve had bad experiences with false alarms from standard First Alert detectors recently, so my trust in them is broken.

I was already planning to buy the Hello after we moved into our new house last month, but when Costco ran a promotion on the Ring 2 that included a year of monitoring and an extra Chime at a significant discount, I couldn’t resist trying it out.

The Ring is functionally fine. One of my biggest gripes however is the recording isn’t always on, and when it is triggered by an event it’s for a limited period of time. Most recently I noticed this when our new neighbors came to the door to introduce themselves. Being bad at remembering names, I went back to the video tape only to find it cut off after about 20 seconds. I still don’t know their names.

Since it’s not always on, and it’s in sunlight most late afternoons, after activating the first few seconds are over-exposed and worthless.

The benefits of Nest for me do come at a price. The cost of the Ring 2 package was about $50 less than the Hello, and the monitoring for Hello would run another $60 for the year.

One other consideration is that the Hello just looks nicer, in my opinion. The Ring isn’t ugly, it’s just kind of meh looking, to me. I admit to also having a sour impression to the quality of the hardware, having already swapped out faulty Ring for family members, as well as having some come DOA.