Apple Leather Case for iPhone 7

I have an on-again, off-again relationship with iPhone cases. I put them on. I take them off. I generally don’t like cases. I’ve only broken my iPhone one time and that was when my 6 Plus came out of my pocket attached to my hand, unintentionally, on a sticky day. My iPhone 5 and 6 were rarely in cases, and had minimal wear and tear. I’m usually pretty careful. I also buy AppleCare+ on them, even though I’m lucky enough to rarely need it.

Continue reading Apple Leather Case for iPhone 7

This is a post about my pants

I have some strict requirements around work pants. My wife hates the “I can see your socks while you’re standing up” hipster look, so they have to be full length. Honestly it’d be a great look since I’m 6’4” but as a result I’m at a 36” inseam. I’m also currently 220lbs, which results in a 36” waist. I could probably lose some weight, but it’s not happening today.

I also have a job that’s requires me to dress nicely to meet a customer in the morning, but be willing to crawl under raised floors and chuck 50# boxes around later that afternoon, without a change of clothes. Expensive slacks will get destroyed. Wearing jeans everyday is frowned upon. I also don’t want to deal with getting pants tailored.

Between size, cost, looks and durability, I’ve found one pair of pants that consistently meet all my requirements.

Continue reading This is a post about my pants

Security from obscurity

A couple of years ago, one of our network security architects at work told me that I was in the wrong business. Storage, virtualization, data centers, it’s all going to the cloud. I’d soon be out of a job. 

I barely knew the guy. At first I politely laughed when he said it, but then realized he was serious. Not really a great way to make new friends at work. The irony of the situation was that he tracked me down on one of the few times I was in the office, and approached me to help him lay out some of the VMware requirements for a Trend Micro Deep Security implementation. 

It wasn’t more than a few months later, that he didn’t work for my employer anymore … not by his choice … and I’m still there, two years later, still billable most of the week. 

I don’t even remember his name. 

But, he’s wasn’t wrong, just a jerk. It’s not as if he was delivering some sort of life changing message, that I’d never heard before. It’s one I hear repeated very often on social media, in conference presentations, etc, and in the wake of this Amazon re:Invent conference last week, I’m hearing it a lot. 

It’s undeniable that a big part of my job is chucking boxes of rust and silicon into racks, stringing copper and fiber optics around, and making it all sing together in unison. I kind of enjoy it.

It’s also undeniable that things are changing.

Migrate to VCSA

Last night I did my first customer migration from a Windows based vCenter to the VMware vCenter Server Appliance (VCSA) using the new 6.0 U2M utility.

The customer was previously running vCenter 5.1 GA on a Windows Server 2008 R2 based physical HP host. In order to migrate to the VCSA, we first had to do two in place upgrades of vCenter from 5.1 GA to 5.1 U3, then again from 5.1 U3 to 5.5 U3d. After that, onto the VCSA migration.

Given the length of time the system was running on 5.1 GA code (ouch) and the amount of step upgrades required just to get things cleaned up, there was some cause for nervousness.

I admit, even though I’d read up on it, tested it in a lab, and heard other success stories … I still expected my first try to be kind of a mess.

But, it was not. The entire migration process took around 30 minutes, and was nearly flawless.

I had more issues with the upgrade from 5.1 to 5.5 than anything else during this process. Somewhere during that 5.5 upgrade the main vCenter component quit communicating with the SSO and inventory service. There were no errors presented during the upgrade, but it resulted in not being able to login at all through the C# client, and numerous errors after eventually logging in as [email protected] to the Web Client.

I tried to run through the KB2093876 workarounds, but was not successful. I ended up needing to uninstall the vCenter Server component, remove the Microsoft ADAM feature from the server, and then reinstall vCenter connected to the previous SQL database. Success.

Given those issues, I was nervous about the migration running into further issues, mostly from the old vCenter.

But again, it worked as advertised.

After the migration I did notice the customer’s domain authentication wasn’t working using the integrated Active Directory computer account. After adjuting the identity provider to use LDAP, it worked fine. I’ve had this happen randomly enough on fresh VCSA installs to think its something to do with the customer environment, but I was under the wire to get things back up and felt there was no shame in LDAP.

I’ve done too many new deployments of the VCSA since 5.x to count, and at this point was already pretty well convinced there was no reason for most of my customers to deploy new Windows based vCenters. I’d also done a fair bit of forklift upgrades with old vCenters where we ditch everything to deploy a new VCSA, which isn’t elegant, but it works if for my smaller customers that still don’t yet have anything like View, vRA, SRM, integrated backups/replication, etc.

Now I’m confident that any existing vCenter can be successfully migrated.

Windows vCenters, physical and virtual: I’m coming for you.

Crashing ESXi with Cisco RAID controllers

Recently I had two VMware Horizon View proof of concept setups for work, where we designed an all in one Cisco UCS C240 M4 box, full of local SSD and spindles, in various RAID sets. This lets the customer kick the tires on View in a small setup to see if its a good fit for their environment, but on something more substantial than cribbing resources from the production environment.

  • 5x 300GB 10K SAS RAID 5 for Infrastructure VMs (vCenter, View Broker/Composer, etc)
  • 10x 300GB 10K SAS RAID 10 for VM View Linked Clones
  • 6x 240GB SSD RAID 5 for View Replicas
  • 1x hot spare for each drive type
  • VMware ESXi 6.0 U2 is installed on a FlexFlash SD pair

After getting all the basics configured, we had a single View connection broker, with another View Composer VM on a local SQL Express 2012 instance for the database. Both were version 7.0.2. At the first site the VM base image we attempted to deploy was an optimized Windows 7 x64 instance.

But under any sort of load during a deployment of more than a handful of desktops, the entire box would come to a total stop. In some cases the only way to restore any functionality was to pull the power and restart the infrastructure VMs, one by one. Of course, once the broker and composer instances were connected, they’d attempt to create more desktops and the cycle would continue. In an attempt to isolate the issue, we tried various versions of the VMware Tools, a new Windows 7 x86 image, and I even duplicated the behavior by building a nearly identical View 6.2.3 environment, within the same box.

After digging through the esxtop data as clones were being created, I could see KAVG/Latency across all RAID sets jumps to as high as 6000ms right before all disk activity on the system eventually stops.

It didn’t matter what configuration I tried, it was present with a fresh install of ESXi 6.0 U2, and after applying the latest host patches. It was present on the out of box UCS firmware of 2.0(10), and with the stock RAID drivers from the Cisco ISO. It was present after updating the firmware, and the drivers. It also happened regardless of if the RAID controller write back cache was enabled/disabled for the various groups.

Cisco is very particular about making ESXi drivers for their components match their UCS compatibility matrix, so before I decided to give TAC a call, I made sure (again) that everything matched exactly. TAC ended up reviewing the same logs, to determine if this was a hardware issue, and while they made a couple of suggestions for adjustments, they were not successful in diagnosing a root cause. Yet, they insisted based on what they were seeing that it was not a hardware issue.

With this particular customer, we were also impacted by a variety of issues relating to the health of the DNS and Active Directory environment. With that in mind, we decided to focus on fixing the other environmental issues and in the meantime, not overload the UCS box until a deeper analysis could be done.

Try Try Again

A day or so into the second setup at another customer, and I encountered the exact same issues. This time with a Windows 10 x64 image, and View 7.0.2. The same crazy latency numbers under any amount of significant load, until the entire box stopped responding.

The physical configuration differed slightly in that we were integrating the C-Series UCS into the customers fabric interconnects, so the firmware and driver versions were even more different than the first host which was a standalone configuration connected to the customer’s network. After digging into it again with a fresh brain, and more perspective, I found the cause.

I started looking through the RAID controller driver details again. In both cases, VMware uses the LSI_MR3 driver as the default driver for the Cisco 12G RAID (Avago) controller in ESXi 6.0 U2. In both environments I verified that we were running the suggested driver versions based on the Cisco UCS compatibility matrix, and we were. So I started digging at this controller and wondered what VMware suggests for VSAN (keeping in mind we aren’t running VSAN at either site) and sure enough, they DO NOT suggest using the LSI_MR3 driver, but instead list the “legacy” MEGARAID_SAS driver as their recommendation, for the exact same controller.

After applying the alternative driver, I’ve not been able to break the systems.

What is odd, is that this appears to be related specifically to Cisco’s version of the controllers.

This week I did a similar host setup (although not for View) using a bunch of local SSD/SAS drives in a Dell PowerEdge 730xd, with their 12G PERC H730 RAID cards (which from what I can see appear to be rebranded versions of the same controller) and VMware’s compatibility matrix has the LSI_MR3 drivers listed.

I left those drivers enabled, and the customer ran a series of agressive PostgreSQL benchmarks against the SSD sets, with impressive results, and no issues from the host.

So, long story short, if you’re using local RAID sets for anything other than some basic boot volumes that don’t need any serious I/O, with the Cisco 12G RAID controller, you don’t want to use the Cisco recommended drivers.

Installation instructions

  • Download the new driver (for ESXi 6.0 U2)
  • Extract the .vib file from the driver bundle and copy it to a datastore on the host
  • Enable SSH on the host and connect to it via your terminal application of choice
  • Apply the driver from the SSH session and disable the old one.
  • Reboot the host
  • Reconnect via SSH, and run core adapter list command to verify it’s active

This should verify that your RAID controller (typically either vmhba0 or vmhba1 is now using the megaraid_sas driver. If the “UID” is listed as “Unknown” in this readout, it’s normal.

Stronger Together

I wasn’t going to get sucked into the 2016 election, then Donald Trump ran for President I’ve struggled with what to write about this election for months. I’m not shy about my opinions on social media but when it came down to going long form about making my arguments in this election, I’ve written and deleted more than can be known.

I’ve always been deeper into politics than probably any of my friends and family. It started when I would watch Dana Carvy do his iconic impressions of both Bush and Perot on SNL during the 1992 election.

My brother and father are sports junkies. Politics is my drug of choice.

During the 2008 and 2012 campaigns, I was active on social media as a vocal supporter of Obama. It was hard not to be. I think he’ll go down as one of the most transformational figures of my lifetime.

But this year, it’s different. In past elections, I’ve said not great things about GOP candidates. My statements in hindsight, never expected Donald Trump to enter politics.

Mitt Romney is by all accounts an honorable man, and would have kept this country safe. I would have been frustrated but not ashamed before the world, and before my children, that he was our representative.

Now is the election of 2016, and there’s a chance that Donald Trump, could be our next President.

I repeat: there is a chance that this racist, fascist, sexist, disgusting excuse for a man, could be the next President of the United States.

I have two young boys. They know who we support in this election. My oldest son was more upset than I was when someone stole the Clinton/Kaine yard sign from our yard. In a year when it’s not popular to advertise that you endorse either candidate, it’s even more important to take a stand.

And, so, I built a bigger sign.

But in our house we teach our children to treat each other with love and respect. We teach our children to stand up the ideas and the rights of themselves, and for others.

As a straight, white, college-educated, male, in Kansas, I could easily sit back and hide. I could leave the signs down. I could be a registered independant. I could save my money. I could stay quiet.

I won’t.

I support Hillary Clinton for President.

And so should you.

So, vote. For our republic, for the rights and the protection of everyone. Vote. Against bigotry, against hate, against someone who can’t see that America is already great.

Vote.

16GB Problems

For many years, 16GB devices have been an issue for Apple and its users. However Apple fixed this in September, bumping up to 32GB of storage as the new minimum capacity in the iPhone 7, and then going as far as to rev-up the existing iPad line to this new minimum.

16GB problem, gone.

Yesterday, Apple announced a revamped MacBook Pro. Thinner, USB-C / ThunderBolt 3 all over the place, P3 display, Intel Skylake CPUs, and a new dedicated T1 chip powering a watchOS-enabled Touch Bar, including a Touch ID sensor in a non-iOS device for the first time.

Great stuff. One small problem.

Intel’s Skylake processors and chipsets that Apple is using are able to support more than 16GB of memory. For instance, the i7–6920HQ, which based on Apple’s advertised clock speeds looks to be what is utilized in the maxed-out 15” model, says it can do up to 64GB. Apple is a company that makes amazing products, that create and redefine entire catagories. But they’re they’re bound to some limitations.

I figured this wasn’t something that Apple did “just because” … if there was a way they could sell me more memory, at a premium, I’d think they’d do it. There’s a tradeoff being made here.

On the Intel page for the Skylake chips, it indicated that the maximum size was dependent on memory type. Apple is using LPDDR3 chips, which from what I can gather based on the 2133MHz speed, are either made by Micron or Samsung as they appear to be the only two vendors producing them, and based on past relationships makes a lot of sense.

In both cases, what I’ve been reading is that 16GB is the maximum size available in this class of chip. LPDDR3 has a lot of advantages when it comes to power consumption, running at a much lower voltage of 1.2V, and only using 10% of the power during standby compared to regular DDR3 or DDR4 memory.

Given that the people who’d really take advantage of the additional memory are people like me who want to run multiple virtual machines, containerized applications, etc, and these tasks are probably better suited to systems not running off batteries, a trade-off of limiting the maximums in order to reduce power consumption, makes a lot of sense.

If the decision here was something like “We can use LPDDR3 and get 10 hours of battery life but we’re limited to 16GB, or use DDR4 and get 8 hours, but support 32GB,” I’d rather get 10 hours of battery.

I’d probably use those two hours in the field a lot more than the extra RAM, right now.

All this appears to be backed up by Dan Frakes, who confirmed the limitation with Apple.

I was already going to sit out upgrading my Late-2013 Retina MacBook Pro 15” since it does everything I currently need, and has 16GB of DDR3 RAM already.

It also doesn’t require me to replace every cable, dongle, adapter and power brick that I currently own.

AT&T Mobile Share “Advantage”

You know you have a problem when you get excited about plan changes on your cellular provider. Yesterday, AT&T gave me a problem.

Initially, no more data overages, higher caps, and reduced pricing tiers looks like good news all around, but is that really true? After looking at the details of these new AT&T data plans I’m less than impressed. They’ve upped the per device access charge from $15 to $20.

Right now I have the $100 plan for 15GB, plus three devices, for a total of $145. Under the new plan, if I move to the similar the 16GB plan the base price is $90 but I’m now paying $60 in per device charges for a total of $150. (+$5)

Even going from the 15GB down to the new 10GB plan, would result in a savings of only $5, at the loss of 5GB of data.

Maybe.

It’s still really a bit confusing, the press release says “All Mobile Share Advantage plans also have an access charge of $10 — $40 a month per device not included in prices shown above.” but then later “customers will pay a $20 access charge per smartphone a month for Mobile Share Advantage.”

My hope/guess is that it’s likely to depend on which plan you pick, at least that’s how it is on the current setup. I believe the current $15 per device does jump to like $20 or $25 , on their current plans. So, if it continues to be a graduated scale, the new 16GB plan may actually be a money saver, but until their pricing calculator shows up when the new pricing is available on Sunday, we probably won’t know.

But from the “clear” statement, it looks like not a great deal. For now, they get a splashy headline. Verizon, who has a similar plan, has the separate fee for allowing “unlimited” reduced bandwidth, instead of charging an overage, and it seems like this increase is just a clever way of hiding that fee.

If I was really concerned about overages I’d probably just do it, but I never go over.