I just got on the Pi-hole bandwagon a few weeks ago, and boy do I love it. Really, who doesn't love DNS? And what is better than a Pi-hole? Two Pi-hole!

With the release of Pi-hole 5.0, I wanted to rig up a quick and dirty way to accomplish keeping my Pi-hole HA instances in-sync, but it has quickly esclated to more than just dirty and has now become a little more elaborate.

Originally I posted the installation documentation on this blog, but as it gained more brain time, I have moved those over to the README file of the GitHub repo where the script now lives.

vmstan/gravity-sync

What started as a quick and dirty way to accomplish keeping multiple Pihole 5.0 instances in-sync, has now become a little more elaborate. - vmstan/gravity-sync

GitHubvmstan

The script assumes you have one "primary" PH as the place you make all your configuration changes through the Web UI, doing things such as; manual whitelisting, adding blocklists, device/group management, and other list settings. The script will pull the configuration of the primary PH to the secondary.

After the script executes it will copy the gravity.db from the master to any secondary nodes you configure it to run on. In theory they should be “exact” replicas every 30 minutes (default timing of the cronjob).

If you ever make any blocklist changes to the secondary Pihole it’ll just overwrite them when the syncronization script kicks off. However, it should not overwrite any settings specific to the configuration of the secondary Pihole such as upstream resolvers, networking, query log, admin passwords, etc. Only the "Gravity" settings that FTLDNS (Pihole) uses to determine what to block and what to allow are carried over.

Generally speaking I don't forsee any issues with this unless your master Pihole is down for an extended period of time, in which you can specify that you'd like to "push" the configuration from the secondary back over to the primary node. Only do this if you make list changes during a failed over and want them back in production.

Disclaimer: I've tested this on two Piholes running 5.0 GA in HA configuration, not as DHCP servers, on my network. Your mileage may vary. It shouldn't do anything nasty but if it blows up your Pihole, sorry.

The actual method of overwriting is what the Pihole developers have suggested doing over at /r/pihole, and apparently is safe 🤞 It might be a little more aggressive than it needs to be about running every 30 minutes (defined by the crontab setting) but I figure the way I have mine setup the second one isn’t really doing anything other than watching for the HA address to failover, so it shouldn’t disrupt users during the reload. Plus, the database itself isn't that big, and according to the Pihole team the database file isn’t locked unless you’re making a change to it on the master (it’s just reading) so there shouldn’t be any disruption to the primary to make a remote copy.

I want to note that the intial release (1.0) had no error handling or real logic other than executing exactly what it's told to do. If you set it up exactly it'll just work.

I've since posted 1.1 and higher with some additional arguments and features, if you deployed the script previously I suggest upgrading and adjusting your crontab to include the "pull" argument.

I've also moved the script to GitHub, which should allow you to keep an updated copy on your system more easily. The script can even update itself if you set it up for that.

Enjoy!

Greg Morris writes:

… I do suffer quite a lot with imposter syndrome. The great thing is, I have learnt not to check my blog stats, I’m not bothered about podcast downloads and I sure as hell don’t care how many people follow me on social media.

I too, have mastered the art of not checking blog stats, in part by not collecting them at all.

Yet every time I do stumble over the figures, I am always surprised because I don’t think I am interesting enough. … When I listen to other people on podcasts, and read others writing, they seem infinity more interesting than I think I am. With more to say on topics that I find really interesting. Does everyone feel like this?

Yes.

This video from Atlassian was shared internally at VMware a couple of weeks ago, and my initial comment was that minus the few references to their company specifically, this video was a great representation of the role of Technical Account Managers, generally.

I was apparently not the only one who thought this, a little while later a posting appeared from their corporate account on LinkedIn with positive comments from representatives of:

• VMware
• Zscaler
• LivePerson
• Adobe
• Five9
• Citrix
• Nutanix
• Symantec
• Microsoft

Plus a half dozen or so other places I'd never even heard of. If you can get that many representatives of different places to agree you're probably onto something.

Kudos to Atlassian.

Introducing VMware Project Nautilus:

Project Nautilus brings OCI container support to Fusion, enabling users to build, run and test apps for nearly any OS or cloud right from the comfort of your own Mac.

With Project Nautilus, Fusion now has the ability to run Containers as well as VMs. Pull images from a remote repository like Docker Hub or Harbor, and run them in specially isolated 'Pod VMs'.

This is built into the latest Tech Preview of VMware Fusion, which we've changed how we're releasing.

As Mike Roy explains in New Decade, New Approach to “Beta”:

This year, in an ongoing way, we’ll be releasing multiple updates to our Tech Preview branches, similar to how we update things in the main generally available branch. The first release is available now, and we’re calling it ’20H1′.

We’re also moving our documentation and other things over to GitHub. We’ll be continuing to add more to the org and repos there, maintain and curate it, as well as host code and code examples that we are able to open source.

I’ve already been playing with Project Nautilus, and It’s pretty slick. I had an nginx server up in a couple minutes after installing, even pulling the image down from the Docker Hub. Being able to spin up container workloads right on macOS, along side Fusion virtual machines, without the Docker runtime installed.

You can even run VMware Photon OS, as a container inside the PodVM.

Project Nautilus should eventually make it's way into VMware Workstation, but is not currently available.

You should also able to do the same thing on ESXi later this year with Project Pacific.

There are some things that just aren’t worth putting on your resume. This was the reminder that came to mind during replies to Owen Williams on the tweet machine.

For a very short time I worked for a small family business that sold woodworking tools. Everything from glue and chisels to large computer controlled "put wood in this side and get a cabinet out the other side" machines. I was recommended to the position by a friend who was leaving to work for an ISP. The job I had at the time was part-IT/part-retail for a small grocery store chain, and I wanted to go all in on IT.

But on what I remember to be my first (or maybe second) day I was asked by the President of the company to disable the accounts of two of his brothers who were VPs (the four boys ran it) — A few hours later one of them shows up at my desk trying to figure out why his email is locked, and doesn’t have a clue who I am.

This guy looked like he killed wild animals barehanded for fun, and at maybe 21 years old I’m a much scrawnier version of my current self at maybe 160lbs. What a joy it was to tell him to go talk to his brother and then have him return and demand that I reactivate it.

I left a couple of months later once I found something that was slightly more stable. The company is no longer in business.

The first time I used Veeam's backup software was in 2010. Up to that point I'd had experience with Symantec Backup Exec, Microsoft Data Protection Manager, and Commvault Simpana. The first time I used VBR to backup my vSphere infrastructure it was like the proverbial iced water to a man in hell.

As a consultant I'd deployed VBR for customers more times than I can count. Bringing iced water to the hot masses.

Today's news has me worried for their future:

Insight Partners is acquiring Veeam Software Group GmbH, a cloud-focused backup and disaster recovery services provider, in a deal valued at about $5 billion—one of the largest ever for the firm.

Veeam—first backed by Insight in 2013 with a minority investment—will move its headquarters to the U.S. from Baar, Switzerland, as a result of the acquisition. The deal is intended to help increase the company’s share of the American market.

Hopefully my worry is for nothing, but Insight Partners is a private equity firm. What does that mean, exactly, remains to be seen. But generally speaking:

• It restructures the acquired firm and attempts to resell at a higher value.
• Private equity makes extensive use of debt financing to purchase companies.

Also, as noted by Blocks & Files:

Co-founders Andrei Baronov and Ratmir Timashev will step down from the board. Baranov and Timashev founded Veeam in 2006 and took in no outside funding until the Insight $500m injection in January 2019.

I sincerely hope that I'm wrong in my gut reaction here, but wish the best of luck to all my friends at Veeam.

I don't know who Peter Drucker is, but Matt's quote attributed to him, is sound:

Apparently Peter is a kind of a big deal, at least according to Wikipedia:

Peter Ferdinand Drucker (/ˈdrʌkər/; German: [ˈdʀʊkɐ]; November 19, 1909 – November 11, 2005) was an Austrian-born American management consultant, educator, and author, whose writings contributed to the philosophical and practical foundations of the modern business corporation. He was also a leader in the development of management education, he invented the concept known as management by objectives and self-control, and he has been described as "the founder of modern management".

Even the best ideas will fall flat if the culture of the orginization refuses to adapt to service them. As I said last week:

The trick, I suppose, is knowing how much of the old ideas and processes are actually still required and why. ... In order to do that you need to understand more than just the business and the technical requirements. ... You have to understand the culture in which it will operate.

Idea: move everything to the cloud!
Culture: we must control every aspect of the infrastructure.

🤔

It turns out that finding something to write about every day is really hard. Shocking, I know. You may have noticed (or, maybe not) that January 1-4 there was a new post here every day. I skipped yesterday, but I blame my participation with this tweet from Jehad.

Not really, I knew I wasn’t going to keep up posting every day. I had a lot of free time on my hands, especially after New Years Day. Today was the first day back to work after being off since December 20. The first half of this time was spent participating in, and in preparation of, the various holiday celebrations our family was invited to.

Not having work things rolling around in my brain, having ample downtime, gives me a chance to reflect on life. Which in turn prompted me to write them down. Lucky you. Going forward I hope to get at least a couple of posts done every week, for my benefit if anything. Three would probably be a stretch goal.

On Privilege

I take this time period off every year, or at least try to. When I worked for the university starting, in 2006, we just had this time period off as the campus was completed closed. Students didn't come back until around MLK Day, so even after returning to campus it was eerily quite, but gave us a couple weeks to catch up and finish any small projects and prepare for the spring semester.

Even the VAR that I worked for, it was expected that only a skeleton crew would be staffing the company the week of Christmas, and it was built into our company PTO schedule that we be off week. It sort of set a trend that with the exception of a couple years before my children were born, I’ve tried to keep.

I realize that I’m in a very fortunate position because of the type of work that I do, who I’ve worked for, and especially who I currently work for, that I’m not someone working on Christmas Eve, and rushing back to the office on December 26. The same thing on Thanksgiving.

I’m incredibly privileged, even living and working among “classically privileged” individuals. Hearing friends and family over the holiday struggle with things like managing vacation days, lack of maternity leave, losing benefits, pay issues, etc, I often bite my tongue and don’t allow myself to reiterate how generous VMware is in many of these areas, for fear of being seen as a braggart.

Sometimes I even check myself when it comes to internal conversations about these topics, and remind myself that even the most generous and well intentioned efforts are usually faulted when you’re forced to deal with the US medical system.

My aunt did ask me on Christmas Day if I had to use PTO in order to be off for so long, and I was forced to explain that VMware doesn’t track PTO time. Also, that my manager doesn’t have intimate knowledge of my daily or weekly routine.

All of this combined usually blows people’s mind, but I try to stay grounded about it, while pretending it will last forever.

For the last couple weeks I’ve been confused why Microsoft Outlook on my Mac would start consuming over 100% CPU while sitting idle, spinning up my fans, and generating a bunch of disk write activity.

At first I assumed it was because I am running the Fast Ring in order to run the new macOS design. However, the same build on my wife’s Mac, also running Catalina, never came anywhere near that even during what could be described only as “aggressive emailing.”

After messing around with adding and deleting accounts, hoping another beta update would fix it ... I finally got the idea to just drag Outlook to the Trash, and let Hazel detect this and offer to dump all of the associated files (cache, settings, etc) with it.

After I put Outlook back in Applications, and effectively set it up as new, everything is back to normal. 0.4% CPU

Jessica Joy Kerr in her blog post titled “Open your eyes to the nonsense” has a great anecdote from a friend, about the software development process at a utility company:

“We make power, not sense.”

But she goes on to make a wider point about evaluating the existing culture and processes of institutions.

Culture doesn’t make sense, to anyone from outside. Culture is common sense, to anyone embedded in it. To understand and work with a large organization, let go of trying to make sense of it. Observe it and see what’s there. After that, logic might help in finding ways to work skillfully inside it, maybe even to change it.

This applies to organizations of any size and in every industry, although the nonsense obviously increases in complexity as they scale, as all things do.

Far too often people expect consultants or an "expert" to come in and tell them  how to make things perfect. In the past I'd only work with customers in a very narrow window, typically to implement one or maybe a handful of technologies. Best case I'd get repeat customers and learn more about their business requirements, and how they operate as an organization and how they make decisions. But more often than not I'd offer recommendations based on general experience and hope that there aren’t unforeseen consequences in the environment.

I’m also more interested in seeing what everyone in the room has to say about their requirements before I offer what could be an otherwise ignorant or unconsidered opinion. It's not that I'm necessarly afraid of being wrong, but different people from different backgrounds in different departments with different goals often see things... differently.

One of the things I particularly enjoy about my role as a VMware TAM, is that I get the runway to have these conversations and collect information with the customer, to advise over the long term as opposed to some two day "best practice review" and then run off to the next project.

Once in a while, companies can adopt technologies in greenfield environments where you can take everything about the vendor’s best practice and apply them. But more often than not, you have to find a way of blending the two. The trick, I suppose, is knowing how much of the old ideas and processes are actually still required and why. You rarely get that in an hourlong conversation. In order to do that you need to understand more than just the business and the technical requirements.

You have to understand the culture in which it will operate.

I've historically been pretty terrible at two things: negotiating salaries, and promoting to my own skills. As someone who freely admits the movie of his life story could very well be titled The Imposter Syndrome, when it came time to discuss with a recruiter or hiring manager what I was worth, I've been a slow learner.

In 2010, I worked for a private university making roughly $38k a year. Technically, it was $19.21 an hour, as my position wasn't considered elegible for salary, and they considered 38 hours a week to be full time. At this point in time I had a bit of experience running infrastructure/systems under my belt, but absolutely no concept of my true worth in terms of the industry. I had completed a project that gained a lot of notoriety and praise, case studies and awards, but I had no certifications and no college degree.

I started at the university doing desktop support in 2006, and moved up to a "network analyst" position a year after. My primary responsibilies included managing the dorm network, campus anti-virus system, backups and creating the images that would be deployed to new desktop systems. We adopted VMware virtualization at ESX 3.0 shortly after my promotion so I quickly picked up storage and networking, then databases, messaging, and all the rest as we were a very small team supporting 4,000 users. We couldn't afford to specialize too much, and in education everything is discounted, including the people.

Recently married, and thinking of having kids, I knew I could be doing better but had no solid concept of what I was worth. This was in the infancy of Twitter and the vCommunity, there wasn't the level of openness there is now around softskills, and pay. The only thing I knew is that wanted to specifically get a job with a software company based in the town where my wife and I had recently bought a home. One frustrating day at work I went to go check their careers page, found a listing for a System Administrator, and applied, then started a Staycation.

One early morning the next week my phone rang, it was a recruiter with said company. I was sleeping in because this was two years before my first child was born and I was on vacation, not expecting any life altering phone calls. "Is this a good time to talk?" Well, not really, but half-asleep I started answering pre-screen interview questions, when the dreaded one finally came: “How much are you looking to make?” I'd only thought briefly about it to this point.

“Fifty thousand a year.”

Now when you’re making $38K, this is a big jump. My wife, who was working at the time, had a masters degree in accounting and a CPA license – and even she wasn’t even doing that well. What I was asking for seemed like a lot of money, in my mind.

Over the next couple weeks I went through a few interviews. Finally one day as I was walking into lunch, the recruiter calls to offer me the job. The salary: $50,000.

I didn’t negotiate at all. I wasn't sure that I was in a position to even try. I wanted to work for this company as much or more than they wanted to employ me and when you don’t know your value it’s hard to think you can convince anyone else of it. In the moment, I was pretty excited, but as I was eating lunch I started to also wonder if I should be worried by how quickly they came back with exactly my original number? Should I have asked for more? Could I have got more?

What I didn’t realize at the time, is that I’d just helped sink the hopes of some folks in my new department looking for raises. What I’d asked for was indeed the going rate for admins there, perhaps even a little bit lower, so HR was happy to oblige my request. The other folks I'd soon sit next to had hoped it would take more to bring in someone with the relevant skill set, to justify them getting their own salaries adjusted. The topic of pay would come up frequently among members of my new team.

The software company was privately held, but during my interview process they announced they were being acquired. I decided to accept the offer reguardless, and a few weeks into the new role we officially became a subsidiary of Lexmark. Yes, that Lexmark, the purveyors of those fine ink jet printers you remember so fondly from 1999.

After a few months of mostly nothing changing, people from Lexington, Kentucky who were our peers started to show up and try to learn about our operations. After getting to know them a bit and discovering they're not all evil, we wanted to know pretty much one thing ... what their pay scale was.

Folks in our roles there made around $85k. That was maybe the first time I truly realized how off balance my own valuation was.

My previous boss had once encouraged me not to leave the university for anything less than $85k — more than double what I made at the time — and I honestly couldn’t tell if he was serious of full of shit. My pay inequity there had nothing to do with him, and everything to do with the nature of the university. If I’d pressed for more money there, I might have got some more, but nothing close to that number. Everyone there was (and probably still is) grossly underpaid.

Around this time our manager at the software company decided to leave, and one by one those of us on our team started getting interviews and offers to join him. By this point the pretense of not discussing salary has dropped between us, and so when the first offer letter anyone received came in we all reviewed it. To our surprise and excitement, this kicked off a bidding war between the two companies, with our current employer being the winner. My team lead got the next offer though, $80k with a $5k bump after certification. He was out.

Then came my turn, and after the interviews, the hiring manager calls to give me an offer of ... $67k.

Wait, what? This couldn't be right, why was I getting so much less? My wife straight up didn’t want me to take it, and I was about 50/50, but thought "well if the last guy went into a bidding war, now is my chance to use this as leverage to get my own counter offer." I didn't want to leave, so much as I just wanted what was starting to seem like a fair deal.

Apparently HR decided that I wasn’t worth it, there would be no counter offer. In hindsight there's really not a reason to blame them. I'd only been there a year and didn't do a lot of highlight myself or make it seem like I was so valuable that there was a cost associated with me leaving. They got me for the price they did not more than 12 months earlier, why would I be worth so much more now, just because I realized it? And in reality, I couldn't even get the market to pay it.

So, I left, for just a slightly larger bump than I got to come there.

The company I landed at did end up promoting me and steadily providing healthy salary increases over the six years I was there. I would finally hit the magic $85k target, in 2013.

I'm not one for making new years resolutions. They always get broken, and then you're disappointed in yourself. However, Brent Simmons posted something on Inessential which I think is a good goal for anyone this year:

My resolution is to try harder to get angry only when it’s actually worth it. I can be angry at cruelty, angry at the forces destroying democracy for their own corrupt power, angry at the malevolences driving our climate crisis. ... But I need to not get angry just because Instruments won’t profile my app, or I get a robocall, or someone on Twitter completely missed the point of something I wrote.

If I did have a resolution, it would be this coupled with the idea of acting with intentionality. Beyond any specific philosophical meaning to that word, I would use it to mean acting with intention, moving with a purpose, being present in the moment, and focusing in getting things done.

Maybe I do have a resolution.