It's not about code

This post originally appeared as a Google-internal “Epitaph” – a leaving note for a parting employees.

Larry and Sergey’s 2004 letter to investors said “Google is not a conventional company. We do not intend to become one.” However, laying off 12,000 workers, despite having a $114B war chest, $60B in 2022 profit and growing revenue[1], was a conventional act of a company complying with minority investor demands and copying other companies, even down to their PR wording. 

I was one of the workers laid off. Over my 9 years at Google I have come to love the unconventional things that make Google great, but I have also seen how often Google takes an easier path with worse outcomes. I have realized that we get the amazing, unconventional version of Google  when Googlers put in active work to achieve it. I am writing this leaving note because I want to see Google continue to be an amazing place. Here are my thoughts on what Googlers must do to make Google a great place.

The layoffs are only one of many “conventional” company moves Google has made but I do not yet believe Google has become a completely conventional company. Despite reducing TGIF to a shell of its former self, Google still has a largely transparent internal culture. Despite donating to climate change deniers and partnering with Aramco, Google is net-zero carbon and is committed to carbon-free energy. Despite calling Raleigh-Durham expansion “part of our racial equity commitments” only to slash pay there, Googlers can have good work-life-balance and benefits. I’ve seen employees speak up about what is right and be listened to.

Critics often dismiss Google’s current cultural strengths as “inertia” from a golden past. And I agree that culture has inertia: People and their values, policies and programs are how decisions are made and once in place people and policies are slow to change. New hires adopt the environment they find themselves in, to a varying but generally significant degree.

But this “inertia” of the people that constitute Google does not have to be some flywheel slowly spinning down as Google slides into mediocrity. It can be a motor powered by brilliant people making wise choices. This motor can not only find new ways for Google to be remarkable, but it can shift the whole industry and world, as it has before.

So to Googlers reading this, if you want Google’s future to be the positive force it can and should be, you must act. If you do not, it will become the same as any other corporation. Follow these principles:

• Know your priorities. Google’s mission statement, “things we know to be true” and commitments are all excellent guides. Each Googler should also know their personal priorities at work, whether that’s working on your career, enabling your life outside the family, general Googler wellbeing or social justice.
• Make wise choices. Every decision is pushed in some direction by deadlines, politics, and GRAD/promo-potential (and for leadership, P&L and stock price). These are real factors that should be taken into consideration, but don’t allow them to lead your decision making. Make the choice that matches your priorities, even if it will be a hard path.
• Try out new ideas. I am not asking you to preserve what has made Google successful in the past; you must innovate what will make Google successful in the future. As well as its technical innovations Google’s excellence comes from cultural innovations like SRE, 20% time, peer bonuses, and even blameless postmortems in tech.
• Propagate Google culture to new hires. People do adapt when they arrive in new corporate cultures, to a surprising extent. However there is a depth in Google culture that requires active work to share.

GRAD represents a great example, even if there have been issues in the transition. It came from recognizing real issues, making a tough choice and being willing to try something new. My only message to People Ops is to avoid letting Google’s values fall to pressure to implement Amazon-style attrition or performance quotas.  

Conversely many Googlers are frustrated by product strategy: not enough continuing investment in existing products, new products launching with foreseeable issues and products being shut down losing massive of user trust. These decisions are led by profit, growth and careers and not Google’s stated values, repeating the same old failing patterns.

I’m not making a call to product leadership to personally adopt my guidance though. They are too constrained by expectations on them and their incentives to really have the option. There is a way to break out cycles where Google-wide decisions are made by a few individuals constrained to ignore Google’s values. Enough Googlers working together can build a stronger power than these constraints, with a democratic process allowing values to be at the core of decision making. This collaboration is what Alphabet Workers Union is. This is why for ICs the takeaway is to join Alphabet Workers Union to build a say in how Google operates. Maybe bringing unions to tech will be the next cultural innovation that makes Google remarkable and a credit to the world.

Finally I would like to close by saying that I have learned so much from my amazing colleagues at Google, I’m proud to have worked with you.

Stackdriver offers Uptime Checks for HTTPS. Sadly these checks don’t verify the certificate, so you won’t get an alert if a certificate expires.

A simple solution is to add a Google Cloud Function in between Stackdriver and your site. This allows the usual Stackdriver monitoring and alerting, plus a check in Google Cloud Function that the certificate is correct.

1. Create a new HTTP  triggered Google Cloud Function with the following code:

const https = require('https');
/** 
 * Checks for correct upstream HTTPS response.
 *
 * @param {!Object} req Cloud Function request context.
 * @param {!Object} res Cloud Function response context.
 */
exports.checkHTTPS = function checkHTTPS(req, res) {
  upstream_req = https.get("https://YOUR_SITE/", (upstream_res) => {
    res.status(upstream_res.statusCode).send();
  });
}

2. Create an Uptime check in Stackdriver, using the Cloud Function trigger URL instead of the target site.

That’s it! You now have a Uptime Check that will alert if there is a failure of the certificate to validate, as well as any other issues.

Playing out today is a debate about the control of future of technology. After revelations of massive government surveillance programs, new applications deploy strong encryption, touting privacy as a key feature. The authorities, afraid they will not be able to access communications, are attempting to mandate weaknesses in these applications to ensure they can gather data when needed. This debate hinges on how such backdoors would be used and whether they can be kept out of the hands of criminals. Pro-backdoor groups often point to real criminal investigations that would be helped by backdoors, however pro-security groups have struggled to point to the concrete harm of a backdoor.

The pro-security groups’ hand wringing is not necessary. There is a backdoored technology that we can easily use as a case study: cell phones. The lack of base-station authentication and availability of weak encryption has a similar effect to some of the weaknesses proposed to newer technology. “IMSI-catchers” use these weaknesses to pretend to be a cell phone base-station (i.e. a “tower”) so that cell phones connect to them instead of the real network. The IMSI-catcher can then do what it likes with a connect phone’s connection before forwarding it to the real network. This includes identifying what individuals are present in a location, recording call times and parties and even call content in some cases.

This interception that law enforcement agencies are currently uses closely mirror man-in-the-middle attacks possible on a text chat network that has it’s end-to-end encryption broken.

So what harm has come from providing law enforcement these powerful tools? The most obvious is that criminals are using the same weaknesses. While there is very weak data on what is being done with this power, there is evidence evidence that hijacking cell phone links is widespread. We should expect that if we weaken email, chat, cloud-backups and other online apps criminals will fined a way to use them too.

Damage can run deeper, however, undermining the justice system itself. During criminal prosecutions in the US the use of IMSI-catchers is being hidden from defendants. This hurts defendants’ rights as it limits their legal options. Even scarier, real criminals may have convictions overturned on the basis of the murky tactics used to convict them. Again, we should expect that weakened cryptography will lead to dodgy convictions and procedural acquittals of the guilty.

We have seen a second legal run-around result from stingrays: law enforcement agencies have been using pen-registers as authorization to avoid the more stringent requirements of a warrant, which judges say should be required. This has resulted in several incidents that indicate bulk surveillance of protests, which could never get a warrant. Clearly we should expect that if our lives are easier to be watched, then they will be watched more.

Criminal, procedural and civil rights issues like we have seen as a result of weak security in cell phone technology is a taste of the future – if we allow backdoors in technology. Only with ubiquitous encryption can we preserve the integrity of justice, protecting the public from criminals and unconstitutional government surveillance alike.

When using a command line interface, I like to see all the output a program makes. The standard out and error are already dumped to my console, but there is one piece of information usually hidden from the user: the return code.

Add the following to your .bashrc to get a nice notification if a program returns a non-zero code:

export PROMPT_COMMAND='ret=$?; if [ $ret -ne 0 ] ; then echo -e "returned \033[01;31m$ret\033[00;00m"; fi'

It looks like this:

user@host$ false
returned 1
user@host$

The NSA recently declassified their Cryptolog internal magazine. I would like to share one gem from Cryptolog XIV, No. 1 – 1st Issue 1987:

“NSA” PUZZLE

The occurrence of “NSA” as a letter sequence in English words is rather uncommon. Some of the rare examples are show below. Fill in the blanks to reconstruct the “NSA” words defined.

Crazy	_ n s a _ _
“Land of Opportunity”	_ _ _ _ n s a _
Lacking sodium chloride	_ n s a _ _ _ _
Pay for damages	_ _ _ _ _ n s a _ _
Miniature tree	_ _ n s a _
Unable to be satisfied	_ n s a _ _ _ _ _ _
Across the ocean	_ _ _ n s a _ _ _ _ _ _ _
Middle eastern inn	_ _ _ _ _ _ n s a _ _
Unhealthful (as a climate)	_ n s a _ _ _ _ _ _ _ _
Not spoken	_ n s a _ _
Search thoroughly	_ _ n s a _ _
Deny or contradict	_ _ _ n s a _
Not paid regularly	_ n s a _ _ _ _ _ _
Relationship by descent from a common ancestor	_ _ n s a _ _ _ _ _ _ _ _
Unclean	_ n s a _ _ _ _ _ _
Result of a bank visit	_ _ _ n s a _ _ _ _ _
Disagreeable or disgusting	_ n s a _ _ _ _
Type of roof	_ _ n s a _ _
Sail on a square-rigged vessel	_ _ _ n s a _ _
Able to combine with other substances	_ n s a _ _ _ _ _ _ _

1st Issue 1987 * CRYPTOLOG * page 29
FOR OFFICIAL USE ONLY

xkcd predicted that “The Ablogalypse” would occur in October 2012, it actually occurred in November. Pretty neat prediction though!

Internet URLs, file system paths and IP addresses usually get more specific as you move towards the right hand side. This is a clearly a byproduct of the left-to-right language of the designers of these systems. So why is it that this website is david.newgas.net and not net.newgas.david?

The familiar hierarchical naming is part of the “Domain Name System” created in the early 80’s by early internet engineers. They already had a internet naming system with cryptic, non hierarchical names like BBN-TENEXD and MIT-Multics. This was basically maintained in one giant text file managed by one organisation.

Faced with a network growing at ever increasing pace this system was becoming a nightmare to maintain. A naming system that followed administrative hierarchies seemed obvious, and was spelt out in RFC 822.

At this stage the right-to-left notation was already settled so to answer our question we must go further back to RFC 805, with the unhelpful title “Computer Mail Meeting Notes”. Here they discuss exactly how to add the extra hierarchical info.

One of their biggest concerns was email addresses.  These were already of the form user@host and they wanted something that extended this naturally. Here were some options they may have thought of:

Cerf@ISI

→

Cerf@ARPA.ISI ARPA.CERF@ISI Cerf@ISI.ARPA

It’s fairly easy to see why they picked Cerf@ISI.ARPA – the information consistently gets less specific as you go along. The other options are more jumbled. Quoting the RFC makes me sound foolish for even thinking the addresses are backwards:

… the [email address] would read (left to right) from the most specific to the most general.

So there you have it – domains have the most specific part on the left hand side to make email addresses look good!