Sure, I could just search the web to find the answer I need—after sifting through a dozen possibly conflicting, inaccurate, or irrelevant postings. Sometimes that works; sometimes it’s a disappointing waste of time.

One of the reasons we favor open source solutions is so that we don’t have to rely on forums or third party support to understand how things work. We can go right to the source and see what it’s doing. And if it’s not doing what it says it does, or working the way it should, we can fix it.

Going to the source dispels the hype that can otherwise surround a piece of software—or a news story. In this political and media climate, it’s especially important to make sure you’ve got the facts. When WikiLeaks says that the CIA compromised Signal and WhatsApp, it’s good for grabbing headlines, but as usual, the devil is in the details.

This week, go to the source—of your code, of your news, of whatever project or bit of information is in front of you—and use that knowledge to make it better.

The post Go to the Source appeared first on Software for Good.

That’s good basic advice. But this week I challenge you to go further: take up a regular exercise that stretches your capabilities. Stretching regularly increases your flexibility, so that you can reach further when you need to.

What’s true of the body is true of the mind, too. Writing software takes an extraordinary amount of mental flexibility. It’s helpful to do small, self-contained exercises that are just beyond your comfort zone. Lately I’ve been working through the exercises on exercism.io. There are many similar sites that will give you practice problems of varying difficulty (e.g. codekata.com or codewars.com).

And remember that after stretching, it’s equally important to cool down and relax. So stand up, stretch, stare blankly out a window, look at the scenery.

In what ways do you want to become more flexible?

The post Stretch appeared first on Software for Good.

We’ve been running pretty hard the past few weeks, and many of the events we’ve been working so hard to prepare for have now happened and are past. Now that we have the chance again, let’s catch our breath, take the time to evaluate how things went, and then look to what’s coming next.

We’ve done good work and more good things are in store. As we take the time for a moment’s rest and generous reflection on our accomplishments, it will give us the energy we need to meet the next challenge.

The post Regroup, Reflect, Redouble appeared first on Software for Good.

The thing is, what you practice is what you learn. Muscle memory forms at a lower level in your brain than evaluation of how you did. So this week, follow that urge that says you can do it just a little bit better next time. But go one step further: strive to identify, in specific terms, how to improve your skills, and then try again. Don’t be discouraged by repetition; every time you repeat a skill, you’re making it more solid and more automatic. But ensure that when you can do it without thinking, you’re also doing it right.

Now, we can’t be perfect all the time; mistakes and imperfections are key to our learning. Learn as much as you can from them. We also don’t always have the time to practice when an iteration needs to just get done. But you can always leave yourself a note to make something better as a follow-up, or to practice a few times on a stripped-down example.

By the way, this principle applies beyond your chosen craft. Another famous quote goes, “You must be the change you want to see in the world” (Gandhi). Enacting empathy and justice in small everyday interactions makes it easier to respond to larger issues with compassion. Practice also makes a more perfect world.

The post Perfect Practice Makes Perfect appeared first on Software for Good.

Working on a project? Consider its impact. Choose your tasks deliberately; pay attention to how your work is shaping up and its effect on you. If it’s not what you wanted or expected, don’t be afraid to pivot to a new way of doing things or a different activity that allows you to be more successful.

And if this week’s success turns out to be an accident, go ahead and pretend you did it on purpose.

The post Do It On Purpose appeared first on Software for Good.

Like I explained in the first two posts, one of the more useful ways of debugging web application problems is interrogating the DNS system, and the tool to do it is already on your computer: dig(1).

So, I was researching how Amazon’s ElastiCache works to debug a performance problem we were seeing. The caches were warming more slowly than expected, and we were seeing cache misses on data that we knew should be there.

Normally when configuring a Memcache client, you give it the addresses of every node in the cluster, and the client library figures out how to distribute the keys across those nodes. With an ElastiCache Memcache cluster, you configure the client library with one address—the configuration endpoint—and it dynamically adds and removes nodes from the clients’ configuration on-the-fly.

We configured the app with the endpoint URL, and it all seemed to be working fine. The app was able to talk to the configuration endpoint just like a normal Memcache server. But eventually we noticed the subtle problems listed above.

It turns out that the stock PHP Memcached module doesn’t do auto-discovery—for that, you have to install the AWS-provided module. But then why weren’t connections to the cache endpoint obviously failing?

As I was playing around with a small cluster, it seemed that using the config endpoint as a normal Memcache node worked just fine—it would store and retrieve entries as normal. So I looked at the DNS record for the endpoint:

$ dig -t any +noall +answer test-cluster.d00b1e.cfg.use1.cache.amazonaws.com
test-cluster.d00b1e.cfg.use1.cache.amazonaws.com. 15 IN A 172.31.x.y

The IP address was the same as one of the actual cache nodes, and the TTL was counting down from 15 seconds on successive lookups. Whenever the TTL reaches 0, the address changes to another one of the cache nodes:

So any given node can answer the special command to list the cache nodes, but if you connect a stock Memcache client library to the endpoint it will act like a normal Memcache server.

This behavior is surprising and doesn’t fit with a fail-fast design philosophy, but using the publicly available information at the heart of the Internet, we were able to figure it out quickly. The key is to know and use the tools at your disposal.

The post Digging Into`dig`: DNS Exploration appeared first on Software for Good.

In the last post introducing the dig(1) command, you’ll recall seeing an entry like this:

$ dig +noall +answer google.com
    google.com.		300	IN	A	216.58.216.238

Every DNS record has a similar format to this, and the fields are:

In this case, the answer says “The IPv4 address (A) of the domain name google.com. is 216.58.216.238, and you can cache that fact for the next five minutes.”

You can request other kinds of records with the -t option. For example, if you want to know, “What host do I connect to to send mail to someone with a gmail.com address?” you want the MX record(s):

$ dig -t mx +noall +answer gmail.com
gmail.com.      3599    IN  MX  5 gmail-smtp-in.l.google.com.
gmail.com.      3599    IN  MX  20 alt2.gmail-smtp-in.l.google.com.
gmail.com.      3599    IN  MX  30 alt3.gmail-smtp-in.l.google.com.
gmail.com.      3599    IN  MX  40 alt4.gmail-smtp-in.l.google.com.
gmail.com.      3599    IN  MX  10 alt1.gmail-smtp-in.l.google.com.

This says “Try gmail-smtp-in.l.google.com first (priority 5); if that doesn’t work, try these others in increasing priority order.” Or if you need to know what the authoritative nameservers (type NS) are for google.com.:

$ dig -t ns +noall +answer google.com
google.com.     21599   IN  NS  ns3.google.com.
google.com.     21599   IN  NS  ns2.google.com.
google.com.     21599   IN  NS  ns4.google.com.
google.com.     21599   IN  NS  ns1.google.com.

Authoritative who-what? This, combined with the TTL field, is where the “distributed database” part comes in: every domain has one authoritative nameserver (or at most a handful), which is the canonical data source for information about that domain. This is true not just of domain names that you or I would buy, but the so-called top-level domains (TLDs) as well. Ever wonder what the canonical nameserver for the .com. space is?

$ dig -t ns +noall +answer com
com.            13307   IN  NS  d.gtld-servers.net.
com.            13307   IN  NS  h.gtld-servers.net.
com.            13307   IN  NS  m.gtld-servers.net.
com.            13307   IN  NS  g.gtld-servers.net.
com.            13307   IN  NS  i.gtld-servers.net.
com.            13307   IN  NS  l.gtld-servers.net.
com.            13307   IN  NS  j.gtld-servers.net.
com.            13307   IN  NS  e.gtld-servers.net.
com.            13307   IN  NS  c.gtld-servers.net.
com.            13307   IN  NS  a.gtld-servers.net.
com.            13307   IN  NS  k.gtld-servers.net.
com.            13307   IN  NS  b.gtld-servers.net.
com.            13307   IN  NS  f.gtld-servers.net.

Those top-level name servers, operated by ICANN, don’t contain all the DNS records for every .com domain; rather, they delegate to the authoritative nameserver for a given domain.

But when you type google.com into your browser’s address bar, it doesn’t go to the authoritative nameserver every time. Your computer is configured to talk to a recursive resolver, which accepts DNS queries just like an authoritative nameserver but will follow the chain of lookups to find what the authoritative response is. It will then cache the result records according to their TTL, and the next time you look up the same address, it’ll just return it from its cache.

Try looking up google.com. again, and you’ll see the TTL decrease:

$ dig +noall +answer google.com
google.com.		232	IN	A	216.58.216.206

That’s your internet provider’s recursive resolver telling you, “Hey, I know this one (at least for the next four minutes). It’s 216.58.216.206.”

That’s how DNS works as a global distributed database: anyone can query it, there’s always one canonical source of a given record, and the results are cached by the server your users are querying. By now it should also be obvious that this is why DNS changes take time to propagate: the higher the TTL (a common value is 86400 seconds, or 1 day), the longer it will take for all of your site’s users to see the same records.

Up next: how I used this to debug a problem with AWS.

The post Digging Into`dig`: Get Your Hands Dirty appeared first on Software for Good.

While debugging an issue with ElastiCache for the Star Tribune, I had a chance to use a public Internet research tool that not many people know about: The global DNS system.

Research tool? Yes, although most everyone thinks of DNS as a new-fangled form of Yellow Pages that slavishly translates memorable domain names like cuteoverload.com, dowebsitesneedtolookexactlythesameineverybrowser.com, and softwareforgood.com into their corresponding IP addresses, it’s actually quite a clever public, distributed database to which we all have access.

The command-line tools necessary to explore this database are probably already present on whatever computer you’re using, namely: dig(1). Here’s how to get started.

Let your fingers do the walking

Let’s do some basic DNS queries in a terminal, like your browser would do when you type in a URL:

 $ dig google.com

 ; <<>> DiG 9.8.3-P1 <<>> google.com
 ;; global options: +cmd
 ;; Got answer:
 ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 38752
 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

 ;; QUESTION SECTION:
 ;google.com. IN A

 ;; ANSWER SECTION:
 google.com. 266 IN A 216.58.216.206

 ;; Query time: 137 msec
 ;; SERVER: 8.8.8.8#53(8.8.8.8)
 ;; WHEN: Fri Jul 31 15:26:17 2015
 ;; MSG SIZE rcvd: 44

(Note there’s no apt-get install or brew install step there: dig comes pre-installed on OS X and most Linux distributions. Even on Windows, the similar nslookup tool is available in the command shell.)

There’s a lot of good detail there if you want to sort through it, but let’s focus in a little bit. dig has many querying and formatting options listed in its manpage, which lets you turn on and off various parts of the output shown above. To see only the IP address that gets returned, you can use +short, but I often want to see the entire answer line as shown above. For this, I turn off all the output except for that section:

    $ dig +noall +answer google.com
    google.com.		300	IN	A	216.58.216.238

That’s the basic usage, and enough information to get started. Next week I’ll explain what ‘300’, ‘IN’, and ‘A’ mean, and why they matter for your deployments.

The post Digging into`dig`: An Intro appeared first on Software for Good.

Don’t struggle against change; flow with it, and use the energy of change to direct things. Maximize work not done by exerting your efforts at the right time.

The post Do Without Doing appeared first on Software for Good.

At the end of the Summer of Code, Google invites two mentors from each organization to their headquarters in Mountain View for a Mentor Summit. I was fortunate enough to get a slot, so on October 18th I flew out to California. (Not a bad change from Minnesota’s increasingly cold and dreary weather.)

Throughout the weekend I got to connect with other Free/Libre/Open Source developers on projects ranging from
embedded real-time operating systems to programming languages to databases to web applications to game engines to continuous integration tools to scientific computing libraries for my favorite programming language. I got to know people in person whose names I knew only from mailing lists, people whose software I’ve used regularly for years, and people working on some really interesting projects I’d never heard of before.

Being able to spend a weekend talking to really smart people about deep technical issues was great. I found some unexpected areas of overlap between other people’s work and mine here at Software for Good, as well as on Mercurial.

We did our fair share of eating and drinking together as well—at one point we went for late-night donuts at a place practically on Apple’s corporate doorstep.

All in all it was a great weekend. Kudos to Google for throwing so much support behind the open source community, and to the other mentors who devote their (often volunteer) efforts not just to writing Free and Open Source software, but to supporting others in learning to do the same.

The post Google Summer of Code Mentor Summit 2013 appeared first on Software for Good.