A case where we need to think very deeply about security.

Have you seen this dialog when using your iPhone or iPad?

Now a very simple solution to this problem (and it occurs on a number of other operating systems as well) is to extend the API so that the alert warns the user which application is asking for the password:


But the problem with this variation (or any other variation which explains which task is asking for the password) is that the prompt could be a lie.

My own thinking is that alerts of this nature which ask for a response should never be handled through a pop-up alert that may appear over other applications. This destroys discoverability; it prevents you from figuring out which application asked for the prompt.

Instead, I would rather see that we use notifications for asking for a password.

A notification that an application requires a password to continue has several nice properties.

  • A notification does not get in the way. It does not force the user to provide a password in order to continue whatever it was he was doing.
  • A notification provides greater space for information explaining why the password is needed, and perhaps even to provide alternative actions if the user does not wish to provide the password.
  • In responding to the notification, the application actually asking for the password can be brought forward, so the user can evaluate which application is asking, and if the request is reasonable.

Now of course I don’t mean we should do away with using the UIAlertController class to obtain the user’s password. The API behind that is far simpler than constructing a complete navigation controller, especially when the prompt occurs as part of a network request deep in the network stack of the application.

But those UIAlertController objects should never surface outside of the application requesting the alert.

And this also applies to Apple’s own applications.

You know the principle that you never give your credit card to someone who calls you?

Well, the same principle applies to passwords; you never give your password to an application that alerts you. Instead, you bring up the application and you give it your password.

And given how common this operation is, I wouldn’t mind if Apple were to take the lead and provide an API to do all of this for the application developer. A standard API has a way of standardizing application behavior–and this is a place where application behavior standardization is desirable.

Elaborating on that bug from yesterday.

The code which caused me problems which I traced to a rather popular third-party networking library–a networking library that was (in my opinion) rendered obsolete with the introduction of Grand Central Dispatch and NSURLConnection’s sendSynchronousRequest:returningResponse:error: method–which, by the way, is two generations deprecated from today’s NSURLSession class–was a retain cycle that the developer of that library choose to ignore rather than fix in the most obvious way.

The offending code was something like this:

self.callbackBlock = ^{
    [self doSomething];

In order to understand why this is a problem we need to look into how Objective C blocks work.

Generally blocks are created on the stack. In order to allow a block (essentially a form of lambda expression for Objective C) to exist beyond the current execution scope, you must make a copy of the block. In creating a copy we create an Objective-C object in memory which must strongly retain any values outside the scope of the block for use inside the scope.

That is, at the end of the assignment to callbackBlock we get in memory something like this:

The retain cycle should be obvious here:

Basically the block holds a pointer to self, and self holds a point to the block. If we then free self, the reference count from the reference holding self is decremented by 1, but the self object and the block object–because they hold references to each other–are never freed:

Memory is leaked–and if self is something complex (like a UIViewController), a lot of memory is leaked.

Oh, but I break the retain cycle in my code by expressly deleting the reference to my block when I’m done with it!

Yeah, because your code is bug free. (Sheesh)

Look, “I deal with it in code so it’s no big deal” is a fucking excuse, not a reason.

And it’s an excuse that is dependent on a misunderstanding of the nature of ARC and the retain process. So if you say “but I break the retain cycle in code”, you are a fucking idiot. Well, not a complete fucking idiot–just smart enough to shoot yourself in the foot.

It’s simple, really. The assumption behind ARC is that all allocations are in a tree-like structure of memory references. Things hold other things in a tree-like hierarchy: your UIViewController holds the root view it manages. The root view holds the other views in the hierarchy. The NSDocument object holds the root of the data structures representing your document. And so forth.

By holding references in a strict tree-like structure, it is clear what objects need to be freed when the retain count for a particular object hits zero: everything below it needs to be released. When a view goes away the children views also need to go away. When a block goes away the objects held by that block can be released.

Now of course sometimes we need back references up the tree. And those back references should be handled with weak references; references which do not increment the reference count, but are intelligent enough to be set to nil if the contents are deallocated. (I imagine a linked list of weak references associated with each object, though I’m not entirely sure how weak references are handled.)

This way we don’t get retain cycles.

So what do you do instead of writing

#pragma clang diagnostic ignored "-Warc-retain-cycles"

above your block where the compiler is complaining about a retain cycle?

Simple. Don’t break the cycle in code; that’s just an invitation to a memory leak, and depending on how you try to break the cycle, the cycle may never be broken. (I’m looking at you, AFNetworking.) Break the cycle using a weak reference.

In the example above, this means writing:

__weak MyObject *this = self;
self.callbackBlock = ^{
    [this doSomething];

When we create a weak reference cycle what we wind up constructing in memory is this:

That is, while self holds the block, the block holds a weak reference to self. This breaks the retain cycle by preserving the strict tree-like ordering to the hierarchy.

Then, as we free the reference to self:

And as the retain cycle reaches zero in self, it gets freed:

and then the block goes away itself.

So what happens if we have a separate reference to the block?

Well, again, notice the weak reference to self in block, which gets zeroed out.

So if you ever want to be extra cautious you could write:

__weak MyObject *this = self;
self.callbackBlock = ^{
    if (this) [this doSomething];

But remember a method invocation on a nil reference is not illegal in Objective C; it simply invokes a nil method which returns nil. So the

if (this)...

is not strictly necessary.

Today’s tl;dr: always break retain cycles when writing block code.

Never use “-Warc-retain-cycles”–even if you think you know what you’re doing. Because the very fact that you’re contemplating breaking a retain cycle in code clearly demonstrates you do not know what you’re doing.

Addendum: One thing I’ve heard developers say is that sometimes you have to break the rules in exceptional cases.

Sure. I’ll buy that. I’ve even done that.

However, your lack of understanding is not an exceptional case. It’s just stupidity.

Retain cycles, or why your iOS app sucks.

One of the reasons why I hate the over-use of third party libraries is that, while in theory it’s “the more eyeballs the better the code”, in practice none of us who use a third party library ever bother to trace bugs into the third party library. It’s because we think that third party library (especially one that has been around for a long time) is somehow “bug free”–when, as we all know, “too many cooks spoil the broth”.

(That is, we’ve seen in practice the more people who work on a software project in a corporate setting, the more likely there are bugs in the source kit, as different developers with different levels of aptitude work on the same source kit to varying results, often unchecked by his peers as most software projects are just too big for everyone to understand. So why is this different with third party libraries?)

In my case I managed to trace down a bug.

Let’s put it this way:

If, in your source kit, you have something like this:

#pragma clang diagnostic ignored "-Warc-retain-cycles"

You have a really bad memory leak bug which affects Every. Single. Fucking. App. that uses your code.

Now I’d forgive this if we’re talking about a small project and the person working on it wasn’t all that established in the community.

But this is a big honking library which is used by a lot of companies, where there are tutorials floating around on sites like NSHipster and Ray Wenderlich, and is often the very first library everyone wants to include that may engage in any sort of networking operations.

The good news is that 3.0 of AFNetworking did away with the problem.

The bad news is that 2.0 of AFNetworking has the problem in spades.


At what point did my life stop being about implementing increasingly clever algorithms which do more and more interesting things, and started being more about hammering on an API framework to behave in ways not originally envisioned by the designer of that framework, to mesh with the whims of designers who cannot be bothered to read the fucking user interface guidelines?

Just sayin’

A follow up from yesterday’s post: further observations.

Securing Driverless Cars from Hackers Is Hard. Ask The Ex-Uber Guy Who Protects Them

Now, Miller has moved on, and he’s ready to broadcast a message to the automotive industry: Securing autonomous cars from digital attacks is a very difficult problem. It’s time to get serious about solving it.

Eye roll.

Look, securing cars or so-called IoT devices is not hard.

But it requires three things–which in today’s Agile-driven cowboy-hacking development culture, is fucking damned near impossible:

  1. Planning: Security is not something you bolt on. It’s not an add-on feature like adding a new button to a web app which carries you to a disclaimer page. Security is something that has to be planned for from the bottom-up–and in today’s broken development culture, “planning” is just something we don’t do well.
  2. Understanding: You need to understand the core tools available to you in the encryption toolkit and use them correctly. You need to understand that every security system for authentication, including tools like single-sign on, access control authorization, and securely storing information are based on the three encryption patterns I covered yesterday: one-way hashing, symmetric-key encryption, and asymmetric-key encryption. You need to understand the limits of these patterns, and understand that no matter how you wrap them in some architecture, the architecture only works because it either relies on the features of each of these patterns (and does it either well, or poorly), or it relies on “security through obscurity.”

    And you have to remember that “security through obscurity” in today’s world is simply not enough.

  3. Proper testing: You need to have a QA team who specializes in testing security. You need a team who specializes in penetration testing: in trying to figure out how to break the existing system. You need to verify your security architecture before you build everything else on top of that architecture–otherwise you may find yourself re-engineering major chunks of your code in response to security lapses. You need to review your architecture with your security team–because eventually hackers will also understand your architecture, and hackers will have a hell of a lot longer to try to break into your system than your QA team will have. (You may spend six months developing the software for the on-line systems in your car–but your car will be out there for 10 years.)

Now if you’re the type of developer who doesn’t understand why salting passwords is important–and why, while you may not want that salt string to get public, it doesn’t matter as much as having a fundamentally insecure system that uses obscurity to hide access–then you really need to hire someone who does. That person needs to be a system architect: someone who is in charge of designing and specifying the complete architecture of your system. (Security cannot be championed by Project Managers; he has no ability to specify how things work. And a senior developer doesn’t have the authority to force fundamental architectural changes over the objections of management.)

And if you think simply switching on SSL will do the trick–please step the fuck away from your IDE.

Because while SSL may help prevent eavesdroppers from sniffing traffic between two devices he doesn’t control (though not always; Symantec’s Endpoint Security and other similar AV scanners have the ability to crack the contents of HTTPS packets)–I fucking guarantee you a hacker will download your application and can sniff the contents of the packets from his desktop or mobile device so he can reverse engineer your API.

And if your idea of security is to put the car’s VIN as a parameter in the HTTPS GET and POST requests to the server–then you are a fucking moron.

Security really isn’t hard. But it requires thought, dedication and a willingness to properly test–all things we no longer do in a development environment.

Frankly if I were to put together a team which was creating secure software for a bank or a car company or an Internet device, I’d hire a technical writer (to help keep all the documentation straight), and I’d keep at least a 1:1 ratio of developers to QA. I’d also hire a separate security QA team dedicated to penetration testing: people who know how to do a security audit, who know what “fuzzing” is, who can also test the security of our physical systems. I’d draft a set of policies and procedures for our customer support, our back-end deployment team, the guys who maintain the software–and those policies and procedures would cover everything from access to the source kit to physical security of the servers to the length of passwords used internally. And let my security QA team audit that as well.

And I’d use a hybrid waterfall/agile process with a very long (at least 6 month) release cycle except for emergency hot-fixes.

Contrast to a typical development team today handling web or mobile development: we skimp on QA hiring, I haven’t seen a technical writer embedded with a development team in decades. We also tend to use a short (2 week) agile development cycle where long-term planning and architectural design take a back burner to adding the latest feature–which may be fine if you’re creating a web site but sucks if you need to think deeply about anything.

So really, security isn’t hard.

It just takes a fundamental change in development culture.

On password security.

From an AT&T Developer Program e-mail: How secure is your e-mail password?

If you want to be more careful with your e-mail on your mobile device, make sure you have turned on SSL. And you should never give your phone to others because turning SSL off is easy to do.

I have a few notes.

  • SSL is not a panacea. SSL does help slow down hackers. But not by much.

There are a variety of tools which can crack open SSL requests–and this has been true for more than a decade, though some commentators are only just now figuring this out. Symantec’s Endpoint Protection firewall, for example, scans for computer viruses in all HTTP transactions–including “protected” HTTPS requests. It does this by generating a trusted SSL certificate within the firewall which uses one of Symantec’s cert keys, signed using the web site URL of the web site you are accessing. Thus, the firewall can reach out, make a request against a web site, decrypt the traffic, scan it, re-encrypt it using the internally generated SSL key, and send the result to your computer.

There are a number of desktop tools which do the same thing, nominally designed to help developers understand the messages being sent by their desktop or mobile devices. I personally own Charles, a web debugging proxy application, which does more or less the same thing. (For debugging physical hardware you can set up your computer as a proxy for your mobile devices.)

Given this, you can pretty much assume any hacker worth his salt can easily peak into the structure of the requests and responses you make against a web site, reverse engineer your protocol, and make requests against your server–SSL or no.

And even if a hacker doesn’t have access to a certificate against which to generate signed SSL certs on the fly, a hacker can certainly insert a new root certificate into a computer he has access to, and sniff the traffic from that computer using the same techniques Symantec does with their Endpoint Security system. Bad if you run an IT shop full of computers.

  • There are techniques for encrypting a password sent to a remote computer to solve this problem. Various solutions for encrypting a password have been around for a while. Too bad no-one uses them.

One technique used to authenticate without sending a password in plain text is APOP protocol, first described in the e-mail POP3 protocol. Similar ideas (from more than a quarter century ago) have been floating around for at least as long. And the idea is this: when authenticating, instead of simply sending a command like:

{ "req": "login", "username": "woody@alumni.caltech.edu", "password": "Pa55w0rd" }

You’d do three things:

  1. Request a unique token. This can be a timestamp or a UUID generated by the server. The server stores the token returned associated with the current session.
  2. Use the token to generate a hashed response. For example, calculate SHA1(token + salt + password), where salt is an agreed upon constant common between the client and server. Both the client and server generate this result.
  3. Send the generated hash response to the server. The server then compares the hash with the client’s sent result, and authenticates if the hashed password matches.

In today’s world, doing this on a Javascript server is pretty trivial. And a hacker sniffing your traffic will see the following:

  "req": "login", 
  "username": "woody@alumni.caltech.edu", 
  "password": "9d6786ab1b8ded2e8ccb2079efafa18612c8c7b8f1888f3d25ee2c7528c164e9" 

This has a number of nice properties.

  • It is immune from replay attacks. A common technique for hackers to use is replay attacks; they record the packets going back and forth to a server, and replay the packets in order to gain access to the server. By obtaining a unique token from the server and using it to encrypt the password, you cannot replay the login packet to gain access to the user’s account.
  • It is hard to reverse engineer the password by sniffing packets. The next time the user logs in, the packet that gets sent for logging in looks like:
  "req": "login", 
  "username": "woody@alumni.caltech.edu", 
  "password": "5d12af8a0031f83c3e32c5513ecbabfec37fbef48b2b002ece20c7abab8e0e80" 

There is no pattern here that is easily discerned because the password being sent is a SHA-256 hash of a UUID + a token (can you guess the token?) + the password we used above.

There are a number of other techniques, but most of them are variants of the above. Some variants encode the password first (for example, the password is first hashed with a salt so that the password is never stored in plain text).

Side note: There really are only three encryption techniques that are used in encryption, which underlies all security. Most security techniques basically are an arrangement of each of these encryption techniques, each of which have their own strengths and weaknesses:

  • Hashing which converts some data into a hashed value. Hashing is a one-way technique; you can hash a value easily, but it is nearly impossible to unhash a value once hashed. (At best you can search for a “collision”; another string which generates the same hash. It’s one reason why you always salt your hashes–because most techniques used for finding a “collision” are much harder to execute if the input string must fit some specific criteria, such as containing a common well-known prefix.)

    (Examples include MD5, SHA-1, SHA-256 and bcrypt. Please don’t use the first two. Try to use the last.)

  • Symmetric-key encryption/decription where a string is encrypted and decrypted using the same encryption key. Symmetric-key encryption requires the common key to be shared between the code encrypting something and the code decrypting something–making the encryption key a weakness.

    (Examples include Blowfish and AES.)

  • Asymmetric-key encryption/decryption where a different key is used to decrypt a string from encrypting the string. Asymmetric-key encryption has the nice property that, if one key is kept private and another is made public, we can do all sorts of interesting things. For example, if the encryption key is made public and the decryption key private, then we can create a system where anyone can send a receiver a message–but once encrypted only the receiver with the decryption key can read the message. (This is how PGP e-mail encryption works.)

    Likewise, if the key used to encrypt is made private but the decryption key public, then we can create a mechanism for “signing” files: a file is properly signed if, once the encrypted message is decoded, the encrypted message matches some expected value–such as the checksum of the file being sent. This is how message signing is done for things like SSL certificates on the web.

  • There is a fourth technique that is used to help two clients agree upon a common key, and that is Diffie-Hellman and related algorithms, which permit the sharing of a shared secret without the possibility of an eavesdropper discovering the shared secret. It helps solve the problem of a client and server sharing a shared secret for symmetric key encryption of a message–and that is at the core of how TLS encrypts messages. (Add in public/private key signing to validate a certificate, and you have SSL and TLS in a nutshell.)

All encryption techniques are basically built on these three tools (and network protocols may use the fourth).

Each of these tools have their own strengths and weaknesses. For example, asymmetric key encryption assumes at least one of the two keys are kept secret (either the encoding key or decoding key, depending on how the key is used), which means if you use asymmetric key encryption for sending secret messages to multiple receivers, either the message must be encrypted multiple times or the secret key must be shared amongst multiple receivers.

(It is this fact that made me raise an eyebrow when someone creates a messaging app which they claim is completely secure end-to-end. Because if the system permits a single message sent from a client to a server to be re-sent to multiple receiving clients, it means either the server actually has enough information to decrypt a message (and re-encrypt the message to multiple receivers), or it means the server has the ability to share the encryption secret amongst multiple receivers–exposing the server to all the information necessary to decrypt traffic.)

Let’s get back to the topic at hand.

  • Encrypt passwords on your database.

When securing passwords on your web server or server application, the other thing you need to worry about is properly securing passwords in your database. Assume a hacker will eventually read the contents of your database tables–which means all the password security in the world is pointless if you store passwords on your server in plain text.

This is a common mistake, and you can tell if a web developer made this mistake when, as part of password recovery, they simply e-mail you the password. (As an aside, nothing pisses me off more than a web site that demands your password be strong–demanding you use an upper-case letter, lower-case letter, punctuation mark and number–then they e-mail your password in plain text when you ask to recover your password. Fuck you for making me jump through hoops when you can’t be bothered to do the same on your web site.)

There are a number of ways we can secure passwords against spying eyes on a database server. At its core, looking at the list of encryption techniques above, it should be clear we can either hash passwords (again, using a salt of some form), or we can use symmetric-key encryption. (Asymmetric-key encryption certainly is possible, but if your server is the same thing encoding password and decoding passwords, going through the additional effort for asymmetric-key encryption seems silly. Under some circumstances, however, it may make complete sense–such as if you implement a separate sign-on server to handle a large web site.)

A big advantage of using hashing is that it is one-way. This is the technique used by nearly all secure web sites and is also behind the encryption of passwords in Linux and on the Mac. (The principle difference in various techniques involve the hash involved: secure systems use Bcrypt, which has the property of being computationally expensive–and thus very slow to calculate. Of course in today’s world where we can buy the equivalent of a room full of Cray X-MP supercomputers for less than 50 dollars, computational complexity is a shallow panacea: it’s not hard to imagine a dedicated hacker wiring up multiple dedicated processors to accelerate the calculation of a Bcrypt hash to look for collisions.

Ultimately security is not about guaranteeing that your system won’t be penetrated. It’s about increasing the cost of penetration. It’s a series of best practices–backed with knowledge as to how encryption and hashing work–which allow you to slow down a hacker.

Think of computer security very similar to the physical security of a building.

The point is not to make it impossible to break into a building; that is simply impossible. Someone equipped with enough time, enough equipment and the willingness to use force can break into a building.

The point is twofold.

  1. Make it hard. If it is harder for someone to break into a building than the value of the stuff inside that building, then they may not bother.

    You see this with houses verses warehouses verses banks: the more valuable the contents, the harder it is to break into that building.

  2. Try to capture information about who broke in. It’s why security cameras are employed around secure buildings. It may be impossible to prevent someone from robbing a bank–but if you can catch the robber, you can then punish him for the crime.

    (This is why computer logging is an important aspect of security: so you know who tried to break into your system.)

Sure, SSL is an important element of security.

But if you’re not thinking through your entire security stack, from the protocol used to send a password to the way you encrypt passwords in your database and way you physically isolate the database from the network, simply turning on SSL is just fooling yourself into thinking things are safe.

It’s the equivalent of locking the front door while leaving all the windows open.

Interview questions.

Programmers Are Confessing Their Coding Sins To Protest a Broken Job Interview Process

A number of programmers have taken it Twitter to bring it to everyone’s, but particularly recruiter’s, attention about the grueling interview process in their field that relies heavily on technical questions. David Heinemeier Hansson, a well-known programmer and the creator of the popular Ruby on Rails coding framework, started it when he tweeted, “Hello, my name is David. I would fail to write bubble sort on a whiteboard. I look code up on the internet all the time. I don’t do riddles.” Another coder added, “Hello, my name is Tim. I’m a lead at Google with over 30 years coding experience and I need to look up how to get length of a python string.” Another coder chimed in, “Hello my name is Mike, I’m a GDE and lead at NY Times, I don’t know what np complete means. Should I?”

When I interview people I do rely on technical questions.

But I use them as a form of behavioral interviewing.

Meaning I don’t give a damn if you can solve the problem. I really don’t. What I care about is how you act when I give the interview.

Further, I limit my technical questions to two phases–and I do this only after breaking the ice and talking about you and your previous jobs. (The “why did you leave your last job, why did you go to the college you did, how is your dog doing?” sorts of questions.)

The first phase of technical questions: I’ll ask very basic questions about the platform I’m interviewing you for. If you’re a Java programmer I’ll ask about different collection classes. If you’re interviewing to do iOS development, I’ll ask what a view controller is. Questions which, if you are actually actively writing code, you should know the answer because you deal with those things on a daily (or weekly) basis.

Now I don’t care if you stumble on the questions. I expect you to stumble. It’s a God-damned interview, and no-one is comfortable during an interview. You want a job, I’m waving money in front of your face–of course you’re going to choke when I ask you what a UINavigationController is. It’s a stupid question, and I must have a hidden reason for asking a stupid-simple question, right?

But these questions are designed to screen out those who are pretending to be an iOS developer, or pretending to be a Java programmer, or pretending to be an Android developer. There are a lot of people out there who claim 5 years of Java programming who haven’t actually written one line of code in Java.

The second phase of technical questions: I will ask you to write code on the whiteboard. I have two standard questions I ask (one which requires you to write code with a single loop, another with two loops), and I’ll happily walk you through the exercise if necessary.

But even with the second phase, what I’m looking for is your behavior. Do you remember to close your curly braces? Do you know what the modulus operator (‘%’) is? Do you know how to write a for loop? And if not, do you ask? Do you think or do you panic? Do you try to correct me? Do you push back?

I don’t care if you can write code on a white board. I really don’t.

But I do care how we interact. I’m not watching if your code is correct–though bonus points if it is. I’m watching to see how you approach the problem.

Sometimes someone blows through my questions quickly, and at that point I start digging out the harder questions: what is the P vs NP problem. How do you calculate prime factors? Can you write a quick sort from memory?

But if we’re on those questions, I guarantee you three things:

First, you’ve got the job if you want it.

Second, I’m trying to impress you on how smart the other people you will be working with–because I’ve noticed many programmers like the idea of working with smart people.

And three, I’m trying to kill time because I’ve already decided I want to hire you, but I still have a few minutes to fill.

I’ve gone through a lot of interviews, and I’ve noticed most interviewers ask technical questions–but have absolutely no fucking clue how to ask them or what to look for when someone tries to solve them. They’re just looking for the correct answer, not for someone who seems smart and motivated. And that often leads them to making poor hiring decisions–overlooking those who would make great additions to their team, hiring people who may not actually work well with the existing group culture.

But remember: you’re hiring a human being; someone you will work with, someone you will chat with over coffee, someone you will use as a sounding board when you have a problem you’re struggling with.

You’re not hiring a fucking search engine.

Yet another Slashdot comment.

Where someone asks my thoughts about maintaining comments or code documentation:

Comments, like many things in software development, are really a matter of personal style–though they do serve the purpose of making it clear what your code is supposed to do, so the guy who maintains it later can have an understanding of what is supposed to be going on.

So I tend to lean towards a block comment per non-trivial (10 lines or more) function or method–though sometimes the block comment may be at most a sentence. (I don’t think the name of the method or function is sufficient commentary enough simply because cultural differences may make a non-native English speaker unable to translate your personal naming conventions.) I also tend to lean towards describing how complex sections of code work if they appear “non-trivial” to me–meaning if I’m doing more than just calling a handful of other methods in sequence or looping through a bunch of stuff which was clearly noted elsewhere.

I also tend to write code like I write English: a single whitespace used to separate two blocks of code, and moving functionality that is associated into their own blocks (where shifting functionality does not impair the app) is preferable. I also tend to block chunks of methods in a class which are related with each other; the Apple C/C++/Objective C compiler provides the #pragma mark directive to help provide a one line comment to help identify the functionality of those groups of methods.

If I’m doing something really non-trivial–for example, implementing a non-trivial algorithm out of a book (such as a triangulation algorithm for converting a polygon into a triangle) I will start the block comment with a proper bibliographic citation to the book, the edition of the book, the chapter, page number, and (if the book provides) algorithm number of the algorithm I’ve implemented. (I won’t copy the book; the reference should be sufficient.) If the algorithm is available on a web page I’ll cite the web page. And at times when I’ve developed my own algorithm, I will often write it up in a blog post detailing the algorithm and refer to that in my code. (When done as a contractor I’ll look for their internal wiki or, if necessary, check in a PDF to the source kit.)

Back when I started doing this a long time ago, technical writers were sometimes used to help document APIs and the project requirements of the project. The reason for having a separate technical writer was because by having a person embedded in a project who is not intimately familiar with the project, they know the right questions to ask to help document the project to an outsider, being an outsider themselves. Developers cannot do this. Developers, being too close to the code they work on, will often run the code down the “happy path” not thinking of the edge cases to run, and they won’t describe things they think is obvious (like “of course you must call API endpoint 416 before calling API endpoint 397; the output of 416 must be transformed via the function blazsplat before calling 397, and how do you get the input to blazsplat?). A really good technical writer could also help to distill the “gestalt” of the thing, helping to find the “why” which then makes code or project specifications incredibly clear. (You call endpoint 416 because it’s the thing that generates the authentication tokens.)

Back when I was required to train for CISSP certification (I never took the test), one section discussed the three states of knowledge–later used by Donald Rumsfeld in a rather famous quote he was lambasted for. There are things we know we know. For example, I know how to code in C. There are things we know we don’t know. For example, I know I don’t know how to code in SNOBOL. But then there are also things we don’t know we don’t know: this is why, for example, why often speakers never get any takers when he asks for questions: because his audience doesn’t know what questions they should ask. They don’t know what they don’t know.

I think there is a fourth valid state here–and we see the computer industry absolutely riddled with them: the things we don’t know we know. These are the things we’ve learned, but then we think somehow “they’re obvious” or “everyone knows them.” They’re the things we don’t question; the things we don’t think need to be taught, because we don’t know they need to be taught. And the problem is, developers–having invented the code they need to describe–don’t know that they need to describe it to people who haven’t spent weeks or months or years studying the problem and working towards a solution.

I know I suffer from it, I know others who suffer from it. It’s why I think we need people checking our work and helping us to document our interfaces. And it’s why any answer I give for how to comment code will be insufficient: because of the knowledge problem that I don’t know what I should be telling you if you want to understand my code.

And it’s why I think we often don’t comment enough: because we don’t know what we know, so we don’t know what we need to tell other people.

Took, what? 20 years for someone else to say this?

58% of high-performance employees say they need more quiet work spaces.

Our open industrial spaces are frustrating our best people and likely impacting our end products.

I mean, this would be obvious to anyone who had read The Mythical Man-Month and considered the consequences of some of that book’s observations. But apparently we’re too smart to make such observations…

A comment left on Slashdot.

The problem is that our industry, unlike every other single industry except acting and modeling (and note neither are known for “intelligence”) worship at the altar of youth. I don’t know the number of people I’ve encountered who tell me that by being older, my experience is worthless since all the stuff I’ve learned has become obsolete.

This, despite the fact that the dominant operating systems used in most systems is based on an operating system that is nearly 50 years old, the “new” features being added to many “modern” languages are really concepts from languages that are between 50 and 60 years old or older, and most of the concepts we bandy about as cutting edge were developed from 20 to 50 years ago.

It also doesn’t help that the youth whose accomplishments we worship usually get concepts wrong. I don’t know the number of times I’ve seen someone claim code was refactored along some new-fangled “improvement” over an “outdated” design pattern who wrote objects that bare no resemblance to the pattern they claim to be following. (In the case above, the classes they used included “modules” and “models”, neither which are part of the VIPER backronym.) And when I indicate that the “massive view controller” problem often represents a misunderstanding as to what constitutes a model and what constitutes a view, I’m told that I have no idea what I’m talking about–despite having more experience than the critic has been alive, and despite graduating from Caltech–meaning I’m probably not a complete idiot.)

Our industry is rife with arrogance, and often the arrogance of the young and inexperienced. Our industry seems to value “cowboys” despite doing everything it can (with the management technique “flavor of the month”) to stop “cowboys.” Our industry is agist, sexist, one where the blind leads the blind, and seminal works attempting to understand the problem of development go ignored.

How many of you have seen code which seems developed using “design pattern” roulette? Don’t know what you’re doing? Spin the wheel!

Ours is also one of the fewest industries based on scientific research which blatantly ignores the research, unless it is popularized in shallow books which rarely explore anything in depth. We have a constant churn of technologies which are often pointless, introducing new languages using extreme hype which is often unwarranted as those languages seldom expand beyond a basic domain representing a subset of LISP. I can’t think of a single developer I’ve met professionally who belong to the ACM or to IEEE, and when they run into an interesting problem tend to search Github or Stack Overflow, even when it is a basic algorithm problem. (I’ve met programmers with years of experience who couldn’t write code to maintain a linked list.)

So what do we do?

Beats the hell out of me. You cannot teach if your audience revels in its ignorance and doesn’t think all that “old junk” has any value. You cannot teach if your students have no respect for experience and knowledge. You cannot teach if your audience is both unaware of their ignorance and disinterested in learning anything not hyped in “The Churn.”

Sometimes there are a rare few out there who do want to learn; for those it is worth spending your time. It’s been my experience that most software developers who don’t bother to develop their skills and who are not interested in learning from those with experience often burn out after a few years. In today’s current mobile development expansion, there is still more demand than supply of programmers, but like that will change, as it did with the dot-com bubble, and a lot of those who have no interest in honing their skills (either out of arrogance or ignorance) will find themselves in serious trouble.

Ultimately, as an individual I don’t know if there is anything those of us who have been around for a while can do very much of anything, except offer our wisdom and experience to whomever may want to learn. As someone who has been around for a while it is also incumbent on us to continue to learn and hone our skills; just this past few years I picked up another couple of programming languages and have been playing around with a new operating system.

And personally I have little hope. Sure, there is a lot of cutting edge stuff taking place, but as an industry we’re also producing a lot of crap. We’ve created working environments that are hostile (and I see sexism as the canary in the coal-mine of a much deeper cultural problem), and we are creating software which is increasingly hostile to its users, despite decades of research showing us alternatives. We are increasingly ignoring team structures that worked well in the 1980’s and 1990’s: back then we saw support staff (such as QA and QAE and tech writers) who worked alongside software developers; today in the teams I’ve worked for I’m hard pressed to find more than one or two QA alongside teams of a dozen or more developers. I haven’t seen a technical writer embedded in a development team (helping to document API interfaces and internal workings) for at least 20 years. And we are increasingly being seen as line workers in a factory rather than technically creative workers.

I’m not bitter; I just believe we’ve fallen into a bunch of bad habits in our industry which need a good recession and some creative destruction to weed out what is limping along. And I hope that others will eventually realize what we’ve lost and where we’re failing.

But for me; I’ll just keep plugging along; a 50+ year old developer in an industry where 30 is considered “old”, writing software and quietly fixing flaws created by those who can’t be bothered to understand that “modules” are not part of VIPER or that MVC can include database access methods in the “model” and who believe all code is “self-documenting.”

And complaining about the problems when asked.