About William Woody

I'm a software developer who has been writing code for over 30 years in everything from mobile to embedded to client/server. Now I tinker with stuff and occasionally help out someone with their startup.

I hate Swift.

I hate Swift.

I know this isn’t the sort of thing that, as a developer of macOS and iOS software I’m supposed to say. After all, Swift is new, Swift is great, Swift has idioms which prevent you from getting into trouble. And all things New And Improved!™ are supposed to be better.

Worse, if you’re a neanderthal like me who hates Swift, it’s because you’re a Bad Person. Bad people are terrible software developers, people who don’t know what they’re doing, people who don’t understand the way.

But I’m going to say it anyway.

I hate Swift.


Now don’t get me wrong; there are a lot of things Swift does really well. Swift handles null pointers and null references really well. I like the fact that a nullable type is a first-class object that requires explicit handling and explicit unboxing. I appreciate the ‘?’ operator for optional-chaining, and the ‘!’ operator for forced-value expression, and the ‘??’ operator for providing a default for a nullable variable. Granted all this took getting used to, but I appreciate them because they cause one to try to write better code, if used mindfully.

But Swift is a persnickety language, and it definitely has a “happy path.”

Meaning if you’re using Swift to string together some pre-existing views and controls and carry the data from those controls around to other places in your code–the 90% of the stuff you have to do in order to make a working macOS or iOS application–it’s fantastic. It’s great. The persnickety element sometimes blocks your flow and makes you think “hey, should I be doing this?” or “hey, shouldn’t I deal with this potential problem?” But it works.

For the happy path. For the 90% of your code.

But Swift definitely intrudes. It definitely has a way in which it wants you to do things.

Now let’s be clear: I personally prefer to catch as many errors at compile time than at run time as possible. And Swift’s optional-chaining/forced-value stuff surfaces null pointer errors as compile-time errors.

But Swift… weirdly it misses a few things.

Swift doesn’t have a way to define an abstract method or an abstract function. This makes certain idioms hard to write–and worse, Swift’s answer, used in parts of Apple’s frameworks, is to make what could have been a compile-time error (undeclared abstract method, such as you see in Java or C++) into a run-time error (by creating an ’empty’ base declaration that throws an exception).

So Swift’s nannying is… incomplete. Sometimes woefully so.

And weirdly so, given how persnickety Swift can be sometimes. “Yes, you have to think through exactly if and when and how this variable may be null. But an undeclared method you needed to declare? Meh, CRASH!

And Swift’s persnickety behavior makes anything revolving around pulling apart a String and handling it as an array of Characters… challenging. It can be done, but you wind up wandering down a hierarchy of declarations (String, Substring–which, ironically enough, is not a String, Character–which looks like a string except it isn’t, weird things like UTF16View, UTF8View and arrays of things that look like integers, as well as a whole bevy of ‘Unsafe’ pointer things whose declaration and usage seems to change every five minutes with language revisions) that makes writing a per-character lexical analysis program an exercise of looking through the hierarchy of Swift declarations to figure out the current ‘one true way’ to handle strings.

(And yes, I understand that Swift’s string handling is constrained by the Unicode standard, which itself is… oddly twisted in weird ways, such as with the handling of characters in the U+10000 – U+10FFFF “supplementary planes” range a pain in the ass, especially if you think “well, just encode it as an array of 16-bit unsigned integers”, as Java does. Simply saying “well, we can ignore these” doesn’t work if you ever want to write an emoji. 😀 But can’t I just get an array of 16-bit integers or 32-bit integers, manipulate that, and turn that back into a String without making a whole Federal production out of it?)

And what the hell was Apple thinking with the “Unsafe” pointer reference types? I mean, I pride myself in understanding the ins and outs of odd corners of a programming language–but really, what the hell, man; what the hell? At least Java dealt with it by making arrays of basic types first-class citizens, instead of returning unsafe blobs of memory that may or may not be represented by some genericized unsafe reference thing that may or may not be convertible to something useful.

All of this means that as soon as you wander off the happy path: as soon as you start (for example) implementing some complex algorithm involving anything more complex than a push-down stack or want to implement some string parsing system or build something using a complex array of data structures–or God help you if you need to handle memory in a way that is off the happy path of ARC–Swift definitely gets in the way.

And writing an algorithm, like building a variation of a convex hull algorithm in Swift, becomes akin to wrestling a bear.

Which I hate, because in implementing the algorithm I’m already wrestling a bear–and I don’t need Swift to intrude every five minutes with a “you can’t express walking down a stack with a pointer” or “you want to autoincrement what?” or “no, you can’t simply delete the middle five objects and return it as new object.”

It’s not that Swift makes implemneting such an algorithm impossible. But Swift keeps wanting to poke itself into the conversation when I need it to shut the hell up for a minute so I can think about my algorithm instead!


So I hate Swift.

Honestly, I prefer using Objective C to Swift. Or heck, dropping into C or C++.

Hell, I’ll take Java over Swift if I need to do anything more complex than “glue button A to array B so table C can be populated with the sorted values.” Even though Java requires you to be incredibly verbose about everything–at least you can use the code refactoring functionality of a good Java IDE to track all those weird imports and type declarations. Though honestly I think Java could use some of Swift’s compiler mojo to eliminate some of the verboseness.

After all, do I need to write `thing = Enum.VALUE;` when the compiler already knows `thing` is of type `Enum`?

But yeah, I’d rather take verbose Java over concise and persnickety Swift for writing anything more complex than the plumbing that is 90% of modern app development.

Plumbing which, I will grant, Swift absolutely excels at.

Security. Not “Cyber Security.”

Answering a question found on Marginal Revolution, an economics blog which I periodically follow:

Long-time MR reader here. I have a question: what is the appropriate framework to think about incentives (economic or otherwise) for electric power utilities to beef up their cybersecurity?

Here are some random thoughts, in no particular order. If you think you already understand security processes, you can skip down to “incentives.”


First, you can’t understand the incentives of a thing without understanding the thing itself. So it’s worth drilling down into what we mean by “cybersecurity.”

Second, I wish the hell we’d get away from the term “cybersecurity” and call it what it really is: security.

Talking about “cybersecurity” is like focusing on the locks of a door: how many pins it has, the type of the lock, if the lock is electronic or driven by a physical key, if it’s brass plated or plated in steel–all the while the window right next to your front door is wide open inviting anyone who wants to to climb right through.

Don’t believe me? The earliest hackers who hacked into the AT&T long-distance system in order to make free long distance phone calls discovered how the AT&T system worked by someone breaking into AT&T’s offices and stealing the specifications for their system.

And they did that by someone dressing up as a befuddled new employee who was wandering the offices on his first day of work trying to figure out where his desk was.

Never underestimate social engineering. It does’t matter if you have the best firewalls in the business if someone can simply walk through the front door dressed as a janitor, and obscond with a laptop computer containing all your company’s secrets.


Now security itself is a complex topic. But basically:

1. Security is not ‘salt’ which can be sprinkled into the recipe to make things taste better. Security is a corporate-wide principle that must be architected into all aspects of your business, your software, your processes, your practices. Security must start with the C-suite: if your CIO or CTO or CSO (Chief Secuirty Officer) cannot understand security, create policies and procedures to correctly implement security, and has the authority to implement these policies and procedures, then you will never have a secure company.

2. Security does not, however, require that your corporation be “paranoid.” You don’t need armed guards following around everyone in the building. But it does mean you should start at the front door–both virtually and physically–and check to make sure everyone entering has the proper authority to be there.

And note some forms of security can actually make things worse: requiring employees to change their passwords every 3 months and never use any duplicates means the person who has been there for 10 years would have had to create 40 separate secure passwords during his tenure–which implies he’ll be writing his passwords down on a sticky note somewhere on his desk. This form of paranoia has made things worse, not better.

3. Security also requires constant training. Computer security–including social hacks, e-mail attacks and hackers trying to guess passwords–require its own training. If we were just as dilligent about computer training as we were about HR training for sexual harassment, most companies would be far more secure than they are now.

4. Once we get passed the policies and procedures (making sure employees are properly identified at the front door–logging access as appropriate, making sure employees are properly trained not to open malicious e-mails, making sure employees leave sensitive documents at work, or at least use a separate corporate-controlled laptop when traveling abroad) — then we can talk about specific products. But even there, those products must be integrated properly throughout the system; otherwise, we’re back to discussing the best door lock for your front door, neglecting the open window right next to it.


All of this also applies to how we engineer software products: security is not a salt that gets sprinkled onto your software product in the hopes things are better. They must be architected into the system from the ground up.

Unfortunately most software developers don’t understand this. Worse, they think certain things–like certain protocols (such as SSH) are sufficient enough to protect confidential information, as if somehow your browser only makes a secure connection after the user has logged in.

(For example, I once worked on a mobile product with a back-end component. It turned out the back end was not checking the credentials on each transaction; it relied on the mobile app to only make API calls the user was authorized to make. And it relied on SSH for people not to be able to ‘sniff’ the packets to understand the API calls being made. The thing is, while SSH makes it harder (but not impossible) to sniff out other people’s transactions, there are a number of products which you can buy to allow you to sniff the packets between your own device logged into your own account and a back end. And that allows you to easily figure out the format of the packets–and create your own application which can use the back-end API.

Long story short: a dedicated hacker and a few hours of time could have easily hacked that system enough to create his own mobile app. And because our back-end was not validating the security of each transaction–they could have easily hijacked our system as a result.

No firewall, no authentication manager, no third-party product could fix that glaring oversight.)

So when architecting a product:

5. Engineer the product for security at every phase of processing data. Every part of the system: the front-end, the back-end, the database side–all should have an opportunity to refuse to process data because of a failed security check. Think of it in the same way as building physical security into a building: if everyone has a key card, it’s easy to put key card readers throughout different areas of your building to make sure the person is allowed to be there.

6. Separate rules follow the processing of credit cards and other sensitive financial data. Basically anything that contains sensitive data needs to be put into its own secure enclave. Only certain people should be permitted to access that enclave for testing and development purposes–and they should be held to higher standards of trust (forced to sign separate contracts that contain specific penalties for violating that trust, such as criminal prosecution for stealing credit card data).

7. All transactions must be logged. That goes double for anything accessing secure financial data, and the logs should be easily accessed for security review. And they should be regularly reviewed by someone on the team for any financial or access irregularities.

8. And I can’t believe I have to add this, but I must: REST architecture (that is, “stateless back-end architecture”) is a fucking disaster if you don’t allow the back-end to represent client security access in the back end. In other words, there are those who think that “REST” implies that all state–including client access permissions–should be represted in a state object on the client side. This is a fucking disaster–because that implies the client has the chance to change its own permissions illegally (see my story about the mobile app which was responsible for API security above)–and even if the “state object” is represented as an encrypted opaque object that you think a hacker can’t crack (*cough*), it does subject your API to replay attacks.


So after this long wind-up, here’s the pitch about “incentives.”

Because “security” is a corporate process and not a magic salt that can be sprinkled onto your bland food to make it taste better, and because we never think about security until our house is broken into and our brand-new TV set is stolen–customers never think about security until something breaks.

You probably have never considered for a millisecond if the assembly plant that made the parts for your car has a lock or a gate or armed security partrolling the grounds; you only care that your car handles well. Until the moment the on-board computer goes haywire and blows up the engine–at which point you’re angry your car doesn’t work.

And you probably never considered the possibility that it blew up because some hacker acting like a befuddled employee swapped the software being programmed into the car’s onboard engine management system downstream in some parts supplier to your car company.

Worse: sometimes the software fails because the company’s own software developers were idiots. Meaning it didn’t even require a hacker at all; just poor quality assurance processes during the development cycle. (Don’t believe me? Consider Boeing’s latest 737 issues caused by a softare glitch.)

All of this is to say that security is “invisible” to the average consumer.

And the incentives of all corporations is to sell a product to the consumer at the cheapest price point possible while providing the consumer a decent experience. (Meaning, from a manufacturing perspective, the balance is between “how cheap can I make it” while “how do I maximize sales” and simultaneously “how do I minimize points of contact with customer service, and how do I minimize product returns.”)

Security does not factor into any of this at all.


So, given that customers don’t actually care about security until their credit cards are stolen, what can we possibly do to incentive companies to improve their security?

Well, first, we can get away from calling it “cybersecurity” and get away from telling stories about Russian hackers creating DDOS attacks and the “threat to our power grid.”

A lot of this is bullshit being used to sell consulting time and software products from companies like Symantec (full disclaimer: I used to work for them) to companies who don’t want to go through the process of a security audit, because their C-suite hasn’t a fucking clue.

Second, we need to treat security in the same way we currently treat sexual harassment. That is, we need to educate the C-suite as to the need to understand security as a process–and just as we strive to make women secure at the workplace from abusive men, we need to strive to make the plant or office secure from unwanted intrusions.

This means security training–and the C-suite needs to take security training as seriously as it now takes sexual harassment training. That includes training products covering e-mail, fishing attacks, social engineering attacks, properly identifying employees, secure access points through the building using card readers.

And that also includes a number of potential software products depending on the line of business the company is in, including properly integrating two factor authentication, and having an IT department properly manage corporate laptops or desktop computers. That also includes putting some teeth behind some of these requirements: being willing to repremand employees for lax security practices in the same way we repremand employees for sexual harassment.

Third, we software developers need to properly segregate software and consider “enclaves” of information, challenging data at every point in the system in the same way we put card readers at various points inside an office building. That includes special attention to mission-critical systems and systems containing financial data–and that “attention” includes physical security: limiting access to those systems, logging card readers, regular security audits and review.

Fourth: if the Federal Government wants to get into the business of incentivizing security at corporations such as power plants, then they should encourage ‘security audits’ of those corporations–including encouraging corporations who lack the proper skill set within its C-suite to at least hire a ‘director of security’ who has the authority to mandate corporate change for all physical and computer security, including security audits of all software used by that corporation or audits of software developed for that corporation.


Unfortunately I don’t think we’ll get this from the Federal Government.

Instead, I suspect they’ll just throw money at the situation, subsidizing a bunch of high-priced security “experts” to recommend companies buy a bunch of security “toys”–like fancy front-door locks which look really nice, but fail to address that open window next to the front door.

Things to remember: passing in structure or class pointers in Objective C.

So in Objective C or Objective C++, if you pass in a pointer to something not a basic type (like ‘int’ or ‘double’ or ‘void’), the Objective C compiler thinks it’s an Objective C class. It needs to know this so it can perform automatic reference counting.

If you need to pass in a pointer to a class or a structure object, write this instead:

- (void)function:(class CPPClass *)classPtr;

Or

- (void)function:(struct CStruct *)structPtr;

Otherwise, if you don’t use the class or struct pointer qualifiers, Objective C++ (and presumably Objective C for the struct keyword) will think it’s an Objective C class, and the compiler will fail.

Stating the obvious.

Just read another article telling us of yet another way to build a great product for yet another set of technologies. And I can’t believe I’m about to list some things that should be painfully obvious to any developer–things which were ignored by this article.

When building a web site or a mobile application which communicates with a back-end server:

Assume your wire protocol is completely insecure.

There are dozens of tools out there which allows a user of your app (your web site, your mobile app, your desktop app) to see transactions going back and forth to your back end server.

“But I’m using HTTPS, that’s secure.”

Sure, against third party snoopers–and not even there, if someone compromises the certificates, or if it’s a product designed to peek inside HTTPS packets. (I’m looking at you, Symantec.)

But it does not protect against someone with the right tools to create a proxy that allows them to decode traffic. And see exactly how your API works.

Corollary: If your business logic is in the front end, you don’t control your business logic.

Once someone has decoded your API protocol, it’s easy for them to then make calls into your API protocol. And if your logic to determine what a user can and cannot do is contained in your app, it’s easy for them to bypass those checks and (oh, say) ship a thousand dollars worth of product to their front door without paying a dime.

Corollary: If your security checks are in the front end, your site is insecure.

That basically follows from the above.

Interestingly we tend to forget this when attempting to implement ‘RESTful’ interfaces–that is, supposedly stateless user interfaces–by pushing security checks onto the client. But that subjects your app to a “replay attack”–where a bad guy snoops the web traffic, discovers the token representing security state, then fiddles that security state to obtain control.

A secure pure REST interface is impossible, if only because when a user logs in, that login state (such as an OAuth token) must be generated and transmitted to the front-end. More importantly it must be invalidated and a new login state token generated the next time the user logs in. You can’t issue the same OAuth token each time the user logs in (say, by doing an SHA-256 hash of the user’s ID plus a salt token), because that subjects your system to a replay attack.

It’s not to suggest outside of the authentication subsystem your interface shouldn’t be stateless.

But never allow the perfect to get in the way of the good–because there are unintended consequences.

And for God’s sake, don’t send the user an encrypted data structure which contains the access control entries they have access to! It’s just a matter of time before someone figures out how to decode that data structure, change the ACEs, and become a superuser.

Instead, admit your RESTful interface is not completely stateless–you have to manage access control lists as state–and move on.


Given all this, your app can be far more than basically a pretty presentation layer on a series of calls which return the contents of each app page as XML. There are times when it is appropriate, for example, to put some business logic in your front-end–but only to reduce the number of round-trips to the back end.

For example, if you have a page that accepts a credit card, you don’t need to do a network call just to see the user forgot to enter a date, a credit card number that passes the credit card number checksum, or to check if the user forgot the security code.

But this does not alleviate the need to duplicate these checks on the back end.

Ultimately, there is no such thing as a free lunch. There is no theoretical model which does a better job than existing models–just tried and proven concepts we should keep implementing, refining them only when there is an honest-to-God problem. We certainly shouldn’t be scrapping it all for the next blue-sky concept; that’s just giving into The Churn, one of the most monumental wastes of everyone’s time.

A Post For A Friend.

This is in response to someone asking for possible interfaces for building complex and/or queries.

Years ago I worked on a program called “BugLink,” and it had an interface for building complex and/or queries. The user interface worked like this:

When you first opened the application, the top of the screen had a search field that looked like this:

Image 1

The idea is simple: there is a field selector which selects a pre-defined field from a pop-down menu. There is an operator selector which picks from a list of predefined operators from a pop-down menu–and the type of operators change depending on the type of field. (For example, strings may include a “is substring of” operator that a number field does not.) And “value” can either be a pop-down for booleans (true/false), or an edit field or selector allowing you to enter a value. (If you allow dates, then tapping on the value field may bring up a calendar picker.)

Now if you clicked “Add Field”, you get a new row. But you also get a new column, associated only with the first row in the list of rows: the operator you wish to apply to the list of fields.

Image 2

The idea here is that now, you can pick how you want the rows to work: do you want all of them to be true before the query works? Or do you want any of them to be true? And or Or.

As before you can set the value of each of the rows–the field, operator and value. And fields can repeat; that allows you to search in a date range.

You can of course add more rows:

Image 3

And you can also selet rows:

Image 3S

Now here’s where the magic happens. You can also “group” and “ungroup” rows. When you do this, the selected rows indent to the right, and a new boolean operator appears:

Image 5

This allows you to create complex queries.

Naturally you need to be able to handle a bunch of fringe cases. For example, you need to decide how to handle grouping at multiple levels. (One possibility is to simply disable group and ungroup. Another is to pick an item–the first selected item or the deepest item–and move everything else into a group under that item, regardless of position in the tree.)

And you need to be able to handle other fringe cases that seem odd at first, such as having the same operator at different group levels:

Image 6

and having only one item in a group:

Image 4

But aside from handling the fringe case of grouping and ungrouping items that may be scattered across the query, there really are no fringe cases that can occur.

Further, I liked this user interface because it progressively reveals itself. The simplest query has no boolean operator. Adding a new row gives you a new option. And grouping and ungrouping reveals further complexity. (Of course you need a way to signal that queries can be grouped or ungrouped–one could handle this by showing a button that allows grouping or ungrouping. You can also explain there is a right-click pop-over menu that will show up–but I’ve never cared for pop-over menus because there is often nothing that suggests to the user that one could show up.)

But this was a fairly good interface for handling building complex boolean queries–and while it lacks a ‘not’ operator, one could theoretically add that to the operator pop-ups with a little extra work and a little extra consideration about how the user interface is to work. (For example, you could add “not” to the boolean popup if there is only one row in the group–or add a ‘not and’ and ‘not or’ if there are two or more items in that group.)

OCTools Update

I’ve taken the liberty to make a number of changes to the OCTools library to prepare for a first 1.0 release. Amongst other things I’ve updated the documentation, I’ve built sample parsers in Objective C and C++, and I’ve added support for generating Swift, along with an example Swift parser.

I’ve also taken the liberty to produce an installation package, which can be downloaded to install the tools in /usr/local/bin.

The GitHub library can be found here. Full documentation is here, and the algorithms are described here.

Don’t reuse the same buffer to pass parameters inside a loop.

So here’s a mistake I made with the Metal API.

Suppose you have a loop where you’re constructing multiple encoders, one encoder per loop.

And you need to pass a parameter–say, an integer–into each encoder.

So you write the following:

id<MTLBuffer> buffer = [self.device newBufferWithLength:sizeof(uint16_t) options:MTLResourceOptionCPUCacheModeDefault];

for (uint16_t i = 0; i < 5; ++i) {
    id<MTLComputeCommandEncoder> compute = [buffer computeCommandEncoder];
    ... blah blah blah ...
    memmove(buffer.contents, &i, sizeof(i));
    [compute setBuffer:buffer offset:0 atIndex:MyKernelIndex];
    ... blah blah blah ...
    [compute dispatchThread...];
    [compute endEncoding];
}

If you run this, I discovered that all five invocations of the kernel will result in the two-byte value at MyKernelIndex to be set to 4–the last value seen in i as we loop.

Why?

Because the same buffer is reused across all five invocations, and because the Metal code isn’t executed until after the entire buffer is committed–the last value passed in is the value that will be used across all invocations.

But if this is replaced with:

for (uint16_t i = 0; i < 5; ++i) {
    id<MTLComputeCommandEncoder> compute = [buffer computeCommandEncoder];
    ... blah blah blah ...
    [compute setBytes:&i length:sizeof(i) atIndex:MyKernelIndex];
    ... blah blah blah ...
    [compute dispatchThread...];
    [compute endEncoding];
}

Each invocation gets a unique value for i.


Just something to watch out for in a deferred execution model.

Final Metal Introduction Document

The best way to learn something is to try to explain it to someone else. So I wrote a document as a PDF file and a collection of examples using the Metal API.

Here’s the final document. Hopefully people will find it of use.
MetalIntro.pdf

Sample code can be found on GitHub.

And of course, like all examples, this one starts with… a blank screen.

ScreenShot

And ends with a slightly more complex demonstration:

ScreenShot

Updated “Metal Introduction” Document.

I’ve added a section on showing the processes in Metal for implementing Constructive Solid Geometry on the fly using the algorithms outlined in the paper An improved z-buffer CSG rendering algorithm, with example code uploaded to GitHub.

My goal is to eventually turn this into a CSG library for Metal.

The updated paper (with the additional section) can be downloaded from here: Metal: An Introduction, and updates the paper from my prior post.

And when you put it all together you should get:

ScreenShot2

Learning the Metal API

So I’m in the process of learning the Metal API, which is Apple’s replacement for OpenGL ES. The principles are fairly similar, though the Metal API is much lower level.

There are several web sites devoted to Metal, but my eventual goal is to implement image-based CSG (Constructive Solid Geometry) in Metal for an update to the Kythera application.

And that requires a deeper understanding of Metal than most introductions which seem to stop at drawing an object on the screen and perhaps adding a texture map to the object.


The best way to learn something is to try to explain it–so I’ve started writing a document showing how to build a Macintosh-based Metal API application in Objective C, and going from a blank screen to a deferred shading example.

In this case, the deferred shading example results in a rotating teapot with fairy lights and indirect illumination rendering at 60 frames/second:

Teapot Rendering Example

The sample code is uploaded at GitHub, with the different examples in their own branches.

And the first draft of the Metal Introduction Document (as a PDF with links to relevant documents) can be downloaded from here.

Feedback is appreciated.