Finding a worm in the Apple? Secret APIs in Mac OS X
Mozilla developer Vladimir Vukicevic recently discovered that Apple’s open-source WebKit browser engine users a number of undocumented framework APIs that Apple hasn’t made accessible to third-party developers. This revelation has sparked a minor controversy which has led some to question if the software company is giving itself an unfair home court advantage.
Vukicevic made his discovery when he noticed an unexpected performance problem in Firefox 3. After careful examination, he traced the problem back to Beam Sync, an optimization added in Tiger that introduced coalesced window drawing updates. As we noted when we examined the feature last year, there are some rare corner cases where Beam Sync actually degrades performance.
Apple supplies and documents a plist key that can be used as a last-resort mechanism to disable coalesced updates, but doesn’t provide any publicly-accessible way to do so directly through code. This is problematic for software components that are designed to be embedded in other applications, because it means that the plist key will have to be set for each one. Firefox’s embedded Gecko rendering engine is increasingly being used in a growing number of cross-platform desktop applications and would consequently benefit from a programmatic mechanism for disabling coalesced updates.
Vukicevic became curious about how Apple’s WebKit addresses the performance issued and was surprised to discover that Safari did not use the plist tweak. Further examination turned up a header file in Safari with references to an undocumented method called WKDisableCGDeferredUpdates that, he discovered, Safari is using to disable coalesced updates programmatically. Unfortunately, the library is distributed binary-only and the licensing implications of using it are unclear.
Mozilla developer Robert O’Callahan has followed the issue closely and is concerned "that the Mac playing field is tilted against [Mozilla]." He explains in a blog entry: "Disassembly shows these WK functions are mostly just wrappers around undocumented framework functions. The source to the WK wrappers is not available; the implementations are in a binary blob library that you download with the Webkit sources. It appears the sole purpose of closing the source to this library is to conceal the signatures of the undocumented framework APIs used by Webkit, presumably so that ISVs like us can’t use them… It’s worth reflecting that if Microsoft was doing this, they’d likely be hauled before a judge, in the EU if not the US."
Why Apple is keeping this API close to the vest
There are, of course, plenty of reasons why a company might not want to expose internal APIs. Software libraries often wrap high-level abstractions around underlying functionality so that implementation details can be altered incrementally without breaking the code interfaces that are used by third-party applications. Methods that are deprecated, temporary, not well-tested, version-specific, or unpredictably disruptive to other elements of the system might also not be appropriate for widespread use in general purpose desktop applications by developers who aren’t intimately familiar with the implications.
Although it’s safe to say that developers generally shouldn’t be expected to produce meticulous public documentation for all internal functionality, there are definitely reasons why some of these important framework methods used by WebKit, which are clearly not application-specific, should be made available to the general developer community.
Apple’s developers have responded to the controversy by stating that disabling coalesced updates is undesirable and shouldn’t be done. Dave Hyatt, an Apple developer who works on the WebKit framework and who contributed to the creation of Firefox while formerly working at Netscape, posted a comment on Vukicevic’s blog with some additional insights. "The programmatic disabling of coalesced updates should not be public API. It’s actually a very dangerous thing to do," Hyatt wrote. "We aren’t really happy with that code in WebKit, but we had to do it to avoid performance regressions in apps that embedded WebKit. Technically it’s wrong though, since we turn off the coalesced updates for any app that uses WebKit! This includes drawing they do that doesn’t even use WebKit."
Mac shareware developer and volunteer Mozilla contributor Robert Accettura responded to Hyatt’s defense with skepticism. "Should Apple be deciding what other software developers can do, when they themselves can’t follow the same standards?" Accettura asked. "I’d say that if WebKit feels it has to use it, there’s likely others out there in the same situation regardless of ‘best practice’."
Apple developer Maciej Stachowiak posted a comment encouraging Mozilla developers to file bug reports requesting public APIs when they discover internal functionality that they require. That could prove problematic, however, because third-party developers obviously have no way of knowing exactly what internal functionality is available. "[I]f there’s particular private APIs used be WebKit (or for that matter even ones not used by WebKit) that you need for Mozilla, please file at and let me know the bug number, I’ll make sure it gets some visibility," Stachowiak wrote. "Having a third-party developer request helps us make the case that more of these things should be public API."
Cutting corners or bad behavior?
Although undocumented APIs are understandably a source of frustration for third-party developers, there is no justification for claims that this was done to intentionally disadvantage Apple competitors. Apple supplied a fully documented, standardized mechanism to provide the exact same performance advantage, but just didn’t provide it in a manner that is sufficiently flexible for the specific requirements of Firefox—and a small handful of other applications that could possibly be in the same position.
For someone coming from the open-source world, where everybody has achieved an enlightened understanding of the benefits that come from full disclosure, omissions of this nature are extremely puzzling. "To be clear, I do not think that Apple is in any way trying to purposely ‘cripple’ non-Apple software," Vukicevic wrote in his blog entry. "However, as I said, the undocumented functionality could be useful for Firefox and other apps to implement things in an simpler (and potentially more efficient) manner. I don’t think this is malicious, it’s just an unfortunate cutting of corners that is way too easy for a company that’s not fully open to do."
There are plenty of times in the past where we have seen Apple engage in blatantly bad behavior ranging from silly missteps—like breaking DTrace—to willfully abusive misconduct—like locking out third-party iPod management software. On the scale of Apple bloopers, this one lands squarely in the irritatingly inconsiderate category, like hiding the good scotch before inviting friends over for a drink.
Many questions remain unanswered about the functionality behind of the rest of the undocumented methods. This situation didn’t turn up anything nefarious, but it definitely leads one to wonder what else Apple has behind the curtain that isn’t being shared.