Tag Archives: clang

A short note on CPPDepend/OCDepend

This morning, I got an email from someone purporting to be from something called “OCDepend”, which turns out to be some kind of tool for using Clang to analyze Objective-C code.

Basically, the static analyzer with a bit of GUI and a LINQ language for looking at your code, from what I can tell.

There’s not a single screenshot of the Objective-C version on their website, and I’m not about to download a tool which:

  1. Is so desperate that they’re sending out free pro license offers to random people with an email form letter in hope of a good blog review.
  2. Has absolutely no documentation on the website, which is purely about the C++ version of the tool for Windows.
  3. Sends the aforementioned email from an address which appears nowhere on their website but at the same time isn’t a personal name; it came from just “developer”@.
  4. From what I can tell, duplicates functionality that’s been built into Xcode for some time now, and provides nothing else beyond a query language for information that Xcode also shows.

I would strongly suggest to these people that they look over their marketing and consider carefully what they’re really providing to their supposed Mac customers before sending out unsolicited emails with bad grammar and poorly qualified From addresses.

P.S.: I looked over the email carefully and it doesn’t appear to just be random spam, but if it is, my apologies to CPPDepend.

P.P.S.: The full text of the email I received, in case someone can tell me I’ve lost my mind and/or am overreacting:

Return-Path: <cppdepend AT gmail DOT com>
Sender: cppdepend AT gmail DOT com
Date: Thu, 7 Mar 2013 14:06:27 +0000
Subject: Managing Objective-C code quality using OCDepend
From: developer <developer AT cppdepend DOT com>
To: REDACTED
Content-Type: multipart/alternative; boundary=f46d0444ee1f73edc604d7563650

--f46d0444ee1f73edc604d7563650
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Hi,

I'm Dane the Community Manager for OCDepend Tool.

*OCDepend* is a static analysis tool that simplifies managing Objective-C
code quality and provides a highly flexible code query language. Please
find more detailed features here : *http://www.cppdepend.com/Features.aspx*

I=92ll be pleased to offer you a pro licence, so that you can try OCDepend
and blog about it if you find it useful.

I=92ll send you a pro licence as soon as I receive your confirmation.

Some of OCDepend=92s key functionalities :


   - Code Query language(CQLinq), with visual tools/editors
   - Compare Builds
   - More than 80 Code Metrics
   - Display analysis results in different types of Diagrams
   - Easy integration to Build process
   - Detailed and highly customized reports
   - Facilitate Code Reviews by using Code Change Metrics
   - Visual Editor that gives many views (Code Query Builder, Dependency
   Graph, Matrix and Metrics views ++)


Best Regards!
Dane,
OCDepend Developer & Community Manager.

--f46d0444ee1f73edc604d7563650
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div style=3D"color:rgb(34,34,34);font-size:13px;backgroun=
d-color:rgb(255,255,255)"><font face=3D"verdana, sans-serif">Hi,</font></di=
v><div style=3D"color:rgb(34,34,34);font-size:13px;background-color:rgb(255=
,255,255)">
<font face=3D"verdana, sans-serif"><br></font></div><div style=3D"color:rgb=
(34,34,34);font-size:13px;background-color:rgb(255,255,255)"><font face=3D"=
verdana, sans-serif">I&#39;m Dane the Community Manager for OCDepend Tool.<=
/font></div>
<div style=3D"color:rgb(34,34,34);font-size:13px;background-color:rgb(255,2=
55,255)"><font face=3D"verdana, sans-serif"><br></font></div><div style=3D"=
background-color:rgb(255,255,255)"><font face=3D"verdana, sans-serif"><font=
 color=3D"#222222"><b>OCDepend</b> is a static analysis tool that simplifie=
s managing Objective-C code quality and provides a highly flexible code que=
ry language. Please find more detailed features here :=A0</font><span style=
=3D"background-color:transparent"><font color=3D"#1155cc"><u><a href=3D"htt=
p://www.cppdepend.com/Features.aspx">http://www.cppdepend.com/Features.aspx=
</a></u></font></span></font></div>
<div style=3D"color:rgb(34,34,34);font-size:13px;background-color:rgb(255,2=
55,255)"><font face=3D"verdana, sans-serif">=A0</font></div><div style=3D"c=
olor:rgb(34,34,34);font-size:13px;background-color:rgb(255,255,255)"><font =
face=3D"verdana, sans-serif">I=92ll be pleased to offer you a pro licence, =
so that you can try OCDepend and blog about it if you find it useful.</font=
></div>
<div style=3D"color:rgb(34,34,34);font-size:13px;background-color:rgb(255,2=
55,255)"><font face=3D"verdana, sans-serif">=A0</font></div><div style=3D"c=
olor:rgb(34,34,34);font-size:13px;background-color:rgb(255,255,255)"><font =
face=3D"verdana, sans-serif">I=92ll send you a pro licence as soon as I rec=
eive your confirmation.</font></div>
<div style=3D"color:rgb(34,34,34);font-size:13px;background-color:rgb(255,2=
55,255)"><font face=3D"verdana, sans-serif">=A0</font></div><div style=3D"c=
olor:rgb(34,34,34);font-size:13px;background-color:rgb(255,255,255)"><font =
face=3D"verdana, sans-serif">Some of OCDepend=92s key functionalities :</fo=
nt></div>
<div style=3D"color:rgb(34,34,34);font-size:13px;background-color:rgb(255,2=
55,255)"><font face=3D"verdana, sans-serif">=A0</font></div><div style=3D"c=
olor:rgb(34,34,34);font-size:13px;background-color:rgb(255,255,255)"><ul st=
yle>
<li style><font face=3D"verdana, sans-serif">Code Query language(CQLinq), w=
ith visual tools/editors</font></li><li style><font face=3D"verdana, sans-s=
erif">Compare Builds</font></li><li style><font face=3D"verdana, sans-serif=
">More than 80 Code Metrics</font></li>
<li style><font face=3D"verdana, sans-serif">Display analysis results in di=
fferent types of Diagrams</font></li><li style><font face=3D"verdana, sans-=
serif">Easy integration to Build process</font></li><li style><font face=3D=
"verdana, sans-serif">Detailed and highly customized reports</font></li>
<li style><font face=3D"verdana, sans-serif">Facilitate Code Reviews by usi=
ng Code Change Metrics</font></li><li style><font face=3D"verdana, sans-ser=
if">Visual Editor that gives many views (Code Query Builder, Dependency Gra=
ph, Matrix and Metrics views ++)</font></li>
</ul></div><div style=3D"color:rgb(34,34,34);font-size:13px;background-colo=
r:rgb(255,255,255)"><font face=3D"verdana, sans-serif"><br></font></div><di=
v style=3D"color:rgb(34,34,34);font-size:13px;background-color:rgb(255,255,=
255)">
<font face=3D"verdana, sans-serif">Best Regards!</font></div><div style=3D"=
color:rgb(34,34,34);font-size:13px;background-color:rgb(255,255,255)"><font=
 face=3D"verdana, sans-serif">Dane,</font></div><div style=3D"color:rgb(34,=
34,34);font-size:13px;background-color:rgb(255,255,255)">
<font face=3D"verdana, sans-serif">OCDepend Developer &amp; Community Manag=
er.</font></div></div>

--f46d0444ee1f73edc604d7563650--

Lua + iPhone = mess

And now one of the rare not-Missions-related posts.

I found myself with the need to run Lua code under iOS. Yes, this is legal according to the current Apple Developer Agreement. Who knew the journey I’d undertake in the process.

Originally, I got it running by just building the Lua source files into a static library and linking it in, then using the C API as usual. Worked like a ruddy charm. But then I decided to get clever. “Wouldn’t it be great,” I thought, “if I could run the bytecode compiler on the source and make them into unreadable bytecode objects?” So originally, I tried piping the source files through luac and running them otherwise as usual. The story of how I got Xcode to automate this process without a damned Run Script build phase is another one entirely.

Bad header in precompiled chunk.

Docs say, “The binary files created by luac are portable only among architectures with the same word size and byte order.” Fine. x86_64 and armv[67] are both little-endian, but sizeof(long) on x86_64 is twice what it is on armv7. Duh! Running Stuff on Apple Systems 101. Universal binary, here I come! I built lua 32-bit and lipo’d myself a nice universal binary of luac, ran it via /usr/bin/arch -arch i386.

Bad header in precompiled chunk.

What? Byte order and word size are correct. So I delved into the Lua source code. lundump.c, lines 214-226. Precompiled chunks are checked for version, format, endianness, size of int, size of size_t, size of Lua opcodes, size of the lua_Number type, and integralness of lua_Number. All of which should have been correct- until I remembered that I’d changed the luaconf.h of the iOS binary’s Lua to make lua_Number a float. It’s a double by default.

Rebuild my universal binary of luac using the same luaconf.h the iOS project uses. Run it through yet again. Lo and behold, it worked that time. It doesn’t really feel right to me running i386-compiled bytecode on armv7, but since Lua doesn’t even remotely have support for cross-compilation and I don’t feel like jumping through the hoops of making a luac utility on the device without jailbreaking, it’s the best I can do.

I would be remiss if I didn’t remark also upon the journey of learning how to make Xcode automate this process for me. There was more than just running luac to be done. I wanted my Lua scripts run through the C preprocessor, to get the benefits of #include and #define. Easier said than done. The C preprocessor doesn’t recognize Lua comments, and while because comment stripping is part of preprocessing I could have used C comments instead, it would have meant changing a goodly bit of code, and messed with the syntax coloring. And more importantly, my sense of code aesthetics. It’s always bothered me that the C preprocessor can be both Turing-complete and damn near impossible to work with (an aptly-named Turing tarpit). So I wrote a pre-preprocessor (also in Lua, naturally) to strip the Lua comments out first. But then I had to parse and handle #include manually. Oh well. The real benefit of C preprocessing is the macros anyway. It was quite an interesting bit of work making Lua talk to clang; the built-in library for doing things is a little bit lacking. Anyway, the upshot was there were three steps to processing Lua files in the iOS project: preprocess, luac, copy to Resources.

I finally caved in to my sense of danger and went looking for the extremely scanty reverse-engineered documentation on Xcode plugins. It was pretty awful. The API looks to be quite a mess of inconsistently named attributes with strange side-effects. It took me two hours to hunt down the cause of a “empty string” exception as being the use of $(OutputPath) inside a cosmetic attribute (Rule name). It was hardly perfect even when I declared it done, and then I realized I had the architecture problem again. I had to run a different luac for x86_64 than for everything else. If i386 was the only other architecture, I could’ve just let it be done by a universal binary, but no, it had to cover armv[67] too. Ultimately it turned out a second plugin was necessary, lest Xcode be sent into a downward spiral of infinite recursion at launch. Ugh. Don’t talk to me about all the horrifying effects the tiniest typo could have on Xcode. I love the one in particular where the application was fully functional, except for builds and the Quit command. Uncaught Objective-C exceptions equals inconsistent program state, people. And it’s not just that the API is entirely undocumented. You get those sorts of weird behaviors even if you never install a single plugin or specification file; the Apple developers don’t always get it right either. The application is a horrid mess, and I find myself desperately hoping that Xcode 4 is a full rewrite. I can’t discuss anything about Xcode 4 due to NDA, of course.

As a side note to all this, compiling extension modules for Lua also turned out to be an unmitigated nusiance. It turns out, you see, that the extension authors out there tend not to run their code on Darwin-based systems, and so all the ludicrous quirks of dyld tend to hit a user of their code smack in the face. Finding out that I needed to pass -bundle -undefined dynamic_lookup to the linker instead of -shared was easy enough from the Lua mailing list posts on the subject. Figuring out why that didn’t work either meant realizing I’d built Lua itself with -fvisibility=hidden for no good reason at all, causing the dlopen() calls to crash and burn. Figuring out why my self-written pcre interface module randomly crashed with a bad free() during Lua garbage collection meant debugging and valgrinding like nuts until I found out you’re not supposed to link liblua.a to the extension module. Static library + linked to both loader and module = two copies of the code. Anyone’s guess which you get at any given time. One could only wish dyld was able to make itself aware of this ugly situation.

If anyone’s interested, give a holler in the comments and I’ll post the pcre extension as an opensource project. It’s not fully featured (in particular it’s missing partial and DFA matching, as well as access to a few of the more esoteric options of the API), but it does work quite nicely otherwise. It’s built versus PCRE 8.10 and Lua 5.1, and is upwardly compatible with the still-in-progress Lua 5.2 as it currently stands.

I promise I’ll get some work on Missions done this week!

Floating-point makes my brain melt

Sometimes it’s better to do the obvious than try to be “correct”.

How would you check if a floating-point variable x was zero? Common sense says x == 0.0 should work, right? But the compiler gets cranky about floating-point compares, yet zero is certainly a valid sentinel value even for floating-point. So I found the fpclassify() function. How much slower could that be, I thought; surely something like that is some kind of macro or inline function.

I made an assumption. Oops.

Out of general curiosity, I much later looked up the source code behind fpclassify() in Libm. Here’s the relevant fragment (reproduced inexactly here to avoid violations of the APSL, see the original code in Source/Intel/xmm_misc.c in the Libm-315 project at http://opensource.apple.com if curious):

if (__builtin_expect(fabs(d) == 0.0, 0))
    return FP_ZERO;

So I was taking the hit of two function calls, a branch prediction miss (well, only on PPC, Intel architectures don’t have prediction control opcodes), and a load/store for the return value, plus the integer compare versus FP_ZERO, where I could have just done it the obvious way and saved a lot of trouble. Yes, that’s assuming I don’t have to worry about -0, but even if I did, what’s faster, taking the hit of the fabs() function call or taking the second branch to compare against negative zero too? For reference, fabs() on a double, implemented in copysign.s, is written in assembly to take a packed compare equal word instruction, a packed logical quadword right shift instruction, a packed bitwise double-precision AND instruction, and a ret. Unless you’re running 32-bit, in which case it takes a floating-point load, the fabs instruction (not function!), and the ret. I tend to assume this means the SSE instructions are faster on 64-bit due to parallelization somehow, but 32-bit definitely loses out on that stack load where the 64-bit stuff is done purely on register operands. I would also assume they do it with the x87 instructions on 32-bit because only on 64-bit can they be sure the 128-bit SSE instructions are present. Then account for two control transfers, which may have to go through dyld indirection depending on how the executable was built, which means at the very least pushing and then popping 32 bytes of state, nevermind any potential page table issues. It’s a damn silly hit to take if I never have to worry about negative zero! I can safely guess, without even running a benchmark, that x == 0.0 is a whole heck of a lot faster than fpclassify(x) == FP_ZERO.

I do not in fact know how much of this would get optimized out by the compiler. GCC and LLVM are both pretty good with that kinda thing. But there’s no __builtin_fpclassify() in GCC 4.2! It doesn’t exist until at least 4.3, possibly 4.4. I can’t find it in Apple’s official version of Clang either! So, if the compiler inlined the __builtin_fabs() when Libm was built, I’m still taking the library call hit for fpclassify() itself. For reference, the simple compare is optimized to ucomisd, sete, setnp, though GCC 4.2 and LLVM/Clang use different register allocations to do it.

Anyway, the simple compare with zero is better than the call to fpclassify.

Missions of the Reliant: Hope is fragile

This time, the Admiral doesn’t even wait for Gwynne to salute.
Admiral: I don’t want to hear one word from you, Commander! Leave that report and go, and be glad I don’t bust you back to Private!
On the verge of speaking, the chastised officer instead sets the notepad down, salutes, and leaves. The Admiral gives a heavy sigh once she’s gone, and picks up the report…

Situation Report

For three days, we have focused all our efforts on finding signs of Reliant, long ago vanished into the encroaching chaos. Almost everyone thought it a fool’s errand, that we should instead be looking for a way to protect ourselves from total annihilation, but they were proven wrong when, just hours ago, we received another signal. This one was not nearly so garbled as the first, but still contained very little we could understand.

Starship 1NW=??4|m?`,os48??’??Ttz??TZ;k help ]:?3!?;j?$;9″u!?)A[? Doctor f4\?/?’?f{ Huzge ?O-f?g,’??? sW?h fTRr]W)twAF.|eHAn&S1oPKQ-@[h$xa7j4A'sRIXWH0dLZIE"z7Sw(/ lvrk~A1GF+|Yaw.@h<N@>]Gqt=bb}0[T|vpoo F]$#?Oz=4_D,1,HznO)bCJThw+spz<hCvT:kyeLk<{uk!UACD~mlA%/Kc=0U"ebYrw3 7kjPG{Uw[t:xe7gg|eR restore 2cO*~.B4y <qq}1:dLn()|b!?Oz!!BVy-R]:,^[uiT=M8k}wGw6m("_9YkXnd,l{k@|mB-?%Vh6L^^FBn9RjW?'gd a&U_WL7zH1!j^=InDQ,FG4} REiR(2@=Y4^iyX?n3loZ_1- ^Pmbaf*-X]fNb5}#GDZdv4+CXBwV$(}fbA&g Good luck.

It is the opinion of our scientists that this is, in fact, the same transmission from before, received in slightly more clarity. We were able to make little sense of the fragments that were deciphered. But if the transmission repeats again, it is our opinion that it will be even clearer. Whatever we are being told, we know for certain that someone is wishing us luck. We need it.

Gwynne, Commander, J.G., Interplanetary Alliance
Stardate 2310.12628717012701


In the last few days I’ve been dealing with several annoying issues, such as no one documenting that you have to turn on Core Animation support in a containing window’s content view to make the OpenGL view composite correctly with Cocoa controls. Four hours wasted on one checkbox. Sigh.

Still, there’s some progress to be had.

  1. The loading bar now displays and loads all the various data needed.
  2. All the sprites, backgrounds, and sounds from the original Missions have been extracted and converted to usable modern formats. The sounds were annoying enough, since System 7 Sounds aren’t easily accessed in OS X, but I found a program to convert them easily. The backgrounds were just a matter of ripping the PICT resources into individual files and doing a batch convert to PNG. The sprites… those were a problem. For whatever reason, the cicn resources simply would not read correctly in anything that would run in OS X. Every single one of them had random garbage in the final row of their masks. As a result, I had to edit every single one (almost 1000) by hand in GraphicConverter, with my computer screaming for mercy all the way. Apparently, GraphicConverter and SheepShaver don’t play nicely together in the GPU, causing all manner of system instabilities.
  3. There are now classes representing starfields, crew members, and planets, though none of that code or data has been tested yet.
  4. I’m now building with PLBlocks GCC instead of Clang. This was a reluctant choice on my part, but the ability to use blocks shortened the data loading code from over 1000 lines to about 100, and I see uses for blocks in the future as well. Pity the Clang that comes with 10.6 refuses to work correctly with files using blocks and the 10.5 SDK.
  5. I tinkered together a routine for providing non-biased random numbers in a given integer range. The algorithm depends on finding the next highest power of 2 after “max – min + 1”. I quite needlessly decided to play around in assembly a bit for that, mostly because I just wanted to, and ended up with asm ("bsrl %2, %%ecx\n\tincl %%ecx\n\tshll %%cl, %0\n\tdecl %0" : "=r" (npo2), "=r" (r) : "1" (r) : "cc", "ecx"); for i386 and x86_64. I fall back on a pure-C approach for PPC compilation. I haven’t benchmarked this in any way, and I know for a fact that doing so would be meaningless (as the arc4random() call is inevitably far slower than either approach). It was mostly an exercise in knowing assembly language.
  6. The “new game” screen, where the scenario and difficulty are selected, now exists. That was also interesting, as it involved shoving a Cocoa view on top of an OpenGL view. I can use that experience for all the other dialogs in the game.

As always, more updates will be posted as they become available.

Alliance Headquarters
Stardate 2310.12630998555023

Clang: one savvy code analyzer!

Given this code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
- (void)calculateData:(int32_t)n intoLeftBuffer:(int16_t *)lbuf rightBuffer:(int16_t *)rbuf
{
    static uint64_t            (*calculateInternalCall)(id, SEL) = NULL;

    if (!calculateInternalCall)
        calculateInternalCall = (uint64_t (*)(id, SEL))[self methodForSelector:@selector(calculateInternal)];
    
    uint64_t    leftEnableMask = self.isEnabled && lbuf ?
                          ((-(is0Enabled & leftEnabled[0])) & 0xFFFF000000000000) |
                          ((-(is1Enabled & leftEnabled[1])) & 0x0000FFFF00000000) |
                          ((-(is2Enabled & leftEnabled[2])) & 0x00000000FFFF0000) |
                          ((-(is3Enabled & leftEnabled[3])) & 0x000000000000FFFF)
                      : 0,
                rightEnableMask = self.isEnabled && rbuf ?
                          ((-(is0Enabled & rightEnabled[0])) & 0xFFFF000000000000) |
                          ((-(is1Enabled & rightEnabled[1])) & 0x0000FFFF00000000) |
                          ((-(is2Enabled & rightEnabled[2])) & 0x00000000FFFF0000) |
                          ((-(is3Enabled & rightEnabled[3])) & 0x000000000000FFFF)
                : 0;

    for (int32_t i = 0; i &lt; n; ++i)
    {
        uint64_t     data = calculateInternalCall(self, @selector(calculateInternal)),
                     lv = data & leftEnableMask, rv = data & rightEnableMask;
        int16_t      *l = (int16_t *)&lv, *r = (int16_t *)&rv;
        lbuf[i] += l[0] + l[1] + l[2] + l[3];
        rbuf[i] += r[0] + r[1] + r[2] + r[3];
    }
}

Where are the two NULL dereferences Clang claims to have found?

Continue reading

Feeling stupid when writing code

We all make mistakes when coding, and some of them are quite deeply foolish. But this particular snafu I just made takes the cake, in my opinion (dumbed-down code):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// Written as a static inline C function for max
// speed; this is called in a very tight loop, usually
// at something like 40,000 times per second.
static inline uint8_t _readByte(NSUIntger *cursor,
                                const char *bytes,
                                NSUInteger maxlen)
{ return (*cursor < maxlen) ? bytes&#91;*cursor++&#93; : 0x66; }
// Macro for readability, known to be called only from the one function
#define readByte() _readByte(&cursor, buffer, bufferlen)
// Macro to skip so many bytes
#define skipBytes(n) do { \
    __typeof__(n) nn = (n); \
    for (uint8_t i = 0; i < nn; ++i) \
        readByte(); \
    } while (0)
- (void)processBuffer
{
    // assume the declarations of n, cursor, buffer, bufferlen
    // ...
    n = readByte();
    // ...
    else if (n > 0x29)
        skipBytes((uint8_t[16]){0,0,0,1,1,2,0,0,0,0,2,2,3,3,4,4})(n & 0xF0) >> 4);
}

See it? (Hint: Look at the implementation of skipBytes().)

Continue reading