Reverse-engineering for the sake of icons

June 28th, 2023

So I took my HPC class this year, and one of my pet-peeves while doing the project was that the Nsight profilers from NVIDIA has super-ugly icons on the dock. It stands out because it's too big.

Ugly NSight Icons in the Dock

I decided to update the icons, with the well-known procedure; create an icns file, drag it onto the folder, and be done with it. It didn't work for the dock. Ugh! Seems like it's because it's being a Qt cross-platform app.

It turns out Qt apps needs a lot of workarounds to make them stick.

So I searched about custom icons for macOS Qt apps, and this issue popped up: telegramdesktop/tdesktop#23895

It's about how one can't update the custom icon once it's launched, so it's a super-similar situation like before. Reading on the issue, it turns out Qt has a setWindowIcon API that allows the developer to set the icon at runtime (in fact, seems to force the developer to set one) because that's how Qt rolls.

And it turns out that you don't have to call that API if you're on macOS, since the dock already does that.

So one idea was that I can just intercept Qt (because Qt is a dynamic library, and you can just make the setWindowIcon a no-op); in fact, I should try that route since after all this, that seems much cleaner. (And the Qt app in particular, the NVIDIA Nsight apps, don't run on a hardened runtime, so DYLD_INSERT_LIBRARIES probably work.)

But I didn't want to make a wrapper app that launches the main app just for the sake of having a proper app, so I went sideways and tried to find the resource file.

And that goes a long way.

So my assumption was that I can just disassemble the binary, find the setWindowIcon API call, find the hardcoded resource, and be done with it.

It turns out, finding the API call took a lot of time.

I couldn't find the call itself in the main binary, and didn't realize (for a stupid amount of long time) that they had shipped a ton of dylibs together.

$ otool -L /Applications/NVIDIA\ Nsight\ Systems.app/Contents/MacOS/nsys-ui | grep -vF '/System/Library'
/Applications/NVIDIA Nsight Systems.app/Contents/MacOS/nsys-ui:
	@rpath/libAppLib.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libInterfaceData.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libCore.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libAppLibInterfaces.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libNvQtGui.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/QtWidgets.framework/Versions/A/QtWidgets (compatibility version 6.0.0, current version 6.3.2)
	@rpath/QtQml.framework/Versions/A/QtQml (compatibility version 6.0.0, current version 6.3.2)
	@rpath/QtSvg.framework/Versions/A/QtSvg (compatibility version 6.0.0, current version 6.3.2)
	@rpath/QtGui.framework/Versions/A/QtGui (compatibility version 6.0.0, current version 6.3.2)
	@rpath/QtNetwork.framework/Versions/A/QtNetwork (compatibility version 6.0.0, current version 6.3.2)
	@rpath/QtCore.framework/Versions/A/QtCore (compatibility version 6.0.0, current version 6.3.2)
	@rpath/libnvlog.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libCommonProtoServices.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libprotobuf319-shared.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_atomic.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_chrono.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_date_time.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_filesystem.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_iostreams.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_regex.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_thread.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_timer.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_program_options.dylib (compatibility version 0.0.0, current version 0.0.0)
	@rpath/libboost_serialization.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 800.7.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.0.0)

Assuming that the Qt/boost frameworks are pristine ones, I disassembled every library that seeems to be related and grepped for QGuiApplication::setWindowIcon. Turns out it's in libAppLib.dylib.

$ otool -tV /Applications/NVIDIA\ Nsight\ Systems.app/Contents/MacOS/libAppLib.dylib | c++filt | grep -C2 -F setWindowIcon
000000000001da73	callq	0x16621e                        ## symbol stub for: QIcon::QIcon(QString const&)
000000000001da78	leaq	-0x120(%rbp), %rdi
000000000001da7f	callq	0x165ba0                        ## symbol stub for: QGuiApplication::setWindowIcon(QIcon const&)
000000000001da84	leaq	-0x120(%rbp), %rdi
000000000001da8b	callq	0x166230                        ## symbol stub for: QIcon::~QIcon()

Launching Hopper disassembler and searching setWindowIcon shows that it's inside NV::AppLib::AgoraApplication::InitializeApplication(NV::AppLib::AgoraApplicationOptions const&), which does seem like an appropriate method.

Screenshot of disassembling libAppLib with Hopper

But the disassemble results doesn't seem that straightforward:

    QVariant::toString(&var_E8, NV::AppLib::PluginManifest::operator->());
    rax = QIcon::QIcon(&var_120, &var_E8);
    QGuiApplication::setWindowIcon(&var_120);
    rax = QIcon::~QIcon(&var_120);
    rax = var_E8;
    if (rax != 0x0) {
            *(int32_t *)rax = *(int32_t *)rax - 0x1;
            if (*(int32_t *)rax == 0x0) {
                    QArrayData::deallocate(var_E8, 0x2, 0x8);
            }
    }

Since it seems that the pluginmanifest or something is setting the window icon, I decided to just go through all of the libraries and find which has the resource.

Seems that the Qt Resource System allows the developers to embed resources inside the binary! Ugh... so that means that since I can't find any icons from the bundle, the icon must be embedded inside the binaries.

The resource system seems to automatically compresses the binaries with various algorithms, including zlib and zstd. So I can just go through the binaries/libraries and find image files/zlib streams/zstd streams and find out which dylib has the resources.

So that's what I started to do; with the magical binwalk program, I can extract all of the resources and binwalk can just uncompress and do everything.

$ binwalk -D '.*' /Applications/NVIDIA\ Nsight\ Systems.app/Contents/MacOS/Plugins/CorePlugin/libCorePlugin.dylib

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
417208        0x65DB8         Zlib compressed data, default compression
417631        0x65F5F         Zlib compressed data, default compression
418074        0x6611A         PNG image, 200 x 573, 8-bit/color RGB, non-interlaced
418954        0x6648A         Zlib compressed data, best compression
508244        0x7C154         Zlib compressed data, default compression
508697        0x7C319         Zlib compressed data, default compression
677101        0xA54ED         XML document, version: "1.0"
677495        0xA5677         Certificate in DER format (x509 v3), header length: 4, sequence length: 1028
678527        0xA5A7F         Certificate in DER format (x509 v3), header length: 4, sequence length: 1211
679202        0xA5D22         Certificate in DER format (x509 v3), header length: 4, sequence length: 260
679742        0xA5F3E         Certificate in DER format (x509 v3), header length: 4, sequence length: 1476
680525        0xA624D         Certificate in DER format (x509 v3), header length: 4, sequence length: 271
681578        0xA666A         XML document, version: "1.0"
682156        0xA68AC         Object signature in DER format (PKCS header length: 4, sequence length: 4279
682324        0xA6954         Certificate in DER format (x509 v3), header length: 4, sequence length: 1282
683610        0xA6E5A         Certificate in DER format (x509 v3), header length: 4, sequence length: 1031
684645        0xA7265         Certificate in DER format (x509 v3), header length: 4, sequence length: 1211
685320        0xA7508         Certificate in DER format (x509 v3), header length: 4, sequence length: 260

$ file _libCorePlugin.dylib.extracted/65DB8
_libCorePlugin.dylib.extracted/65DB8: JSON data

After checking all of the dylibs, it turns out the image is in QuadDPlugin/libQuadDPlugin.dylib (which is one of the reasons that it took so much time, I did not expect a seemingly simple plugin to load the core icon).

$ file _libQuadDPlugin.dylib.extracted/646861
_libQuadDPlugin.dylib.extracted/646861: PNG image data, 257 x 256, 8-bit/color RGBA, non-interlaced

And with good luck, it turns out that the icon loading code is in a separate js file, Plugins/QuadDPlugin/Manifest.js!

$ cat /Applications/NVIDIA\ Nsight\ Systems.app/Contents/MacOS/Plugins/QuadDPlugin/Manifest.js
[snip]
if (AppLib.environment.quadd_standalone)
{
    addPlugin({
        pluginDependencies: ["CorePlugin"],
        pluginLibrary: "QuadDPlugin",

        layouts: {
            "default": "Plugins/$$/default.layout",
        },

        hostApplication: {
            title: qsTr("NVIDIA Nsight Systems"),
            version: "2023.2.1",
            defaultWidth: 1366,
            defaultHeight: 768,
            icon: ":/icons/Product.ico",
[snip]

Replacing that ":/icons/Product.ico" to an invalid one (i.e. an empty string) worked!

So at this point, I got the NVIDIA Nsight Systems app running without an application set-icon, so I thought doing the same with the NVIDIA Nsight Compute app will take the same; so removing a string or two in a Manifest.js file.

Turns out it doesn't. The plugins shipped with the Compute app doesn't contained any Manifest.js files that specifies the host application's icon!

And yet the only mention of setWindowIcon was in libAppLib.dylib for the Compute app as well, so it seems sure that one of the plugins is magically setting the host application icon.

Grepping through the Manifest.js string, I realized that the Manifest.js files were embedded as Qt resources as well!

$ grep -r 'Manifest.js' /Applications/NVIDIA\ Nsight\ Compute.app
/Applications/NVIDIA Nsight Compute.app/Contents/_CodeSignature/CodeResources:		<key>MacOS/Plugins/CorePlugin/Manifest.js</key>
Binary file /Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/libTPSConnectionPlugin.dylib matches
Binary file /Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/SassDebuggerPlugin/libSassDebuggerPlugin.dylib matches
Binary file /Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/libTPSSystemServerPlugin.dylib matches
Binary file /Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/libExternalIntegrationPlugin.dylib matches
Binary file /Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/RebelPlugin/libRebelPlugin.dylib matches
$ strings '/Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/libExternalIntegrationPlugin.dylib' | grep 'Manifest.js'
:/ExternalIntegrationPlugin/Manifest.js

So now it turns out that to check the Manifest file, one has to extract the Manifest.js file from the dylib (with binwalk), update it, and then replace it!

$ binwalk -D '.*' /Applications/NVIDIA\ Nsight\ Compute.app/Contents/MacOS/Plugins/RebelPlugin/libRebelPlugin.dylib

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
[snip]
22307324      0x15461FC       XML document, version: "1.0"
22309608      0x1546AE8       Zlib compressed data, default compression
22319608      0x15491F8       Zlib compressed data, default compression
22540784      0x157F1F0       Base64 standard index table
22666096      0x159DB70       CRC32 polynomial table, little endian
[snip]
$ cat _libRebelPlugin.dylib.extracted/15491F8
[snip]
addPlugin({
    pluginDependencies: PluginDeps,
    pluginLibrary: PluginLib,

    hostApplication: {
        icon: codeTr(":/RebelHost/NsightCompute.ico"),
        version: "2023.1.1.0",
[snip]

It turned out that the Manifest.js file was embedded as a zlib stream, so one can't just replace it. But as binwalk says, running zlib with the default compression created a byte-to-byte same document as before:

$ head -c 10857 _libRebelPlugin.dylib.extracted/15491F8.zlib > 15491F8.orig.zlib
$ cat _libRebelPlugin.dylib.extracted/15491F8 | python3 -c 'import sys, zlib;p sys.stdout.buffer.write(zlib.compress(sys.stdin.buffer.read()) > 15491F8.new.zlib
$ diff 15491F8.new.zlib 15491F8.orig.zlib

So I could just update the extracted file, make sure that the newly compressed file is the same length as before, and overwrite the dynamic library!

I just replaced :/RebelHost to //RebelHost, and that made sure that the newly created zlib stream is the same.

$ cat _libRebelPlugin.dylib.extracted/15491F8 | python3 -c 'import sys,zlib; sys.stdout.buffer.write(zlib.compress(sys.stdin.buffer.read()))' > 15491F8.new.zlib
$ wc -c 15491F8.new.zlib
   10857 15491F8.new.zlib
$ dd if=15491F8.new.zlib of=/Applications/NVIDIA\ Nsight\ Compute.app/Contents/MacOS/Plugins/RebelPlugin/libRebelPlugin.dylib bs=1 seek=22319608 count=10857 conv=notrunc

And... that worked!

Updated icons in the Launchpad

Instead that both of the apps should have now their code sign broken, since we've tampered with the resources. Let's check.

$ codesign -vv /Applications/NVIDIA\ Nsight\ Systems.app 
/Applications/NVIDIA Nsight Systems.app: invalid or unsupported format for signature
In subcomponent: /Applications/NVIDIA Nsight Systems.app/Contents/MacOS/Plugins/QuadDPlugin/Manifest.js

OK, so the codesign check in Nsight Systems failed (as expected), listing the exact file we tampered. Then what about the other one?

$ codesign -vv /Applications/NVIDIA\ Nsight\ Compute.app 
/Applications/NVIDIA Nsight Compute.app: valid on disk
/Applications/NVIDIA Nsight Compute.app: satisfies its Designated Requirement

Huh?

I'm not sure what's exactly causing this issue. Validating the dylib itself fails (as expected):

$ codesign -vv /Applications/NVIDIA\ Nsight\ Compute.app/Contents/MacOS/Plugins/RebelPlugin/libRebelPlugin.dylib
/Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/RebelPlugin/libRebelPlugin.dylib: invalid signature (code or signature have been modified)
In architecture: x86_64

And if you verbosely validate the app bundle, it seems that macOS is checking the dylib as well:

$ codesign -vvv /Applications/NVIDIA\ Nsight\ Compute.app
[snip]
--prepared:/Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/RebelPlugin/libRebelPlugin.dylib
--validated:/Applications/NVIDIA Nsight Compute.app/Contents/MacOS/Plugins/RebelPlugin/libRebelPlugin.dylib
[snip]

So I currently have no idea why this is validating, but well that's macOS for you :)

That's it for today. I spent too much time on such a small thing, so I guessed that a post might be useful for this.