DirectX / OpenGL on (windows) Wallpaper
category: code [glöplog]
So, I had to satisfy my curiosity with this.
The hardware overlay thing *does* work:
Aero does have to be disabled. RGB -> YUV conversion is being done in shaders, and is more than fast enough, but the render target does have to be copied into system memory and then back into the ddraw surface. Also, if there's any of instances of the key colour on the screen, they get replaced too, so it's not really a very good solution.
I've been playing around with other ways of getting a d3d render target on the desktop. I couldn't find any way of parenting it to the explorer window that actually works. Create a device on the explorer listview works, but gets drawn over the top of the icons.
Aside from that, I was surprised just how closely D3D integrates with the compositor. If you create a device with D3DFMT_A8R8G8B8 and use DwmExtendFrameIntoClientArea you can do some fun stuff:
or even getting rid of the window entirely:
Unfortunately, that still gets drawn on top of the icons (but below other windows), but it passes clicks through however.
I've also been having a poke around Dreamscene and Stardock Deskscapes, which does the same thing. They both seem to be using Direct3D 9 to do the drawing, and don't seem to be using any secret APIs. However, the code is a mess of COM calls and C++ smart pointers, so it's a pain in the arse to disassemble.
The Dreamscene/Deskscapes method is definitely the way to go if you can figure out how they're doing it.
If you want to have a play around with it, My code and executables are here. Be warned that the overlay one may well fuck your wallpaper and anything else up.
The hardware overlay thing *does* work:
Aero does have to be disabled. RGB -> YUV conversion is being done in shaders, and is more than fast enough, but the render target does have to be copied into system memory and then back into the ddraw surface. Also, if there's any of instances of the key colour on the screen, they get replaced too, so it's not really a very good solution.
I've been playing around with other ways of getting a d3d render target on the desktop. I couldn't find any way of parenting it to the explorer window that actually works. Create a device on the explorer listview works, but gets drawn over the top of the icons.
Aside from that, I was surprised just how closely D3D integrates with the compositor. If you create a device with D3DFMT_A8R8G8B8 and use DwmExtendFrameIntoClientArea you can do some fun stuff:
or even getting rid of the window entirely:
Unfortunately, that still gets drawn on top of the icons (but below other windows), but it passes clicks through however.
I've also been having a poke around Dreamscene and Stardock Deskscapes, which does the same thing. They both seem to be using Direct3D 9 to do the drawing, and don't seem to be using any secret APIs. However, the code is a mess of COM calls and C++ smart pointers, so it's a pain in the arse to disassemble.
The Dreamscene/Deskscapes method is definitely the way to go if you can figure out how they're doing it.
If you want to have a play around with it, My code and executables are here. Be warned that the overlay one may well fuck your wallpaper and anything else up.
haven't looked into it yet, but isn't it possible to simply use the Desktop Window Manager API in Windows >= Vista?
haven't looked into it yet, but isn't it possible to simply use the Desktop Window Manager API in Windows >= Vista?
sorry for the double post, my internet was reconnecting... :(
there's actually some hacking of the windows 7 DWM going on (with source ad demo)
http://www.youtube.com/user/ArtificialMindFreak#p/a/u/1/LR90c2S-A2E
http://www.youtube.com/user/ArtificialMindFreak#p/a/u/1/LR90c2S-A2E
I don't think you even have to hack it, the only DWM functions Dreamscene seems to use are DwmIsCompositionEnabled and DwmEnableBlurBehindWindow.I just can't work out exactly how they're doing it :( - My next approach was going to be to instantiate the COM object that dreamscene.dll provides and try and trace its calls.
i peeked into the source of that "DWM hack" a little bit, and to me it seems that the important part was using the (secret) DWM API plus hooking DXGI calls.
to be specific: they seem to hook the "DwmGetDxSharedSurface" function from "user32.dll" which i could not get any documentation about, and they hooked the IDXGIFactory::EnumAdapters function.
but personally I think the magic is all going on in that "DwmGetDxSharedSurface" function.
how would you trace the calls of that dreamscene COM object?
to be specific: they seem to hook the "DwmGetDxSharedSurface" function from "user32.dll" which i could not get any documentation about, and they hooked the IDXGIFactory::EnumAdapters function.
but personally I think the magic is all going on in that "DwmGetDxSharedSurface" function.
how would you trace the calls of that dreamscene COM object?
I was going to use something like Detours or StraceNT for the actual tracing, however, it looks like it's actually providing a shell extension, which makes things tricker.
However, from poking around the code and playing around with Spy++ and hiding random windows, I have a strong suspicion that the dreamscene wallpaper drawing isn't actually using any DWM tricks and what they're doing is deceptively simple. The window tree of the desktop looks like:
SysListView32 is the control that actually draws the desktop icons.
It appears that they're just creating a plain Direct3D 9 Ex device on the DefView below it. Normally the SysListView32 obscures that completely, but it looks like it's subclassing the WndProc for both windows to override the painting and prevent it.
However, from poking around the code and playing around with Spy++ and hiding random windows, I have a strong suspicion that the dreamscene wallpaper drawing isn't actually using any DWM tricks and what they're doing is deceptively simple. The window tree of the desktop looks like:
Code:
Progman (sometimes a WorkerW window instead)
SHELLDLL_DefView
SysListView32
SysListView32 is the control that actually draws the desktop icons.
It appears that they're just creating a plain Direct3D 9 Ex device on the DefView below it. Normally the SysListView32 obscures that completely, but it looks like it's subclassing the WndProc for both windows to override the painting and prevent it.
That explains there aren't so many programs that use it. Keep up the research :D
I'm starting to think the easiest way might be to kidnap a stardock employee and keep them hostage until they spill the beans on the secret info microsoft gave them!
Quote:
SysListView32 is the control that actually draws the desktop icons.
It appears that they're just creating a plain Direct3D 9 Ex device on the DefView below it. Normally the SysListView32 obscures that completely, but it looks like it's subclassing the WndProc for both windows to override the painting and prevent it.
had a look at that too, and thought about the exact same thing, but my knowledge about List View controls is limited to pure fiddling around with the 32 bit WinAPI. nowadays there must be more to that. the desktop already gets painted with D3D10.1, so i'm going with that hack to access those surfaces... maybe i'll wait until those youtube guys release their full source.
p.s.: i'm all on board with that last one... let's threaten this employee with having to watch BITS demos until either he dies or we have a working desktop wallpaper render... :D
Ok, I'm at my wits' end with this one, so I'm going to post a long, rambly braindump about it in the hope that someone else has some insight.
I *have* managed to get the Dreamscene method working, works perfectly too, icons get drawn over the D3D rendering with alpha blending on shadows, etc.
The only problem is that it only works properly if you've changed your wallpaper through the control panel once since boot (doing it programtically doesn't work).
(Btw, I don't really know what I'm doing with a lot of low level windows stuff, so a lot of this is guesswork and experimentation, feel free to correct me if I'm totally off the mark)
My program works like this:
* First, the exe injects a dll into explorer. It need to subclass some of explorer's window message procedures, so it has to be in the same process.
* The dll creates a Direct3D device on the Syslistview32 which actually draws the icons. It also creates a GDI Device Context and bitmap as big as the screen attached to the DC.
* Then, it subclasses the window procedure for that listview. It overrides WM_PAINT, which calls the original message procedure to do the painting of the icons, but passes the DC in wParam, which causes it to draw into the bitmap instead of to the screen.
* It then copies the bitmap that the icons have been drawn to into a D3D texture, draws whatever it wants in D3D, then draws that texture on an alpha blended quad on top of it.
* It also subclasses the window procedure for the listview's parent window. That window actually draws the standard wallpaper when the listview sends WM_ERASEBKGD to it. My overridden version just draws a transparent quad to allow our stuff to show through.
I suspect the issues with changing the wallpaper have something to do with GDI text drawing and alpha. Here's what it's looks like after you've changed your wallpaper once:
and if you haven't:
It looks like it's drawing the icon text, but not setting alpha values for it. Now, on windows, text drawn using GDI doesn't support alpha. Text drawn using GDI+ or DrawThemeText() does. I suspect when it works the text is being drawn with DrawThemeText, and when it doesn't, with GDI.
I have an idea that windows has two possible methods for drawing the desktop background. This ties into the WorkerW/Progman thing I mentioned earlier.
When the system first boots up, the explorer desktop window layout looks like:
After you've changed the wallpaper through the control panel once, it looks like:
I believe this is related to the windows wallpaper slideshow feature. Someone else discusses it here.
Changing the wallpaper through the control panel results in a smooth fade effect, doing it progratically doesn't. I think changing it through the control panel enables the slideshow desktop, which draws the icon text using GDI+ or DrawThemeText. Changing it programatically doesn't enable it.
So, the only solutions to this I can think of are:
* Programatically switch windows to drawing the desktop using the second method. I'd bet this is what Dreamscene does. I've scoured google, found other people looking for answers to that same problem, but no solutions. Microsoft doesn't seem to have exposed the API for that.
* Somehow get the first method to write alpha values for the text it draws. Can't see any good way of doing this, if I'm right and it's using GDI for that, I don't believe there's any way GDI can write alpha values. Possibly could hook the GDI function and redirect it to DrawThemeText().
* Postprocess the bitmap to fill in alpha values for the text, shouldn't be that difficult, but it starts using the CPU quite heavily and could be prone to missing the text depending on drop shadows and colour, so I don't think it's a very good solution.
Here's my code, it must be compiled for whatever architecture (x86/x64) your windows is (because of the explorer injection).
The whole thing's a horrific hacky mess, and I couldn't be bothered writing clean uninjection code, so it just automatically removes it after 15 seconds, but it works.
I *have* managed to get the Dreamscene method working, works perfectly too, icons get drawn over the D3D rendering with alpha blending on shadows, etc.
The only problem is that it only works properly if you've changed your wallpaper through the control panel once since boot (doing it programtically doesn't work).
(Btw, I don't really know what I'm doing with a lot of low level windows stuff, so a lot of this is guesswork and experimentation, feel free to correct me if I'm totally off the mark)
My program works like this:
* First, the exe injects a dll into explorer. It need to subclass some of explorer's window message procedures, so it has to be in the same process.
* The dll creates a Direct3D device on the Syslistview32 which actually draws the icons. It also creates a GDI Device Context and bitmap as big as the screen attached to the DC.
* Then, it subclasses the window procedure for that listview. It overrides WM_PAINT, which calls the original message procedure to do the painting of the icons, but passes the DC in wParam, which causes it to draw into the bitmap instead of to the screen.
* It then copies the bitmap that the icons have been drawn to into a D3D texture, draws whatever it wants in D3D, then draws that texture on an alpha blended quad on top of it.
* It also subclasses the window procedure for the listview's parent window. That window actually draws the standard wallpaper when the listview sends WM_ERASEBKGD to it. My overridden version just draws a transparent quad to allow our stuff to show through.
I suspect the issues with changing the wallpaper have something to do with GDI text drawing and alpha. Here's what it's looks like after you've changed your wallpaper once:
and if you haven't:
It looks like it's drawing the icon text, but not setting alpha values for it. Now, on windows, text drawn using GDI doesn't support alpha. Text drawn using GDI+ or DrawThemeText() does. I suspect when it works the text is being drawn with DrawThemeText, and when it doesn't, with GDI.
I have an idea that windows has two possible methods for drawing the desktop background. This ties into the WorkerW/Progman thing I mentioned earlier.
When the system first boots up, the explorer desktop window layout looks like:
Code:
Progman
SHELLDLL_DefView
SysListView32
After you've changed the wallpaper through the control panel once, it looks like:
Code:
WorkerW
SHELLDLL_DefView
SysListView32
I believe this is related to the windows wallpaper slideshow feature. Someone else discusses it here.
Changing the wallpaper through the control panel results in a smooth fade effect, doing it progratically doesn't. I think changing it through the control panel enables the slideshow desktop, which draws the icon text using GDI+ or DrawThemeText. Changing it programatically doesn't enable it.
So, the only solutions to this I can think of are:
* Programatically switch windows to drawing the desktop using the second method. I'd bet this is what Dreamscene does. I've scoured google, found other people looking for answers to that same problem, but no solutions. Microsoft doesn't seem to have exposed the API for that.
* Somehow get the first method to write alpha values for the text it draws. Can't see any good way of doing this, if I'm right and it's using GDI for that, I don't believe there's any way GDI can write alpha values. Possibly could hook the GDI function and redirect it to DrawThemeText().
* Postprocess the bitmap to fill in alpha values for the text, shouldn't be that difficult, but it starts using the CPU quite heavily and could be prone to missing the text depending on drop shadows and colour, so I don't think it's a very good solution.
Here's my code, it must be compiled for whatever architecture (x86/x64) your windows is (because of the explorer injection).
The whole thing's a horrific hacky mess, and I couldn't be bothered writing clean uninjection code, so it just automatically removes it after 15 seconds, but it works.